Dependencies and Pre-requisites

Pre-requisites for self-hosting Pipekit

Pipekit depends on the following tools being available in your environment:

Kubernetes

• We recommend Kubernetes 1.32 or greater.

Setup

• The installation makes use of ingresses to route traffic to the various services. You will need to have an ingress controller installed in your cluster. We recommend using the ingress-nginx controller.

• OTEL or Prometheus will allow you to monitor the pipekit services on their defined metrics ports. This is not a strict requirement but is recommended.

• An in-cluster log aggregation tool such as Loki or Fluentd is recommended to collect logs from the various services. This is not a strict requirement but is recommended.

• The Pipekit services make use of HPAs to scale under load. You will need to have the metrics server installed in your cluster in order to make use of this.

Postgres

• We recommend you use a managed Postgres service such as AWS RDS or Azure Postgres.

• We recommend Postgres 17 or greater.

Setup

• The Pipekit services require a Postgres database to store their data. You will need to create a database and user for the id and users services to use:

  CREATE DATABASE users;
  CREATE DATABASE id;

Redis

• Can be inside the cluster or a managed service such as AWS Elasticache.

• We recommend Redis 7 or greater running in a high availability setup with sentinel.

Setup

No special configuration is required for Redis. We would recommend that it is configured to be highly available. It would also be advisable to configure Redis to use a password which can be passed to the Pipekit Helm Chart.

Grafana Loki

• Can be hosted within the Kubernetes cluster, or externally.

• We recommend Loki 3.5 or greater.

Setup

No special configuration is required for Loki. We would recommend that it is configured to be highly available.

Git Provider (GitHub, GitLab)

You can optionally connect your Git provider to Pipekit in order to make use of the GitOps features. This is optional, but highly recommended.

Setup

Follow the instructions in the Git Provider documentation to connect your Git provider to Pipekit.