Git Provider Integration

Integrating with a Git provider such as GitHub, or GitLab

Pipekit integrates with Git providers such as GitHub and GitLab so users can trigger Workflows from commits and pull requests. This guide will walk you through the steps to integrate them with Pipekit.

These examples assume you are self-hosting Pipekit on the subdomain pipekit.example.com. Replace this with your actual domain.

GitHub

1. Go to https://github.com/organizations/example/settings/apps replacing example with your github organization.

2. Click on "New GitHub App"

3. Fill in the details:

   • GitHub App name: [Choose a unique App name]

   • Homepage URL: https://pipekit.example.com

   • Callback URL: https://pipekit.example.com/accounts/github

   • Add a second Callback URL: https://pipekit.example.com/api/id/v1/sso/github/callback

     • Tick "Request user authorization (OAuth) during installation"

   • Webhook URL: https://pipekit.example.com/api/events-handler/v1/events/github

   • Repository permissions:

     • Administration: Read & Write

     • Checks: Read & Write

     • Contents: Read & Write

     • Deployments: Read & Write

     • Pull requests: Read & Write

     • Webhooks: Read & Write

   • Subscribe to events:

     • Create

     • Pull request

     • Push

     • Repository

4. Click "Create GitHub App"

5. Generate a private key and download it as prompted.

6. Note down the App name, App ID, Client ID and generate a new Client Secret.

7. Populate the Helm Chart Values with the collected information and install Pipekit.

8. Install Pipekit and log in with the Break Glass User account.

9. Create an organization and cluster. Go back to the organization and click settings and then press the GitHub button to authenticate the Org with GitHub.

10. Install the Github App into the repo(s) you wish Pipekit to interact with.

GitLab

1. Enable gitProviders.gitlab.enabled in the Helm Chart Values to configure Pipekit to use GitLab.

2. Go to your Organization settings in Pipekit and click on "GitLab" to authenticate your Organization with GitLab.

3. Enter a PAT that has the following scopes:

• api

• read_user

• read_repository

1. Choose a random string for the gitProviders.gitlab.webhookSecret and apply it to the Helm chart values. This will be used to automatically create a webhook in GitLab to post events to Pipekit.

BitBucket Data Center

1. Enable gitProviders.bitbucket.enabled in the Helm Chart Values to configure Pipekit to use BitBucket.

2. Ensure a secure gitProviders.bitbucket.webhookSecret is set. This is used to automatically create webhooks in BitBucket to notify Pipekit.

3. Go to your Organization settings in Pipekit and click on "BitBucket Data Center" to authenticate your Organization with GitLab.

4. Enter your HTTP access token that has the following rights:

• PROJECT_READ

• REPO_ADMIN

Azure DevOps

Pipekit supports both Azure DevOps Services (cloud, dev.azure.com/{org}). One Pipekit account corresponds to one Azure DevOps organization; projects within that organization are surfaced in Pipekit's repo picker grouped under their project name.

Helm chart configuration

1. Enable gitProviders.azureDevOps.enabled in the Helm Chart Values.

2. Set gitProviders.azureDevOps.webhookBasicAuthUser and gitProviders.azureDevOps.webhookBasicAuthPass to a username/password pair of your choice. Pipekit configures every Service Hook subscription it creates to deliver with these Basic auth credentials and verifies them on each incoming delivery.

Authentication: Personal Access Token (PAT)

Create a PAT with the permissions in the table below, then connect the account in Pipekit by submitting the organization URL (https://dev.azure.com/{org} for Services, your server URL for Server) and the PAT.

Permission	Why Pipekit needs it	Required or optional
Code (Read)	Clone the repo and read files, refs, tags, and the default branch, and list repositories in the picker	Required
Code (Status)	Post pipe run status back to commits and pull requests	Optional (only when commit statuses are enabled)
Project & Team (Read)	Resolve a project name to its GUID when registering Service Hook subscriptions	Optional (only when Pipekit creates the hooks)
Edit subscriptions (project permission, not a PAT scope)	Create and delete Service Hook subscriptions on the repo	Optional (only when Pipekit creates the hooks)

The last three rows are gated by the two write toggles described in Limiting Pipekit's writes. With manual service hooks, both Project & Team (Read) and Edit subscriptions become unnecessary; drop Code (Status) too if you disable commit statuses. A fully read-only setup (manual hooks and no commit statuses) needs only Code (Read).

Edit subscriptions is a project-level permission, not a PAT scope. It is held by a Project Administrator or granted directly to the identity. Microsoft made the old vso.hooks* PAT scope private, so this permission is the only way to authorize hook creation.

Pipekit never writes to your repository contents. Its only writes to Azure DevOps are Service Hook subscriptions and commit statuses, and both can be turned off per account.

Service Hooks

Pipekit creates three Service Hook subscriptions per repo on first attach: git.push, git.pullrequest.created, and git.pullrequest.updated. Pull request updates are filtered to source-branch push notifications. Branch creation, tag creation, and tag updates are all carried by git.push. Cleanup on detach removes all three. There is no global Service Hook subscription — every repo manages its own.

Webhook delivery URL

https://<your-pipekit-host>/api/events-handler/v1/events/azure-devops

Limiting Pipekit's writes to Azure DevOps

Pipekit never writes to your repository contents. Its only writes are Service Hook subscriptions and commit statuses, and each can be disabled per account from the Connect Azure DevOps dialog:

• I'll create the service hooks manually stops Pipekit from creating or deleting Service Hook subscriptions. Enable this if you cannot grant the Edit subscriptions permission. You must then create the subscriptions yourself (see below), otherwise commits and pull requests will not trigger pipes.

• Don't post commit statuses to Azure DevOps stops Pipekit from posting run status back to commits and pull requests. Enable this if you do not want to grant the Code (Status) scope.

Creating service hooks manually

Azure DevOps service hooks are created per repository, per event. Every repository you use in a pipe needs all three of these subscriptions: git.push, git.pullrequest.created, and git.pullrequest.updated. There is no project-wide or account-wide hook, so for N repositories you create 3 × N subscriptions in total.

For each repository, create all three of these subscriptions:

• Code pushed (git.push) — also carries branch and tag creation, so no separate hook is needed for those.

• Pull request created (git.pullrequest.created).

• Pull request updated (git.pullrequest.updated) — also set the change filter to the source-branch push option (PushNotification) so that only new commits trigger a run, not reviewer or vote changes.

All three share the same URL and Basic auth; only the trigger differs. Create each one as follows:

1. In Azure DevOps, open Project settings (the gear icon at the bottom of the left sidebar).

2. Under General, select Service hooks, then click the + (plus) icon.

3. For the service, select Web Hooks, then click Next.

4. Select the trigger and set its Repository filter to the target repository, then click Next. You must set the Repository filter; leaving it unset delivers events for every repository in the project, and Azure DevOps bills per delivery.

5. In URL, enter the webhook delivery URL.

6. In Basic authentication username and Basic authentication password, enter the pair you configured for Pipekit (see the note below). Optionally click Test to confirm Pipekit returns 200, then click Finish.

Repeat these steps for all three triggers, then for every other repository you attach to a pipe.

The Basic authentication username and password are not an Azure DevOps user. They are an arbitrary pair you invent purely to secure the webhook payloads. Pipekit's events handler holds the same pair, set through the gitProviders.azureDevOps.webhookBasicAuthUser and gitProviders.azureDevOps.webhookBasicAuthPass Helm values (exposed to the service as the AZURE_DEVOPS_WEBHOOK_BASIC_AUTH_USER and AZURE_DEVOPS_WEBHOOK_BASIC_AUTH_PASS environment variables), and uses it to verify that each incoming delivery is authentic. The pair must be identical in every subscription and in your Helm values, and neither value may be empty, or Pipekit rejects the delivery with 401 Unauthorized.

Config-as-code and namespace-permission triggers use their own hooks. If you use config-as-code, create all three subscriptions again with the ?type=config-as-code variant of the delivery URL. If you use namespace permissions, create only a Code pushed (git.push) subscription with the ?type=permission variant. Do not create pull request subscriptions for the ?type=permission variant. Pipekit handles pull request events the same way regardless of the type parameter, so the extra hooks would trigger duplicate runs.

Limiting Pipekit's requests to your git provider

You can cap how many API requests per minute Pipekit sends to your git provider. Set the limit in Settings > Accounts using the API rate limit (requests/min) field. Leave the field empty for no limit.

The limit applies to the connected account as a whole: every Pipekit component that talks to your git provider shares the same budget, no matter how many replicas are running. Requests beyond the budget wait for the next available slot instead of failing, so short bursts are smoothed out rather than rejected. Waiting is capped at 2 minutes: a request that cannot get a slot within that time (or before its own timeout) fails with a timeout error rather than queueing indefinitely.

Setting a limit adds a Redis check to every git provider request. If Redis is unavailable, requests for rate-limited accounts fail rather than exceed the configured cap.

Next Steps

If you haven't done so, continue to create a cluster and then create a pipe to start using Pipekit.

You have now successfully integrated your git provider with your Organization. This will allow you to use a number of Pipekit's features:

• Create Pipes that run Workflows stored in your Git repositories.

• Create and manage Pipes that use Run Conditions to trigger Workflows on GitHub events.

• The WorkflowTemplates features of Pipekit.

• Manage Workflow Priorities

• Manage Namespace Permissions through GitOps.

Populating the repository cache in Pipekit

If you enable a Git Provider, a Kubernetes cron job will be installed into the cluster that will run every night to query your git provider for an updated list of Pipekit-accessible repositories and store them in the Pipekit database. This is done to ensure that Pipekit has the most up-to-date list of repositories to work with and prevents excessive API calls to your git provider.