Getting Started
How to run a pipeline on Pipekit
The following sign-up flow is in private beta. Please shoot us an email at [email protected] if you'd like to be added to the Pipekit waitlist or to request a live demo.
To install Pipekit, you will need the following items available:
- 1.A Kubernetes cluster, either with Argo Workflows already installed, or with the desire to install Argo Workflows into that cluster.
- 1.
- 2.Fill out your basic user information and hit Submit
- 3.Check your email for a welcome email containing a link to confirm your account
- 4.Click the link to confirm your account and log in.
If you are not the administrator of the organization, check your email for an invite link. Upon clicking the link, you will be registered for the organization and can either connect a cluster or run a pipeline.
If you are the administrator of the organization using Pipekit, take the following steps:
- 1.On the, "Orgs" page, click the "+ Add Org" button.
- 2.When prompted, give a name and description for the organization, and click, "Submit".
- 3.Click on your newly created organization in the list.
- 4.You will then be taken to a page to configure users, add any users you would like by typing their email in the text input and then click the, "Invite" button. From here, you can either connect a GitHub or Gitlab account or go to the next step to connect a cluster.
Under your organization view, there is a tab called, "Settings". Click on it, and you will see a section for connecting your organization's GitHub or Gitlab account.
- 1.Click the button for GitHub. It will lead you through the process to authorize Pipekit's GitHub App.
- 2.When prompted by GitHub, select either, "All repositories" or select the list of repositories you would like to grant Pipekit access to using the dropdown.
- 3.Click, "Authorize"
For Gitlab, you will need to create a Personal Access Token. We recommend creating a service account user in your organization on Gitlab that has access to any repositories that you want Pipekit to read from.
To connect Pipekit, click the button for Gitlab, input the Personal Access Token that you created, and click, "Submit"
Now that you've created an organization, we will connect Pipekit to Kubernetes clusters where we want to run workflows. Navigate to the, "Clusters" page using the navigation bar. From here, take the following steps:
- 1.Click the, "+ Connect Cluster" button to be taken to the connect cluster page.
- 2.Select the organization you want to use to connect the cluster.
- 3.Add the name for the cluster. For ease of use, we recommend keeping the name all lower case with no whitespace.
- 4.Add the namespace that you would like to install the Pipekit Agent in. Common namespaces are
argoandpipekit, although you can choose any namespace you wish, including the namespace where Argo Workflows resides (by default,argo). - 5.Add a description and click, "Submit".
- 6.You will then be given the option to either install the Pipekit Agent on your cluster using either a Kubernetes YAML Manifest or Helm Chart. Install it with whichever method you would like. At this point you can also choose to install Argo Workflows using the same installer.
- 7.Upon successful installation, you will be redirected to the Pipes page to configure a pipeline.
In Pipekit, we use the term Pipe to refer to an Argo Workflow and all of the runs associated with that Workflow. We can create and run Pipes using three methods:
- 1.The Pipekit CLI
- 2.The GitHub/Gitlab account we configured above
- 3.Directly in a Hera Workflow
The CLI is the quickest method to run a Pipe. Take the following steps:
- 1.
- 2.Navigate to an Argo Workflows file. We've included a basic example below
- 3.Login through the CLI using
pipekit login - 4.Submit the Argo Workflow to the cluster you connected above using the command
pipekit submit --cluster-name=your-cluster-name argo-workflow.yaml
Sample Argo Workflow file:
dag-diamond.yaml
1KB
Code
dag-diamond.yaml
After connecting your GitHub/Gitlab, click the + icon in the navigation bar to get to the Create Pipe page. Configure the pipeline by filling out the form using these steps:
- 1.Add a name for your pipeline. For now, let's call our pipeline DAG Example
- 2.Add a description. We'll use this description: A directed, acyclic graph
- 3.Select the repo where your configuration yaml file will live on GitHub/Gitlab
- 4.Enter the location of the pipeline configuration file. We'll be using dag-diamond.yaml for this example.
- 5.In the "Run Conditions" section, specify when a pipeline should run. Right now, set the condition to be Branch and main. You can also run pipelines off Git tags and use regex matching.
- 6.Click submit to create your pipeline
On your computer, go to the repo you used in the section above. Add the
dag-diamond.yaml file below, commit the change, and push it to your main branch.dag-diamond.yaml
1KB
Code
dag-diamond.yaml
git add .
git commit -m "Run pipeline"
git push origin master
Then click on the example pipeline in the pipes page. You will then see a list of pipeline runs that now has one entry. Click where it says "BRANCH: main" to see the pipeline result and the log output.
It is best practice to set a Namespace and Service Account for your Argo Workflow. Depending on how your organization has the cluster set up, it may be mandatory. If you submit a workflow without a namespace or service account, the Pipekit Agent will use the
default namespace and service account for the cluster, which may not have the permissions needed to run the workflow.If you are using the Pipekit Agent Helm Chart to install Argo Workflows, the default Service Account Name is
argo-workflow and the namespace is the namespace you chose to install Argo Workflows into.This sample Argo Workflow file is the same as above, but has the namespace set to
argo and the serviceAccount set to argo-workflow. Adjust for your needs:dag-diamond-namespace-sa.yaml
1KB
Code
dag-diamond-namespace-sa.yaml
Last modified 29d ago