CLI
The Pipekit CLI allows you to interact with Pipekit services without leaving your terminal.
Installation
To install the Pipekit CLI, fetch the precompiled binaries or packages from our releases page, or use the following install instructions, depending on your OS:
MacOS & Linux (Homebrew)
brew install pipekit/tap/cli
Windows (Scoop)
scoop bucket add pipekit https://github.com/pipekit/scoop.git
scoop install pipekit/cli
Release Notes
Release notes are published on the Pipekit Releases site.
Docker Container
We produce a Wolfi-based Docker image that contains the Pipekit CLI. To use it you can run a docker run
command appended with the pipekit arguments you require. You can log in to pipekit non-interactively by providing the required environment variables:
docker run \
-e PIPEKIT_USERNAME=myusername \
-e PIPEKIT_PASSWORD=mypassword \
pipekit13/cli \
pipekit login && \
pipekit list clusters
The latest
tag is always pinned to the latest released version of the Pipekit CLI. To find the a specific version, check the tags on Docker Hub.
Containers are available in linux/amd64 and linux/arm64 variants. Both use Wolfi Linux as the base image.
Or you can copy the binary into a container that you control. See "Used within another container" for an example.
Used within a Workflow
It would be more common to use the container within a workflow. An example of this is shown below:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
name: pipekit-cli
spec:
entrypoint: main
templates:
- name: main
dag:
tasks:
- name: pipekit-cli
template: pipekit-cli
- name: pipekit-cli
env:
- name: PIPEKIT_USERNAME
valueFrom:
secretKeyRef:
name: pipekit-credentials
key: username
- name: PIPEKIT_PASSWORD
valueFrom:
secretKeyRef:
name: pipekit-credentials
key: password
container:
image: pipekit13/cli
command:
- sh
- -c
- |
pipekit login
pipekit list clusters
resources:
requests:
memory: 10Mi
cpu: 10m
Used within another container
You can also use the Pipekit CLI within another container. This example Dockerfile shows how you could install the Pipekit CLI into a Jupyter notebook container:
FROM jupyter/scipy-notebook
COPY --from=pipekit13/cli /usr/local/bin/pipekit /usr/local/bin/pipekit
Pipekit CLI container Software Bill of Materials (SBOM)
An SBOM for the CLI container is embedded within the container image in SPDX format. Further information is available on the SBOM page.
Authentication
You need to authenticate before using the Pipekit CLI to submit workflows. This is done using your username & password, or using single sign-on (SSO), depending on how your account was created. Running the login
command, you're prompted to choose one of the two mentioned methods:
$> pipekit login
? Select a login method: [Use arrows to move, type to filter]
> Login with username and password
Single sign-on (SSO)
After you've successfully logged in, either you provided your username and password or you've gone through the SSO pathway, selecting your identity provider and following the redirects through your browser, you should see the following message in your terminal:
successfully logged in, token expires on: 2023-02-23T12:14:52+01:00
Pipekit CLI stores your access token in your home directory, under ~/.pipekit/
.
Non-Interactive Login
You can also log in non-interactively, by providing your username and password as arguments to the login
command:
$> pipekit login -u username -p password
or
$> pipekit login --user username --password password
Alternatively, you can provide your credentials using environment variables:
export PIPEKIT_USERNAME=username
export PIPEKIT_PASSWORD=password
$> pipekit login
Submitting a workflow
To submit a workflow yaml for execution, use the submit
command. Upon submission, Pipekit will run this workflow on the specified cluster; creating a new Pipe if none of the existing Pipes have the required name. Pipes can be automatically created when a workflow is submitted, in that case, they will be named after the value of Argo Workflow's generateName
property. All future submissions of the workflow with the same generateName
will create new runs grouped under the same Pipe.
To submit
a workflow, you need to provide the path to the workflow yaml as an argument, and a list of flags, out of which only cluster-name
is required, example:
$> pipekit submit --namespace staging --cluster-name=staging --pipe-name=workflowPipe -p param1=value1 -p param2=value2 ./path/to/workflow.yaml
--namespace
-n
string
name of the namespace to submit the workflow to
--cluster-name
-c
string
name of the cluster to submit the workflow to
--open-ui
bool
open UI for the submitted workflow
--parameter
-p
stringArray (key=value)
pass input parameter(s)
--wait
-w
bool
wait for the submitted workflow to complete
--pipe-name
-d
string
name of the pipe to submit the workflow to
--json
bool
output the run details in JSON format
Getting information about a run (executed workflow)
To get information about a run, use the get
command. For example:
$> pipekit get run --run-uuid=6b0b0b1a-1b1a-4b1a-9b1a-1b1a1b1a1b1a
Run details:
RunUUID: b713faf9-5d7d-45cf-ae30-a63c6bcc9e3f
Status: completed
Created At: 2023-03-29 10:54:50.628945 +0000 UTC
You can also get information about a run by providing the cluster name, namespace and workflow name. In case there are multiple runs for the given workflow name, the latest one will be returned. For example:
$> pipekit get run --cluster-name=clusterA --namespace=default --workflow-name=workflowA
Run details:
RunUUID: b713faf9-5d7d-45cf-ae30-a63c6bcc9e3f
Status: completed
Created At: 2023-03-29 10:54:50.628945 +0000 UTC
--run-uuid
-r
string
run UUID of the workflow you want to get
--cluster-name
-c
string
name of the cluster where workflow is located
--namespace
-n
string
k8s namespace where workflow is/was running
--workflow-name
-w
string
name of the workflow
Listing clusters, pipes and runs
The Pipekit CLI offers a list
command that can be used to fetch and list pipekit entities, such as cluster, Pipes and runs.
Listing clusters
$> pipekit list clusters
[CLUSTER NAME] [CLUSTER UUID] [ACTIVE] [REGISTERED]
clusterB 53fe39bc-b163-4d22-8f9e-d9da3dfa0d08 true true
Listing pipes
To list all Pipes that have at least one run on a given cluster:
$> pipekit list pipes --cluster-name=clusterB
[PIPE NAME] [PIPE UUID] [ENABLED]
diamond afba5ea5-34c1-42a3-abc3-af0aa7bd8941 true
--all
-A
bool
get all Pipes, both enabled and disabled
--cluster-name
-c
string
the name of the cluster to list from (required)
--enabled
bool
enabled/disabled switch (default true)
Listing runs
To list all runs on a given cluster:
$> pipekit list runs --cluster-name=clusterB
NAMESPACE NAME STATUS AGE DURATION PRIORITY MESSAGE UI LINK
default diamond-m91tv RUNNING 2h 2h 0 http://localhost:30000/pipes/afba5ea5-34c1-42a3-abc3-af0aa7bd8941/runs/dabd11d2-730c-4d28-969f-5c6ad9007725
default diamond-89sfy RUNNING 2h 2h 0 http://localhost:30000/pipes/afba5ea5-34c1-42a3-abc3-af0aa7bd8941/runs/c5e36594-d19b-477e-9fb6-9f4aadc6e8bf
--cluster-name
-c
string
name of the cluster to list from
--all
-A
bool
get all runs, regardless of the status
--statuses
-s
stringArray
get runs with a given status (defaults to running)
Run actions
There are certain actions that can be applied to runs after the workflow is submitted, those are stop
, terminate
and restart
.
Stop and Terminate
To stop a running workflow:
$> pipekit stop --run-uuid b6129dbf-30c2-4135-bac5-7347a1928c22
To terminate a running workflow (stop immediately without running the exit handlers):
$> pipekit terminate --run-uuid b6129dbf-30c2-4135-bac5-7347a1928c22
Both of these commands have the same flags:
--run-uuid
-r
string
run UUID of the running workflow you want to stop/terminate (required)
Restart
To restart a run, resubmitting the workflow for execution:
$> pipekit restart --run-uuid b6129dbf-30c2-4135-bac5-7347a1928c22
--run-uuid
-r
string
run UUID of the run you want to restart (required)
--open-ui
bool
open the UI of the resubmitted workflow
--wait
-w
bool
wait for the workflow to complete
Import from Workflow Archive
An admin can import archived workflows from an Argo Workflow Archive to Pipekit. This action will import the workflows and their metadata into Pipekit.
The import of Workflow Archive Logs is not supported.
Use the import workflows
command in the Pipekit CLI:
$> pipekit import workflows --cluster-name test-cluster
--cluster-name
-c
string
name of the cluster to assign workflows to (required)
--argoServerURI
string
Argo Server URI (defaults to localhost:2746)
--argoAuthToken
string
Argo Server authentication token
--batchSize
integer
number of workflows to fetch from the Workflow Archive per API call (defaults to 10)
--argoServerSubPath
string
sub path used when connecting to Argo Server
--insecure
-k
bool
skip TLS verification (defaults to false)
--useSSL
bool
use SSL when connecting to Argo Server (defaults to true)
Hera
Users can run the pipekit hera
command to get a token to run their Hera
workflows (and CronWorkflows) through Pipekit using the Pipekit Python SDK.
Use the hera
command in the pipekit CLI:
$> pipekit hera
Token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
See the Pipekit Python SDK documentation for more details and examples.
Logs
To observe pod logs directly in the Pipekit CLI, use the logs
command:
$> pipekit logs -r b6129dbf-30c2-4135-bac5-7347a1928c22 -p diamond-18261664 -c main -f
Logs of a given pipe run can be filtered by the pod name and container name.
--run-uuid
-r
string
UUID of the run (required)
--container
-c
string
name of the container to filter logs by
--pod
-p
string
name of the pod to filter logs by
--node-id
-n
string
id of the workflow node to filter logs by
--follow
-f
bool
follow logs stream until run is finished
Last updated