CLI

The Pipekit CLI allows you to interact with Pipekit services without leaving your terminal.

Installation

To install the Pipekit CLI, fetch the precompiled binaries or packages from our releases page, or use the following install instructions, depending on your OS:

MacOS & Linux (Homebrew)

brew install pipekit/tap/cli

Homebrew homepage

Windows (Scoop)

scoop bucket add pipekit https://github.com/pipekit/scoop.git
scoop install pipekit/cli

Scoop homepage

Release Notes

Release notes are published on the Pipekit Releases site.

Docker Container

We produce an alpine-based Docker image that contains the Pipekit CLI. To use it you can run a docker run command appended with the pipekit arguments you require. You can log in to pipekit non-interactively by providing the required environment variables:

docker run \
-e PIPEKIT_USERNAME=myusername \
-e PIPEKIT_PASSWORD=mypassword \
pipekit13/cli \
pipekit login && \
pipekit list clusters

The latest tag is always pinned to the latest released version of the Pipekit CLI. To find the a specific version, check the tags on Docker Hub.

Containers are available in linux/amd64 and linux/arm64 variants. Both use Alpine Linux as the base image. They execute using the default alpine container user, root. If you need to run the container as a non-root user, you can use the --user flag and set it to the in-built pipekit user and group:

docker run \
-e PIPEKIT_USERNAME=myusername \
-e PIPEKIT_PASSWORD=mypassword \
-u pipekit:pipekit \
pipekit13/cli \
pipekit login && \
pipekit list clusters

Or you can copy the binary into a container that you control. See "Used within another container" for an example.

Used within a Workflow

It would be more common to use the container within a workflow. An example of this is shown below:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: pipekit-cli
spec:
  entrypoint: main
  templates:
  - name: main
    dag:
      tasks:
        - name: pipekit-cli
          template: pipekit-cli
  - name: pipekit-cli
    env:
      - name: PIPEKIT_USERNAME
        valueFrom:
          secretKeyRef:
            name: pipekit-credentials
            key: username
      - name: PIPEKIT_PASSWORD
        valueFrom:
          secretKeyRef:
            name: pipekit-credentials
            key: password
    container:
      image: pipekit13/cli
      command:
        - sh
        - -c
        - |
          pipekit login
          pipekit list clusters
      resources:
        requests:
          memory: 10Mi
          cpu: 10m

Used within another container

You can also use the Pipekit CLI within another container. This example Dockerfile shows how you could install the Pipekit CLI into a Jupyter notebook container:

FROM jupyter/scipy-notebook
COPY --from=pipekit13/cli /usr/local/bin/pipekit /usr/local/bin/pipekit

Pipekit CLI container Software Bill of Materials (SBOM)

An SBOM for the CLI container is embedded within the container image in SPDX format. Further information is available on the SBOM page.

Authentication

You need to authenticate before using the Pipekit CLI to submit workflows. This is done using your username & password, or using single sign-on (SSO), depending on how your account was created. Running the login command, you're prompted to choose one of the two mentioned methods:

$> pipekit login
? Select a login method:  [Use arrows to move, type to filter]
> Login with username and password
  Single sign-on (SSO)

After you've successfully logged in, either you provided your username and password or you've gone through the SSO pathway, selecting your identity provider and following the redirects through your browser, you should see the following message in your terminal:

successfully logged in, token expires on: 2023-02-23T12:14:52+01:00

Pipekit CLI stores your access token in your home directory, under ~/.pipekit/.

You can also log in non-interactively, by providing your username and password as arguments to the login command:

$> pipekit login -u username -p password

$> pipekit login --user username --password password

Alternatively, you can provide your credentials using environment variables:

export PIPEKIT_USERNAME=username
export PIPEKIT_PASSWORD=password
$> pipekit login

Submitting a workflow

To submit a workflow yaml for execution, use the submit command. Upon submission, Pipekit will run this workflow on the specified cluster; creating a new Pipe if none of the existing Pipes have the required name. Pipes can be automatically created when a workflow is submitted, in that case, they will be named after the value of Argo Workflow's generateName property. All future submissions of the workflow with the same generateName will create new runs grouped under the same Pipe.

To submit a workflow, you need to provide the path to the workflow yaml as an argument, and a list of flags, out of which only cluster-name is required, example:

$> pipekit submit --namespace staging --cluster-name=staging --pipe-name=workflowPipe -p param1=value1 -p param2=value2 ./path/to/workflow.yaml

Flag	Shorthand	Type	Description
--namespace	-n	string	name of the namespace to submit the workflow to
--cluster-name	-c	string	name of the cluster to submit the workflow to
--open-ui		bool	open UI for the submitted workflow
--parameter	-p	stringArray (key=value)	pass input parameter(s)
--wait	-w	bool	wait for the submitted workflow to complete
--pipe-name	-d	string	name of the pipe to submit the workflow to

Flag

Shorthand

Type

Description

--namespace

-n

string

name of the namespace to submit the workflow to

--cluster-name

-c

string

name of the cluster to submit the workflow to

--open-ui

bool

open UI for the submitted workflow

--parameter

-p

stringArray (key=value)

pass input parameter(s)

--wait

-w

bool

wait for the submitted workflow to complete

--pipe-name

-d

string

name of the pipe to submit the workflow to

Getting information about a run (executed workflow)

To get information about a run, use the get command. For example:

$> pipekit get run --run-uuid=6b0b0b1a-1b1a-4b1a-9b1a-1b1a1b1a1b1a
Run details:
RunUUID: b713faf9-5d7d-45cf-ae30-a63c6bcc9e3f
Status: completed
Created At: 2023-03-29 10:54:50.628945 +0000 UTC

You can also get information about a run by providing the cluster name, namespace and workflow name. In case there are multiple runs for the given workflow name, the latest one will be returned. For example:

$> pipekit get run --cluster-name=clusterA --namespace=default --workflow-name=workflowA
Run details:
RunUUID: b713faf9-5d7d-45cf-ae30-a63c6bcc9e3f
Status: completed
Created At: 2023-03-29 10:54:50.628945 +0000 UTC

Flag	Shorthand	Type	Description
--run-uuid	-r	string	run UUID of the workflow you want to get
--cluster-name	-c	string	name of the cluster where workflow is located
--namespace	-n	string	k8s namespace where workflow is/was running
--workflow-name	-w	string	name of the workflow

Flag

Shorthand

Type

Description

--run-uuid

-r

string

run UUID of the workflow you want to get

--cluster-name

-c

string

name of the cluster where workflow is located

--namespace

-n

string

k8s namespace where workflow is/was running

--workflow-name

-w

string

name of the workflow

Listing clusters, pipes and runs

The Pipekit CLI offers a list command that can be used to fetch and list pipekit entities, such as cluster, Pipes and runs.

Listing clusters

$> pipekit list clusters
[CLUSTER NAME]   [CLUSTER UUID]                         [ACTIVE]   [REGISTERED]
clusterB         53fe39bc-b163-4d22-8f9e-d9da3dfa0d08   true       true

Listing pipes

To list all Pipes that have at least one run on a given cluster:

$> pipekit list pipes --cluster-name=clusterB
[PIPE NAME]   [PIPE UUID]                            [ACTIVE]
diamond       afba5ea5-34c1-42a3-abc3-af0aa7bd8941   true

Flag	Shorthand	Type	Description
--cluster-name	-c	string	name of the cluster that has pipe runs (required)
--all	-A	bool	get both active and inactive pipes
--active		bool	active/inactive switch (defaults to true)

Flag

Shorthand

Type

Description

--cluster-name

-c

string

name of the cluster that has pipe runs (required)

--all

-A

bool

get both active and inactive pipes

--active

bool

active/inactive switch (defaults to true)

Listing runs

To list all runs on a given cluster:

$> pipekit list runs --cluster-name=clusterB
NAMESPACE   NAME            STATUS    AGE   DURATION   PRIORITY   MESSAGE   UI LINK
default     diamond-m91tv   RUNNING   2h    2h         0                    http://localhost:30000/pipes/afba5ea5-34c1-42a3-abc3-af0aa7bd8941/runs/dabd11d2-730c-4d28-969f-5c6ad9007725
default     diamond-89sfy   RUNNING   2h    2h         0                    http://localhost:30000/pipes/afba5ea5-34c1-42a3-abc3-af0aa7bd8941/runs/c5e36594-d19b-477e-9fb6-9f4aadc6e8bf

Flag	Shorthand	Type	Description
--cluster-name	-c	string	name of the cluster to list from
--all	-A	bool	get all runs, regardless of the status
--statuses	-s	stringArray	get runs with a given status (defaults to running)

Flag

Shorthand

Type

Description

--cluster-name

-c

string

name of the cluster to list from

--all

-A

bool

get all runs, regardless of the status

--statuses

-s

stringArray

get runs with a given status (defaults to running)

Run actions

There are certain actions that can be applied to runs after the workflow is submitted, those are stop, terminate and restart.

Stop and Terminate

To stop a running workflow:

$> pipekit stop --run-uuid b6129dbf-30c2-4135-bac5-7347a1928c22

To terminate a running workflow (stop immediately without running the exit handlers):

$> pipekit terminate --run-uuid b6129dbf-30c2-4135-bac5-7347a1928c22

Both of these commands have the same flags:

Flag	Shorthand	Type	Description
--run-uuid	-r	string	run UUID of the running workflow you want to stop/terminate (required)

Flag

Shorthand

Type

Description

--run-uuid

-r

string

run UUID of the running workflow you want to stop/terminate (required)

Restart

To restart a run, resubmitting the workflow for execution:

$> pipekit restart --run-uuid b6129dbf-30c2-4135-bac5-7347a1928c22

Flag	Shorthand	Type	Description
--run-uuid	-r	string	run UUID of the run you want to restart (required)
--open-ui		bool	open the UI of the resubmitted workflow
--wait	-w	bool	wait for the workflow to complete

Flag

Shorthand

Type

Description

--run-uuid

-r

string

run UUID of the run you want to restart (required)

--open-ui

bool

open the UI of the resubmitted workflow

--wait

-w

bool

wait for the workflow to complete

Hera

Users that want to run their Hera workflows (and CronWorkflows) through Pipekit just need to update the token for the one provided by Pipekit and use the Pipekit Python SDK. To get the necessary information use the hera command in the pipekit CLI:

$> pipekit hera
Token:   eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

Hera Example

The following example runs the coinflip workflow in the argo namespace using the argo-workflow service account via Pipekit.

from hera.workflows import Container, Step, Steps, Workflow, script
from pipekit_sdk.service import PipekitService

pipekit = PipekitService(token='eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c')

@script(image="python:alpine3.6", command=["python"], add_cwd_to_sys_path=False)
def flip_coin() -> None:
    import random

    result = "heads" if random.randint(0, 1) == 0 else "tails"
    print(result)

with Workflow(
    generate_name="coinflip-",
    annotations={
        "workflows.argoproj.io/description": (
            "This is an example of coin flip defined as a sequence of conditional steps."
        ),
    },
    entrypoint="coinflip",
    namespace="argo",
    service_account_name="argo-workflow"
) as w:
    heads = Container(
        name="heads",
        image="alpine:3.6",
        command=["sh", "-c"],
        args=['echo "it was heads"'],
    )
    tails = Container(
        name="tails",
        image="alpine:3.6",
        command=["sh", "-c"],
        args=['echo "it was tails"'],
    )

    with Steps(name="coinflip") as s:
        fc: Step = flip_coin()

        with s.parallel():
            heads(when=f"{fc.result} == heads")
            tails(when=f"{fc.result} == tails")

pipekit.submit(w, "clusterB")

Logs

To observe pod logs directly in the Pipekit CLI, use the logs command:

$> pipekit logs -r b6129dbf-30c2-4135-bac5-7347a1928c22 -p diamond-18261664 -c main -f

Logs of a given pipe run can be filtered by the pod name and container name.

Flag	Shorthand	Type	Description
--run-uuid	-r	string	UUID of the run (required)
--container	-c	string	name of the container to filter logs by
--pod	-p	string	name of the pod to filter logs by
--follow	-f	bool	follow logs stream until run is finished

Flag

Shorthand

Type

Description

--run-uuid

-r

string

UUID of the run (required)

--container

-c

string

name of the container to filter logs by

--pod

-p

string

name of the pod to filter logs by

--follow

-f

bool

follow logs stream until run is finished

Last updated 26 days ago

Installation

MacOS & Linux (Homebrew)

Windows (Scoop)

Release Notes

Docker Container

Used within a Workflow

Used within another container

Pipekit CLI container Software Bill of Materials (SBOM)

Authentication

Non-Interactive Login

Submitting a workflow

Getting information about a run (executed workflow)

Listing clusters, pipes and runs

Listing clusters

Listing pipes

Listing runs

Run actions

Stop and Terminate

Restart

Hera

Hera Example

Logs