LogoLogo
PipekitPricingBlogPipekit StatusRelease Notes
  • Introduction
  • Getting Started
  • CLI
    • Cron Workflows
  • Pipekit Agent
    • Helm Install
  • Pipekit
    • Authentication
      • Okta
    • Runs
    • Pipes
      • Managing Pipes
        • Run Conditions
        • Secrets
        • Alerting
      • Pipe Runs
        • Run Graph (DAG)
        • Pod Logs
        • Workflow Logs
        • Workflow YAML
      • Cron Workflows
      • Externally Triggered Workflows
    • Metrics
    • Templates
    • Clusters
    • Organization
      • Creating an Organization
      • Managing Users
      • Managing Alert Providers
      • Settings
      • Permissions
  • Python SDK
    • Jupyter Notebooks
    • Python Scripts
  • Self-Hosting Pipekit
    • Dependencies and Pre-requisites
    • Container Images
    • Kubernetes Permissions
    • Self-Hosted Pipekit Helm Chart
    • License Key
    • Initial Login and Break Glass Account
    • Integrating with your Git Provider
    • Configuring SSO
  • Additional Information
    • Free Trial Cluster
  • REST API
Powered by GitBook
On this page
  • Get a token
  • Submission
  • Log Streaming
  1. Python SDK

Jupyter Notebooks

Using the Pipekit Python SDK in Jupyter Notebooks

Last updated 9 months ago

Get a token

Use the to generate a token using pipekit hera (ensure you are logged in first).

You can then pass this token to a PipekitService:

from pipekit_sdk.service import PipekitService

# Create a Pipekit service that is used to talk to the Pipekit API
pipekit = PipekitService(token="<token>")

Submission

Create a Workflow to submit:


from hera.workflows import DAG, Workflow, script

@script(image="python:3.12")
def echo(message):
    print(message)

with Workflow(
    generate_name="dag-diamond-",
    entrypoint="diamond",
    namespace="argo",
    service_account_name="argo-workflow",
) as w:
    with DAG(name="diamond"):
        A = echo(name="A", arguments={"message": "A"})
        B = echo(name="B", arguments={"message": "B"})
        C = echo(name="C", arguments={"message": "C"})
        D = echo(name="D", arguments={"message": "D"})
        A >> [B, C] >> D

Then use submit to run the Workflow on Pipekit, remember to pass in your cluster name.

# Submit the Workflow to Pipekit
pipe_run = pipekit.submit(w, "<cluster-name>")
print("pipe_run", pipe_run, "\n")

Log Streaming

After you have submitted a Workflow as above, you will be holding a "pipe_run" - you can stream the logs to your Notebook using print_logs on the uuid:

pipekit.print_logs(pipe_run["uuid"])
Pipekit CLI