Workflow of Workflows

Workflow of Workflows

A Workflow of Workflows is a parent Workflow that starts other Workflows. The Workflows it starts are its children.

Pipekit keeps the children with their parent. Every child run lives in the same Pipe as the parent, so you get one Pipe for the whole Workflow of Workflows instead of one Pipe per child.

Before you start

Pipekit does not trigger the child Workflows, so the Pipekit Agent reads them from your cluster. Turn on externally-triggered Workflows first. The children will not show up without it.

Run one

1. Start the parent Workflow with pipekit submit or your own method.

2. Open the Runs dashboard. Only the parent run shows in the list.

3. Open the parent run to see its child runs and their status.

4. Open a child run to read its logs, then use the parent link to go back.

Link a node to its child

A parent run can open many children, so Pipekit cannot always tell which node started which child on its own. Annotate each child with the name of the node that created it and Pipekit draws a direct link.

Add the workflows.pipekit.io/parent_node annotation to the child Workflow and set its value to {{node.name}}. Argo replaces {{node.name}} with the name of the parent node that creates the child.

- name: submit-child
  resource:
    action: create
    setOwnerReference: true
    manifest: |
      apiVersion: argoproj.io/v1alpha1
      kind: Workflow
      metadata:
        generateName: my-child-
        annotations:
          workflows.pipekit.io/parent_node: "{{node.name}}"
      spec:
        # ...

Open the parent run, open the graph or step list, and select the node that started a child. The node panel shows a Child Run button that opens that child.

The annotation is optional. Use an annotation, not a label: node names contain characters that Kubernetes rejects in label values. Without it, you still reach the children from the Children section of the parent run.

See the metrics

Open the parent's Pipe and go to the Performance tab. Parent runs and child runs each get their own sub-tab. Use the Children sub-tab to look at the child runs on their own.

Start the parent with pipekit submit. A child that starts before Pipekit has registered the parent lands in its own Pipe instead of the parent's.