> For the complete documentation index, see [llms.txt](https://docs.pipekit.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.pipekit.io/organizations-and-access/byo-logs.md).

# Bring Your Own Logging

Pipekit allows you to connect your self-hosted Loki instance to the Pipekit control plane. Once you add your Loki configuration to Pipekit, you will have to use a log collector on your Kubernetes clusters in order to send the logs from your cluster to your Loki instance.

## Connect a Loki Instance

Go to the [Org tab in Pipekit](https://pipekit.io/organization). Then navigate to the `BYO Logs` tab. Enter the address of your Loki instance along with a username and password that you use for authenticating at the gateway/ingress layer. Then submit the information. You'll have to make sure that the address is accessible to the Internet and that the following endpoints are reachable:

```yaml
/loki/api/v1/query
/loki/api/v1/query_range
/loki/api/v1/tail
/loki/api/v1/series
```

## Configuring a Log Collector

In order to get the logs from your Kubernetes clusters running Argo Workflows into your Loki instance, you'll have to configure a log collector. We use and test with [Fluent Bit](https://fluentbit.io/).

### Labels

Pipekit appends three labels to pods run by workflows on clusters where the Pipekit agent is present and configured to either submit workflows or read externally submitted workflows. This works for both Pipekit-submitted and externally submitted workflows. The labels are the following:

```yaml
workflows.pipekit.io/org_uuid
workflows.pipekit.io/pipe_uuid
workflows.pipekit.io/run_uuid
```

Map these to `orgUUID`, `pipeUUID`, and `runUUID` in Loki.

Pipekit also reads three more labels that your log collector must set on each log stream: `nodeId`, `podName`, and `containerName`. `podName` is the name of the pod and `containerName` is the name of the container. `nodeId` is the Argo workflow node ID. Argo sets this ID on each pod as the `workflows.argoproj.io/node-id` annotation, so read the value from that annotation.

{% hint style="warning" %}
`nodeId` must hold the Argo workflow node ID, not the name of the Kubernetes cluster node that the pod ran on. Pipekit matches logs to a workflow step by `nodeId`. If you map it to the cluster node name, the logs for that step do not appear in Pipekit.
{% endhint %}

### Fluent Bit Config

Here's a sample fluent bit config that we'd recommend. Note that this configuration reads logs directly from the container file system:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentbit-config
  namespace: <<REPLACE_WITH_NAMESPACE>>
data:
  fluent-bit.conf: |
    [SERVICE]
        Flush         1
        Log_Level     info
        Daemon        off
        Parsers_File  parsers.conf
    @INCLUDE input.conf
    @INCLUDE output.conf

  input.conf: |
    [INPUT]
        Name                tail
        Parser              json
        Path                /tmp/*.log # set to whatever path your logs will be stored in the filesystem
        Mem_Buf_Limit       100MB
        Skip_Long_Lines     On
        Refresh_Interval    5
        Rotate_Wait         15

  output.conf: |
    [OUTPUT]
        name                    loki
        match                   *
        host                    <<REPLACE_WITH_LOKI_HOST>>
        port                    <<REPLACE_WITH_LOKI_PORT>>
        labels                  orgUUID=$orgUUID, pipeUUID=$pipeUUID, runUUID=$runUUID, podName=$podName, containerName=$containerName, nodeId=$nodeId
        tenant_id_key           orgUUID # this line is only needed if you're doing multi-tenancy
        remove_keys             orgUUID, pipeUUID, runUUID, podName, containerName, createdAt, nodeId
        line_format             key_value # must be set to key_value, otherwise character escapes will appear
        drop_single_key         on

  parsers.conf: |
    [PARSER]
        Name        json
        Format      json
```