# Managing Bring Your Own Logging Backend

Pipekit allows you to connect your self-hosted Loki instance to the Pipekit control plane. Once you add your Loki configuration to Pipekit, you will have to use a log collector on your Kubernetes clusters in order to send the logs from your cluster to your Loki instance.

## Connect a Loki Instance

Go to the [Org tab in Pipekit](https://pipekit.io/organization). Then navigate to the `BYO Logs` tab. Enter the address of your Loki instance along with a username and password that you use for authenticating at the gateway/ingress layer. Then submit the information. You'll have to make sure that the address is accessible to the Internet and that the following endpoints are reachable:

```yaml
/loki/api/v1/query
/loki/api/v1/query_range
/loki/api/v1/tail
/loki/api/v1/series
```

## Configuring a Log Collector

In order to get the logs from your Kubernetes clusters running Argo Workflows into your Loki instance, you'll have to configure a log collector. We use and test with [Fluent Bit](https://fluentbit.io/).

### Labels

Pipekit will append the following labels to pods ran by workflows on clusters where the Pipekit agent is present and configured to either submit workflows or read externally submitted workflows. The labels are the following:

```yaml
workflows.pipekit.io/org_uuid
workflows.pipekit.io/pipe_uuid
workflows.pipekit.io/run_uuid
```

These labels will need to be mapped to `orgUUID`, `pipeUUID`, and `runUUID` in Loki.

### Fluent Bit Config

Here's a sample fluent bit config that we'd recommend. Note that this configuration reads logs directly from the container file system:

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentbit-config
  namespace: <<REPLACE_WITH_NAMESPACE>>
data:
  fluent-bit.conf: |
    [SERVICE]
        Flush         1
        Log_Level     info
        Daemon        off
        Parsers_File  parsers.conf
    @INCLUDE input.conf
    @INCLUDE output.conf

  input.conf: |
    [INPUT]
        Name                tail
        Parser              json
        Path                /tmp/*.log # set to whatever path your logs will be stored in the filesystem
        Mem_Buf_Limit       100MB
        Skip_Long_Lines     On
        Refresh_Interval    5
        Rotate_Wait         15

  output.conf: |
    [OUTPUT]
        name                    loki
        match                   *
        host                    <<REPLACE_WITH_LOKI_HOST>>
        port                    <<REPLACE_WITH_LOKI_PORT>>
        labels                  orgUUID=$orgUUID, pipeUUID=$pipeUUID, runUUID=$runUUID, podName=$podName, containerName=$containerName, nodeId=$nodeId
        tenant_id_key           orgUUID # this line is only needed if you're doing multi-tenancy
        remove_keys             orgUUID, pipeUUID, runUUID, podName, containerName, createdAt, nodeId
        line_format             key_value # must be set to key_value, otherwise character escapes will appear
        drop_single_key         on

  parsers.conf: |
    [PARSER]
        Name        json
        Format      json
```