Skip to content

Commit

Permalink
Logging: add labels_app label in the fluent bit log forwarding config
Browse files Browse the repository at this point in the history
This label makes it easier to single out streams related to a Software
Factory, in case a fluent bit collector is deployed to collect more
than just a SF's logs.

Change-Id: I938000e897591f3406e7c2270839dc6ce5328215
  • Loading branch information
mhuin committed Jun 20, 2024
1 parent b0ce2e1 commit 6a2fac7
Show file tree
Hide file tree
Showing 7 changed files with 10 additions and 4 deletions.
1 change: 1 addition & 0 deletions controllers/libs/logging/logging.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ func CreateForwarderEnvVars(name string, extraLabels []FluentBitLabel) []apiv1.E
base.MkEnvVarFromFieldRef("K8S_NAMESPACE", "metadata.namespace"),
base.MkEnvVarFromFieldRef("K8S_PODIP", "status.podIP"),
base.MkEnvVar("K8S_LABELS_RUN", name),
base.MkEnvVar("K8S_LABELS_APP", "sf"),
}
for i := range extraLabels {
var v = base.MkEnvVar("K8S_"+extraLabels[i].Key, extraLabels[i].Value)
Expand Down
1 change: 1 addition & 0 deletions controllers/static/mariadb/fluentbit/fluent-bit.conf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
add podname ${K8S_PODNAME}
add ip ${K8S_PODIP}
add labels_run mariadb
add labels_app sf
add component mariadb
add container mariadb
{{- range .ExtraKeys }}
Expand Down
1 change: 1 addition & 0 deletions controllers/static/nodepool/fluentbit/fluent-bit.conf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
add podname ${K8S_PODNAME}
add ip ${K8S_PODIP}
add labels_run dib
add labels_app sf
add component nodepool
add container nodepool-builder
{{- range .ExtraKeys }}
Expand Down
1 change: 1 addition & 0 deletions controllers/static/zookeeper/fluent-bit.conf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
add podname ${K8S_PODNAME}
add ip ${K8S_PODIP}
add labels_run zookeeper
add labels_app sf
add component zookeeper
add container zookeeper
{{- range .ExtraKeys }}
Expand Down
7 changes: 4 additions & 3 deletions doc/deployment/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ Please refer to your cluster's documentation to find out what existing logging s

## Log Forwarding to a Fluent Bit collector

Native Cluster Logging solutions **should always** be your first choice for managing the logs of
Native Cluster Logging solutions **should always** be your first choice for managing the logs of
Zuul and Nodepool. This is almost always the best solution in terms of cluster resource usage,
and the most straightforward to implement as an application deployer.

If **however** no such solution is available, or if the default solution does not suit your
If **however** no such solution is available, or if the default solution does not suit your
needs, the SF-Operator can set up Zuul and Nodepool to forward application logs to a
[Fluent Bit](https://fluentbit.io/) collector. The collector must be configured to expose an
[Fluent Bit](https://fluentbit.io/) collector. The collector must be configured to expose an
[HTTP input](https://docs.fluentbit.io/manual/pipeline/inputs/http) that your deployment
will be able to reach.

Expand All @@ -62,6 +62,7 @@ The log JSON payload that is sent to your Fluent Bit collector is enriched with
| key | description or value |
|--------|------------------|
| labels_run | the "run" label in the pod's metadata; ie the name of the microservice emitting the log line |
| labels_app | the "app" label, it is always set to "sf" |
| podip | the IP address of the pod |
| namespace | the namespace where the pod is running |
| nodename | the node where the pod is running |
Expand Down
1 change: 1 addition & 0 deletions doc/reference/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ All notable changes to this project will be documented in this file.
### Added

- CLI - add the 'debug' flag to set the LogLevel to DEBUG
- Log Forwarding - add preset label "labels_app" by default, its value is "sf"

### Changed
### Deprecated
Expand Down
2 changes: 1 addition & 1 deletion tools/loki/test-fluentbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ data:
name loki
match *
host loki-http
labels namespace=$namespace, podIP=$podip, nodeName=$nodename, application=$component, podName=$podname, labels_run=$labels_run, labels_app=sf, container=$container
labels namespace=$namespace, podIP=$podip, nodeName=$nodename, application=$component, podName=$podname, labels_run=$labels_run, labels_app=$labels_app, container=$container
kind: ConfigMap
metadata:
creationTimestamp: null
Expand Down

0 comments on commit 6a2fac7

Please sign in to comment.