Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pod UID as a shard key for "loki.source" clustering #2189

Open
ptodev opened this issue Nov 28, 2024 · 1 comment
Open

Use pod UID as a shard key for "loki.source" clustering #2189

ptodev opened this issue Nov 28, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@ptodev
Copy link
Contributor

ptodev commented Nov 28, 2024

Request

Alloy has two components for gathering logs which support clustering - loki.source.kubernetes and loki.source.podlogs. In order for those components to divide the work among members of the cluster, they currently hash some of the k8s service discovery labels:

  • loki.source.kubernetes uses both public and non-public labels, but since Fix logs cluster #1716 it only takes into account some of them.
  • loki.source.podlogs uses a hash of all the "public" labels (the ones which don't start with __).

It would be better if the only label we use is the one whose value is the pod UID. For example, __meta_kubernetes_pod_uid. This is enough to uniquely identify a pod. If there is no such label, and if clustering is enabled, then no logs should be sent.

Use case

This has a few advantages:

  • It's safer. A pod UID is guaranteed to identify a pod instance uniquely. There's no possibility of the same pod instance being picked up by several Alloy instances.
  • It's more performant. No need to extract all the labels and hash them.
  • No possibility of hash collisions.
  • Simpler to document and to understand.

The only disadvantage is that it's a breaking change. Some users may need to change their configs.

@ptodev ptodev added the enhancement New feature or request label Nov 28, 2024
@ptodev
Copy link
Contributor Author

ptodev commented Nov 28, 2024

I wonder if it's worth using the container ID instead, from a label such as __meta_kubernetes_pod_container_id ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant