[Kubernetes provider]Update kubernetes provider README #5990

MichaelKatsoulis · 2024-11-11T12:54:13Z

What does this PR do?

This PR updates the README of Kubernetes Provider. It was created years ago and is out of date. It includes high level information and configuration options of Kubernetes Provider.

Why is it important?

The information provided in the README are out of date and can confuse a lot of readers.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in ./changelog/fragments using the changelog tool
I have added an integration test or an E2E test

…rovider-readme

mergify · 2024-11-11T12:54:56Z

This pull request does not have a backport label. Could you fix it @MichaelKatsoulis? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

mergify · 2024-11-11T12:54:56Z

backport-v8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

pkoutsovasilis · 2024-11-12T09:11:00Z

internal/pkg/composable/providers/kubernetes/README.md


-### Template based autodiscover
+a. In  **standalone** elastic agent, user needs to edit the [elastic-agent-standalone-kubernetes.yaml](https://github.com/elastic/elastic-agent/blob/f994f5bfdf68db27902a4175c3b655b4d611cf7c/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml#L28)
+b. In **fleet managed** elastic agent, kubernetes provider is enabled by default with default values. User needs to follow the steps described in the [documentation](https://www.elastic.co/guide/en/fleet/current/advanced-kubernetes-managed-by-fleet.html) to configure it.


I feel like a blank line is missing from here (between a. and b.)

Does that cause it to render different? I don't think so.

it actually does, now a. and b. are in the same paragraph which makes it kind hard to distinguish the two. However if you add an empty line between a. and b. they appear properly separated 🙂

Then absolutely fix before merge.

pkoutsovasilis · 2024-11-12T09:11:46Z

internal/pkg/composable/providers/kubernetes/README.md

+
+### Conditions based autodiscover
+
+

 Example:


with the new write up below Example: feels redundant

pkoutsovasilis · 2024-11-12T09:15:01Z

internal/pkg/composable/providers/kubernetes/README.md

+
+As an example we will use the redis module.
+
+To automatically identify a Redis Pod and monitor it with the Redis integration, add the following input configuration inside the [Elastic Agent Standalone manifest](https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml):


qq: for the Conditions-based autodiscover to work correctly is any kubernetes provider required to be enabled? If yes please provide the necessary settings of the provider in this example

The default settings of the kubernetes provider should be enough for conditions to work.

pkoutsovasilis · 2024-11-12T09:17:26Z

internal/pkg/composable/providers/kubernetes/README.md

+    co.elastic.hints/info.period: 5m
+```
+
+In order to enable hints based autodiscover, user needs to uncomment the following lines in the [Elastic Agent Standalone manifest](https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml)


point to the lines in the k8s yaml (https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml#L31) or even better, if you can, capture which agent providers configuration is needed and then point to the lines

pkoutsovasilis · 2024-11-12T09:19:36Z

internal/pkg/composable/providers/kubernetes/README.md

+The init container will start before the elastic agent pod and will donwload all the templates of packages [supported](https://github.com/elastic/elastic-agent/tree/main/deploy/kubernetes/elastic-agent-standalone/templates.d). 
+Elastic agent will then collect from the pods running(through the watchers mechanism) all the hints annotations and will try to match them with the correct package.
+In the redis example, it will use the [redis template](https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-standalone/templates.d/redis.yml) and substitute the template variables with the values specified in the annotations. Default values will be used for variables not provided.
+A new input will be then created for redis and reds data will start being collected.


Suggested change

A new input will be then created for redis and reds data will start being collected.

A new input will be then created for redis and redis-specific data will start being collected.

blakerouse

A very nice improvement, thank you.

blakerouse · 2024-11-12T19:33:47Z

internal/pkg/composable/providers/kubernetes/README.md

+
+As an example we will use the redis module.
+
+To automatically identify a Redis Pod and monitor it with the Redis integration, add the following input configuration inside the [Elastic Agent Standalone manifest](https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml):


The default settings of the kubernetes provider should be enough for conditions to work.

blakerouse · 2024-11-12T19:34:08Z

internal/pkg/composable/providers/kubernetes/README.md


-### Template based autodiscover
+a. In  **standalone** elastic agent, user needs to edit the [elastic-agent-standalone-kubernetes.yaml](https://github.com/elastic/elastic-agent/blob/f994f5bfdf68db27902a4175c3b655b4d611cf7c/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml#L28)
+b. In **fleet managed** elastic agent, kubernetes provider is enabled by default with default values. User needs to follow the steps described in the [documentation](https://www.elastic.co/guide/en/fleet/current/advanced-kubernetes-managed-by-fleet.html) to configure it.


Does that cause it to render different? I don't think so.

swiatekm · 2024-11-13T11:55:16Z

internal/pkg/composable/providers/kubernetes/README.md


-### Template based autodiscover
+a. In  **standalone** elastic agent, user needs to edit the [elastic-agent-standalone-kubernetes.yaml](https://github.com/elastic/elastic-agent/blob/f994f5bfdf68db27902a4175c3b655b4d611cf7c/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml#L28)


Isn't this provider enabled by default in general? I believe this is the case for all providers.

swiatekm · 2024-11-13T11:56:37Z

internal/pkg/composable/providers/kubernetes/README.md

+
+The Kubernetes provider watches for Kubernetes resources and generates mappings from them (similar to events in beats provider). The mappings include those variables([list of variables](https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html#_provider_for_pod_resources)) for each k8s resource with unique value for each one of them.
+
+Agent composable controller which controls all the providers receives these mappings and tries to match them with the input blogs of the configurations.


Suggested change

Agent composable controller which controls all the providers receives these mappings and tries to match them with the input blogs of the configurations.

Agent composable controller which controls all the providers receives these mappings and tries to match them with the input blocks of the configurations.

Also, maybe add a link to the controller module here?

swiatekm · 2024-11-13T11:57:34Z

internal/pkg/composable/providers/kubernetes/README.md

+
+### Hints based autodiscover
+
+Standalone elastic agent supports autodiscover based on hints collected from the [Kubernetes Provider](https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html). The hints mechanism looks for hints in kubernetes pod annotations that have the prefix `co.elastic.hints`. As soon as the container starts, elastic agent checks it for hints and launches the proper configuration for the container. Hints tell elastic agent how to monitor the container by using the proper integration.


Suggested change

Standalone elastic agent supports autodiscover based on hints collected from the [Kubernetes Provider](https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html). The hints mechanism looks for hints in kubernetes pod annotations that have the prefix `co.elastic.hints`. As soon as the container starts, elastic agent checks it for hints and launches the proper configuration for the container. Hints tell elastic agent how to monitor the container by using the proper integration.

Standalone elastic agent supports autodiscover based on hints collected from the [Kubernetes Provider](https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html). The hints mechanism looks for hints in kubernetes pod annotations that have the prefix `co.elastic.hints`. As soon as the Pod is ready, elastic agent checks it for hints and launches the proper configuration for the container. Hints tell elastic agent how to monitor the container by using the proper integration.

MichaelKatsoulis added 2 commits November 11, 2024 14:50

Update kubernetes provider documentation

fc4b6c7

Merge remote-tracking branch 'upstream/main' into update-kubernetes-p…

c07bb50

…rovider-readme

MichaelKatsoulis added the skip-ci label Nov 11, 2024

MichaelKatsoulis requested a review from a team as a code owner November 11, 2024 12:54

MichaelKatsoulis requested review from tetianakravchenko and constanca-m November 11, 2024 12:54

MichaelKatsoulis self-assigned this Nov 11, 2024

MichaelKatsoulis requested a review from gizas November 11, 2024 12:54

mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Nov 11, 2024

ycombinator requested review from pkoutsovasilis and swiatekm November 12, 2024 00:15

pkoutsovasilis reviewed Nov 12, 2024

View reviewed changes

blakerouse approved these changes Nov 12, 2024

View reviewed changes

swiatekm reviewed Nov 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Kubernetes provider]Update kubernetes provider README #5990

[Kubernetes provider]Update kubernetes provider README #5990

MichaelKatsoulis commented Nov 11, 2024

mergify bot commented Nov 11, 2024

mergify bot commented Nov 11, 2024

pkoutsovasilis Nov 12, 2024 •

edited

Loading

blakerouse Nov 12, 2024

pkoutsovasilis Nov 12, 2024

blakerouse Nov 12, 2024

pkoutsovasilis Nov 12, 2024

pkoutsovasilis Nov 12, 2024

blakerouse Nov 12, 2024

pkoutsovasilis Nov 12, 2024

pkoutsovasilis Nov 12, 2024

blakerouse left a comment

blakerouse Nov 12, 2024

blakerouse Nov 12, 2024

swiatekm Nov 13, 2024

swiatekm Nov 13, 2024

swiatekm Nov 13, 2024


		As an example we will use the redis module.

		To automatically identify a Redis Pod and monitor it with the Redis integration, add the following input configuration inside the [Elastic Agent Standalone manifest](https://github.com/elastic/elastic-agent/blob/main/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml):

	A new input will be then created for redis and reds data will start being collected.
	A new input will be then created for redis and redis-specific data will start being collected.


		### Template based autodiscover
		a. In standalone elastic agent, user needs to edit the [elastic-agent-standalone-kubernetes.yaml](https://github.com/elastic/elastic-agent/blob/f994f5bfdf68db27902a4175c3b655b4d611cf7c/deploy/kubernetes/elastic-agent-standalone-kubernetes.yaml#L28)


		The Kubernetes provider watches for Kubernetes resources and generates mappings from them (similar to events in beats provider). The mappings include those variables([list of variables](https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html#_provider_for_pod_resources)) for each k8s resource with unique value for each one of them.

		Agent composable controller which controls all the providers receives these mappings and tries to match them with the input blogs of the configurations.


		### Hints based autodiscover

		Standalone elastic agent supports autodiscover based on hints collected from the [Kubernetes Provider](https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html). The hints mechanism looks for hints in kubernetes pod annotations that have the prefix `co.elastic.hints`. As soon as the container starts, elastic agent checks it for hints and launches the proper configuration for the container. Hints tell elastic agent how to monitor the container by using the proper integration.

[Kubernetes provider]Update kubernetes provider README #5990

Are you sure you want to change the base?

[Kubernetes provider]Update kubernetes provider README #5990

Conversation

MichaelKatsoulis commented Nov 11, 2024

What does this PR do?

Why is it important?

Checklist

mergify bot commented Nov 11, 2024

mergify bot commented Nov 11, 2024

pkoutsovasilis Nov 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blakerouse left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkoutsovasilis Nov 12, 2024 •

edited

Loading