Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve cluster detection for entity and app signals processors #1525

Open
wants to merge 4 commits into
base: feature-custom-metrics-entity
Choose a base branch
from

Conversation

musa-asad
Copy link
Contributor

@musa-asad musa-asad commented Feb 3, 2025

Description of the issue

When running on a Kubernetes cluster, the only way the CloudWatch Agent can detect the cluster name is:

  1. If it's provided in the configuration:
"kubernetes": {
  "cluster_name": "<cluster-name>",
  "enhanced_container_insights": true
},
"application_signals": {
  "hosted_in": "<cluster-name>"
}
  1. By making a ec2:DescribeTags call.

Ideally, the agent should minimize external API calls, so we should implement a global way to retrieve the cluster name in case it isn't provided in the configuration. We can fetch the K8S_CLUSTER_NAME environmental variable, which is populated by the Amazon CloudWatch Observability EKS add-on / Helm chart.

Description of changes

  • Create GetHostedIn function, which looks for cluster name under app signals.
  • Create GetClusterName function, which looks for cluster name under kubernetes, in K8S_CLUSTER_NAME environmental variable, or by a ec2:DescribeTags call.
  • Reference these functions in entity and app signals processors.
  • Organize entity processor so that it only calls these functions when running in Kubernetes.
  • Add unit tests for entity processor.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

In an EKS cluster, I added a print statement to verify the environmental variable's value is accessed during translation:
Screenshot 2025-02-03 at 2 14 23 AM

Requirements

Before commit the code, please do the following steps.

  1. Run make fmt and make fmt-sh
  2. Run make lint

@musa-asad musa-asad changed the base branch from main to custom-metrics-entity February 3, 2025 05:20
@musa-asad musa-asad self-assigned this Feb 3, 2025
@musa-asad musa-asad requested review from Paramadon, dricross, JayPolanco and zhihonl and removed request for Paramadon and JayPolanco February 3, 2025 06:49
@musa-asad musa-asad marked this pull request as ready for review February 3, 2025 07:14
@musa-asad musa-asad requested a review from a team as a code owner February 3, 2025 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant