Implement JMX E2E Tests in Terraform Framework

[musa-asad]

Description of the issue

The CloudWatch Agent now supports JMX metrics on EKS. We need to add testing to verify the behavior for JMX metrics collection in EKS environments.

Description of changes

Co-PR: aws/amazon-cloudwatch-agent#1463.

Revision 1

• Implemented test/e2e/jmx/jmx_test.go with the following functions:
  • ApplyHelm: Applies a helm release based on the specified resources passed in as variables, which sets up the CloudWatch Agent and deploys an annotated sample application.
  • TestResources: Tests to see if the appropriate resources have been deployed on the EKS cluster.
  • TestMetrics: Tests to see if metrics are being emitted properly based on the agent configuration file passed in.
• Added test/e2e/jmx/resources/cwagent_configs/ and test/e2e/jmx/resources/sample_apps/ for custom agent configurations and sample applications to deploy.
  • Added a check to see if tomcat.sessions and tomcat.rejected_sessions increases above 0 accordingly.
• Added GetMetricMaximum to util/awsservice/cloudwatchmetrics.go in order to find the maximum value for a metric being emitted.
• Added nodes to generator/test_case_generator.go and generator/resources/eks_e2e_test_matrix.json to be able to use this as a configurable value from the generated matrix.

Revision 2

• Use testify's require / assert methods instead of t.Fatal / t.Error.
• Add destroy logic for Kubernetes resources by uninstalling helm and deleting test namespace.
• Removed --values from Helm installation since it isn't required.
• Use kubectl wait for operator deployment instead of time.Sleep to continue as soon as the operator is ready.
• Removed -deployment from deployment names due to redundancy.
• Set the namespace once for testXMetrics functions due to redundancy.
• Switch to using our own ECR images to keep stable images we control.
• Replaced GetMetricMaximum with CheckMetricAboveZero, which exits as soon as it finds a metric above zero.
• Added variables for ami_type and instance_type to have control over those values for future testing.
• Added kafka and kafka_producer metrics.
• Moved test environment initialization and resource management logic into util/common/kubernetes.go.
• Added check for jmx in agent configuration.
• Added check for jmx environmental variables in application pod.
• Cleaned up code by adding comment blocks and helper functions.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

View Workflow Run #12452251638