Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing runtime metrics from cAdvisor #1667

Open
stevehipwell opened this issue Feb 14, 2024 · 1 comment
Open

Missing runtime metrics from cAdvisor #1667

stevehipwell opened this issue Feb 14, 2024 · 1 comment

Comments

@stevehipwell
Copy link
Contributor

stevehipwell commented Feb 14, 2024

What happened:
The kubelet cAdvisor metrics are missing the runtime values meaning we can't see the node usage.

What you expected to happen:
I'd expect to see the runtime values as part of the kubelet metrics.

How to reproduce it (as minimally and precisely as possible):
Run kubectl get --raw "/api/v1/nodes/${NODE_NAME}/proxy/stats/summary" | jq -r '.node.systemContainers[] | .name' and you'll see the following response.

kubelet
pods

Anything else we need to know?:
This appears to be broken because the --runtime-cgroups=/runtime.slice/containerd.service kubelet argument hasn't been set. I can confirm that if this argument has been set the outcome of the command above is a as follows.

kubelet
runtime
pods

Environment:

  • AWS Region: n/a
  • Instance Type(s): n/a
  • EKS Platform version: n/a
  • Kubernetes version: n/a
  • AMI Version: v20240209
  • Kernel: n/a
  • Release information: n/a

<!-- If this is a security issue, please do not discuss on GitHub. Please report any suspected or confirmed security issues to AWS Security https://aws.amazon.com/security/vulnerability-reporting/ -->
@stevehipwell
Copy link
Contributor Author

CC @cartermckinnon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant