[Feature] Support for new customMetrics value in DCGM Exporter

[chipzoller]

NVIDIA/dcgm-exporter#351 adds support for user-definable custom metrics directly in the Helm values file which greatly simplifies the user experience needed when deploying DCGM Exporter with emission of custom metrics. Adding support for this field in the operator's values file would be very useful. Without it, users would have to resort to the (values-file undocumented) field in the ClusterPolicy at config, which accepts the name of a ConfigMap, meaning users have to pre-create this prior to deploying the operator (see also template ref here).

Please consider adding support for the new customMetrics field in the ClusterPolicy CRD and, by extension, in the Helm values of the operator to allow users to deploy the operator and DCGM Exporter with a list of custom metrics upfront.

[chipzoller]

cc @francisguillier

[chipzoller]

Relates to #935

[cdesiniotis]

@chipzoller thanks for the feature request. Our helm chart offers a similar experience for the k8s-device-plugin and mig-manager configuration files, where the user can provide the ConfigMap data directly via the values.yaml file, as opposed to creating a ConfigMap beforehand. See https://github.com/NVIDIA/gpu-operator/blob/v24.6.1/deployments/gpu-operator/values.yaml#L263-L291 and https://github.com/NVIDIA/gpu-operator/blob/v24.6.1/deployments/gpu-operator/values.yaml#L361-L399. We may want to provide a similar set of fields in the values file for dcgm-exporter.

PRs are welcome if you are interested in taking this on.

cc @tariq1890

[chipzoller]

Hi @cdesiniotis, yep, totally, but because everything behind the scenes is handled by templating the ClusterPolicy custom resource, the changes needed here are more extensive as the API needs to be extended in the CRD to accommodate a new field (and other changes). If it were just a simple matter of extending the Helm values that'd be no problem, so unfortunately someone else may be better to pick this up.

[cdesiniotis]

The required changes should be contained to just the helm values and templates. We shouldn't need to extend the CRD at all. See the PR which added this feature to the mig-manager component: #803

[chipzoller]

I see. Looks like it should piggyback off the config object I'm attempting to document in #935. I was thinking the CR would need a new field to "natively" implement this as a block scalar value. I'll take another look at this and see if I can send a PR.

[chipzoller]

@cdesiniotis, can you help unblock me here?

[chipzoller]

Thanks for your help, @cdesiniotis. PR is up at #949.