Bug: [Jsonnet] Statefulsets have SecurityContext.runAsUser(0) set

[EoinFarrell]

What is the bug?

Hi Grafana Labs Team,

The jsonnet configuration for Mimir Statefulsets configures the containers to run as the root user, link to code.

The Grafana Mimir documentation states:

Grafana Mimir does not require any special permissions on the hosts that it runs on. Because of this, you can deploy it in environments that enforce the Kubernetes Restricted security policy.

But setting containers to run as root is against the Kubernetes Restricted security policy, exert:

Running as Non-root user (v1.23+)

Containers must not set runAsUser to 0

Restricted Fields

spec.securityContext.runAsUser
spec.containers[].securityContext.runAsUser
spec.initContainers[].securityContext.runAsUser
spec.ephemeralContainers[*].securityContext.runAsUser

Allowed Values

any non-zero value
undefined/null

The Flusher job also has this configured, link to code.

From looking at the git history it looks like this piece of code has been in place since this repo was started, so if it is not required, I would guess it was originally put in place to allow Cortex to bind to port 80 (restricted port, which can block non root users from binding to it), as Cortex defaulted to port 80, unlike Mimir which defaults to port 8080, see migrating-from-cortex.md.

For reference, it does not look like the default Helm chart values have this issue (runAsUser: 10001 is set in some places which is not a security concern). Though I did find a similar issue raised previously related to the Helm charts: #5758.

How to reproduce it?

n/a

What did you think would happen?

Run as root should not be set

What was your environment?

Kubernetes jsonnet

Any additional context to share?

No response

[EoinFarrell]

I've opened a draft PR to remove the config, if its agreed it can be removed: #10339

[narqo]

Thank you for the detailed investigation of the issue, and for the PR. We were discussing it internally and the only concern/question that popped up was about changing existing stateful sets:

That is, could removing the runAsUser: 0 from an existing STS — e.g. the ingesters — cause any issues, where a new rollout, with the security context removed, couldn't read/write the data in the attached PVC, created by the previous version, under the root? From a quick test in one of the environments, this didn't seem to be the case. The rollout worked just fine. But given how critical the ingesters' data is for both read- and write-path, we want to be extra cautious about this change (or at least have a migration plan for the production systems).