configuration for securityContext in the CRD for all pods created by operator

[adabuleanu]

Problem Statement

I want to be able to set securityContext so pods can be set up to run as NonRoot for security reasons. There are a couple of places where there is no such configuration possible, although there do not seem to be obvious restrictions. Example: the CruiseControler monitoring config init-container is configured using .spec.monitoringConfig which does not support a securityContext. There are most likely other places where security configuration is not possible to add.

Proposed Solution

Add ability to configure the securityContext to all containers and init-containers created by the operator.

Alternatives Considered

Ideally it would be to have the ability to configure the KafkaCluster CR so that it would pass the restricted profile of Pod Security Standards.

Additional Context

A similar initiative is tracked in #430 and #138 and implemented in #527 but does not covered all scenarios (see above described scenario). An ideal test would be that all containers pass PSS.

[bartam1]

Hello @adabuleanu !
Pull request is welcome :)