Question - does modifying services to contact non-headless Mimir services affect anything?

[dorkamotorka]

Consider this - The mimir-distributed helm chart has the Mimir.config option, where you can among other things configure frontend_worker.scheduler_address, frontend_worker.frontend_address and frontend.scheduler_address. By default these values have the corresponding values:

frontend_worker.scheduler_address: {{ template "mimir.fullname" . }}-query-scheduler-headless.{{ .Release.Namespace }}.svc:{{ include "mimir.serverGrpcListenPort" . }}

frontend_worker.frontend_address: {{ template "mimir.fullname" . }}-query-frontend-headless.{{ .Release.Namespace }}.svc:{{ include "mimir.serverGrpcListenPort" . }}

frontend.scheduler_address: {{ template "mimir.fullname" . }}-query-scheduler-headless.{{ .Release.Namespace }}.svc:{{ include "mimir.serverGrpcListenPort" . }}

If I remove the *-headless in all the 3 configuration the setup (I mean the Mimir deployment) works without an issue - AT LEAST FOR NOW. As to my understanding headless services in Kubernetes are meant to retrieve the pods names which you can then directly contact from your application separately, compared to the "normal service (non-headless)" that acts as the load-balancer between the three. I would think that by contacting the non-headless service, it should work ONLY IF Mimir internally handles that among the three pods none of the forwarding(read/write operations), beside the locking of the stored/retrieved files happen concurrently, or if this is in a way handled by Mimir application code, so there are no race conditions occurrences. Is there such a thing or is my application bound to fail with such configuration?

In other words, where is the HA, if I connect to the headless service and consequently limit my application to only be able to forward metrics to a particular Mimir pod? If this Pod is down, there's no HA, because my application will have to wait for that same Pod to come back alive.

Primarily posted here, but with no reply so far.