Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Kafka consumer groups not working in GMS #553

Open
rospe opened this issue Feb 11, 2025 · 0 comments
Open

Custom Kafka consumer groups not working in GMS #553

rospe opened this issue Feb 11, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@rospe
Copy link

rospe commented Feb 11, 2025

Describe the bug
GMS is producing errors, when rolled out with limited access rights. In our case, we only allowed GMS to use groups with a specific prefix.
I found that GMS is not aware of those custom groups, even when they are set in the global kafka properties.

Reason being is here: https://github.com/acryldata/datahub-helm/blob/master/charts/datahub/subcharts/datahub-gms/templates/deployment.yaml#L299

We do use standalone consumers and we want to use custom groups at the same time.

Workaround
Added these consumer group names to the ENV manually.

Log extract
2025-02-11 11:36:26,065 [pool-6-thread-1] ERROR c.d.m.d.throttle.KafkaThrottleSensor:252 - Error fetching consumer group offsets.
Caused by: org.apache.kafka.common.errors.GroupAuthorizationException: Group authorization failed.

@rospe rospe added the bug Something isn't working label Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant