Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bitnami/Airflow] Add Job Support to RBAC for Airflow Chart #30804

Open
siddjellali opened this issue Dec 5, 2024 · 4 comments
Open

[Bitnami/Airflow] Add Job Support to RBAC for Airflow Chart #30804

siddjellali opened this issue Dec 5, 2024 · 4 comments
Assignees
Labels
airflow tech-issues The user has a technical issue about an application triage Triage is needed

Comments

@siddjellali
Copy link

siddjellali commented Dec 5, 2024

Name and Version

bitnami/airflow:latest

What architecture are you using?

None

What steps will reproduce the bug?

Hello Bitnami Team,

I am currently using the [Airflow chart](https://github.com/bitnami/charts/tree/main/bitnami/airflow) and noticed that the default RBAC configuration does not include permissions for managing Kubernetes Job resources.

This can be useful for scenarios where additional jobs need to be scheduled or managed as part of an Airflow deployment. For example, workflows might spawn Kubernetes jobs for specific tasks outside the Airflow ecosystem.

What do you see instead?

kubernetes.client.exceptions.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': '6760a41c-f165-436f-b817-d354a5af13b8', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': '91e0e32e-d27c-44e4-bb42-f6c4440372c0', 'X-Kubernetes-Pf-Prioritylevel-Uid': '980f753d-da38-40ee-b95b-e7209628fdcf', 'Date': 'Thu, 05 Dec 2024 13:40:30 GMT', 'Content-Length': '319'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch is forbidden: User \"system:serviceaccount:app-airflow:airflow\" cannot create resource \"jobs\" in API group \"batch\" in the namespace \"app-airflow\"","reason":"Forbidden","details":{"group":"batch","kind":"jobs"},"code":403}
[2024-12-05T13:40:30.510+0000] {local_task_job_runner.py:266} INFO - Task exited with return code 1
[2024-12-05T13:40:30.668+0000] {taskinstance.py:3895} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2024-12-05T13:40:30.677+0000] {local_task_job_runner.py:245} INFO - ::endgroup::

Additional information

To resolve this, I propose adding the necessary rules to the Role or ClusterRole template to support Job resources.

Proposed Changes

In the templates/rbac/role.yaml file, add permissions for the Job resource:

rules:
  - apiGroups: ["batch"]
    resources: ["jobs"]
    verbs: ["create", "delete", "get", "list", "watch"]

This change would allow Airflow pods to interact with Job resources if necessary.

Benefits

  • Expands the flexibility of Airflow in Kubernetes environments.
  • Enables advanced use cases such as dynamically creating Kubernetes jobs from Airflow tasks.
  • Aligns the chart with common Kubernetes practices.
@siddjellali siddjellali added the tech-issues The user has a technical issue about an application label Dec 5, 2024
@siddjellali siddjellali changed the title Add Job Support to RBAC for Airflow Chart [Bitnami/Airflow] Add Job Support to RBAC for Airflow Chart Dec 5, 2024
@github-actions github-actions bot added the triage Triage is needed label Dec 5, 2024
@carrodher
Copy link
Member

Thank you for bringing this issue to our attention. We appreciate your involvement! If you're interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

@siddjellali
Copy link
Author

siddjellali commented Dec 6, 2024

Please approve PR: #30817

@siddjellali
Copy link
Author

waiting PR approval....

@carrodher
Copy link
Member

Thank you for your contribution! Please note that due to the Christmas holidays, there might be a delay in the usual cadence of the team reviewing community Pull Requests. We appreciate your patience and understanding during this time, and we'll get back to your submission as soon as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
airflow tech-issues The user has a technical issue about an application triage Triage is needed
Projects
None yet
Development

No branches or pull requests

3 participants