Skip to content

Use Kustomize HelmChart for Spark Operator #3139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Use Kustomize HelmChart for Spark Operator #3139

wants to merge 13 commits into from

Conversation

kunal-511
Copy link
Contributor

@kunal-511 kunal-511 commented May 21, 2025 •
edited
Loading

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

Updated the Spark Operator deployment within apps/spark/spark-operator/base to utilize Kustomize's native helmCharts feature.
Updated the dynamically fetching and rendering the Helm chart, replacing the previous method of using a pre-rendered resources.yaml

✅ Contributor Checklist

  • I have tested these changes with kustomize. See Installation Prerequisites.
  • All commits are signed-off to satisfy the DCO check.
  • I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

@kunal-511 kunal-511 marked this pull request as ready for review May 21, 2025 09:48
@kunal-511
Copy link
Contributor Author

@juliusvonkohout
Have a look

@google-oss-prow Google OSS Prow
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kimwnasptd for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@juliusvonkohout
Copy link
Member

I am a bit confused. spark is totally independent of the KSERVE models web application.

juliusvonkohout
juliusvonkohout reviewed May 21, 2025
View reviewed changes
apps/kserve/models-web-app/base/kustomization.yaml Outdated
@@ -15,8 +17,14 @@ configMapGenerator:
- literals:
- APP_DISABLE_AUTH="True"
name: kserve-models-web-app-config
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmCharts:
Copy link
Member

@juliusvonkohout juliusvonkohout May 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this belongs to https://github.com/kubeflow/manifests/tree/master/apps/spark/spark-operator spark and kserve have noting to do with each other.

Copy link
Contributor Author

@kunal-511 kunal-511 May 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I know that kserve and spark are not related to each other

@google-oss-prow google-oss-prow bot added size/XXL and removed size/S labels May 21, 2025
@kunal-511 kunal-511 requested a review from juliusvonkohout May 21, 2025 16:59
@kunal-511 kunal-511 changed the title Integrates the Spark Operator as a dependency for the models-web-app Use Kustomize HelmChart for Spark Operator May 21, 2025
@kunal-511
Copy link
Contributor Author

kunal-511 commented May 27, 2025 •
edited
Loading

@juliusvonkohout is there anything else pending in this to be added ?

juliusvonkohout
juliusvonkohout reviewed May 27, 2025
View reviewed changes
.github/workflows/manifests_example_test.yaml
@@ -19,4 +19,4 @@ jobs:

- name: Unit Test
run: |
kustomize build example
kustomize build --enable-helm example
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not change the default installation for now due to politcial reasons. Lets do the spark -operator stuff under experimental/helm for now. and keep
apps/spark/spark-operator as it is

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can leave the PR open, just create a separate one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juliusvonkohout juliusvonkohout juliusvonkohout left review comments

@kimwnasptd kimwnasptd Awaiting requested review from kimwnasptd

Assignees
No one assigned
Labels
size/XXL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@kunal-511 @juliusvonkohout