Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional Spark Properties via Configmap #388

Open
therealslimjp opened this issue Apr 12, 2024 · 0 comments
Open

Add additional Spark Properties via Configmap #388

therealslimjp opened this issue Apr 12, 2024 · 0 comments

Comments

@therealslimjp
Copy link

As a user I want to be able to pass spark properties, which might be the same over multiple sparkApplications (e.g. kerberos settings), via a configmap to the spark-submit command. Using this, I save effort and complexity repeatedly defining the same spark properties over and over when writing sparkApplications.

I tried to do some hacks to mock this behavior with mounting a spark-defaults.conf (which won't work because the spark-driver already mounts on /opt/spark/conf and thus I can't mount it there because the location is not unique) or passing a --properties-file (which also does not work because a --properties-file is already passed inherently)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants