Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support custom GCS bucket subdirectories for Dataproc #521

Open
3 tasks done
pbotros opened this issue Jan 7, 2025 · 1 comment
Open
3 tasks done

[Feature] Support custom GCS bucket subdirectories for Dataproc #521

pbotros opened this issue Jan 7, 2025 · 1 comment
Labels
pkg:dbt-bigquery Issue affects dbt-bigquery triage:product In Product's queue type:enhancement New feature request

Comments

@pbotros
Copy link

pbotros commented Jan 7, 2025

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-bigquery functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Currently, DBT-Bigquery writes Python model files for Dataproc to gs://<gcs bucket>/<model file name>, which can conflict if multiple users (e.g. dev users, CI runs, etc.) concurrently utilize the same GCS bucket.

Instead, it'd be nice for a subdirectory to be optionally configured, to write to gs://<gcs bucket>/<gcs subdirectory>/<model file name> instead.

Describe alternatives you've considered

N/A

Who will this benefit?

Users that want to share a single GCS bucket across multiple concurrent use cases (CI, dev, etc.)

Are you interested in contributing this feature?

Yes

Anything else?

No response

@pbotros pbotros added type:enhancement New feature request triage:product In Product's queue labels Jan 7, 2025
@pbotros
Copy link
Author

pbotros commented Jan 7, 2025

Let me know if this feature request sounds reasonable; if so, I'm happy to contribute it. I anticipate this should be a small change.

@mikealfare mikealfare added the pkg:dbt-bigquery Issue affects dbt-bigquery label Jan 14, 2025
@mikealfare mikealfare transferred this issue from dbt-labs/dbt-bigquery Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg:dbt-bigquery Issue affects dbt-bigquery triage:product In Product's queue type:enhancement New feature request
Projects
None yet
Development

No branches or pull requests

2 participants