Support BigQuery oauth credentials

[lewish]

Currently BigQuery can only be authorized through a service account, which adds significant overhead to setting up a BQ project.

It should be possible to provide oauth credentials when executing against BigQuery.

• During init-creds add an option to use OAuth for BigQuery projects

• Send the user to the oauth connection page with a callback to localhost:someport

• Spin up a small server to handle the oauth callback

• Write the oauth credentials to the .df-credentials.json file

• Create a Dataform OAuth client ID

• Support OAuth credentials in the BigQuery adapter and extend the credentials proto to store this

• Implement the new init-creds flow above

[dorzey]

👍 we would definitely like to have this

[thinhha]

Downloading service account keys is not very good security practice. The better approach is to use OAUTH as described in this article: https://medium.com/@jryancanty/stop-downloading-google-cloud-service-account-keys-1811d44a97d9

One way to do this is to use GOOGLE_APPLICATION_CREDENTIALS as described in:
#1164

Another way is for Dataform to expect a short-lived OAuth 2.0 access token, either as input to the CLI or from an environment variable like as done by terraform:
https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#access_token

[jamiekt]

I wholeheartedly agree with @thinhha , access tokens would be an ideal mechanism here.

[jamiekt]

This is a very relevant reference: https://cloud.google.com/iam/docs/creating-short-lived-service-account-credentials

[thinhha]

more details on how DBT did this here: dbt-labs/dbt-core#2672