Add project config override cli options

[Ekrekr]

Fixes #1295

[BenBirt]

we should clarify (for all of these) that they override any settings in dataform.json

[Ekrekr]

Thread above - this is for init, so does not override anything, and is a required flag in this case.

[sukhamritboparai]

hi team, when is this PR merge or feature available?

[Ekrekr]

Examples:

$ bazel run //packages/@dataform/cli:bin init -- --help
...
dataform init <warehouse> [project-dir]

Create a new dataform project.

Positionals:
  warehouse    The project's data warehouse type. If unset, the value from dataform.json is used.  [required] [choices: "bigquery", "presto", "postgres", "redshift", "snowflake", "sqldatawarehouse"]
  project-dir  The Dataform project directory.  [default: "."]

Options:
  --help              Show help  [boolean]
  --version           Show version number  [boolean]
  --default-database  The default database to use. For BigQuery, this is a Google Cloud Project ID.  [string]
  --default-location  The default BigQuery location to use. See https://cloud.google.com/bigquery/docs/locations for supported values.
  --skip-install      Whether to skip installing NPM packages.  [default: false]

$ bazel run //packages/@dataform/cli:bin run -- --help
...
dataform run [project-dir]

Run the dataform project's scripts on the configured data warehouse.

Positionals:
  project-dir  The Dataform project directory.  [default: "."]

Options:
  --help                Show help  [boolean]
  --version             Show version number  [boolean]
  --dry-run             If set, built SQL is not run against the data warehouse and instead is printed to the console.  [boolean]
  --run-tests           If set, the project's unit tests are required to pass before running the project.  [boolean]
  --full-refresh        Forces incremental tables to be rebuilt from scratch.  [boolean] [default: false]
  --actions             A list of action names or patterns to run. Can include '*' wildcards.  [array]
  --tags                A list of tags to filter the actions to run.  [array]
  --include-deps        If set, dependencies for selected actions will also be run.  [boolean]
  --include-dependents  If set, dependents (downstream) for selected actions will also be run.  [boolean]
  --credentials         The location of the credentials JSON file to use.  [default: ".df-credentials.json"]
  --json                Outputs a JSON representation of the compiled project.  [boolean] [default: false]
  --timeout             Duration to allow project compilation to complete. Examples: '1s', '10m', etc.  [string] [default: null]
  --warehouse           The project's data warehouse type. If unset, the value from dataform.json is used.  [choices: "bigquery", "presto", "postgres", "redshift", "snowflake", "sqldatawarehouse"]
  --default-database    The default database to use. For BigQuery, this is a Google Cloud Project ID. If unset, the value from dataform.json is used.  [string]
  --default-schema      Override for the default schema name. If unset, the value from dataform.json is used.
  --default-location    The default BigQuery location to use. See https://cloud.google.com/bigquery/docs/locations for supported values. If unset, the value from dataform.json is used.
  --assertion-schema    Default assertion schema. If unset, the value from dataform.json is used.
  --vars                Override for variables to inject via '--vars=someKey=someValue,a=b', referenced by `dataform.projectConfig.vars.someValue`. If unset, the value from dataform.json is used.  [string] [default: null]
  --database-suffix     Default assertion schema. If unset, the value from dataform.json is used.
  --schema-suffix       A suffix to be appended to output schema names. If unset, the value from dataform.json is used.
  --table-prefix        Adds a prefix for all table names. If unset, the value from dataform.json is used.

[BenBirt]

Please delete yarn-error.log.

[BenBirt]

I still find this structure confusing.

If we were only ever using these flags to construct ProjectConfig, I'd buy it. But we don't; some flags are used for that sometimes, but for other uses other times.

I think this namespacing is more confusing than it's worth, to be honest.