Skip to content

acc: factcheck postgres_projects update_mask=* and =spec - #6767

Merged
denik merged 1 commit into
mainfrom
denik/investigate-star-mask
Sep 22, 2026
Merged

denik merged 1 commit into
mainfrom
denik/investigate-star-mask

Conversation

@denik

@denik denik commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Why

Raw-$CLI acceptance tests that document, against the real API, what the Postgres project Update endpoint actually accepts — no bundle deploy involved. These pin the behavior the direct engine's mask logic has to work around.

Changes

Two tests under postgres_projects/factcheck/:

  • update_mask_star: create accepts a minimal body (display_name + pg_version; the server materializes the rest), but that same body with update_mask=* is rejected field by field (Field '...' is in update_mask but not provided). Which field comes first isn't contractual, so a Repl normalizes it.
  • update_mask_spec: update_mask=spec applies the body but, unlike *, does not require every field — a body that omits history_retention_duration succeeds and the omitted scalar is preserved, not reset. (So spec is a lenient patch, not a full replace.)

Both inherit Cloud = true, so they also run against a real workspace; verified on AWS.

To make the local run match cloud, validateUpdateMask in the testserver now handles the wildcard: * is accepted, then every leaf of the resource's update-mask path list must be present in the body (a oneof group satisfied by any member), returning the first missing. Previously the fake rejected * as an unknown path.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 30ac1fa

Run: 35724280208

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux 280 15 4:30
✅​ aws windows 282 13 3:34
✅​ azure linux 275 17 4:35
✅​ azure windows 277 15 3:59
✅​ gcp linux 276 17 5:56
🔄​ gcp windows 1 277 15 6:25
Test Name gcp windows
🔄​ TestSyncIncrementalSyncPythonNotebookToFile 🔄​f
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:24 gcp windows TestAccept
3:21 azure windows TestAccept
3:12 aws windows TestAccept

@denik
denik force-pushed the denik/investigate-star-mask branch 3 times, most recently from db06fe2 to d102dd8 Compare September 21, 2026 13:55
Comment thread scripts/update_mask_spec.sh Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like these are the scripts that seeded the acc tests?

If so they can be omitted.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the other way around - I made tests and then these scripts so lakebase team can run them easily. They won't be merged.

Two raw-$CLI acceptance tests (no bundle) that run the same sequence of updates,
one under update_mask=* and one under update_mask=spec, so the goldens document
where the masks agree and differ against the real API:

- create accepts a minimal body (the server materializes the rest);
- a partial body under * is rejected field by field, while under spec it is
  applied and the omitted fields are preserved (spec is a patch, not a replace);
- after a field is set non-default, a body that omits it is again rejected by *,
  while spec 500s — masking the whole spec resets the omitted sub-message and its
  emptied autoscaling limits fail validation; the other fields are applied first,
  so it is a partial write (the display change persists, only the response errors).

The sequence is inlined in each test's script (they differ only by the mask); a
header comment shows how to run it standalone against a real workspace by stubbing
the harness helpers. It uses `postgres update-project NAME UPDATE_MASK`, which
waits for the operation, so every mutation is committed before the next read and
the outcomes are deterministic; the update result is piped through project_fields
so a success records an allow-listed project and a failure (empty stdout) is a
no-op with its error shown. After each update, print_requests.py prints the exact
PATCH sent — including update_mask=* verbatim, confirming the wire value. Both
inherit Cloud = true.

Testserver: validateUpdateMask now accepts the * wildcard and then requires every
leaf of the resource's mask-path list (a oneof group satisfied by any member),
matching the API's field-by-field rejection. PostgresProjectUpdate reproduces the
whole-spec-drops-a-non-default-sub-message 500 as a partial write, with a 32-hex
trace id so goldens normalize like cloud.

Co-authored-by: Isaac
@denik
denik force-pushed the denik/investigate-star-mask branch from 269b04b to 30ac1fa Compare September 22, 2026 11:55
@denik
denik enabled auto-merge September 22, 2026 11:57
@denik
denik added this pull request to the merge queue Sep 22, 2026
Merged via the queue into main with commit 25fdfe5 Sep 22, 2026
32 checks passed
@denik
denik deleted the denik/investigate-star-mask branch September 22, 2026 12:38
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 25fdfe5

Run: 35728472499

Env ❌​FAIL 🔄​flaky ✅​pass 🙈​skip Time
❌​ aws linux 24 1633 1084 256:22
🔄​ aws windows 3 1555 1109 275:57
✅​ azure linux 1466 1135 221:12
🔄​ azure windows 1 1366 1160 242:00
✅​ gcp linux 1452 1139 234:24
🔄​ gcp windows 2 1351 1164 260:09
29 interesting tests: 24 FAIL, 5 flaky
Test Name aws linux aws windows azure windows gcp windows
❌​ TestAccept ❌​F 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/debug/fetch-repository-info/DATABRICKS_BUNDLE_ENGINE=direct/DMS= ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/debug/fetch-repository-info/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true ✅​p ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/invariant/destroy_idempotent ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=app.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=instance_pool.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=instance_pool.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=job_cross_resource_ref.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=job_cross_resource_ref.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=job_permission_ref.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=model_serving_endpoint.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=model_serving_endpoint.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=model_with_permissions.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=model_with_permissions.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=pipeline.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/destroy_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=pipeline.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=app.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=instance_pool.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=instance_pool.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=instance_pool.yml.tmpl/READPLAN= ❌​F ✅​p ✅​p ✅​p
❌​ TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=instance_pool.yml.tmpl/READPLAN=1 ❌​F ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/jobs/cluster_policy/config_conflicts_policy ✅​p 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/jobs/cluster_policy/config_conflicts_policy/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true ✅​p 🔄​f ✅​p ✅​p
🔄​ TestSyncIncrementalFileOverwritesFolder ✅​p ✅​p 🔄​f ✅​p
Top 50 slowest tests (at least 2 minutes):
duration env testname
13:20 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
13:00 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
12:38 gcp linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
12:33 gcp windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:16 aws linux TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:13 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
9:03 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:52 aws linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:49 aws linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:47 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:44 aws windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:14 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
8:12 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
8:06 aws windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:58 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:56 azure windows TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:48 aws linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:45 gcp windows TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:38 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
7:37 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:37 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
7:34 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
7:28 gcp linux TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:25 gcp linux TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:22 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:11 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
7:10 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
7:09 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:08 azure linux TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:04 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
7:02 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
6:59 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:52 gcp windows TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:48 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
6:41 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:40 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:30 aws windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:21 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
6:14 aws windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
6:08 gcp windows TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
6:05 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:59 azure windows TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
5:55 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
5:50 azure linux TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
5:41 gcp linux TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:33 aws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:30 aws linux TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct/DMS=
5:27 aws windows TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform/DMS=
5:27 gcp windows TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true
5:20 azure windows TestAccept

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants