Skip to content

[test](regression) Pin row counts in bucket shuffle set operation tests - #70546

Open
924060929 wants to merge 1 commit into
apache:masterfrom
924060929:codex/stabilize-bucket-shuffle-set-operation
Open

924060929 wants to merge 1 commit into
apache:masterfrom
924060929:codex/stabilize-bucket-shuffle-set-operation

Conversation

@924060929

Copy link
Copy Markdown
Contributor

Background

The bucket-shuffle set-operation suite checks physical plan shapes as well as query results. INTERSECT input ordering and bucket-shuffle anchor selection depend on estimated row counts.

What problem does this PR solve?

Issue Number: None
Related PR: #65129 (enabled the suite and added bucket-shuffle coverage)
PR Type: Test

Problem Summary:

The three initial tables each contain three rows, but asynchronous tablet statistics updates can leave one estimated at three rows and another at one row. This changes INTERSECT input order or the UNION anchor and causes intermittent shape mismatches. Inject row_count=3 for both columns of these tables before checking plans. Keep the existing queries, assertions, and golden results.

Details

  • Failure: INTERSECT children can swap, or the join above UNION ALL can change from colocated to bucket shuffle.
  • Cause: The plan checks relied on asynchronously reported row counts immediately after creating and loading the tables.
  • Stability evidence: The complete suite passed 10 consecutive runs with default settings and another 10 with enable_spill=true. Only row counts are specified: adding detailed column statistics changes an unrelated join choice and bypasses the parent-hash-request coverage. All existing shape and result checks remain intact, including the fixtures with larger injected row counts later in the suite.

Testing

On a local macOS instance with one FE and one ASAN BE, built from b325dcd:

  • Single complete suite run passed.
  • ./run-regression-test.sh --run -d query_p0/set_operations -s bucket_shuffle_set_operation -parallel 1 -times 10: 10/10 passed with default settings.
  • The same command with global enable_spill=true: 10/10 passed.
  • Regression framework compilation and its 50 unit tests passed.
  • FE build passed with zero Checkstyle violations; BE build passed.
  • git diff --check passed. No golden output changes.

Release note

None

Check List (For Author)

  • Behavior changed: No
  • Does this need documentation: No
  • Pre-submission status: Ready

AI Disclosure

  • Tool: Codex
  • Generated: Test setup and PR description.
  • Human review: Pending maintainer review.

### Background

The bucket-shuffle set-operation suite checks physical plan shapes as well as query results. INTERSECT input ordering and bucket-shuffle anchor selection depend on estimated row counts.

### What problem does this PR solve?

Issue Number: None
Related PR: apache#65129 (enabled the suite and added bucket-shuffle coverage)
PR Type: Test

Problem Summary:

The three initial tables each contain three rows, but asynchronous tablet statistics updates can leave one estimated at three rows and another at one row. This changes INTERSECT input order or the UNION anchor and causes intermittent shape mismatches. Inject row_count=3 for both columns of these tables before checking plans. Keep the existing queries, assertions, and golden results.

### Details

- **Failure:** INTERSECT children can swap, or the join above UNION ALL can change from colocated to bucket shuffle.
- **Cause:** The plan checks relied on asynchronously reported row counts immediately after creating and loading the tables.
- **Stability evidence:** The complete suite passed 10 consecutive runs with default settings and another 10 with enable_spill=true. Only row counts are specified: adding detailed column statistics changes an unrelated join choice and bypasses the parent-hash-request coverage. All existing shape and result checks remain intact, including the fixtures with larger injected row counts later in the suite.

### Testing

On a local macOS instance with one FE and one ASAN BE, built from b325dcd:

- Single complete suite run passed.
- `./run-regression-test.sh --run -d query_p0/set_operations -s bucket_shuffle_set_operation -parallel 1 -times 10`: 10/10 passed with default settings.
- The same command with global enable_spill=true: 10/10 passed.
- Regression framework compilation and its 50 unit tests passed.
- FE build passed with zero Checkstyle violations; BE build passed.
- `git diff --check` passed. No golden output changes.

### Release note

None

### Check List (For Author)

- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Ready

### AI Disclosure

- Tool: Codex
- Generated: Test setup and PR description.
- Human review: Pending maintainer review.
@924060929

Copy link
Copy Markdown
Contributor Author

run buildall

@924060929

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static review of PR #70546 at head ad96168 found no new findings. The only change adds six fixed-name ALTER TABLE ... MODIFY COLUMN ... SET STATS ('row_count'='3') statements after creating and loading three-row OLAP fixtures. Source tracing confirms each statement validates synchronously, writes and synchronizes the column statistic, marks the base-index table row count as user-injected, and is consumed by Nereids when selecting the bucket-shuffle basic child. The injected count matches the data, and count-only stats leave NDV untrustable, so no result or unrelated join-plan coverage defect was substantiated. The change is focused and follows existing regression-test syntax and setup conventions; no concurrency, lifecycle, configuration, protocol, persistence, or source-data path is introduced. Existing inline threads: none. User review focus: none. Static validation covered the complete authoritative diff, changed-path list, existing review context, and upstream/downstream statistics flow; no builds or runtime tests were executed under the static-review contract. Review converged in round 1 because both assigned review passes returned NO_NEW_VALUABLE_FINDINGS and the only risk focus was dismissed with evidence.

Review converged after 1 round(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant