Skip to content

refactor(hash-aggr): grouping sets routing - #24261

Draft
2010YOUY01 wants to merge 3 commits into
apache:mainfrom
2010YOUY01:split-aggr-03-grouping-sets-routing
Draft

refactor(hash-aggr): grouping sets routing#24261
2010YOUY01 wants to merge 3 commits into
apache:mainfrom
2010YOUY01:split-aggr-03-grouping-sets-routing

Conversation

@2010YOUY01

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Part of #22710
Depends on #24260

Only the last commit belongs to this PR. I’ll rebase onto main after the dependent PR is merged.

Rationale for this change

The key changes are in datafusion/physical-plan/src/aggregates/mod.rs

  • Route execution for aggregation with grouping sets to the migrated (see epic issue for background) aggregation path
  • If input is ordered by group keys, and grouping set is also enabled, fallback to regular unordered aggregation
    • legacy implementation can somehow enable the ordering optimization for grouping sets, I'm quite skeptical if it get it correct. There will be a lot of extra implementation complexity if we want to apply ordering optimization for grouping sets, so now this PR just separating them for simplicity.

What changes are included in this PR?

Are these changes tested?

Existing tests + UT

Are there any user-facing changes?

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.28571% with 89 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.99%. Comparing base (634f0b3) to head (aec7b70).
⚠️ Report is 42 commits behind head on main.

Files with missing lines Patch % Lines
...sical-plan/src/aggregates/ordered_single_stream.rs 86.74% 49 Missing and 10 partials ⚠️
datafusion/physical-plan/src/aggregates/mod.rs 88.19% 5 Missing and 14 partials ⚠️
...plan/src/aggregates/aggregate_hash_table/common.rs 88.37% 0 Missing and 5 partials ⚠️
.../aggregates/aggregate_hash_table/common_ordered.rs 60.00% 0 Missing and 2 partials ⚠️
...gates/aggregate_hash_table/ordered_single_table.rs 94.44% 0 Missing and 2 partials ⚠️
...c/aggregates/aggregate_hash_table/partial_table.rs 0.00% 0 Missing and 1 partial ⚠️
...rc/aggregates/aggregate_hash_table/single_table.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24261      +/-   ##
==========================================
- Coverage   81.06%   80.99%   -0.07%     
==========================================
  Files        1107     1108       +1     
  Lines      382191   384679    +2488     
  Branches   382191   384679    +2488     
==========================================
+ Hits       309805   311583    +1778     
- Misses      54083    54727     +644     
- Partials    18303    18369      +66     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants