Skip to content

feat: add ASOF joins to DataFrame API - #23831

Merged
2010YOUY01 merged 8 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-dataframe
Sep 11, 2026
Merged

feat: add ASOF joins to DataFrame API#23831
2010YOUY01 merged 8 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-dataframe

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 23, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Rationale for this change

This is the DataFrame API layer of the ASOF JOIN stack. It exposes the logical contract already available on main through the fluent API without coupling DataFrame callers to SQL.

What changes are included in this PR?

  • Add DataFrame::join_asof with an optional equality expression and one ordered match expression.
  • Add DataFrame::join_asof_using for same-named equality keys.
  • Cover both entry points with a DataFrame execution test.

Are these changes tested?

Yes:

  • cargo fmt --all
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo test -p datafusion --test core_integration dataframe::join_asof --all-features
  • The extended workspace test command from the contributor guide

Are there any user-facing changes?

Users can build ASOF joins through DataFrame::join_asof and DataFrame::join_asof_using. Both methods use the same expression validation and left-preserving semantics as the logical builder and SQL API.

@github-actions github-actions Bot added sql SQL Planner logical-expr Logical plan and expressions optimizer Optimizer rules core Core DataFusion crate substrait Changes to the substrait crate proto Related to proto crate physical-plan Changes to the physical-plan crate auto detected api change Auto detected API change labels Jul 23, 2026
@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.66667% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.90%. Comparing base (4048898) to head (7dbc424).
⚠️ Report is 33 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/core/src/dataframe/mod.rs 86.66% 0 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23831      +/-   ##
==========================================
+ Coverage   81.80%   81.90%   +0.10%     
==========================================
  Files        1130     1132       +2     
  Lines      417754   420600    +2846     
  Branches   417754   420600    +2846     
==========================================
+ Hits       341754   344503    +2749     
+ Misses      55875    55773     -102     
- Partials    20125    20324     +199     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the physical-expr Changes to the physical-expr crates label Aug 14, 2026
@Xuanwo
Xuanwo force-pushed the xuanwo/asof-dataframe branch from 90448bd to 7713821 Compare August 20, 2026 17:56
@Xuanwo
Xuanwo force-pushed the xuanwo/asof-dataframe branch from 7713821 to 7ef360b Compare August 28, 2026 05:40
@github-actions github-actions Bot removed physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate labels Aug 28, 2026
@github-actions github-actions Bot added the physical-plan Changes to the physical-plan crate label Aug 30, 2026
# Conflicts:
#	datafusion/expr/src/logical_plan/builder.rs
#	datafusion/expr/src/logical_plan/plan.rs
#	datafusion/substrait/tests/cases/serialize.rs
@Xuanwo
Xuanwo marked this pull request as ready for review September 9, 2026 12:18
@Xuanwo

Xuanwo commented Sep 9, 2026

Copy link
Copy Markdown
Member Author

@2010YOUY01 & @jayzhan211, this PR is ready, let's go!

@github-actions github-actions Bot removed the auto detected api change Auto detected API change label Sep 9, 2026

@jayzhan211 jayzhan211 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Xuanwo , some non-blocking suggestions

Comment thread datafusion/core/tests/dataframe/mod.rs Outdated
Comment thread datafusion/core/src/dataframe/mod.rs Outdated
@github-actions github-actions Bot removed sql SQL Planner logical-expr Logical plan and expressions labels Sep 10, 2026
@github-actions github-actions Bot removed optimizer Optimizer rules substrait Changes to the substrait crate proto Related to proto crate physical-plan Changes to the physical-plan crate labels Sep 10, 2026

@2010YOUY01 2010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you.

@2010YOUY01
2010YOUY01 added this pull request to the merge queue Sep 11, 2026
@Xuanwo

Xuanwo commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Thank you @jayzhan211 and @2010YOUY01, let's rock!

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 11, 2026
@Xuanwo

Xuanwo commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Cc @2010YOUY01 for another merge queue

@2010YOUY01
2010YOUY01 added this pull request to the merge queue Sep 11, 2026
Merged via the queue into apache:main with commit 4d87baf Sep 11, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants