Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP][Proposal] PARQUET-2430: Add parquet joiner #1273

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
Open

[WIP][Proposal] PARQUET-2430: Add parquet joiner #1273

wants to merge 24 commits into from

Commits on Jan 28, 2024

  1. add initial ParquetJoiner implementation

    maxim_konstantinov committed Jan 28, 2024
    Configuration menu
    Copy the full SHA
    f5144b2 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2024

  1. add initial ParquetJoiner implementation

    maxim_konstantinov committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    01a08dd View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. Merge remote-tracking branch 'origin/master' into add-parquet-joiner

    maxim_konstantinov committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    28c987c View commit details
    Browse the repository at this point in the history

Commits on Feb 17, 2024

  1. refactor ParquetJoiner implementation

    maxim_konstantinov committed Feb 17, 2024
    Configuration menu
    Copy the full SHA
    7ae3505 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. extend the main test for multiple files on the right

    maxim_konstantinov committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    05eb22a View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. extend the main test for multiple files on the right

    maxim_konstantinov committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    6bb950d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into add-parquet-joiner

    maxim_konstantinov committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    87b923c View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2024

  1. converge join logic, crate a draft of options and rewriter

    maxim_konstantinov committed Feb 23, 2024
    Configuration menu
    Copy the full SHA
    f9536c3 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. move ParquetJoinTest logic to ParquetRewriterTest

    maxim_konstantinov committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    d7f11d9 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2024

  1. improve Parquet stitching test

    maxim_konstantinov committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    e8e7ffe View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. remove custom ParquetRewriter constructor

    maxim_konstantinov committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    3ee946c View commit details
    Browse the repository at this point in the history

  2. remove custom ParquetRewriter constructor

    maxim_konstantinov committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    fd409c4 View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. refactor ParquetRewriter

    maxim_konstantinov committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    5a98219 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. apply spotless and address PR comments

    maxim_konstantinov committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    7b2fd1a View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. move extra column writing into processBlocksFromReader

    maxim_konstantinov committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    8da8291 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2024

  1. add getInputFiles back

    maxim_konstantinov committed Mar 16, 2024
    Configuration menu
    Copy the full SHA
    68e41ba View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'fork/master' into add-parquet-joiner

    maxim_konstantinov committed Mar 16, 2024
    Configuration menu
    Copy the full SHA
    98b9b23 View commit details
    Browse the repository at this point in the history

  3. fix extra ParquetRewriter constructor so tests can pass

    maxim_konstantinov committed Mar 16, 2024
    Configuration menu
    Copy the full SHA
    6d2c222 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. remove not needed TODOs

    maxim_konstantinov committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    883e935 View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2024

  1. address PR comments

    maxim_konstantinov committed Mar 24, 2024
    Configuration menu
    Copy the full SHA
    8ef36b5 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. Merge remote-tracking branch 'origin/master' into add-parquet-joiner

    maxim_konstantinov committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    79cc2b8 View commit details
    Browse the repository at this point in the history

  2. rename inputFilesR to inputFilesToJoin

    maxim_konstantinov committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    0bbf72f View commit details
    Browse the repository at this point in the history

  3. rename inputFilesR to inputFilesToJoinColumns

    maxim_konstantinov committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    ca53bff View commit details
    Browse the repository at this point in the history

  4. add getParquetInputFiles listing to the rewrite start logging

    maxim_konstantinov committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    1e7998a View commit details
    Browse the repository at this point in the history