Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v24.1.x] Transforms: Fix deploy-time timequery bug #20139

Open
wants to merge 2 commits into
base: v24.1.x
Choose a base branch
from

Commits on Jun 25, 2024

  1. xform: Fix cold-start timequery bug

    Default behavior is for transforms to start processing input from an offset
    equivalent to "the latest committed offset at deploy time". However, in
    situations where no records are committed between the moment the transform
    was deployed and the moment the transform starts, the timequery servicing
    this logic returns an empty result, which is handled by returning offset::min,
    which signals the transform to begin processing at the beginning of the topic.
    
    This commit adjusts the logic in source::offset_at_timestamp to return nullopt
    for an empty timequery result, and the handling of that nullopt to resolve
    the transform start offset to latest.
    
    Signed-off-by: Oren Leiman <[email protected]>
    (cherry picked from commit 2401fa3)
    oleiman committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5f01cf4 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. dt/xform: Test that transforms consume from end of topic

    Specifically in situations where no records are produced between
    deploy time and start time.
    
    Signed-off-by: Oren Leiman <[email protected]>
    (cherry picked from commit 0f49b21)
    oleiman committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    bda09b8 View commit details
    Browse the repository at this point in the history