[WIP] [Rhythm] Block builder test updates #4510
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
This updates the block builder tests and fake kafka cluster setup to be closer to real kafka, and less specific to the current block builder implementation. The goal is to make the tests more general and stable against future block builder changes.
Changes:
kfake
directly, wrap with another struct...Why does this need to change:
To support consumer groups, we use custom callback via
ControlKey(OffsetCommit,...)
. And another one for tests like_committingFails
to simulate a commit failure. Howeverkfake
stores these callbacks in a map and cannot guarantee their execution order. Therefore the only way to reliably test is combine all with one callback. Using the wrapper struct is overall easier than the alternative plumbing in the previous setup.Related: #4480
Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]