Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-16733: Add share group record support to OffsetsMessageParser #17282

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from

Conversation

AndrewJSchofield
Copy link
Contributor

@AndrewJSchofield AndrewJSchofield commented Sep 26, 2024

KIP-932 added a bunch of new record schemas to the consumer offsets topic which need to be added to the OffsetsMessageParser in kafka-dump-log.sh.

Here's an example of the output:

% bin/kafka-dump-log.sh --offsets-decoder --files /tmp/kraft-combined-logs/__consumer_offsets-0/00000000000000000000.log 
Dumping /tmp/kraft-combined-logs/__consumer_offsets-0/00000000000000000000.log
Log starting offset: 0
baseOffset: 0 lastOffset: 5 count: 6 baseSequence: 0 lastSequence: 5 producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 0 isTransactional: false isControl: false deleteHorizonMs: OptionalLong.empty position: 0 CreateTime: 1727354207209 size: 480 magic: 2 compresscodec: none crc: 2390172074 isvalid: true
| offset: 0 CreateTime: 1727354207209 keySize: 50 valueSize: 42 sequence: 0 headerKeys: [] key: {"type":"10","data":{"groupId":"console-share-consumer","memberId":"Tsnir8MZRt27WL47QhcKVA"}} payload: {"version":"0","data":{"rackId":null,"clientId":"console-share-consumer","clientHost":"/127.0.0.1","subscribedTopicNames":["T1"]}}
| offset: 1 CreateTime: 1727354207209 keySize: 26 valueSize: 29 sequence: 1 headerKeys: [] key: {"type":"9","data":{"groupId":"console-share-consumer"}} payload: {"version":"0","data":{"topics":[{"topicId":"xfugYJTERGetNxY2KH_JFA","topicName":"T1","numPartitions":1,"partitionMetadata":[]}]}}
| offset: 2 CreateTime: 1727354207209 keySize: 26 valueSize: 7 sequence: 2 headerKeys: [] key: {"type":"11","data":{"groupId":"console-share-consumer"}} payload: {"version":"0","data":{"epoch":1}}
| offset: 3 CreateTime: 1727354207209 keySize: 50 valueSize: 26 sequence: 3 headerKeys: [] key: {"type":"13","data":{"groupId":"console-share-consumer","memberId":"Tsnir8MZRt27WL47QhcKVA"}} payload: {"version":"0","data":{"topicPartitions":[{"topicId":"xfugYJTERGetNxY2KH_JFA","partitions":[0]}]}}
| offset: 4 CreateTime: 1727354207209 keySize: 26 valueSize: 7 sequence: 4 headerKeys: [] key: {"type":"12","data":{"groupId":"console-share-consumer"}} payload: {"version":"0","data":{"assignmentEpoch":1}}
| offset: 5 CreateTime: 1727354207209 keySize: 50 valueSize: 35 sequence: 5 headerKeys: [] key: {"type":"14","data":{"groupId":"console-share-consumer","memberId":"Tsnir8MZRt27WL47QhcKVA"}} payload: {"version":"0","data":{"memberEpoch":1,"previousMemberEpoch":0,"state":0,"assignedPartitions":[{"topicId":"xfugYJTERGetNxY2KH_JFA","partitions":[0]}]}}

These are the records to show that a new share group has been created as a new member was assigned a single partition.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@github-actions github-actions bot added the core Kafka Broker label Sep 26, 2024
@AndrewJSchofield
Copy link
Contributor Author

3 unrelated test failures (features command).

Copy link
Contributor

@mumrah mumrah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AndrewJSchofield, LGTM!

Can you include some sample output in the PR description?

@mumrah mumrah added the tools label Sep 27, 2024
@chia7712
Copy link
Contributor

@AndrewJSchofield Could you please rebase code to run CI again?

@AndrewJSchofield
Copy link
Contributor Author

Closing and re-opening PR to rerun the build.

@mumrah
Copy link
Contributor

mumrah commented Sep 30, 2024

@AndrewJSchofield just merging trunk into the PR should be enough to trigger CI. If there's no commits to merge in, you can either push an empty commit or manually re-run the job. Note that re-running a job does not pull in the latest trunk changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants