[ISSUE #10966] Fix TransactionalOpBatchService busy loop when deleteContext is drained - #10967
[ISSUE #10966] Fix TransactionalOpBatchService busy loop when deleteContext is drained#10967waterWang wants to merge 1 commit into
Conversation
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
This PR fixes a critical busy-loop bug in TransactionalOpBatchService where the thread consumes ~100% CPU after transaction-message workloads go idle. The fix is minimal, well-documented, and includes a unit test that reproduces the issue.
Analysis
Root Cause (Accurate):
deleteContextentries are added viaputIfAbsentbut never removed- After draining, contexts remain with stale
lastWriteTimestamp batchSendOpMessage()returns 0 when all contexts are empty- Returning 0 causes immediate re-execution → busy loop
Fix Quality:
- ✅ Minimal change (11 lines + clear comments)
- ✅ Returns
currentTime + intervalinstead of 0, breaking the loop - ✅ New deletes still wake the thread promptly via
wakeup() - ✅ No breaking changes, backward compatible
- ✅ Unit test reproduces the bug and validates the fix
Code Review:
The fix is correct and addresses the root cause. The comment clearly explains the "why" (not just "what"), which helps future maintainers understand the edge case. The test case properly validates the fix by ensuring the return value is a future timestamp.
Performance Impact:
This eliminates unnecessary CPU consumption in production environments with intermittent transaction-message workloads. A significant operational improvement.
Verdict
LGTM — Solid bug fix with proper test coverage. Ready to merge.
Automated review by github-manager-bot
…eleteContext is drained When every entry in deleteContext has already been drained (offsets batched and sent) but is still present in the map, batchSendOpMessage() falls through to the stale lastWriteTimestamp branch and returns 0L. TransactionalOpBatchService treats 0 as "run again immediately", so waitForRunning(0) returns at once and the thread busy-loops at ~100% CPU after any transaction-message workload goes idle. Fix by returning System.currentTimeMillis() + transactionOpBatchInterval when no op message was batched in the round (sendMap == null), so the thread sleeps until the next scheduled scan. New deletes still wake it promptly via deletePrepareMessage() -> wakeup(). Signed-off-by: waterWang <waterwang@proton.me>
What
Fixes #10966 — the
TransactionalOpBatchServicethread busy-loops at ~100% CPU after a transaction-message workload goes idle.Root cause
deleteContextentries are only everputIfAbsentindeletePrepareMessage()and never removed. Once a context has been fully drained (all offsets batched and sent), it stays in the map forever with a stalelastWriteTimestamp.In
batchSendOpMessage(), when every context is empty, the scan falls through to the stale-firstTimestampbranch:firstTimestamp = min(startTime, staleLastWrite)→ stale valuewakeupTimestamp = stale + interval < startTime→ thewakeupTimestamp > startTimeguard fails0LBack in
TransactionalOpBatchService.run():→ tight loop, one CPU core pinned at ~100% indefinitely.
Fix
When no op message was batched in the round (
sendMap == null), returnSystem.currentTimeMillis() + transactionOpBatchIntervalinstead of0L, so the thread sleeps until the next scheduled scan. New deletes still wake it promptly viadeletePrepareMessage() -> transactionalOpBatchService.wakeup().Verification
testBatchSendOpMessage_noPendingDataReturnsFutureWakeup— places a drained context (stalelastWriteTimestamp = 0) indeleteContextand asserts the returned wakeup time is in the future.mvn -am -pl broker test -Dtest=TransactionalMessageServiceImplTest→ Tests run: 9, Failures: 0, Errors: 0 (8 pre-existing + 1 new).TransactionalOpBatchService.run()+batchSendOpMessage(): before fix 12,126,584 iterations/2s (busy loop), after fix 13 iterations/2s (sleeps normally).