retryable_indexes allocated and dropped by scheduler #4157

apfitzge · 2024-12-17T20:05:42Z

Problem

Communication pattern between the scheduler and worker thread allocations is a bit off.
Currently, the scheduler sends a ConsumeWork and the workers respond with FinishedConsumeWork which holds an additional Vec<usize> representing the retryable transactions
This leads to the worker threads consistently allocating vectors that are then dropped by the scheduler thread

Summary of Changes

Pre-allocate retryable_indexes on work message which is then populated by the workers and returned for inspection/deallocation by the scheduler thread

Fixes #

apfitzge · 2024-12-17T20:07:31Z

core/src/banking_stage/consume_worker.rs

+        // Copy retryable indexes back to return message
+        work.retryable_indexes.extend(
+            output
+                .execute_and_commit_transactions_output
+                .retryable_transaction_indexes
+                .iter()
+                .cloned(),
+        );


Copying the indexes here - possible to improve further by allowing the consumer to fill this in directly. But for now this PR just gets rid of the wonky allocation/deallocation pattern

retryable_indexes allocated and dropped by scheduler

0899b46

apfitzge self-assigned this Dec 17, 2024

apfitzge commented Dec 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

retryable_indexes allocated and dropped by scheduler #4157

retryable_indexes allocated and dropped by scheduler #4157

apfitzge commented Dec 17, 2024

apfitzge Dec 17, 2024

retryable_indexes allocated and dropped by scheduler #4157

Are you sure you want to change the base?

retryable_indexes allocated and dropped by scheduler #4157

Conversation

apfitzge commented Dec 17, 2024

Problem

Summary of Changes

apfitzge Dec 17, 2024

Choose a reason for hiding this comment