Tracking sync sources shouldn't wait for headers

[Mirko-von-Leipzig]

Tracking sources for non-headers currently wait for a new header (or state diff for classes), and then download that one block via p2p. This is quite suboptimal as the header is only required to turn the item stream into block-of-items stream - which means there is an unecessary delay to downloading the items.

A better layout is to have all sources to follow the header source design: monitor the latest onchain block and stream out all items for the missing blocks. In other words, headers should be downloaded concurrently with the rest of the data instead of headers first and then the rest.

This does mean that non-headers cannot be turned into block-of-items yet, but this can be done in follow-up stages e.g. using BufferStage from #2066.

[Mirko-von-Leipzig]

There is an issue with this model that I have been unable to properly solve. Each source somehow needs to track which block's it has succesfully handled, however peers are allowed to send less data than the fully requested range - however this would only be "figured" out after the blockify stage. The exception is the header source since it of course has the block number bundled in - and each item covers a single block.

Maybe having the block number as a delimited message inside the p2p stream wasn't such a bad idea after all..

One possible solution is to have feedback from the blockify stage. Of course this somewhat sucks since now the system flow isn't in a single direction. It also requires the entire system to complete fully i.e. blockifies input must have been fully flushed. This implies the above drawing is more of a happy path - we need synchronized feedback. One way to accomplish this is by embedding a oneshot feedback channel as a message - once it is processed we know what the status is.