[garbage_colection] Fix crash in clear_resharding_data while fetching prev_epoch_id

[shreyan-gupta]

We had an issue in the clear_resharding_data while fetching the prev_epoch_id. The implementation of prev_epoch_id relied on fetching the block_info of the last block of prev_epoch to get the epoch_id. This unfortunately failed for the case of GC as the block_info was already garbage collected.

The new implementation here relies on using the block_header to get the epoch_id instead of block_info.

This was unfortunately only caught in mocknet and not integration tests as having a small enough epoch_length lead to the block_info being cached in the epoch_manager (even though it was GC'd)

Zulip post: https://near.zulipchat.com/#narrow/stream/295558-pagoda.2Fcore/topic/Master.20binary.20Can't.20clear.20old.20data/near/402240517

Pending: Figure out a way to include test from this PR: #10193
We would need to enable no_cache feature for the test.

[wacban]

LGTM

[wacban]

It's a bit confusing that the comment says we're using the headers and immediately in the first line after you get the block info :) Maybe clarify for which blocks we expect to have the info and for which only the headers?

[shreyan-gupta]

Updated comments with more detailed explanation.

[posvyatokum]

This should work from gc point of view even after we eventually implement partial BlockHeader garbage collection.
But I wonder, do you really want to wait at least 3 (default 5) epochs to gc resharding data? Currently clear_data is only called for blocks that fall out of gc_num_epochs_to_keep boundary. I thought you were thinking about deleting this right after resharding epoch is finished.
If so, it will require bigger refactoring of garbage collection (can be done outside of this PR).

[shreyan-gupta]

This should work from gc point of view even after we eventually implement partial BlockHeader garbage collection. But I wonder, do you really want to wait at least 3 (default 5) epochs to gc resharding data? Currently clear_data is only called for blocks that fall out of gc_num_epochs_to_keep boundary. I thought you were thinking about deleting this right after resharding epoch is finished. If so, it will require bigger refactoring of garbage collection (can be done outside of this PR).

I'd made this change from the perspective of it being the least disruptive and most aligned with the current GC behavior at the cost of storage for 3 to 5 epochs. If we'd like to delete the old shard data as soon as resharding is done, we can do that as well. I however see a thread on Zulip to reduce the GC window to basically when we finalize block (for BP)