feat: ledger replay with 'fake' validator keypair

[thlorenz]

Summary

Adding ability to prepare a ledger for replay in a diagnostics/test scenario.

Originally @GabrielePicco requested a way to basically ignore the keypair until after ledger
replay. However from looking at the way that the validator is initialized I realized this would
turn out to be a heavy handed solution to the problem.

Instead I decided to add a new command to the genx CLI that would allow for the keypair to be
overwritten right before we want to replay a specific legger. This is even more secure since
we're sure to wipe the original validator keypair from disk before starting the replay.

This also allows downloading the ledger without the keypair in the first place.

It is also safe to not cause any changes on chain (as a result of replay or txs after) since
the validator keypair is incorrect and cannot sign any on-chain transactions.

FIXES: #299

NOTE: I'm basing this off the previous PR (#300) since builds up on it. Once that is merged we
need to rebase this PR on master to merge.

Details

Here is an excerpt from the genx README explaining how to overwrite the validator keypair with a
random one when replaying the ledger to diagnose/test.

test-ledger

Prepares a ledger for replay in a diagnostics/test scenario. At this point it ensures that the
original keypair of the validator operator is not used. Instead it is overwritten with a random
keypair. That keypair is then printed to the terminal so that the VALIDATOR_KEYPAIR env var
can be set to it when running the epehemeral validator to replay the ledger.

One can auto-set that env var using the output of this command as follows:

export VALIDATOR_KEYPAIR=$(cargo run --bin genx -- test-ledger ./ledger)

Then we can just do the following to replay the ledger (i.e. providing the above config):

cargo run --release ledger-config.toml

Greptile Summary

This PR adds a new test-ledger command to the genx CLI tool that enables ledger replay testing without requiring the original validator keypair, improving testing and diagnostic capabilities.

• Added test-ledger command in tools/genx/src/main.rs that overwrites validator keypair with a random one for testing
• Added ledger_replay_test.rs module to handle keypair generation and file management for test scenarios
• Updated README with clear documentation and examples for using the new test-ledger functionality
• Builds on previous PR (feat: log ledger replay progress update #300) which added progress logging for ledger replay operations

[greptile-apps]

_{3 file(s) reviewed, 4 comment(s)
Edit PR Review Bot Settings | Greptile}

[GabrielePicco]

Testing replay on a realistic ledger with a different keypair fails on the first transaction that requires a clone.

Changes needed

[thlorenz]

Testing replay on a realistic ledger with a different keypair fails on the first transaction that requires a clone.

OK so we can improve on the idea, but there are other important changes in here which are needed to successfully simulate ledger replay locally, namely handling a huge amount of accounts.

The only piece that is not desired at this point is the ledger keypair overwrite. However that is already a separate command, I just need to improve the docs on that.

So once I did the docs part we should merge this anyways (I'll put a note about the limitations with the docs).
What do you think?

[GabrielePicco]

I would rename it to OverrideKeypair (same for the command name), since is not preparing for tests

[GabrielePicco]

Ditto, I would rename to OverrideKeypair

[GabrielePicco]

I'm not against merging the PR, it adds a functionality to override the keypair.
I would rename the new command to something more explicit as mentioned in the command to avoid confusion and we will address properly the re-play with another approach

[thlorenz]

LedgerReplayTest

The idea of the name was that in the future this command would do any extra preparation of the ledger to make it usable in tests. But I can rename for now if you prefer

[GabrielePicco]

Closing as it doesn't fix the issue. We will re-open with a proper fix for the replay