Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add from-genesis subcommand #342

Merged
merged 19 commits into from
Mar 28, 2023
Merged

Conversation

morgsmccauley
Copy link
Contributor

@morgsmccauley morgsmccauley commented Feb 1, 2023

This PR adds the from-genesis subcommand which does the following:

  1. Downloads the genesis file from S3 and stores the accounts/access keys defined in it
  2. Starts indexing from the genesis block

This does not delete the genesis file after downloading it, this job will be left to the executor.

Majority of this code has been copied from the master-nearcore version of storing genesis records, the obvious difference being that we now need to download the genesis file. I've commented inline where other things differ.

Closes #327

@morgsmccauley morgsmccauley changed the title feat: Add from-genesis command feat: Add from-genesis subcommand Feb 1, 2023
indexer/src/main.rs Outdated Show resolved Hide resolved
@morgsmccauley morgsmccauley marked this pull request as ready for review February 1, 2023 19:15
@morgsmccauley morgsmccauley requested review from telezhnaya and khorolets and removed request for telezhnaya February 1, 2023 19:15
khorolets
khorolets previously approved these changes Feb 3, 2023
Copy link
Member

@khorolets khorolets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall! I've left a few comments. I don't want to be a blocker here, so I approve it while expecting you to solve my remarks. Thanks!

indexer/src/configs.rs Show resolved Hide resolved
indexer/src/main.rs Outdated Show resolved Hide resolved
indexer/src/configs.rs Show resolved Hide resolved
database/src/adapters/genesis.rs Show resolved Hide resolved
@khorolets
Copy link
Member

@ecp88 I remember we talked about disabling buildkite jobs for all the branches except master-nearcore. Do you have any updates on that?

database/src/adapters/genesis.rs Outdated Show resolved Hide resolved
database/src/adapters/genesis.rs Outdated Show resolved Hide resolved
@morgsmccauley morgsmccauley force-pushed the feat/start-from-genesis branch from c412f6f to 93b057d Compare March 15, 2023 01:02
Copy link
Member

@khorolets khorolets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know when you finished and I can start a review (disregard my current approval, please)

indexer/src/main.rs Outdated Show resolved Hide resolved
indexer/src/main.rs Outdated Show resolved Hide resolved
@kampleforth
Copy link

when is this slated to merge? I'm having trouble with the from-block subcommand, it always fails with database query errors

@morgsmccauley
Copy link
Contributor Author

@kampleforth I am working on this now, you can expect it soon

@morgsmccauley morgsmccauley force-pushed the feat/start-from-genesis branch from 5a32102 to e3d9630 Compare March 24, 2023 02:34
indexer/src/main.rs Outdated Show resolved Hide resolved
@khorolets khorolets self-requested a review March 24, 2023 08:24
@khorolets
Copy link
Member

when is this slated to merge? I'm having trouble with the from-block subcommand, it always fails with database query errors

There is no guarantee your issue is related to this pull request. Please, consider firing an issue describing the trouble you have with from-block and what exact "database query errors` you struggle with.

@khorolets khorolets dismissed their stale review March 24, 2023 12:39

We rethought the process

@morgsmccauley morgsmccauley force-pushed the feat/start-from-genesis branch from e3d9630 to 6537660 Compare March 27, 2023 22:07
@morgsmccauley
Copy link
Contributor Author

morgsmccauley commented Mar 27, 2023

@khorolets this is ready for re-review now. I've resolved most comments, but left a couple open
because I want to ensure you are onboard with the solutions there :).

Copy link
Member

@khorolets khorolets left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@morgsmccauley morgsmccauley merged commit b64a423 into master Mar 28, 2023
@morgsmccauley morgsmccauley deleted the feat/start-from-genesis branch March 28, 2023 08:03
This was referenced Apr 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store genesis file with Lake based indexer
5 participants