Skip to content

Use RPC providers to speed up syncing from old snapshots #5504

@lemmih

Description

@lemmih

Summary

Syncing from an old snapshot requires downloading a lot of tipsets (2880 per day the snapshot is behind). We download the tipsets from the libp2p swarm in batches of 16, and we're limited to a single batch at a time. This sequential download is often slower than the validation of the tipsets, and validation cannot begin until all the tipsets have been downloaded.

RPC providers offer a ChainGetTipSetByHeight method that returns a tipset for a given height. We can use this method as an alternate, non-libp2p source of tipsets. Instead of sequentially fetching a single range of tipsets from the swarm, we can use RPC providers to split the range into smaller ranges fetched in parallel.

Requirements

Fetching headers starting from a 1-week-old snapshot should at least be 10x faster.

Motivation

I want syncing from older snapshots to be faster.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: RequestRequests to implement features or make other changes

    Type

    No type

    Projects

    Status

    Ready

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions