Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose bitfield #7

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

Frando
Copy link
Contributor

@Frando Frando commented Aug 11, 2020

This adds two optional fields bitfield and length to the has method request. If set, the response includes a run-length encoded bitfield of the downloaded blocks from seq for length.

On the client side, a new getBitfield method is added. It resolves to the decoded bitfield, wrapped in a bitfield instance.

This, together with #5, solves #2 and should allow indexers to work reliably and efficiently on sparsely synced hypercores.

@mafintosh
Copy link
Contributor

Whats your usecase for getting the bitfields out? Since this is adding the semver'ed api surface, we're per default a little conservative about adding apis

@Frando
Copy link
Contributor Author

Frando commented Aug 13, 2020

I have an indexer that wants to index all downloaded blocks for a list of (sparse) feeds.Therefore, it tracks which key-seq pairs it has indexed already. With hyperspace, I cannot ensure that there's no downloads happening while the indexer is not running. So, to check for that, on each start of the indexer, without this patch I'd have to do one async has request for each seq for each feed I'm tracking. With this patch, it's just one async request per feed, and then the bitfield comparison can happen in a regular sync loop.

@Frando
Copy link
Contributor Author

Frando commented Sep 7, 2020

@mafintosh if a new method getBitfield sounds better to you I'm fine with that as well. I thought it fitted nicely with the has method (to keep API surface smaller) but don't really mind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants