Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Pluggable gc with exemptions #17

Merged
merged 9 commits into from
Nov 28, 2024
Merged

Conversation

rklaehn
Copy link
Collaborator

@rklaehn rklaehn commented Nov 25, 2024

Description

This adds gc to blobs so it does not have to be driven externally. A big complication is that we want to have the ability to add exemptions to gc, e.g. from docs or willow.

So the gc is initially not running when the Blobs object is created. We have the ability to add exemption callbacks in this stage. Once the setup is done you can (but don't have to!) start the gc loop, at which point adding exemptions is no longer possible.

Todo:

  • try out with docs

Breaking Changes

Notes & open questions

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.

also add ability to add excepmtions for gc before gc is started.
@rklaehn rklaehn changed the base branch from main to import-exapmles November 25, 2024 10:24
Copy link

github-actions bot commented Nov 25, 2024

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh-blobs/pr/17/docs/iroh_blobs/

Last updated: 2024-11-28T13:24:20Z

Base automatically changed from import-exapmles to main November 26, 2024 11:35
@rklaehn rklaehn marked this pull request as ready for review November 26, 2024 14:06
@rklaehn rklaehn requested a review from matheus23 November 26, 2024 14:12
@rklaehn
Copy link
Collaborator Author

rklaehn commented Nov 26, 2024

The place where I tried this out is https://github.com/n0-computer/iroh-docs/tree/try-out-gc

@dignifiedquire
Copy link
Collaborator

new api looks good

Copy link
Member

@matheus23 matheus23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly confident this API will work for willow.

tests/blobs.rs Outdated Show resolved Hide resolved
rklaehn and others added 4 commits November 28, 2024 10:46
different way to deal with the clone madness

Co-authored-by: Philipp Krüger <[email protected]>
first protect, then delete tag
# Conflicts:
#	src/net_protocol.rs
@rklaehn rklaehn merged commit 25525a8 into main Nov 28, 2024
23 of 24 checks passed
@rklaehn rklaehn deleted the pluggable-gc-with-exemptions branch November 28, 2024 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants