wip: zstd:chunked support via oci-client #137
Conversation
This code pulls images via containers-image-proxy-rs (which is a wrapper around skopeo). It's a bit too large to have it in the top-level lib.rs, and we're about to add an alternate implementation, so split it out. We keep a pull() function in place for compatibility reasons to avoid needing to update all callers: we will start using this function to dispatch to the correct implementation depending on the reference format. Signed-off-by: Allison Karlitskaya <[email protected]>
This is based on the "pull" example in the zstd-chunked-rs repository. Signed-off-by: Allison Karlitskaya <[email protected]>
|
There's some cool code here, and I think it's really useful actually to have this as an option even if it's hidden/experimental just to use as a comparison. The main issue is really c/image (the important part of skopeo here) vs oci-client as expanded on in https://github.com/containers/containers-image-proxy-rs/?tab=readme-ov-file#why and it's for that reason I can't imagine using this by default in bootc at least anytime in the nearly visible future. Things like ECR credential helpers are highly load bearing. OTOH, other folks could reasonably make other choices. |
|
Ya. My first instinct is to stick this behind a feature flag (and maybe also do the same with the skopeo and potential-future-oci-dir paths)... it could be useful for the kind of scenario where we don't need the skopeo features. In terms of those features, I feel like:
I feel like we could maybe close some of these gaps by adding some code to skopeo to do the work of resolving registry aliases, taking care of the initial authentication, etc. and call out to that. I'm also sort of of the (optimistic) opinion that maybe we can have a Rust-based stack for all of this stuff at some point, and honestly, the sooner the better... and although this piece of code here leaves a lot of the other questions unanswered, it's maybe one part of moving towards a world where we could image a feasible solution to that puzzle... |
→ #138 |
|
So the biggest unanswered question here is what to do about the sha256-to-fsverity mapping. It's currently stored as raw My initial impulse was to store symlinks in the repository, but: this is just about the worst possible thing we can do. There's no actual benefit to being able to follow sha256 symlinks into the object files and this is only used for zstd:chunked. I'd actually prefer something more like an sqlite database storing sha256->fs-verity mappings. Also: it might make sense to store that in @alexlarsson is currently trying to solve a very similar issue for the ostree mapping... |
No description provided.