Receive a built image through the content store

[MayCXC]

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update

Motivation and Context

A build exports its image somewhere, and exporting it into the store the runtime reads from is what makes the image runnable without a trip through a tar. The exporter writes the blobs and the build records the images naming them. The builder-side half is apple/container-builder-shim#98, which explains what the tar path costs: three copies of every image live at once at the end of a build, every byte crossing whether the caller holds it or not, and an unbounded gRPC buffer pool writing the stream over virtiofs.

A blob between arriving and being claimed belongs to nobody. The sweep that reclaims blobs no image references takes it, so a build running alongside image rm lost the manifest it had just exported. The blobs a build exports gather in an ingest session, which the store does not hold and no sweep lists, and the store takes the whole set in the same held lock that records the images naming them: the blobs appear at a moment when no sweep can run, and the records claiming them are there when the lock is released. Pulling an image and loading an OCI layout land their content the same way and for the same reason.

The lock is what makes that a moment rather than a window. The store's operations suspend mid-flight and the actor admits other calls at every suspension, so a sweep overlapping a landing computed its keep set without the arriving image; it is held across each whole mutating operation now, the way containerd guards the same window with leases: https://github.com/containerd/containerd/blob/main/docs/garbage-collection.md

Only an export into the store gathers, so only then is there a session, and that export is what hands it over: a build that reaches its end has nothing left to give back, and one that does not is what the catch returns it for.

Two answers the store owes a writer follow from where the blobs are. A writer asks what the store holds before writing and skips whatever it says it has, so a blob it asks about is one this build will not write and still needs: the gathering takes a copy, the way importing an image copies the blobs the store already holds into its own ingest. And a blob already gathered is answered for out of the gathering, since the store does not hold it yet, which is how importing an image resolves a blob it may itself have only just written.

A blob is named for its digest alone under the store's algorithm directory, which is the name a later read resolves; a name carrying the algorithm again lands a file nothing looks for.

Dependency and Merge Order

Depends on #2183 (carried in this branch, and through it on #2020) for the pipeline this extends, and on apple/container-builder-shim#98 for the exporter that writes into the proxy. Merge order: apple/container-builder-shim#88, #2020, apple/container-builder-shim#99, #2183, apple/container-builder-shim#98, then this.

The lock this holds across mutating store operations is the same one #2171 introduces, for the same reason; whichever lands first, the other rebases onto it.

Testing

• Tested locally
• Added/updated tests
• Added/updated docs

New unit coverage for the write half of the content proxy: what the store answers about a blob it holds, a blob only gathered, and a blob it has never seen. Verified end to end against a shim carrying apple/container-builder-shim#98: a build lands its blobs in the store, the image registers from the root descriptor, a rebuild moves only the changed layer, and a concurrent image rm no longer takes the manifest a build has just exported.

Integration suite: 397 passed. Unit suite: 772 passed. make fmt, make check clean.