Carry buildkit's environment, daemon flags, and socket to the builder - #2188
Closed
MayCXC wants to merge 5 commits into
Closed
Carry buildkit's environment, daemon flags, and socket to the builder#2188MayCXC wants to merge 5 commits into
MayCXC wants to merge 5 commits into
Conversation
The builder wraps buildkit and exposed almost none of it: two color variables crossed from the caller's environment into the container, the daemon's flags were unreachable, and its socket never left the VM. Every BUILDKIT_ variable in the caller's environment now rides into the builder, so BUILDKIT_HOST points the shim's client at a daemon of the operator's choosing, the BUILDKIT_TLS set carries that address's credentials, and whatever buildkit documents next needs no new plumbing; NO_COLOR rides along as the conventional outlier the color handling already honored, and a change to any of them recreates the builder the way the color variables always did. --buildkitd-flags takes one string of daemon flags, buildx's own contract for the same name, handed to the shim after -- where it passes them to buildkitd verbatim. --publish-buildkit-socket puts the daemon's socket on the host through the runtime's socket publishing, where buildctl and buildx dial it directly, the daemon-socket convention Docker Desktop, colima, and podman machine follow on macOS; remote machines reach the same socket through buildkit's own ssh scheme.
One builder start carries a BUILDKIT_ environment variable, daemon flags after --buildkitd-flags, and --publish-buildkit-socket. The test reads each where its consumer does: the shim's environment at /proc/1/environ, buildkitd's argv for the passed flags, and the host filesystem for the published socket, then runs a build with all three applied.
The builder's process environment mixes the image's own variables with the ones start injects; moby/buildkit's image carries BUILDKIT_SETUP_CGROUPV2_ROOT, so a start that reads the whole environment back sees variables it never injected and recreates a builder that already matches its target, and concurrent builds then race one another's delete and create cycles. The injected set rides a label on the container record, stamped at create and compared on the next start, the way docker compose stamps com.docker.compose.config-hash on the services it manages.
A start whose inputs match the running builder returns it untouched; a start whose managed environment differs replaces it. A file written into the builder's filesystem tells the cases apart: it survives a kept builder and is absent from a recreated one.
A builder replaced mid-flight interrupts every build sharing it, so the replacement names its reason: the drifted settings print to stderr beside the progress output before the stop.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of Change
Motivation and Context
Closes #2187. Closes #2186.
Exposing buildkit. The builder wraps buildkit and exposed almost none of it: two color variables crossed from the caller's environment into the container, the daemon's flags were unreachable, and its socket never left the VM.
Every
BUILDKIT_variable in the caller's environment now rides into the builder, soBUILDKIT_HOSTpoints the shim's client at a daemon of the operator's choosing, theBUILDKIT_TLSset carries that address's credentials, and whatever buildkit documents next needs no new plumbing;NO_COLORrides along as the conventional outlier the color handling already honored, and a change to any of them recreates the builder the way the color variables always did.--buildkitd-flagstakes one string of daemon flags, buildx's own contract for the same name, handed to the shim after--where it passes them to buildkitd verbatim.--publish-buildkit-socketputs the daemon's socket on the host through the runtime's socket publishing, wherebuildctlandbuildxdial it directly, the daemon-socket convention Docker Desktop, colima, and podman machine follow on macOS; remote machines reach the same socket through buildkit's own ssh scheme.The drift comparison. The builder's process environment mixes the image's own variables with the ones start injects;
moby/buildkit's image carriesBUILDKIT_SETUP_CGROUPV2_ROOT, so a start that reads the whole environment back sees variables it never injected and recreates a builder that already matches its target, and concurrent builds then race one another's delete and create cycles. The injected set rides a label on the container record, stamped at create and compared on the next start, the way docker compose stampscom.docker.compose.config-hashon the services it manages.A builder replaced mid-flight interrupts every build sharing it, so the replacement names its reason: the drifted settings print to stderr beside the progress output before the stop.
The concurrent delete-and-create racing above is the same window #2174 serializes; the two compose, and this removes most of the occasions for the race rather than the race itself.
Testing
One builder start carries a
BUILDKIT_environment variable, daemon flags after--buildkitd-flags, and--publish-buildkit-socket. The test reads each where its consumer does: the shim's environment at/proc/1/environ, buildkitd's argv for the passed flags, and the host filesystem for the published socket, then runs a build with all three applied.A second case covers the drift decision: a start whose inputs match the running builder returns it untouched; a start whose managed environment differs replaces it. A file written into the builder's filesystem tells the cases apart, surviving a kept builder and absent from a recreated one.
Integration suite: 397 passed. Unit suite: 772 passed.
make fmt,make checkclean.