Reach a builder over ssh or tcp instead of the local vsock - #2190
Closed
MayCXC wants to merge 1 commit into
Closed
Conversation
The build command's builder is hardwired to this machine's: the only acquisition dials the local builder's vsock and starts one when the dial fails. A second machine with disk and cycles to spare could hold the builder, but nothing could reach it. buildkit reaches remote daemons through BUILDKIT_HOST, whose ssh helper executes buildctl dial-stdio on the peer and speaks over the command's standard streams, the ssh channel carrying authentication and encryption with no TLS in the path; tcp reaches a listener directly, where mTLS is the hardening buildkitd documents. https://github.com/moby/buildkit/blob/master/client/connhelper/ssh/ssh.go builder dial-stdio is that command for this CLI: it proxies the standard streams to the builder's vsock, so a peer needs sshd and this CLI and nothing else. The build command resolves the address from --builder, then the CONTAINER_BUILD_REMOTE environment variable, then build.remote in the configuration, the way buildctl's --addr rides over BUILDKIT_HOST, in the same address grammar: ssh addresses spawn ssh running the peer's dial-stdio with a socketpair end as both of the child's streams, tcp addresses connect directly, and the connection feeds the same Builder the vsock path feeds. A remote builder's lifecycle is its machine's, so the local start and retry paths stay local.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of Change
Motivation and Context
Closes #2189.
The build command's builder is hardwired to this machine's: the only acquisition dials the local builder's vsock and starts one when the dial fails. A second machine with disk and cycles to spare could hold the builder, but nothing could reach it.
buildkit reaches remote daemons through
BUILDKIT_HOST, whose ssh helper executesbuildctl dial-stdioon the peer and speaks over the command's standard streams, the ssh channel carrying authentication and encryption with no TLS in the path; tcp reaches a listener directly, where mTLS is the hardening buildkitd documents. https://github.com/moby/buildkit/blob/master/client/connhelper/ssh/ssh.gobuilder dial-stdiois that command for this CLI: it proxies the standard streams to the builder's vsock, so a peer needs sshd and this CLI and nothing else. The build command resolves the address from--builder, then theCONTAINER_BUILD_REMOTEenvironment variable, thenbuild.remotein the configuration, the waybuildctl's--addrrides overBUILDKIT_HOST, in the same address grammar: ssh addresses spawn ssh running the peer'sdial-stdiowith a socketpair end as both of the child's streams, tcp addresses connect directly, and the connection feeds the sameBuilderthe vsock path feeds.A remote builder's lifecycle is its machine's, so the local start and retry paths stay local.
Complementary to #2188, which publishes the local daemon's socket for a client on the same machine; this is the half that crosses machines and speaks this CLI's own protocol rather than buildkit's.
Testing
A build driven against a second Mac over
ssh://runs on that machine's builder, with the context served from the invoking machine as usual; atcp://address reaches a listener directly. With no address configured, the local vsock path is unchanged, including its start-on-failure behaviour.Integration suite: 397 passed. Unit suite: 772 passed.
make fmt,make checkclean.