Repository navigation
relay: add static-IP egress relay service (TLS 1.3 + yamux) - #21
Merged
Merged
Conversation
Adds `vprox relay`, the server side of the macOS userspace egress dataplane. A macOS host's egress proxy opens one TLS 1.3 session per VM, authenticates with VPROX_PASSWORD in a session hello, and multiplexes the VM's TCP and UDP flows over yamux; the relay dials each destination bound to the granted static IP. - lib/relayproto: protocol v1 (hello, stream header with proto, refusal status codes, UDP framing) with hex fixtures shared with FastActions/fa. - lib/relay: source allow-list, bearer auth, static-IP ownership check, SSRF destination policy (RFC1918, loopback, link-local, CGNAT, multicast, 240/4, own addresses, extra CIDRs/ports), per-session and per-source limits, idle timeouts, Prometheus metrics, SIGTERM drain via GoAway. - cmd/relay: flags, /metrics endpoint, --print-spki-pin for the client pin. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
End-to-end numbers with this relay deployed on |
Dial timing is exported through vprox_relay_dial_seconds, so carrying it in every stream reply duplicates data the host proxy never uses. Trim the reply to the status byte and fold the shared post-dial bookkeeping of the TCP and UDP paths into session.dialed. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Mirror of the fa egressproxy/relayproto change: the hello carries HMAC-SHA256(password, "client" || TLS-exporter binding) and a successful reply carries the matching "server" proof, so the host proxy can trust the self-signed relay without an SPKI pin and the password never crosses the wire. Drops --print-spki-pin. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A fixed SO_RCVBUF disables receive autotuning, so a single download stream is capped at buffer/RTT: 4 MiB at ~100 ms RTT measured ~40 MB/s against ~205 MB/s with autotuning on the same host and path. tcp_rmem/tcp_wmem already bound the autotuned buffers. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
shreyas-blacksmith
marked this pull request as ready for review
September 28, 2026 14:40
- a hello that arrives while draining is refused with HelloUnavailable (the client retries the next relay) instead of HelloSessionLimit - the session's yamux mux is published under a mutex and a GoAway requested before the mux exists is delivered once it is, so a session admitted in the hello window still learns about the drain - one Server serves one listener; --listen is no longer repeatable and the unused Server.ln field is gone Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9f7a274. Configure here.
…ut too Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
bruce-y
approved these changes
Sep 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Server side of the macOS static-IP userspace egress dataplane (FastActions/fa
agent/docs/macos-egress-proxy-design.md§5, PR 1 of the §8 split). Replacesroute-to→ gif/ESP → IPIP for Mac static IPs with:Port of the
benchmarks/vzbench/relayprototype; the prototype's bench-only mTLS andKindMuxbyte are replaced by a versioned session hello with bearer auth (phase 1, parity with/connect-ipip). Client-cert mTLS is the phase-2 follow-up.Protocol v1 (
lib/relayproto) — new, not in the prototypeAll big-endian. A byte-identical copy with the same hex fixtures ships in FastActions/fa (
agent/egressproxy/relayproto); the fixtures are the cross-module compatibility contract since vprox can't import fa.lib/relay— §5 hardening as requirements--allow-source, required): checked on the raw accept, before the TLS handshake;vprox_relay_source_rejected_total.--tls-cert/--tls-keyoverride).vprox relay --print-spki-pinprints the base64 SHA-256 SPKI the host proxy pins instead ofInsecureSkipVerify.subtle.ConstantTimeCompareagainstVPROX_PASSWORD. Hello must arrive within 10s of accept.static_ipmust be an address of the relay host (interface scan, 10s TTL) or the hello is refusedbad_static_ip; the reply'segress_ipis what outbound sockets bind to (Dialer.LocalAddr/DialUDP(laddr)).Policy.Check, runs before every dial — the FORWARD chain doesn't apply to a local process):0/8, RFC1918,100.64/10,127/8,169.254/16,224/4,240/4, IPv6, port 0, every address currently on the host's interfaces (so tunnel/vprox0/new static IPs are covered without restart), plus--deny-cidr(pass the WG block + internal CIDR) and--deny-port. Own-address discovery failing never disables the static list.session_limit), streams 4096 per session (limit), UDP 1024 per session (udp_limit), yamux window 256 KiB, accept backlog 256, dial timeout 5s, TCP idle 6h, UDP idle 60s. Refused streams don't tear down the session.GoAway(host proxy seesErrRemoteGoAwayand reconnects to the restarted relay) → in-flight streams get--drain-timeout(15s) to finish → cut.vprox_relay_draininggauge.--metrics-addr, default127.0.0.1:9444): sessions open/total, hello refusals by status, streams open/total by proto, refusals by status, bytes by dir+static_ip, dial latency histogram by proto, Go/process collectors.Wiring not in this repo
The systemd unit, UFW rule (relay port restricted to
ipip_client_cidrs, same as/connect-ipip), and Alloy scrape target land in FastActions/fainfra/ansible/setup_vprox_server.yaml(PR E of the split), since that playbook is what clones this repo.Tests
go test ./...(relay + relayproto): protocol fixtures and round-trips; policy table; hello refusals (bad password / foreign IP / garbage); TCP echo >1 MiB with half-close; dial-failed vs policy-denied on a live session; UDP echo incl. zero-length datagram; UDP idle close; stream and UDP limits; per-source session limit; source allow-list drop; drain = GoAway + in-flight stream completes + exit once drained. CI on this repo is build-only.§6 acceptance (design doc)
Pending. Items 1–14 need the full stack (this relay on staging vprox
.247, the fa egress-proxy, supervisor andvm/vzproxy mode on.16). Will be run once fa PR D exists and the numbers posted here. Prototype reference (same host pair): P1 11.9s (~9.1 Gbit/s) vs OLD 62.7s (~1.7 Gbit/s), P2 10.3s vs 10.6s, P4 8.3s vs 8.2s.Link to Devin session: https://app.devin.ai/sessions/3732a5e335164324a71ec27bf19595aa
Open in Devin Desktop: https://app.devin.ai/desktop/session/3732a5e335164324a71ec27bf19595aa?variant=devin
Requested by: @shreyas-blacksmith
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.