Repository navigation
Stop flooding other guests' unicast frames to isolated TAPs - #563
Draft
sjmiller609 wants to merge 1 commit into
Draft
sjmiller609 wants to merge 1 commit into
sjmiller609 wants to merge 1 commit into
Conversation
Port isolation blocks guest-to-guest forwarding, but frames for a MAC the bridge has no entry for are still flooded to every port, so a guest in promiscuous mode can capture other guests' inbound traffic. Pin each guest MAC to its TAP as a static FDB entry and turn off unknown-unicast flooding and learning on isolated ports.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port isolation stops guest-to-guest forwarding, but when the bridge has no FDB entry for a destination MAC it still floods the frame to every port. A guest that puts its NIC in promiscuous mode can therefore capture other guests' inbound unicast traffic on the same host.
For every TAP on an isolated network,
createTAPDevicenow:bridge fdb replace <mac> dev <tap> master static), so delivery to the guest never depends on flooding;Notes:
NUD_NOARP(static), notNUD_PERMANENT.NUD_PERMANENTmakes the bridge treat the MAC as its own (local) rather than an address behind the port.CreateAllocationandRecreateAllocationpass the allocated MAC. Forked restores reconfigure the guest to the allocated MAC before the fork call returns. The kernel removes a port's FDB entries when the TAP is deleted, so no cleanup path is needed.Testing
Unit test
TestHardenIsolatedPortStopsUnicastLeak(root only, skips otherwise) builds a bridge in a throwaway network namespace with two hardened ports and one port left at kernel defaults. It transmits frames through the bridge device, as the host stack does for guest-bound traffic, and checks:static;Run as root: passes 5/5. Mutation check: turning flooding back on, turning learning back on, or using
NUD_PERMANENTeach fails the test.go test ./lib/network/andGOOS=darwin go vet ./lib/network/pass.Before/after on a dev hypeman host
Both revisions (
before= merge basea096e53,after= this branch) ran ondev-yul-hypeman-0(AMD EPYC 9334, XFS on md RAID) in a dedicated network namespace with their own data dirs, bridge, and subnet. Only one revision ran at a time, alternating by round. The host's staging Hypeman service was not touched. Image:onkernel/chromium-headless-private:869aaa2, Firecracker, file-backed snapshots. Forks follow the production shape:POST /snapshots/{id}/forkwithtarget_state=Running, apply fork identity, CDP/json/version, confirm identity.Leak repro. One guest sniffs in promiscuous mode while the host sends 20 unicast frames to a MAC that has no FDB entry:
The same result reproduced in a second run.
Latency. Every operation passed after one harness fix: the first run used a 10s per-attempt identity timeout, and identity calls stalled longer than that under burst load on both revisions.
Hypeman-side time (fork, create, restore) is unchanged. Most of the end-to-end gap is guest-side identity apply, and it comes from each revision having its own seed snapshot. Crossing binaries and templates (40 single forks per cell):
The template accounts for ~65–100ms. What's left that's attributable to this change is ~25ms average, which is within the round-to-round spread of this harness (±20–40ms). Tail stalls of ~14s under 25-way bursts showed up on both revisions.
Not done: staging QA through CI.