Allow limited inbound ICMP to Nexus, add ICMP type/code filters to firewall rules #8194

FelixMcFelix · 2025-05-21T12:14:37Z

This PR adds more detail to firewall protocol filters. Rather than being specified purely as a string->enum mapping, each protocol filter is now a standard tagged enum like many other types in the Nexus API. Here, this allows us to express more control over ICMP traffic. For instance:

all ICMP ([{"type": "icmp"}]),
all ICMP echo ([{"type": "icmp", "value": { icmp_type: 0 }}, {"type": "icmp", "value": { icmp_type: 8 }}])
a subset of ICMP destination unreachable ({"type": "icmp", "value": { icmp_type: 0, code: "0-4" }}).

Building on this, Nexus now has a firewall entry to permit the receipt of e.g., Destination Unreachable messages. I've added a new system endpoint to control whether this is enabled or disabled.

As part of this, I've converted the CRDB representation from a fixed ENUM[] to a STRING(32)[]. There are a couple of reasons for this:

ICMPv6 will need a similar level of specificity,
Users will need to specify arbitrary (ranges) of L4 protocols, once OPTE is more permissive (Allow/support arbitrary L4 protocols in guest traffic opte#609).
Future protocols we give first-class support to might also require/want in-depth filters.

Should close #7998.

TODO:

Bump maghemite OPTE version on main.

That seems to be be the primary reason for the entire test suite having a Very Bad Time.

Thanks, CRDB.

FelixMcFelix · 2025-05-23T11:08:44Z

I've setup these bits on dublin this morning. After performing the schema update, everything seems happily functional, and posting true/false to the new system endpoint correctly enables/disables the ICMP allowance as expected. Console is broken when we go to the firewall rules tab, but that is to be expected given the OpenAPI changes.

On the actual MTU dscovery front, it looks as though both this PR and main are doing the correct thing?

If from the web console I perform a 'Reload (Override Cache)' we can kick off large enough TCP transfers to run into LSO. When we trace this through:

Dublin

BRM23230018 # dtrace -n 'xde_mc_tx:entry{ this->m = (mblk_t *)arg1; this->mss = this->m->b_datap->db_struioun.cksum.pad } xde_mc_tx:entry/this->m->b_datap->db_struioun.cksum.flags && this-
>mss != 0xBADD && this->mss/{printf("packet with MSS %d", this->m->b_datap->db_struioun.cksum.pad);}'
dtrace: description 'xde_mc_tx:entry' matched 2 probes
CPU     ID                    FUNCTION:NAME
111   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
  1   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
 69   5301                  xde_mc_tx:entry packet with MSS 1368
^C

BRM23230018 # dtrace -n 't4_eth_tx:entry{ this->m = (mblk_t *)arg1; this->mss = this->m->b_datap->db_struioun.cksum.pad } t4_eth_tx:entry/this->m->b_datap->db_struioun.cksum.flags && this-
>mss != 0xBADD && this->mss/{printf("packet with MSS %d", this->m->b_datap->db_struioun.cksum.pad);}'
dtrace: description 't4_eth_tx:entry' matched 2 probes
CPU     ID                    FUNCTION:NAME
  9   6360                  t4_eth_tx:entry packet with MSS 8928
 34   6360                  t4_eth_tx:entry packet with MSS 8928
117   6360                  t4_eth_tx:entry packet with MSS 8928
120   6360                  t4_eth_tx:entry packet with MSS 8928
110   6360                  t4_eth_tx:entry packet with MSS 8928
120   6360                  t4_eth_tx:entry packet with MSS 8928
 86   6360                  t4_eth_tx:entry packet with MSS 8928
 86   6360                  t4_eth_tx:entry packet with MSS 8928
 77   6360                  t4_eth_tx:entry packet with MSS 8928
 78   6360                  t4_eth_tx:entry packet with MSS 8928
 87   6360                  t4_eth_tx:entry packet with MSS 1368
  8   6360                  t4_eth_tx:entry packet with MSS 8928
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 78   6360                  t4_eth_tx:entry packet with MSS 8928
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 43   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 43   6360                  t4_eth_tx:entry packet with MSS 1368
 43   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 78   6360                  t4_eth_tx:entry packet with MSS 8928
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 69   6360                  t4_eth_tx:entry packet with MSS 1368
 43   6360                  t4_eth_tx:entry packet with MSS 1368
 43   6360                  t4_eth_tx:entry packet with MSS 1368
^C

The latter query is also catching underlay traffic, hence the 8928B entries. An MSS of 1368 matches what I'd maybe expect for a reduced MTU of 1402 bytes (resp. 1448 for a 1500B MTU). Do we have any hits on the new rules?

BRM23230018 # opteadm dump-layer -p opte0 firewall
Port opte0 - Layer firewall
======================================================================
Inbound Flows
----------------------------------------------------------------------
PROTO  SRC IP        SPORT  DST IP      DPORT  HITS  ACTION
TCP    172.20.17.94  50056  172.30.2.6  443    0     no-op
TCP    172.20.17.94  50062  172.30.2.6  443    0     no-op

Outbound Flows
----------------------------------------------------------------------
PROTO  SRC IP      SPORT  DST IP        DPORT  HITS  ACTION
TCP    172.30.2.6  443    172.20.17.94  50056  1     no-op
TCP    172.30.2.6  443    172.20.17.94  50062  1     no-op

Inbound Rules
----------------------------------------------------------------------
ID   PRI    HITS  PREDICATES                               ACTION
126  65534  2     inner.ip.proto=TCP                       "Stateful Allow"
                  inner.ulp.dst=80,=443

125  65534  0     inner.icmp.type=time exceeded            "Stateful Allow"
124  65534  0     inner.icmp.type=destination unreachable  "Stateful Allow"
                  inner.icmp.code=4

DEF  --     427   --                                       "deny"

Outbound Rules
----------------------------------------------------------------------
ID   PRI  HITS  PREDICATES  ACTION
DEF  --   2     --          "stateful allow"

Apparently not, although it's hard to tell given that the reconcile task is completely reinstating the rules periodically so we're losing those counters. Note that all the deny entries here are from my own ping https://recovery.sys.dublin.eng.oxide.computer command.

Dogfood

BRM44220011 # dtrace -n 'xde_mc_tx:entry{ this->m = (mblk_t *)arg1; this->mss = this->m->b_datap->db_struioun.cksum.pad } xde_mc_tx:entry/this->m->b_datap->db_struioun.cksum.flags && this-
>mss != 0xBADD && this->mss/{printf("packet with MSS %d", this->m->b_datap->db_struioun.cksum.pad);}'
dtrace: description 'xde_mc_tx:entry' matched 2 probes
CPU     ID                    FUNCTION:NAME
 41   4868                  xde_mc_tx:entry packet with MSS 1448
  9   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
127   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 68   4868                  xde_mc_tx:entry packet with MSS 1368
 83   4868                  xde_mc_tx:entry packet with MSS 1368
 83   4868                  xde_mc_tx:entry packet with MSS 1368
 83   4868                  xde_mc_tx:entry packet with MSS 1368
 83   4868                  xde_mc_tx:entry packet with MSS 1368
^C

BRM44220011 # dtrace -n 't4_eth_tx:entry{ this->m = (mblk_t *)arg1; this->mss = this->m->b_datap->db_struioun.cksum.pad } t4_eth_tx:entry/this->m->b_datap->db_struioun.cksum.flags && this->mss != 0xBADD && this->mss/{printf("packet with MSS %d", this->m->b_datap->db_struioun.cksum.pad);}'
dtrace: description 't4_eth_tx:entry' matched 2 probes
CPU     ID                    FUNCTION:NAME
 18   5927                  t4_eth_tx:entry packet with MSS 8928
 52   5927                  t4_eth_tx:entry packet with MSS 8928
 52   5927                  t4_eth_tx:entry packet with MSS 1448
113   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 76   5927                  t4_eth_tx:entry packet with MSS 8928
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1448
 44   5927                  t4_eth_tx:entry packet with MSS 1448
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 18   5927                  t4_eth_tx:entry packet with MSS 1368
 18   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
120   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
^C
 76   5927                  t4_eth_tx:entry packet with MSS 8928
 95   5927                  t4_eth_tx:entry packet with MSS 1368
 78   5927                  t4_eth_tx:entry packet with MSS 1368
 43   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 43   5927                  t4_eth_tx:entry packet with MSS 1368
 43   5927                  t4_eth_tx:entry packet with MSS 1368
 60   5927                  t4_eth_tx:entry packet with MSS 1368
 43   5927                  t4_eth_tx:entry packet with MSS 1368
 60   5927                  t4_eth_tx:entry packet with MSS 1368
 60   5927                  t4_eth_tx:entry packet with MSS 1368
  1   5927                  t4_eth_tx:entry packet with MSS 1368
113   5927                  t4_eth_tx:entry packet with MSS 1368
  1   5927                  t4_eth_tx:entry packet with MSS 1368
 69   5927                  t4_eth_tx:entry packet with MSS 1368
 86   5927                  t4_eth_tx:entry packet with MSS 1368
 52   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 42   5927                  t4_eth_tx:entry packet with MSS 1368
 42   5927                  t4_eth_tx:entry packet with MSS 1368
 42   5927                  t4_eth_tx:entry packet with MSS 1368
103   5927                  t4_eth_tx:entry packet with MSS 1368
 19   5927                  t4_eth_tx:entry packet with MSS 1368
 19   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368
 68   5927                  t4_eth_tx:entry packet with MSS 1368

There's a mix in there, but I assume we have something on the lab-side hitting the API periodically, since I see those 1448 entries passively. But dogfood appears to be discovering and using the reduced MTU just fine.

Fixing oxidecomputer/opte#748 may have been sufficient to get this working, and that PR has been pulled into main already. As a fun bonus, due to this work I can now run traceroute from the Nexus zone, so we might want to expand the scope of the allow rule across external DNS too. The NTP zones can't really benefit since they're behind a SNAT.

FelixMcFelix

I think this is good for review, I've pointed out some open design points I'm unsure on.

.github/buildomat/jobs/deploy.sh

FelixMcFelix · 2025-05-27T10:13:48Z

illumos-utils/src/opte/firewall_rules.rs

+                                // Port assignments are incompatible with non
+                                // TCP/UDP protocols.
+                                if matches!(
+                                    proto,
+                                    ProtoFilter::Tcp | ProtoFilter::Udp
+                                ) {
+                                    filters.set_ports(ports.clone());
+                                }


I'm surprised we haven't run into this one yet -- if someone were to install a rule on [TCP, ICMP] x [port 80, port 443] this would previously be installed as the two rules:

proto = TCP && (port = 80 || port = 443), [valid]

proto = ICMP && (port = 80 || port = 443). [invalid]

In OPTE, a port match will always return false unless it is applied to TCP/UDP traffic.

FelixMcFelix · 2025-05-27T10:17:53Z

nexus/db-fixed-data/src/vpc_firewall_rule.rs

+        targets: vec![VpcFirewallRuleTarget::Subnet(
+            super::vpc_subnet::NEXUS_VPC_SUBNET.name().clone(),
+        )],


Do we want to include any more services here?

FelixMcFelix · 2025-05-27T10:19:17Z

nexus/db-fixed-data/src/vpc_firewall_rule.rs

+            protocols: Some(vec![
+                VpcFirewallRuleProtocol::Icmp(Some(VpcFirewallIcmpFilter {
+                    // Type 3 -- Destination Unreachable
+                    icmp_type: 3,
+                    // Code 4 -- Fragmentation needed
+                    code: Some(4.into()),
+                })),
+                VpcFirewallRuleProtocol::Icmp(Some(VpcFirewallIcmpFilter {
+                    // Type 11 -- Time Exceeded
+                    icmp_type: 11,
+                    code: None,
+                })),
+            ]),


This gives us PMTUD (although it looks as though the TCP stack can figure it out in the absence of Fragmentation Needed messages?), and the ability to use traceroute from Nexus/DNS. Is the latter needed? Are there any other ICMP families which would be crucial?

@jclulow Do you have any thoughts here on any missing types/codes which might be needed (or on whether to extend the scope to, e.g., external DNS)?

From the Network sync today, the disposition seems to be a definite yes for extending this to external DNS (making sure we don't have any poor interactions with remote IP filters), and possibly allowing through ICMP redirect messages. I'll investigate the latter for a bit.

@rcgoodfellow I've extended the set here to include ICMP redirects and port unreachable messages (on top of the ones discussed initially), which I think is a good minimum set.

For both classes of zone, this ignores the nexus host IP filter for the same reason that we can't apply it to external DNS (i.e., it's very hard to know a priori which nodes can/will send us error messages). I've put a comment explaining this where this logic occurs today:

omicron/nexus/networking/src/firewall_rules.rs

Lines 320 to 325 in 63e1f70

// Note that inbound ICMP is not currently governed by this filter,

// as error-type ICMP messages can arrive from any host which a

// Nexus/DNS zone reaches out to *or* a gateway/router on the path to

// that destination.

(None, Some(allowed_ips)) => {

if allowlist_applies_to_firewall_rule(rule) {

nexus/external-api/src/lib.rs

nexus/src/app/vpc.rs

common/src/api/external/mod.rs

nexus/db-model/src/vpc_firewall_rule.rs

hawkw · 2025-05-28T18:45:51Z

Looks like you need to re-run integration_tests::unauthorized::test_unauthorized with EXPECTORATE=overwrite https://buildomat.eng.oxide.computer/wg/0/details/01JWBXVQWGZD1RRK5D3D1QEHT3/4jM5GiYbhdgOFpRJWfRHNBFflAiqzBgnrIo7ndUvW5ZbrRsJ/01JWBXW3KJD328SS2FYWZGXMK2#S7450

common/src/api/external/mod.rs

This change reduces the binary size of opteadm and brings in the deadlock fix for oxidecomputer/opte#758. Backports a few pieces from #8194 to support some minor rework of the ioctl interface. - [x] Swap back to maghemite main. - [x] Await creation of lab-helios image.

A few types (target & filter specifications, ranges) associated with firewall rules and routes rely on a custom string-based representation in CRDB. However, when deserializing them today we end up cycling through `PgValue`->`String`->`ParsedEnumType`, needlessly allocating and copying strings per element. This PR prevents us from doing so by generating these `impl`s for any type which implements the existing `DatabaseString` trait. While there are more types in `nexus-db-model` using `FromSql<Text, _>`, most seem to make use of a `String` internally regardless, so there's no immediate benefit (other than consistency) in changing them up at this time. Motivated by some of the discussion in #8194.

Redirect came out of our earlier discussion, port unreachable should be useful in the limit too.

A few types (target & filter specifications, ranges) associated with firewall rules and routes rely on a custom string-based representation in CRDB. However, when deserializing them today we end up cycling through `PgValue`->`String`->`ParsedEnumType`, needlessly allocating and copying strings per element. This PR prevents us from doing so by generating these `impl`s for any type which implements the existing `DatabaseString` trait. While there are more types in `nexus-db-model` using `FromSql<Text, _>`, most seem to make use of a `String` internally regardless, so there's no immediate benefit (other than consistency) in changing them up at this time. Motivated by some of the discussion in #8194.

FelixMcFelix · 2025-06-26T18:43:47Z

@hawkw Checking in on this, are we happy with this from an API/DB/Control-plane point of view? I'll fixup the merge conflict once dogfood's updated.

Other than that, this looks good to me, but I don't know if I'm the best person to answer the questions about whether there are other ICMP messages we might want to allow, so I'd hold off on merging until that's nailed down.

@taspelund / @rcgoodfellow , are we happy with this as a starting point for ICMP? I appreciate there's more we can do on giving operators precise control inclusive of e.g. ping (https://github.com/oxidecomputer/customer-support/issues/334), but then we need to worry more about the interaction with nexus IP allow lists etc.

hawkw · 2025-06-26T19:08:11Z

@hawkw Checking in on this, are we happy with this from an API/DB/Control-plane point of view? I'll fixup the merge conflict once dogfood's updated.

Yup, all good on my end!

hawkw

Looks good to me provided everything seems right from the networking perspective!

rcgoodfellow · 2025-06-26T22:23:16Z

@taspelund / @rcgoodfellow , are we happy with this as a starting point for ICMP? I appreciate there's more we can do on giving operators precise control inclusive of e.g. ping (https://github.com/oxidecomputer/customer-support/issues/334), but then we need to worry more about the interaction with nexus IP allow lists etc.

To make sure I understand this correctly, we're only letting DU through to Nexus? We must be careful not to introduce changes that would make Nexus more visible to scanners and the like.

FelixMcFelix · 2025-06-26T22:42:24Z

To make sure I understand this correctly, we're only letting DU through to Nexus? We must be careful not to introduce changes that would make Nexus more visible to scanners and the like.

The set here is:

DU (only of code Port Unreachable, Fragmentation Needed)
Redirect
Time Exceeded

This reverts commit 5a5308f. It looks as though something is gummed up in pkg, since 0.37.381 is the last available version there.

FelixMcFelix added 3 commits May 21, 2025 13:09

Initial plumbing.

f315451

More OpteHdl fixups.

8a33170

Parity.

38bd2ff

FelixMcFelix force-pushed the felixmcfelix/icmp-for-nexus branch from 5594fc0 to 38bd2ff Compare May 21, 2025 12:46

FelixMcFelix added 13 commits May 21, 2025 13:59

Merge branch 'main' into felixmcfelix/icmp-for-nexus

c5aec33

CRDB didn't like *that* migration.

283fc56

TODO: migration test.

3d13775

Merge branch 'main' into felixmcfelix/icmp-for-nexus

da4c157

Iterating on launch failures.

29db3dd

serde_json::from_str 🫠

01f0f4c

That seems to be be the primary reason for the entire test suite having a Very Bad Time.

Merge branch 'main' into felixmcfelix/icmp-for-nexus

3f8d07e

Simple migration query.

14804cb

An idempotent column rename

983e6b7

Thanks, CRDB.

Some control over ICMP allowance.

3d1a58e

Help the deploy job along, a little.

62a8b3d

Last test fixup.

564424f

fmt.

0c19f5f

FelixMcFelix mentioned this pull request May 23, 2025

Nexus ICMP paranoia breaks Path MTU Discovery #7998

Closed

FelixMcFelix added 2 commits May 23, 2025 20:20

Renaming, and some serde/integration tests.

32a4914

Self-review

6b9fa7a

FelixMcFelix commented May 27, 2025

View reviewed changes

FelixMcFelix marked this pull request as ready for review May 27, 2025 10:39

FelixMcFelix added this to the 16 milestone May 27, 2025

hawkw reviewed May 27, 2025

View reviewed changes

nexus/external-api/src/lib.rs Outdated Show resolved Hide resolved

nexus/src/app/vpc.rs Outdated Show resolved Hide resolved

common/src/api/external/mod.rs Outdated Show resolved Hide resolved

nexus/db-model/src/vpc_firewall_rule.rs Outdated Show resolved Hide resolved

Review feedback

5c7caed

hawkw reviewed May 28, 2025

View reviewed changes

common/src/api/external/mod.rs Outdated Show resolved Hide resolved

common/src/api/external/mod.rs Outdated Show resolved Hide resolved

common/src/api/external/mod.rs Outdated Show resolved Hide resolved

Review feedback, missed test fixup.

1fc7b2b

FelixMcFelix mentioned this pull request May 29, 2025

External API FromStrs could have api::external::Error as their error type #8238

Open

FelixMcFelix mentioned this pull request Jun 10, 2025

&str-based FromSql impls for firewall/route types #8303

Merged

FelixMcFelix added 4 commits June 13, 2025 12:04

Merge branch 'main' into felixmcfelix/icmp-for-nexus

ef9f701

Feedback: allow limited ICMP into external DNS zones.

21efccf

Extend ICMP allow to Redirect and Port Unreachable

63e1f70

Redirect came out of our earlier discussion, port unreachable should be useful in the limit too.

Bump Deploy job image version (pre-emptively).

2323892

FelixMcFelix added 4 commits June 16, 2025 17:27

Merge branch 'main' into felixmcfelix/icmp-for-nexus

fe82893

Update VpcFirewallRuleProtocol to benefit from alloc-less FromSql

fccba56

Merge branch 'main' into felixmcfelix/icmp-for-nexus

f676507

Remove OPTE_COMMIT override

0d2fa6e

hawkw approved these changes Jun 26, 2025

View reviewed changes

sudomateo mentioned this pull request Jun 26, 2025

Handle updated networking enums oxidecomputer/oxide.go#299

Open

FelixMcFelix added 3 commits June 27, 2025 09:06

Merge branch 'main' into felixmcfelix/icmp-for-nexus

ffaf268

Incredibly silly missed schema versions

6eec7a0

Bump OPTE to latest patch.

5a5308f

FelixMcFelix mentioned this pull request Jun 28, 2025

Bump OPTE to v0.37.383 oxidecomputer/maghemite#515

Merged

FelixMcFelix added 3 commits June 28, 2025 13:15

Revert "Bump OPTE to latest patch."

f652de1

This reverts commit 5a5308f. It looks as though something is gummed up in pkg, since 0.37.381 is the last available version there.

Bump Maghemite to main.

299aca0

Merge branch 'main' into felixmcfelix/icmp-for-nexus

2c64f3c

rcgoodfellow approved these changes Jul 1, 2025

View reviewed changes

FelixMcFelix merged commit 4a67564 into main Jul 1, 2025
19 checks passed

FelixMcFelix deleted the felixmcfelix/icmp-for-nexus branch July 1, 2025 15:28

askfongjojo mentioned this pull request Jul 10, 2025

Vpc firewall rule rendering needs update oxidecomputer/console#2851

Closed

	// Note that inbound ICMP is not currently governed by this filter,
	// as error-type ICMP messages can arrive from any host which a
	// Nexus/DNS zone reaches out to or a gateway/router on the path to
	// that destination.
	(None, Some(allowed_ips)) => {
	if allowlist_applies_to_firewall_rule(rule) {

Allow limited inbound ICMP to Nexus, add ICMP type/code filters to firewall rules #8194

Allow limited inbound ICMP to Nexus, add ICMP type/code filters to firewall rules #8194

Uh oh!

Conversation

FelixMcFelix commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FelixMcFelix commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dublin

Dogfood

Uh oh!

FelixMcFelix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

FelixMcFelix May 27, 2025

Choose a reason for hiding this comment

Uh oh!

FelixMcFelix May 27, 2025

Choose a reason for hiding this comment

Uh oh!

FelixMcFelix May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FelixMcFelix May 29, 2025

Choose a reason for hiding this comment

Uh oh!

FelixMcFelix Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

FelixMcFelix Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hawkw commented May 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FelixMcFelix commented Jun 26, 2025

Uh oh!

hawkw commented Jun 26, 2025

Uh oh!

hawkw left a comment

Choose a reason for hiding this comment

Uh oh!

rcgoodfellow commented Jun 26, 2025

Uh oh!

FelixMcFelix commented Jun 26, 2025

Uh oh!

Uh oh!

Uh oh!

FelixMcFelix commented May 21, 2025 •

edited

Loading

FelixMcFelix commented May 23, 2025 •

edited

Loading

FelixMcFelix May 27, 2025 •

edited

Loading