feat(kubernetes): launch isolated sandbox and supervisor pods

Signed-off-by: Drew Newberry <anewberry@nvidia.com>

Author: drew

.agents/skills/helm-dev-environment/SKILL.md

@@ -69,28 +69,12 @@ mise run helm:skaffold:dev
 mise run helm:skaffold:run
 ```
 
-**Supervisor sidecar topology** (build once and leave running):
-```bash
-mise run helm:skaffold:run:sidecar
-```
-
-**Supervisor sidecar topology with TLS/mTLS enabled** (build once and leave running):
-```bash
-mise run helm:skaffold:run:sidecar-mtls
-```
-
-Both commands build the `gateway` and `supervisor` images and deploy the OpenShell Helm
-chart. The sidecar profile renders an `openshell-network-init` init container for
-nftables setup and an `openshell-supervisor-network` runtime sidecar for proxying.
-Binary-aware policy mode runs that sidecar as UID 0 with `SYS_PTRACE` and
-`DAC_READ_SEARCH`; relaxed mode can run it as the configured proxy UID, which
-must be at least `1000` and distinct from the workload UID. The
-sidecar-mTLS profile reuses `ci/values-sidecar.yaml` and restores
-`server.disableTls=false` inline for Skaffold. The `pkiInitJob` hook (a pre-install
-Job that runs `openshell-gateway generate-certs`) generates mTLS secrets on first
-install. The default Skaffold values export gateway and Kubernetes-driver traces to
-the collector service installed by `helm:k3s:create`. Envoy Gateway opt-in; see the
-Optional Add-ons section below.
+Both commands build the gateway and trusted runtime image and deploy the
+OpenShell Helm chart. Kubernetes sandboxes always use the proxy-pod topology:
+a capability-free sandbox Pod behind an empty-egress NetworkPolicy and a
+separate capability-free supervisor Deployment. The default Skaffold values
+export gateway and Kubernetes-driver traces to the collector service installed
+by `helm:k3s:create`. Envoy Gateway is opt-in; see the Optional Add-ons section.
 
 The gateway Service uses ClusterIP. Access is via Envoy Gateway (port `8080`) or
 the unified local forwarding task:
@@ -102,8 +86,8 @@ mise run helm:k3s:forward
 The task forwards OTLP/gRPC to `http://127.0.0.1:4317` and the trace UI to
 `http://127.0.0.1:18888`. When Skaffold has deployed a Kubernetes gateway, it
 also forwards the gateway to `http://127.0.0.1:8090`; otherwise it continues
-with the collector ports only. A successful plaintext `helm:skaffold:run` or
-`helm:skaffold:run:sidecar` registers the gateway under the worktree-specific
+with the collector ports only. A successful plaintext `helm:skaffold:run`
+registers the gateway under the worktree-specific
 k3d cluster name and selects it as the active gateway. Keep the forwarding
 task running while using those endpoints.
 
@@ -134,8 +118,7 @@ create the Secret named `openshell-ha-pg` with a `uri` key, then run
 ### TLS behaviour
 
 `ci/values-skaffold.yaml` sets `server.disableTls: true`, so Skaffold-based deploys run
-plaintext by default. To test sidecar topology with TLS enabled, use
-`mise run helm:skaffold:run:sidecar-mtls`.
+plaintext by default. Remove that override to test gateway TLS/mTLS locally.
 
 | Mode | `server.disableTls` | Gateway scheme |
 |------|---------------------|----------------|
@@ -188,12 +171,6 @@ openshell sandbox list --gateway-endpoint https://localhost:8090
 mise run helm:skaffold:delete
 ```
 
-For a sidecar-profile deployment:
-
-```bash
-mise run helm:skaffold:delete:sidecar
-```
-
 ### Delete the cluster entirely
 
 ```bash
@@ -259,7 +236,7 @@ SPIFFE JWT-SVIDs for dynamic provider token grants:
 `openshell.local` and adds a `ClusterSPIFFEID` that maps sandbox pod
 annotations to `spiffe://openshell.local/openshell/sandbox/<sandbox-id>`.
 OpenShell mounts the SPIFFE CSI Workload API socket at
-`/spiffe-workload-api/spire-agent.sock` into sandbox pods for provider token
+`/spiffe-workload-api/spire-agent.sock` only into supervisor Pods for provider token
 grants. Supervisor-to-gateway authentication remains on the Kubernetes
 ServiceAccount bootstrap and gateway-minted sandbox JWT path; the selected
 Kubernetes compute driver validates the projected token before the gateway
@@ -320,7 +297,6 @@ for dependencies still declared in `Chart.yaml`.
 | `deploy/helm/openshell/ci/values-gateway.yaml` | Envoy Gateway GRPCRoute + Gateway overlay |
 | `deploy/helm/openshell/ci/values-high-availability.yaml` | HA test overlay (`replicaCount: 2` with external PostgreSQL Secret) |
 | `deploy/helm/openshell/ci/values-keycloak.yaml` | Keycloak OIDC overlay |
-| `deploy/helm/openshell/ci/values-sidecar.yaml` | Supervisor sidecar topology overlay for Kubernetes e2e/dev |
 | `deploy/helm/openshell/ci/values-spire.yaml` | SPIFFE/SPIRE provider token grant overlay |
 | `deploy/helm/openshell/ci/values-spire-stack.yaml` | SPIRE hardened chart values for local dev |
 | `deploy/helm/openshell/ci/values-tls-disabled.yaml` | Lint-only: TLS + auth disabled (reverse-proxy edge termination) |

.github/workflows/branch-e2e.yml

@@ -377,8 +377,8 @@ jobs:
             extra_helm_values: ""
           - agent_sandbox_api: v1beta1
             agent_sandbox_version: v0.5.0
-            topology: sidecar
-            extra_helm_values: deploy/helm/openshell/ci/values-sidecar.yaml
+            topology: proxy-pod
+            extra_helm_values: deploy/helm/openshell/ci/values-proxy-pod.yaml
     permissions:
       actions: read
       contents: read

AGENTS.md

@@ -38,7 +38,8 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
 | `crates/openshell-conformance/` | CLI conformance library | Reusable driver-agnostic scenarios and command runner |
 | `crates/openshell-conformance-cli/` | Conformance CLI | Distributable `list` and `run` entrypoint for gateway conformance |
 | `crates/openshell-server/` | Gateway server | Control-plane API, sandbox lifecycle, auth boundary |
-| `crates/openshell-sandbox/` | Sandbox runtime | Container supervision, policy-enforced egress routing |
+| `crates/openshell-sandbox/` | Sandbox runtime | Capability-free workload launcher, process identity, and seccomp-mediated I/O |
+| `crates/openshell-supervisor/` | Supervisor runtime | Gateway session, policy evaluation, credentials, and upstream networking |
 | `crates/openshell-binary-identity/` | Binary identity | Shared trusted procfs executable identity resolution for isolation backends |
 | `crates/openshell-isolation-interface/` | Isolation backend interface | RFC 0012 `IsolationBackend` trait + types; the supervisor-facing runtime contract for the boundary |
 | `crates/openshell-policy/` | Policy engine | Filesystem, network, process, and inference constraints |
@@ -59,7 +60,6 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
 | `crates/openshell-driver-db-credstore/` | Database credential driver | In-process `CredentialDriver` backend for gateway database credential storage |
 | `crates/openshell-driver-kubernetes/` | Kubernetes compute driver | In-process `ComputeDriver` backend for K8s sandbox pods |
 | `crates/openshell-driver-docker/` | Docker compute driver | In-process `ComputeDriver` backend for local Docker sandbox containers |
-| `crates/openshell-driver-mxc/` | MXC compute driver | Windows in-process `ComputeDriver` backend for MXC sandbox execution |
 | `crates/openshell-driver-podman/` | Podman compute driver | In-process `ComputeDriver` backend for local Podman sandbox containers |
 | `crates/openshell-driver-vm/` | VM compute driver | Standalone libkrun-backed `ComputeDriver` subprocess (embeds its own rootfs + runtime) |
 | `crates/openshell-driver-mxc/` | Microsoft MXC compute driver | In-process Windows AppContainer and isolation-session compute backend |

Cargo.lock

@@ -12,7 +12,6 @@ pub const OPT_ROOT: &str = "/opt/openshell";
 pub const ETC_ROOT: &str = "/etc/openshell";
 pub const TLS_ROOT: &str = "/etc/openshell-tls";
 pub const RUN_ROOT: &str = "/run/openshell";
-pub const SIDECAR_RUN_ROOT: &str = "/run/openshell-sidecar";
 pub const NETNS_MOUNT_ROOT: &str = "/run/netns";
 pub const NETNS_IPROUTE2_ROOT: &str = "/var/run/netns";
 
@@ -34,7 +33,6 @@ pub const CONTROL_ROOTS: &[&str] = &[
     ETC_ROOT,
     TLS_ROOT,
     RUN_ROOT,
-    SIDECAR_RUN_ROOT,
     NETNS_MOUNT_ROOT,
     // The supervisor currently uses the conventional iproute2 spelling.
     NETNS_IPROUTE2_ROOT,
@@ -52,9 +50,6 @@ pub const CONTAINER_POLICY_PATH: &str = "/etc/openshell/policy.yaml";
 pub const POLICY_ADVISOR_SKILL_PATH: &str = "/etc/openshell/skills/policy_advisor.md";
 
 pub const SSH_SOCKET_PATH: &str = "/run/openshell/ssh.sock";
-pub const SIDECAR_CONTROL_SOCKET: &str = "/run/openshell-sidecar/control.sock";
-pub const SIDECAR_TLS_DIR: &str = "/etc/openshell-tls/proxy";
-pub const SIDECAR_CLIENT_TLS_DIR: &str = "/etc/openshell-tls/proxy/client";
 pub const CLIENT_TLS_DIR: &str = "/etc/openshell-tls/client";
 pub const SUPERVISOR_CA_CERT_PATH: &str = "/etc/openshell-tls/openshell-ca.pem";
 pub const SUPERVISOR_CA_BUNDLE_PATH: &str = "/etc/openshell-tls/ca-bundle.pem";
@@ -92,9 +87,6 @@ mod tests {
             CONTAINER_POLICY_PATH,
             POLICY_ADVISOR_SKILL_PATH,
             SSH_SOCKET_PATH,
-            SIDECAR_CONTROL_SOCKET,
-            SIDECAR_TLS_DIR,
-            SIDECAR_CLIENT_TLS_DIR,
             CLIENT_TLS_DIR,
             SUPERVISOR_CA_CERT_PATH,
             SUPERVISOR_CA_BUNDLE_PATH,

crates/openshell-core/src/container_paths.rs

@@ -296,7 +296,6 @@ mod tests {
             "/etc/openshell-tls",
             "/run",
             "/run/openshell/cache",
-            "/run/openshell-sidecar/control.sock",
             "/run/netns/project",
             "/var/run/netns/project",
         ] {

crates/openshell-core/src/driver_mounts.rs

@@ -4,7 +4,7 @@
 //! Shared state controlling agent-driven policy proposals.
 //!
 //! Initialised once during sandbox start from the `agent_policy_proposals_enabled`
-//! setting and updated by the policy poll loop or authoritative sidecar control
+//! setting and updated by the policy poll loop or authoritative supervisor
 //! when the setting changes. Read by the `policy.local` route handler and by
 //! the skills installer to gate the agent-controlled mutation surface.
 
@@ -15,7 +15,7 @@ use std::sync::atomic::{AtomicBool, Ordering};
 ///
 /// Clones point at the same atomic value, so the sandbox orchestrator can pass
 /// this into the process and network supervisors and then update it from the
-/// settings poll loop or sidecar control.
+/// settings poll loop or supervisor control.
 #[derive(Clone, Debug)]
 pub struct AgentProposals {
     enabled: Arc<AtomicBool>,

crates/openshell-core/src/proposals.rs

@@ -165,8 +165,8 @@ impl ProviderCredentialState {
     /// Build a static provider state from an already-prepared child
     /// environment snapshot.
     ///
-    /// Kubernetes sidecar topology uses this in the process-only supervisor:
-    /// the network sidecar owns provider credential resolvers and sends the
+    /// The Kubernetes proxy-pod topology uses this in the sandbox process:
+    /// the supervisor owns provider credential resolvers and sends the
     /// workload-facing env map over a local control channel. The process leaf
     /// must inject that map into child processes without re-placeholderizing it
     /// or holding the gateway-side resolver material.

crates/openshell-core/src/provider_credentials.rs

@@ -130,20 +130,13 @@ impl MainProcessConfig {
 /// Deployment-controlled telemetry toggle propagated to the sandbox supervisor.
 pub const TELEMETRY_ENABLED: &str = "OPENSHELL_TELEMETRY_ENABLED";
 
-/// Supervisor pod/runtime topology. Kubernetes sidecar mode sets this to
-/// `"sidecar"`; the default combined supervisor path omits it.
-pub const SUPERVISOR_TOPOLOGY: &str = "OPENSHELL_SUPERVISOR_TOPOLOGY";
-
 /// The isolation backend admitted by the deployment configuration (RFC 0012).
 ///
 /// Delivered on a channel separate from the topology descriptor so descriptor
 /// verification against the admitted backend is not self-referential. Required
 /// whenever a topology descriptor is supplied.
 pub const ADMITTED_ISOLATION_BACKEND: &str = "OPENSHELL_ADMITTED_ISOLATION_BACKEND";
 
-/// Network enforcement backend selected by the compute driver.
-pub const NETWORK_ENFORCEMENT_MODE: &str = "OPENSHELL_NETWORK_ENFORCEMENT_MODE";
-
 /// Comma-separated runtime networking capabilities supplied by the compute
 /// driver. Capabilities describe substrate the shared supervisor may activate;
 /// they never move policy evaluation into the driver.
@@ -152,20 +145,6 @@ pub const NETWORK_RUNTIME_CAPABILITIES: &str = "OPENSHELL_NETWORK_RUNTIME_CAPABI
 /// Driver capability for policy-gated DNS and transparent TCP interception.
 pub const POLICY_DNS_TRANSPARENT_TCP_CAPABILITY: &str = "policy-dns-transparent-tcp";
 
-/// Whether network policy evaluation must bind requests to the peer binary.
-///
-/// The default when unset is `"required"`. Kubernetes sidecar experiments may
-/// set this to `"relaxed"` to enforce endpoint and L7 policy without per-binary
-/// `/proc` identity binding.
-pub const NETWORK_BINARY_IDENTITY: &str = "OPENSHELL_NETWORK_BINARY_IDENTITY";
-
-/// Unix socket used by Kubernetes sidecar topology for local coordination.
-///
-/// The network sidecar owns gateway credentials and serves policy/provider
-/// state over this socket instead of exposing gateway credentials to the agent
-/// container.
-pub const SIDECAR_CONTROL_SOCKET: &str = "OPENSHELL_SIDECAR_CONTROL_SOCKET";
-
 /// Optional TLS server name override used when connecting to the gateway.
 pub const GATEWAY_TLS_SERVER_NAME: &str = "OPENSHELL_GATEWAY_TLS_SERVER_NAME";
 

crates/openshell-core/src/sandbox_env.rs

@@ -16,6 +16,7 @@ path = "src/main.rs"
 
 [dependencies]
 openshell-core = { path = "../openshell-core", default-features = false }
+openshell-isolation-interface = { path = "../openshell-isolation-interface" }
 openshell-otel = { path = "../openshell-otel" }
 openshell-policy = { path = "../openshell-policy" }
 
@@ -37,6 +38,8 @@ tracing-subscriber = { workspace = true }
 thiserror = { workspace = true }
 miette = { workspace = true }
 notify = "8"
+rand = { workspace = true }
+rcgen = { workspace = true }
 
 [dev-dependencies]
 openshell-otel-test-support = { path = "../openshell-otel-test-support" }