refactor: streamline Android display streaming around shared video

README.md

@@ -39,7 +39,7 @@ view inside the editor.
 
 ## Features
 
-- Supports streaming both iOS simulators and Android emulators
+- Supports native H.264 streaming for both iOS simulators and Android emulators
 - Full simulator control & inspection using private iOS accessibility APIs and Android UIAutomator - available using `simdeck` CLI
 - Real-time screen `describe` command using accessibility view tree - available in token-efficient format for agents
 - Profiling built-in: CPU, memory, disk writes, network throughput, hang signals, and stack sampling
@@ -89,6 +89,9 @@ Normal service restarts preserve that token so paired clients stay connected.
 Use `simdeck service reset` only when you want to rotate the service token and
 restart the LaunchAgent.
 The service uses port 4310 unless you pass `-p` or `--port`.
+SimDeck-owned Android emulator boots use host GPU rendering by default; use
+`simdeck service restart --android-gpu auto` or
+`--android-gpu swiftshader_indirect` only as a machine-specific fallback.
 Use `simdeck service kill` when you want to stop every SimDeck service process,
 including services started from another checkout or installed binary.
 
@@ -130,7 +133,6 @@ simdeck pasteboard get
 simdeck screenshot --output screen.png
 simdeck screenshot --with-bezel --output screen-bezel.png
 simdeck record --seconds 5 --output screen-recording.mp4
-simdeck stream --frames 120 > stream.h264
 simdeck describe
 simdeck describe --format agent --max-depth 4
 simdeck describe --format agent --max-depth 4 --interactive

docs/api/health.md

@@ -21,6 +21,7 @@ Example:
   "serverKind": "launchAgent",
   "timestamp": 1714094761.234,
   "videoCodec": "auto",
+  "androidGpu": "host",
   "lowLatency": false,
   "realtimeStream": true,
   "localStreamFps": 60,
@@ -46,6 +47,7 @@ Important fields:
 | `httpPort`      | Port serving UI and API                                 |
 | `serverKind`    | `launchAgent` or `standalone`                           |
 | `videoCodec`    | Requested codec mode: `auto`, `hardware`, or `software` |
+| `androidGpu`    | Android emulator renderer mode for SimDeck-owned boots  |
 | `streamQuality` | Active stream profile and limits                        |
 | `webRtc`        | ICE settings the browser should use                     |
 
@@ -90,7 +92,7 @@ Content-Type: application/json
   "clientId": "browser-ABC",
   "kind": "viewport",
   "udid": "9D7E5BB7-...",
-  "codec": "h264",
+  "codec": "video/H264/103",
   "decodedFps": 59.7,
   "droppedFps": 0.0,
   "latestRenderMs": 6.2

docs/api/rest.md

@@ -165,15 +165,14 @@ Performance query parameters:
 
 ## Live video
 
-| Method | Path                                  | Purpose                                |
-| ------ | ------------------------------------- | -------------------------------------- |
-| `POST` | `/api/simulators/{udid}/webrtc/offer` | WebRTC offer/answer stream setup       |
-| `GET`  | `/api/simulators/{udid}/h264`         | H.264 WebSocket fallback               |
-| `GET`  | `/api/simulators/{udid}/input`        | Input WebSocket for fallback transport |
-| `GET`  | `/api/simulators/{udid}/control`      | Alias for input control WebSocket      |
-| `POST` | `/api/simulators/{udid}/refresh`      | Request a fresh frame or keyframe      |
-
-For normal clients, copy the browser behavior instead of hand-coding a raw decoder. The UI supports WebRTC first and H.264 WebSocket fallback.
+| Method | Path                                  | Purpose                               |
+| ------ | ------------------------------------- | ------------------------------------- |
+| `POST` | `/api/simulators/{udid}/webrtc/offer` | WebRTC offer/answer stream setup      |
+| `GET`  | `/api/simulators/{udid}/input`        | Input WebSocket fallback for controls |
+| `GET`  | `/api/simulators/{udid}/control`      | Alias for input control WebSocket     |
+| `POST` | `/api/simulators/{udid}/refresh`      | Request a fresh frame or keyframe     |
+
+For normal clients, copy the browser behavior instead of hand-coding a raw decoder. The UI uses the WebRTC offer endpoint for live video. Android emulator IDs use the same WebRTC endpoint; their H.264 frames are produced from the emulator `-share-vid` display surface, not screenshot polling.
 
 Minimal WebRTC request:
 

docs/cli/commands.md

@@ -207,12 +207,6 @@ simdeck logs --seconds 30 --limit 200
 simdeck chrome-profile
 ```
 
-Diagnostic iOS H.264 stream:
-
-```sh
-simdeck stream --frames 120 > stream.h264
-```
-
 ## Studio and providers
 
 For hosted Studio workflows:

docs/cli/flags.md

@@ -26,18 +26,19 @@ When `service restart` is run without `--port`, it preserves the installed
 LaunchAgent port or the current singleton service port before falling back to
 `4310`.
 
-| Flag                         | Default        | Notes                                                                             |
-| ---------------------------- | -------------- | --------------------------------------------------------------------------------- |
-| `--port <port>` / `-p`       | `4310`         | HTTP port; `service restart` preserves the existing service port when omitted     |
-| `--bind <ip>`                | `127.0.0.1`    | Use `0.0.0.0` or `::` for LAN access                                              |
-| `--advertise-host <host>`    | detected       | Host printed for remote browsers                                                  |
-| `--client-root <path>`       | bundled client | Static client directory                                                           |
-| `--video-codec <mode>`       | `auto`         | `auto`, `hardware`, or `software`                                                 |
-| `--stream-quality <profile>` | `full`         | `full`, `balanced`, `economy`, `low`, `tiny`, `ci-software`, and related profiles |
-| `--local-stream-fps <fps>`   | `60`           | Local stream frame target                                                         |
-| `--low-latency`              | off            | Conservative software H.264 profile                                               |
-| `--open`                     | off            | Open the browser after starting the service                                       |
-| `--autostart` / `-a`         | off            | Register the service as a macOS LaunchAgent                                       |
+| Flag                         | Default        | Notes                                                                                                                  |
+| ---------------------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
+| `--port <port>` / `-p`       | `4310`         | HTTP port; `service restart` preserves the existing service port when omitted                                          |
+| `--bind <ip>`                | `127.0.0.1`    | Use `0.0.0.0` or `::` for LAN access                                                                                   |
+| `--advertise-host <host>`    | detected       | Host printed for remote browsers                                                                                       |
+| `--client-root <path>`       | bundled client | Static client directory                                                                                                |
+| `--video-codec <mode>`       | `auto`         | `auto`, `hardware`, or `software`                                                                                      |
+| `--android-gpu <mode>`       | `host`         | Android emulator renderer: `host`, `auto`, `software`, `lavapipe`, `swiftshader`, `swangle`, or `swiftshader_indirect` |
+| `--stream-quality <profile>` | `full`         | `full`, `balanced`, `smooth`, `economy`, `low`, `tiny`, `ci-software`, and related profiles                            |
+| `--local-stream-fps <fps>`   | `60`           | Local stream frame target                                                                                              |
+| `--low-latency`              | off            | Conservative software H.264 profile                                                                                    |
+| `--open`                     | off            | Open the browser after starting the service                                                                            |
+| `--autostart` / `-a`         | off            | Register the service as a macOS LaunchAgent                                                                            |
 
 ## `describe`
 

docs/extensions/browser-client.md

@@ -38,7 +38,6 @@ Force a stream transport while debugging:
 
 ```text
 http://127.0.0.1:4310?stream=webrtc
-http://127.0.0.1:4310?stream=h264
 ```
 
 Use the default URL for normal operation.

docs/guide/architecture.md

@@ -26,7 +26,7 @@ This is why a long-lived service feels faster than repeatedly calling lower-leve
 
 ## Video flow
 
-The browser opens a live stream for the selected device. SimDeck sends fresh frames, drops stale ones when a client falls behind, and lets the browser request refreshes. The UI can use WebRTC or H.264-over-WebSocket fallback depending on browser support and network behavior.
+The browser opens a live WebRTC stream for the selected device. SimDeck sends fresh frames, drops stale ones when a client falls behind, and lets the browser request refreshes. iOS frames come from the native display bridge and are encoded on the Mac; Android frames come from the emulator `-share-vid` shared display surface and are encoded on the Mac.
 
 Tune this from the user-facing controls or with:
 

docs/guide/service.md

@@ -63,15 +63,16 @@ LaunchAgent port or the current singleton service port before falling back to
 These options are accepted by `simdeck`, `service start`, `service restart`,
 `service on`, and `service reset`:
 
-| Flag                         | Default     | Use it when                                |
-| ---------------------------- | ----------- | ------------------------------------------ |
-| `--port <port>` / `-p`       | `4310`      | You want a specific service port           |
-| `--bind <ip>`                | `127.0.0.1` | You need LAN access with `0.0.0.0` or `::` |
-| `--advertise-host <host>`    | detected    | Remote browsers need a specific host or IP |
-| `--video-codec <mode>`       | `auto`      | You need to force encoder behavior         |
-| `--stream-quality <profile>` | `full`      | You want lower CPU or bandwidth use        |
-| `--local-stream-fps <fps>`   | `60`        | You want a different local stream target   |
-| `--client-root <path>`       | bundled UI  | You are serving a custom static client     |
+| Flag                         | Default     | Use it when                                       |
+| ---------------------------- | ----------- | ------------------------------------------------- |
+| `--port <port>` / `-p`       | `4310`      | You want a specific service port                  |
+| `--bind <ip>`                | `127.0.0.1` | You need LAN access with `0.0.0.0` or `::`        |
+| `--advertise-host <host>`    | detected    | Remote browsers need a specific host or IP        |
+| `--video-codec <mode>`       | `auto`      | You need to force encoder behavior                |
+| `--android-gpu <mode>`       | `host`      | You need to change Android emulator GPU rendering |
+| `--stream-quality <profile>` | `full`      | You want lower CPU or bandwidth use               |
+| `--local-stream-fps <fps>`   | `60`        | You want a different local stream target          |
+| `--client-root <path>`       | bundled UI  | You are serving a custom static client            |
 
 ## Restart CoreSimulator
 

docs/guide/troubleshooting.md

@@ -137,12 +137,14 @@ curl http://127.0.0.1:4310/api/metrics
 
 If `frames_dropped_server` keeps climbing, the client or network cannot keep up. Move closer to the host, reduce quality, or switch to software encoding.
 
+For Android emulator streams, SimDeck uses the emulator `-share-vid` shared display surface. If Android video never starts, confirm `adb devices` shows the emulator as `device`, that it has fully booted, and that externally launched emulators were started with `-share-vid`. SimDeck-owned Android boots add the flag automatically and default to `--android-gpu host`; try `simdeck service restart --android-gpu auto` or `--android-gpu swiftshader_indirect` only when host rendering is unstable.
+
 ### Browser cannot establish WebRTC
 
-Force the H.264 WebSocket fallback while testing:
+Use `?stream=webrtc` while testing to make transport selection explicit:
 
 ```text
-http://127.0.0.1:4310?stream=h264
+http://127.0.0.1:4310?stream=webrtc
 ```
 
 For routed remote sessions, configure TURN as described in [Video and streaming](/guide/video#remote-browsers).

docs/guide/video.md

@@ -1,14 +1,19 @@
 # Video and streaming
 
-SimDeck streams live device video to the browser. Local sessions default to high quality. Remote or constrained sessions can trade detail for lower CPU and latency.
+SimDeck streams live device video to the browser. Local sessions default to full-resolution 60 fps. Remote or constrained sessions can trade detail for lower CPU and latency.
 
 iOS simulator H.264 uses VideoToolbox for hardware encoding and x264 for software encoding.
+Android emulator H.264 uses the emulator `-share-vid` display surface. SimDeck reads BGRA frames from the `videmulator<console-port>` shared memory region and encodes them on the Mac, so normal Android live video stays on the native shared display path.
 
 ## When encoding runs
 
-SimDeck starts encoding when a browser stream needs H.264 frames. The server
-requests an initial keyframe to answer the WebRTC or H.264 WebSocket viewer,
-then keeps a shared refresh pump active while frame subscribers exist.
+SimDeck starts encoding when a browser stream needs H.264 frames. For iOS, the
+server requests an initial keyframe to answer the WebRTC viewer, then keeps a
+shared refresh pump active while frame subscribers exist.
+For Android, SimDeck starts emulators with `-share-vid`, maps the shared display
+region, and feeds changed BGRA frames into the native host H.264 encoder.
+SimDeck-owned Android boots also default to `-gpu host`, matching the native
+emulator app's accelerated renderer while staying in headless shared-video mode.
 
 The browser reports whether the page and stream canvas are foreground. When all
 known viewers are hidden or the last frame subscriber disconnects, the native
@@ -35,7 +40,8 @@ Common profiles:
 
 | Profile       | Use it for                              |
 | ------------- | --------------------------------------- |
-| `full`        | Local browser on a fast Mac             |
+| `full`        | Default local full-resolution 60 fps    |
+| `smooth`      | Full-size 60 fps with lower bitrate     |
 | `balanced`    | Good local quality with less bandwidth  |
 | `economy`     | Remote browser or busy machine          |
 | `low`         | Slower Wi-Fi or shared hosts            |
@@ -44,6 +50,18 @@ Common profiles:
 
 The browser also has stream controls for transport, resolution, FPS, and refresh.
 
+## Pick an Android GPU mode
+
+SimDeck-owned Android emulator boots use host GPU acceleration by default:
+
+```sh
+simdeck service restart --android-gpu host
+```
+
+Use `auto` to let the Android emulator choose the renderer. Use
+`swiftshader_indirect`, `swiftshader`, `software`, `lavapipe`, or `swangle` only
+when host rendering is unstable on a specific machine.
+
 ## Pick a codec
 
 ```sh
@@ -58,6 +76,12 @@ simdeck service restart --video-codec software
 | `hardware` | Dedicated local machines where VideoToolbox hardware H.264 is reliable.             |
 | `software` | x264 software H.264 for CI, screen recording conflicts, or hardware encoder stalls. |
 
+The codec setting controls simulator host encoding. Android emulator streams use
+the same service codec by default for shared display frames; set
+`SIMDECK_ANDROID_VIDEO_CODEC=auto`, `hardware`, or `software` before starting the
+service only when you need an Android-specific encoder override. Stream quality
+controls the encoded Android frame size.
+
 When multiple simulator streams run at the same time, `auto` keeps one active
 stream on the hardware encoder path and routes additional active auto streams to
 software encoding. This avoids saturating the shared VideoToolbox hardware
@@ -69,15 +93,14 @@ For very constrained software sessions:
 simdeck service restart --video-codec software --low-latency
 ```
 
-## WebRTC and fallback
+## WebRTC
 
-The browser tries WebRTC first. If WebRTC cannot render a frame, the UI can fall back to H.264 over WebSocket when the browser supports WebCodecs.
+The browser uses WebRTC for live video. SimDeck no longer exposes a separate H.264 WebSocket video transport.
 
 Force a mode while debugging:
 
 ```text
 http://127.0.0.1:4310?stream=webrtc
-http://127.0.0.1:4310?stream=h264
 ```
 
 ## Remote browsers

packages/client/src/api/types.ts

@@ -50,7 +50,7 @@ export interface SimulatorMetadata {
   isBooted: boolean;
   android?: {
     avdName?: string;
-    grpcPort?: number;
+    consolePort?: number;
     serial?: string;
   };
   privateDisplay?: PrivateDisplayInfo;