RFC: Message Chunking

[colbylwilliams]

Summary

Add a transport-agnostic message-segmenting primitive to AHP so a single logical JSON-RPC message can be split across multiple WebSocket (or other transport) frames and reassembled by the receiver before normal dispatch. Two new primitives:

1. A reserved control notification — ahp/messageSegment — that carries one segment of a larger message.
2. A capabilities.chunking object exchanged during initialize (and again on reconnect when the transport changes) so each side declares per-direction limits.

The wire-format payload is base64-encoded UTF-8 bytes of the original JSON-RPC message. The receiver concatenates the bytes of all segments in a group, decodes once, and dispatches the result as if it had arrived as a single message.

Motivation

AHP today assumes any one JSON-RPC message can fit in a single transport frame. That assumption holds for direct WebSocket connections between a client and a daemon, where browsers and servers allow multi-megabyte frames in practice.

It breaks once a managed pub/sub transport with a hard frame cap sits in the path. Concretely, Azure Web PubSub — adopted as the relay substrate in github/copilot-host#105 — rejects any WebSocket frame larger than 1 MB with a connection close. The reliable subprotocol has no service-side chunking.

The set of AHP messages that can plausibly exceed 1 MB on a long-running session:

Message	When it gets large
reconnect result snapshot	A long session whose state has accumulated turns, tool calls, customizations, and changesets
action notification carrying session/customizationsChanged	Clients with rich customization blobs
action notification carrying session/toolCallComplete	Tools that return large blobs (file reads, large HTTP responses, gh pr view --json output)
action notification carrying session/changesetsChanged	Sessions that touched many files
action notification carrying terminal/data	cat huge_file, verbose build logs, large structured CLI output
dispatchAction params	A user paste of a large prompt
resourceWrite params	Writing a large file body through the host

Direct WebSocket transports do not hit the 1 MB cap today, but a protocol-level primitive lets every transport with a frame ceiling — present or future — work uniformly. It also gives implementations a place to put per-transport tuning (smaller chunks for high-latency mobile links, larger chunks for LAN).

Non-goals

• Streaming text into state. session/delta already streams partial text into the session reducer. This proposal does not change that. The word segment is chosen specifically to avoid confusion with the existing "chunk" of streaming text.
• Exactly-once semantics for chunked requests across reconnect. Idempotency is the caller's responsibility, as it is for unchunked requests today.
• Resumable transfers across transport disconnect. Reassembly state is dropped on disconnect; the server-side replay path handles missed actions naturally (see §Reconnection).
• Binary frame transports. This proposal targets JSON text frames. A future companion proposal can add a parallel binary form once we have a transport that needs it.
• Compression. Out of scope; compose with transport-level compression (e.g. WebSocket permessage-deflate) or a future application-level codec.

Design overview

Segmenting layers above JSON-RPC. From the application's point of view, nothing changes — a notification is still a notification, a response is still a response. The framing layer underneath the dispatcher hides reassembly:

sequenceDiagram
    participant App as Application<br/>(reducer, handler)
    participant Frame as Segmenting layer
    participant Wire as Transport (1 frame ≤ N bytes)
    participant FrameR as Segmenting layer
    participant AppR as Application<br/>(reducer, handler)

    App->>Frame: dispatch ActionEnvelope (3 MB)
    Frame->>Frame: split into 4 base64 segments
    Frame->>Wire: ahp/messageSegment #0
    Frame->>Wire: ahp/messageSegment #1
    Frame->>Wire: ahp/messageSegment #2
    Frame->>Wire: ahp/messageSegment #3
    Wire-->>FrameR: ahp/messageSegment #0..#3
    FrameR->>FrameR: concat + base64-decode
    FrameR->>AppR: dispatch reassembled ActionEnvelope

Loading

A small in-flight reassembly table on the receiver holds partial groups keyed by groupId. Once a group is complete, the assembled bytes are parsed as a single JSON-RPC message and dispatched through the normal path.

Wire format

ahp/messageSegment

ahp/messageSegment is a JSON-RPC notification. It is not an application notification: it is a control notification consumed by the segmenting layer before normal JSON-RPC dispatch, and it does not carry a channel: URI.

interface MessageSegmentParams {
  /**
   * Opaque sender-chosen identifier that scopes one in-flight reassembly.
   *
   * MUST be a non-empty string of at most 128 UTF-8 bytes. Senders SHOULD
   * use a random 16-byte value encoded as hex or base64url; receivers MUST
   * NOT interpret the value.
   *
   * `groupId` and JSON-RPC `id` are independent namespaces. A `groupId`
   * MUST be unique among the sender's currently in-flight reassemblies on
   * a single connection; it MAY be reused once the receiver has either
   * fully reassembled or discarded that group.
   */
  readonly groupId: string;

  /**
   * 0-based position of this segment within the group. MUST be a
   * non-negative integer < total and < 2^31. MUST be strictly monotonically
   * increasing within a single `groupId`: receivers MUST reject duplicate
   * or out-of-order indices as a protocol error.
   */
  readonly index: number;

  /**
   * Total number of segments in this group. MUST be ≥ 1, MUST be < 2^16
   * (65,536), MUST be stable across every segment of the group. The first
   * segment establishes the total; subsequent segments with a different
   * `total` MUST be rejected as a protocol error.
   */
  readonly total: number;

  /**
   * Base64-encoded slice of the UTF-8 bytes of the original JSON-RPC
   * message being reassembled.
   *
   * `data` uses the standard base64 alphabet (RFC 4648 §4) with padding.
   * Concatenating the decoded bytes of segments 0..total-1 in order MUST
   * produce a UTF-8 byte sequence that parses as exactly one JSON-RPC
   * request, response, or notification.
   */
  readonly data: string;
}

interface MessageSegmentNotification {
  readonly jsonrpc: '2.0';
  readonly method: 'ahp/messageSegment';
  readonly params: MessageSegmentParams;
}

A new ControlNotificationMap registry sits alongside ClientNotificationMap and ServerNotificationMap for messages of this kind:

export interface ControlNotificationMap {
  'ahp/messageSegment': { params: MessageSegmentParams };
}

Control notifications travel in either direction. Receivers MUST handle them whether the peer is a client or a server.

Base64 over raw text

data is base64 of the message's UTF-8 bytes — not the raw text. The rationale:

• Byte-exact size accounting. Senders sizing segments against a hard frame ceiling need to know the wire size before sending. Embedding raw JSON text inside another JSON string re-escapes every ", \, and control character, so the same logical content can land at very different wire sizes depending on its contents. Base64 has a fixed 4:3 expansion factor and no escaping.
• No codepoint-boundary ambiguity. Splitting a JSON string at a UTF-8 byte boundary that falls inside a multi-byte codepoint is a bug source. Base64 sidesteps it.
• Forward-compatibility with non-UTF-8 future encodings. If a future protocol version adopts CBOR or another binary message form, the same segmenting primitive applies unchanged.

The price is a flat ~33% data-overhead inflation. That is an acceptable cost for a primitive that exists specifically to fit under a hard wire ceiling.

Capability negotiation

Segmenting is opt-in per direction. Each side advertises what it is willing to receive. The peer only segments outgoing messages if the receiver advertised support.

A new top-level capabilities field appears in both InitializeParams and InitializeResult:

export interface ClientCapabilities {
  readonly chunking?: ChunkingCapability;
}

export interface ServerCapabilities {
  readonly chunking?: ChunkingCapability;
}

export interface ChunkingCapability {
  /**
   * Maximum size, in UTF-8 bytes, of any single transport frame this side
   * is willing to receive — for both unchunked messages and individual
   * `ahp/messageSegment` notifications. Includes the JSON-RPC envelope
   * overhead.
   */
  readonly maxIncomingFrameBytes: number;

  /**
   * Maximum size, in UTF-8 bytes, of a fully reassembled JSON-RPC message
   * this side is willing to receive. MUST be ≥ `maxIncomingFrameBytes`.
   */
  readonly maxIncomingMessageBytes: number;

  /**
   * Maximum number of concurrent in-flight reassembly groups this side
   * will hold open. MUST be ≥ 1. Defaults to an implementation-defined
   * value if unset.
   */
  readonly maxIncomingGroups?: number;

  /**
   * Maximum time, in milliseconds, this side will hold an incomplete
   * group before discarding it as abandoned. Defaults to an
   * implementation-defined value if unset.
   */
  readonly groupTimeoutMs?: number;
}

// Additive in InitializeParams / InitializeResult / ReconnectParams.
export interface InitializeParams extends BaseParams {
  // ...existing fields...
  readonly capabilities?: ClientCapabilities;
}

export interface InitializeResult {
  // ...existing fields...
  readonly capabilities?: ServerCapabilities;
}

export interface ReconnectParams extends BaseParams {
  // ...existing fields...
  readonly capabilities?: ClientCapabilities;
}

Capabilities MUST also be exchanged in ReconnectParams because reconnect runs on a fresh transport whose limits the server has no other way to learn. InitializeResult is not re-issued; the server SHOULD assume its previously-negotiated capabilities still apply, but MAY downgrade by simply choosing not to segment. The client MAY include a fresh capabilities block on reconnect if its transport limits have changed.

Capability matching

Once both sides have exchanged capabilities:

• If the receiver in a direction did not advertise chunking, the sender in that direction MUST NOT send ahp/messageSegment. It MUST send messages as-is; if any single message would exceed the transport's frame ceiling, the sender MUST fail the operation locally (or, for action notifications, MAY drop the connection with a defined error — see §Reconnection).
• If both sides advertised chunking, segmenting is enabled. The sender MUST respect the receiver's maxIncomingFrameBytes (for every frame it puts on the wire — segmented or not) and maxIncomingMessageBytes (for any message it segments).

Sender behavior

For every outgoing JSON-RPC message M:

1. Serialize M to UTF-8 bytes — call the result B.
2. If len(B) plus the surrounding transport overhead is ≤ receiver's maxIncomingFrameBytes, send M as a single frame and stop.
3. Otherwise:
   1. If the receiver did not advertise chunking, fail the send (see §Reconnection).
   2. If len(B) > maxIncomingMessageBytes, fail the send.
   3. Mint a unique groupId.
   4. Compute the per-segment payload size S such that, after base64 encoding and inclusion in a MessageSegmentNotification, the resulting frame fits within maxIncomingFrameBytes. (Base64 expands by 4/3; allow for JSON envelope overhead.)
   5. Split B into ceil(len(B) / S) chunks, in order.
   6. Encode each chunk with base64 and send it as ahp/messageSegment with the correct index, total, and groupId.

The sender SHOULD send all segments of a group contiguously, but MAY interleave segments from different groups for fairness — see Interleaving.

A sender MUST NOT segment an ahp/messageSegment (no recursion).

Receiver behavior

The receiver maintains a small table:

type ReassemblyTable = Map<string, {
  total: number;
  nextIndex: number;
  bufferedBytes: number;
  segments: Uint8Array[];
  startedAt: number;
}>;

On every ahp/messageSegment notification:

1. Look up groupId in the table.
   • If absent and the table already holds maxIncomingGroups entries, reject as a protocol error.
   • If absent, validate index === 0 and total ≥ 1; insert a new entry.
   • If present, validate total matches the previously-recorded total and index === nextIndex.
2. Decode data from base64. Reject if decoding fails or expansion exceeds the receiver's maxIncomingFrameBytes.
3. Append the decoded bytes to the group's buffer. If bufferedBytes exceeds maxIncomingMessageBytes, reject.
4. Increment nextIndex.
5. If nextIndex === total:
   1. Concatenate the segments into a single Uint8Array.
   2. Decode UTF-8 → JSON-RPC parse. Reject if either step fails.
   3. Reject if the reassembled message is itself an ahp/messageSegment.
   4. Dispatch the reassembled message through the normal handler path.
   5. Remove the entry from the table.

The receiver MUST run a periodic sweep that removes entries older than groupTimeoutMs (default implementation-defined, see §Limits and DoS protections). Sweep does not generate protocol errors; the sender will retry on reconnect.

On transport disconnect, the receiver MUST drop the entire reassembly table.

Validation rules (errors)

ahp/messageSegment is a notification, so there is no JSON-RPC response path for errors. Any of the following conditions are protocol errors and the receiver MUST close the transport with a defined close reason (e.g. WebSocket close code 4400 with reason "invalid messageSegment"). The client SHOULD reconnect.

Condition	Notes
groupId is missing, empty, or > 128 bytes
index is not a non-negative integer
index >= total
index !== nextIndex for an existing group	Out-of-order delivery is impossible under a reliable, ordered transport; this is either a sender bug or interleaving with the same groupId
total < 1 or total >= 2^16
total changes mid-group
data is missing or not a base64-encoded string
Decoded segment exceeds the receiver's maxIncomingFrameBytes
Accumulated bytes for the group exceed maxIncomingMessageBytes
Active groups exceed maxIncomingGroups
Reassembled bytes are not valid UTF-8 / not a valid JSON-RPC message
Reassembled message is itself ahp/messageSegment	No recursion
groupId already in flight when a new index === 0 segment arrives

Interleaving

A sender MAY interleave segments from different groupIds — e.g., to keep a small, latency-sensitive terminal/data notification flowing while a multi-segment session/customizationsChanged is mid-transmission. Receivers MUST support up to maxIncomingGroups concurrent groups.

Senders SHOULD:

• Avoid starvation of small messages behind a bulk transfer.
• Bound the number of concurrent groups they originate at the receiver's maxIncomingGroups.
• Send segments of a single group in index order (the receiver requires this).

A simple sender that always sends groups contiguously is conformant.

Channel-invariant exception

AHP's overview specifies:

Every notification's params carries a top-level channel: URI.

This invariant lets receivers dispatch by (method, params.channel) without per-method deserialisation, and it is compile-time-checked in types/version/message-checks.ts.

ahp/messageSegment is a control notification — it belongs to the framing layer, not to any subscribable resource — so it does not carry a channel. Faking channel: 'ahp-root://' would imply incorrect routing semantics (root-channel handlers would receive segments) and would mislead future readers.

The proposal therefore introduces a third, narrow registry — ControlNotificationMap — that sits next to ClientNotificationMap and ServerNotificationMap. The compile-time invariant is updated to read: every entry in ClientNotificationMap or ServerNotificationMap carries channel; entries in ControlNotificationMap do not.

We expect ControlNotificationMap to stay tiny. Anything that is genuinely about a subscribable resource belongs in the existing two registries.

Limits and DoS protections

Recommended default limits, intended as guidance not normative requirements:

Limit	Default
maxIncomingFrameBytes	900 KB on Web PubSub-relayed paths; 4 MB elsewhere
maxIncomingMessageBytes	32 MB
maxIncomingGroups	8
groupTimeoutMs	30 000
groupId length	≤ 128 UTF-8 bytes
total	< 2^16 (65 536)

Implementations MUST enforce some finite value for each. A peer that omits maxIncomingGroups or groupTimeoutMs from its advertised capability is asking the other side to apply its own defaults; it is not asking for unbounded resources.

Specific attack vectors and mitigations:

• total: 2^31 slow-drip group. Rejected by the total < 2^16 validation rule.
• Many half-finished groups. Bounded by maxIncomingGroups plus groupTimeoutMs sweep.
• Bytes that expand after JSON unescaping. Base64 has no nested unescaping; rejected at decode time if the segment exceeds maxIncomingFrameBytes.
• Huge groupId strings. Rejected by the 128-byte cap.
• Same groupId reused for two concurrent groups by a malicious peer. Rejected by the "duplicate groupId in flight" rule.

Reconnection

The interaction with the existing reconnection flow is the trickiest part of this design. Two cases:

Server → client action envelopes

The server assigns serverSeq when it produces an ActionEnvelope. The client tracks lastSeenServerSeq and MUST update it only after fully reassembling, parsing, and dispatching the envelope — never on individual segment receipt.

If the transport drops mid-group:

1. The client's reassembly table is discarded.
2. The client reconnects with lastSeenServerSeq still pointing at the last fully processed envelope.
3. The server's existing replay path resends every action with serverSeq > lastSeenServerSeq, which includes the partially-delivered envelope. It will be segmented again on the new transport (possibly differently, since limits may have been re-negotiated).

No per-segment acknowledgement is required.

Outstanding client → server (or server → client) requests

If a request is mid-segmentation when the transport drops, the request id is no longer meaningful on the new connection. The caller's promise (or equivalent) MUST be rejected with a transport-disconnect error. Whether the request is safe to retry is the caller's responsibility — chunking adds no exactly-once guarantee for non-idempotent operations. Callers SHOULD NOT blindly retry createSession, resourceWrite, or any other state-mutating request without an idempotency mechanism appropriate to that command. This matches the pre-existing semantics for non-segmented requests across reconnect.

Fail-closed when the receiver did not advertise chunking

If a sender produces a message that would exceed the receiver's maxIncomingFrameBytes and the receiver did not advertise chunking:

• For requests, the sender MUST fail the local call with a defined error rather than send a frame the receiver would reject.
• For responses, the sender MUST return a JSON-RPC error of code -32011 (MessageTooLarge, new code introduced by this proposal).
• For notifications, the sender MAY drop the notification and log; the receiver will fall back to the natural recovery mechanism for the affected channel (reconnect + state catch-up).

Examples

Example A — chunked action envelope

A 2.4 MB session/toolCallComplete action over a Web PubSub link with the receiver advertising { maxIncomingFrameBytes: 900 000, maxIncomingMessageBytes: 33 554 432, maxIncomingGroups: 8 }:

{ "jsonrpc": "2.0", "method": "ahp/messageSegment",
  "params": {
    "groupId": "b9f1c6c2-3f4d-4f8e-9a2c-7d4c0e9a8e15",
    "index": 0, "total": 3,
    "data": "eyJqc29ucnBjIjoiMi4wIiwibWV0aG9kIjoiYWN0aW9uIiwicGFyYW1z..."
  }
}
{ "jsonrpc": "2.0", "method": "ahp/messageSegment",
  "params": {
    "groupId": "b9f1c6c2-3f4d-4f8e-9a2c-7d4c0e9a8e15",
    "index": 1, "total": 3,
    "data": "OiB7ImNoYW5uZWwiOiJhaHAtc2Vzc2lvbjovYWJjLTEyMyIsImFjdGlvbi..."
  }
}
{ "jsonrpc": "2.0", "method": "ahp/messageSegment",
  "params": {
    "groupId": "b9f1c6c2-3f4d-4f8e-9a2c-7d4c0e9a8e15",
    "index": 2, "total": 3,
    "data": "ifSwgInNlcnZlclNlcSI6NDIxLCAib3JpZ2luIjpudWxsfQ=="
  }
}

Concatenating the base64-decoded data payloads, in index order, produces the original UTF-8 bytes of:

{ "jsonrpc": "2.0", "method": "action",
  "params": { "channel": "ahp-session:/abc-123", "action": { /* … large */ }, "serverSeq": 421, "origin": null }
}

which is then dispatched through the receiver's normal notification handler.

Example B — chunked reconnect result

The server's reconnect response for a long-running session might carry a multi-megabyte snapshot[]. The framing layer segments the entire JSON-RPC response (including the id correlation field) the same way:

{ "jsonrpc": "2.0", "method": "ahp/messageSegment",
  "params": { "groupId": "g7", "index": 0, "total": 5, "data": "..." }
}
// ... segments 1..4 ...
{ "jsonrpc": "2.0", "method": "ahp/messageSegment",
  "params": { "groupId": "g7", "index": 4, "total": 5, "data": "..." }
}

After reassembly the client's JSON-RPC layer sees a normal:

{ "jsonrpc": "2.0", "id": 17, "result": { "type": "snapshot", "snapshots": [ /* … */ ] } }

and correlates it with the in-flight reconnect request id: 17.

Example C — capability advertisement

// Client → Server
{
  "jsonrpc": "2.0", "id": 1, "method": "initialize",
  "params": {
    "channel": "ahp-root://",
    "protocolVersions": ["0.3.0", "0.2.0"],
    "clientId": "client-abc",
    "capabilities": {
      "chunking": {
        "maxIncomingFrameBytes": 900000,
        "maxIncomingMessageBytes": 33554432,
        "maxIncomingGroups": 8,
        "groupTimeoutMs": 30000
      }
    }
  }
}

// Server → Client
{
  "jsonrpc": "2.0", "id": 1,
  "result": {
    "protocolVersion": "0.3.0",
    "serverSeq": 0,
    "snapshots": [ /* … */ ],
    "capabilities": {
      "chunking": {
        "maxIncomingFrameBytes": 900000,
        "maxIncomingMessageBytes": 16777216,
        "maxIncomingGroups": 4,
        "groupTimeoutMs": 30000
      }
    }
  }
}

Both sides advertised support, so segmenting is enabled in both directions. The effective limits are the receiver's for each direction: outbound from client to server uses the server's 16 MB / 900 KB; outbound from server to client uses the client's 32 MB / 900 KB.

Alternatives considered

A. Put chunk metadata on ActionEnvelope directly

The original sketch in github/copilot-host#105 suggested adding chunk { index, total, groupId } to ActionEnvelope itself. Rejected because it covers only the action notification — not reconnect results, subscribe results, dispatchAction params, resourceWrite params, or any other message that can grow large — and it leaks transport concerns into application action semantics.

B. Layer segmenting below JSON-RPC, as a transport-binding concern

Cleaner separation of concerns, but it pushes the problem onto every transport binding. Each transport (WebSocket, TCP-with-framing, in-process) would need its own segmenting format, and JSON-RPC-aware middleware (proxies, logs, debuggers) could not see across the seam. A reserved control notification is portable.

C. Raw UTF-8 substring instead of base64

Rejected — see Base64 over raw text. The hidden re-escaping inside the outer JSON envelope makes wire-size accounting unreliable, and codepoint-boundary splitting is a footgun.

D. Per-segment ack with replay-from-segment

A reliable-delivery layer on top of the existing reliable transport. Rejected as redundant: AHP's transport requirements already guarantee ordered, reliable delivery within a single connection, and the existing reconnect-with-replay path handles transport drops cleanly. Per-segment acks would double message volume on the metered paths this proposal is trying to make tractable.

E. Mandatory, no capability advertisement

Simpler, but a strict older implementation receiving an unsolicited ahp/messageSegment would silently ignore it (JSON-RPC notification semantics), causing silent message loss. Capability advertisement gates the sender so this never happens.

Migration and versioning

This is an additive, capability-gated change. Older peers can interoperate with newer peers without modification, provided the older side never advertises chunking (which it cannot, because it has never heard of the capability).

The recommended rollout:

1. Bump the AHP protocol version to introduce MessageSegmentParams, the ControlNotificationMap registry, the capabilities field on InitializeParams / InitializeResult / ReconnectParams, and the -32011 MessageTooLarge error code.
2. New implementations advertise chunking only when speaking the new protocol version. Implementations speaking older protocol versions MUST NOT advertise or use chunking.
3. Sender-side fallback: when a peer does not advertise chunking but a sender produces an oversized message, the sender follows the fail-closed rules.

A version bump (rather than capability-only) is recommended because:

• It marks the introduction of a new control-notification registry — a structural change to the message taxonomy.
• It prevents accidental silent message loss to a non-capable peer that happens to ignore unknown capabilities fields.
• It lets implementations advertise the full extension surface (capabilities + control notification + error code + reconnect-time renegotiation) as one coherent feature.

Open questions

1. Should maxIncomingFrameBytes apply to every frame, or only to ahp/messageSegment frames?
   Currently the proposal says every frame. This is the safer default — Web PubSub will close the connection regardless — but it means even small messages from a sender are bounded. Consider documenting it as "the smaller of the negotiated cap and the underlying transport's hard cap, whichever applies."

2. Does maxIncomingGroups need separate inbound and outbound advertisements?
   Today only the receive side advertises. A peer might want to declare it will only originate K groups concurrently to avoid the other side's table filling up, but that is implementation discipline rather than a wire concern.

3. Should there be a cancelGroup control notification?
   For abandoning a partially-sent group without waiting for the receiver's groupTimeoutMs sweep. Probably yes for very long timeouts; probably no for the default 30 s.

4. Compression as a sibling control notification?
   A future ahp/messageCompressed could carry one compressed message; combined with segmenting, this would reduce wire volume meaningfully. Out of scope for this proposal; flagged here so the control-notification registry is designed with room.

5. Should groupTimeoutMs be enforceable by the sender too?
   A misbehaving sender could open a group and never finish it. Receiver sweep handles this. Adding a sender-side obligation is largely advisory.

Acknowledgements

The motivating real-world constraint and design conversations are documented in github/copilot-host#105. The pattern of a reserved JSON-RPC notification namespace for protocol-private control messages is well-established in LSP's $/ convention; this proposal uses the AHP-native ahp/ namespace for the same purpose.