Skip to content

feat(client): pass through by-handle payload segments - #171

Closed
faresobeid wants to merge 1 commit into
mainfrom
feat/payload-passthrough
Closed

faresobeid wants to merge 1 commit into
mainfrom
feat/payload-passthrough

Conversation

@faresobeid

@faresobeid faresobeid commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

What

generate() returns the choice's payload (by-handle segments of a shared file that the prime-rl inference server writes instead of returning routed_experts / sampling_mask inline). Pass-through only. The inline fields and the base64 fast path are unchanged.

Companion PRs: PrimeIntellect-ai/verifiers#2765 (segments on traces) and PrimeIntellect-ai/prime-rl#3902 (writer and consumers; GPU checks there).

Landing order

  1. perf(client): read prime-rl's packed completion_logprobs #174 and perf(v1): accept packed CSR sampling masks from prime-rl's /generate verifiers#2778 (packed /generate, with perf(inference): pack /generate per-token payloads and tune API-server GC prime-rl#3900)
  2. This PR and feat(v1): by-handle payload segments on traces verifiers#2765
  3. feat: pass router-replay ids and sampling masks by handle on multi-node runs prime-rl#3902 (re-pin deps/renderers / deps/verifiers to the merged commits first)
  4. Disaggregated P/D: feat(inference): router replay on the V2 runner everywhere prime-rl#3830, fix(pd): honor routed_experts dtype in inline merge router#61 then feat(pd): stitch by-handle routed_experts payload segments router#60

This PR and #170 both add a key after "sampling_mask" in generate()'s result, so the second to land needs a one-line rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant