ProviderConfig.maxOutputTokens not applied to the request for a custom provider + unknown model

[robsteen]

Summary

With @github/copilot-sdk@1.0.0 (@github/copilot CLI v1.0.59, protocolVersion 3), setting
ProviderConfig.maxOutputTokens for a custom OpenAI (BYOK) provider has no effect when the
model is not in the SDK's known-model-limits table. The value is accepted on the config but
never reaches the outbound request body, so output is not capped.

Environment

• @github/copilot-sdk@1.0.0, @github/copilot CLI v1.0.59, protocolVersion 3
• Custom OpenAI provider (BYOK relay):
  ProviderConfig = { type: 'openai', baseUrl: '…/v1/openai', wireApi: 'responses', apiKey, maxOutputTokens: 32000 }
• Model: a custom Azure deployment (not a SDK-recognized model name)

What works

wireApi: 'responses' on the same ProviderConfig is honored — the CLI logs
Using custom provider … wireApi=responses and the outbound request is Responses-shaped
(instructions / input / text / reasoning).

Expected

max_output_tokens (Responses API) appears in the outbound request, capping output at 32000,
per the ProviderConfig.maxOutputTokens doc ("Overrides the resolved model's default max output tokens").

Actual

The outbound Responses request contains no max_output_tokens. Instrumentation confirms
provider.maxOutputTokens = 32000 is present on the session config immediately before
createSession() (alongside the wireApi that does take effect) — so the value reaches the SDK,
but the SDK does not emit it.

Likely cause

The request is built as roughly:

max_output_tokens: e.maxOutputTokens ?? <modelLimits>.max_output_tokens

The custom model isn't in the SDK's hard-coded limits map (the claude-* / gpt-5-mini / … table),
and ProviderConfig.maxOutputTokens doesn't appear to populate e.maxOutputTokens for a custom
provider — so the field is omitted entirely. This makes maxOutputTokens effectively a no-op for
any model the SDK doesn't already recognize.

Questions

1. Should ProviderConfig.maxOutputTokens be honored for custom providers / models not in the
   SDK's known-limits table? Today it only seems to apply to recognized models.
2. If not, what's the supported way to cap output tokens for a custom Azure deployment via the SDK,
   without a request-body/header shim?
3. Does the same limitation apply to the wireApi: 'completions' path?

Happy to share instrumented logs or a minimal repro.

[github-actions]

Investigation Findings

Thanks for the detailed report. I've investigated this against the SDK source code and can confirm this is a bug.

What I looked at

• nodejs/src/types.ts — the user-facing ProviderConfig type
• nodejs/src/generated/rpc.ts — the generated wire-format ProviderConfig type
• nodejs/src/client.ts — how provider config is forwarded in session.create / session.resume RPC calls
• nodejs/test/client.test.ts and python/test_client.py — unit tests for provider config forwarding

What I found

The SDK correctly forwards maxOutputTokens. Both the user-facing ProviderConfig (in types.ts) and the generated RPC ProviderConfig (in generated/rpc.ts) include maxOutputTokens, and the SDK passes config.provider directly in the RPC payload without transformation. Unit tests at lines 793–816 in client.test.ts (and equivalent in test_client.py) confirm the value is present in the session.create wire payload.

The bug is in the CLI layer, not the SDK itself. The SDK correctly serializes maxOutputTokens into the JSON-RPC session config — the CLI receives it. However, when building the outbound HTTP request to the LLM API, the CLI applies provider.maxOutputTokens only when the model is a well-known one (present in its internal model-limits table). For custom/unrecognized Azure deployments or other arbitrary models, the CLI falls back to a lookup that returns undefined, causing max_output_tokens to be omitted from the request body entirely.

Additionally, there is a documentation gap in the README. The ProviderConfig section in nodejs/README.md (lines 698–706) lists type, baseUrl, apiKey, bearerToken, wireApi, and azure — but does not mention maxOutputTokens or maxPromptTokens at all. The only documentation for these fields is the JSDoc in types.ts, which says "Overrides the resolved model's default max output tokens" with no caveat about the known-model restriction.

Why this is a bug

The documented intent of ProviderConfig.maxOutputTokens is to override the max output tokens for the session. There is no documented limitation that this only applies to SDK-recognized models. Users of BYOK providers with custom model names have no other supported mechanism to cap output tokens.

Answers to your questions

1. Should maxOutputTokens be honored for custom/unknown models? Yes — this is the documented intent and the behavior should be fixed in the CLI to apply the value unconditionally when set, regardless of whether the model is in the known-limits table.
2. Workaround? There is no clean workaround within the current public API. Using modelCapabilities (SessionConfig.modelCapabilities) via the limits.max_output_tokens path is the closest alternative — but max_output_tokens is not currently exposed in the user-facing ModelCapabilities.limits type in types.ts (it's in the generated RPC type only), so this is not a supported path either.
3. Does the same limitation apply to wireApi: 'completions'? Almost certainly yes, since the same model-lookup path is used for both wire APIs.

Generated by Bug Handler for issue #1627 · sonnet46 3.4M · ◷