feat(provider): Add Kimi K3 support across applicable providers#933
feat(provider): Add Kimi K3 support across applicable providers#933navedmerchant wants to merge 3 commits into
Conversation
📝 WalkthroughWalkthroughAdds Kimi K3 to Moonshot and OpenCode Go catalogs, normalizes gateway metadata, sends required reasoning settings, omits unsupported temperature, and preserves reasoning/tool calls across provider requests. ChangesKimi K3 provider support
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant ProviderHandler
participant AI SDK
participant ProviderAPI
Client->>ProviderHandler: submit Kimi K3 messages
ProviderHandler->>AI SDK: serialize reasoning and tool calls
ProviderHandler->>ProviderAPI: send reasoning_effort and token limits without temperature
ProviderAPI-->>ProviderHandler: stream response deltas
ProviderHandler-->>Client: emit text, reasoning, and tool results
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (3)
src/api/providers/vercel-ai-gateway.ts (1)
109-112: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover the new streaming-output and non-streaming request paths in both gateways.
The new
createMessagerequest parameters are tested, but reasoning-delta emission andcompletePromptserialization remain uncovered.
src/api/providers/vercel-ai-gateway.ts#L109-L112: test thatreasoning_contentandreasoningdeltas yield reasoning chunks.src/api/providers/vercel-ai-gateway.ts#L143-L170: test Kimi K3 and optional-reasoningcompletePromptpayloads.src/api/providers/zoo-gateway.ts#L258-L261: test that reasoning deltas yield reasoning chunks.src/api/providers/zoo-gateway.ts#L305-L332: test Kimi K3 and optional-reasoningcompletePromptpayloads.As per coding guidelines, use package-local unit tests for request construction and error handling.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/vercel-ai-gateway.ts` around lines 109 - 112, Expand package-local unit tests for src/api/providers/vercel-ai-gateway.ts:109-112 to verify reasoning_content and reasoning deltas yield reasoning chunks, and for src/api/providers/vercel-ai-gateway.ts:143-170 to verify Kimi K3 and optional-reasoning completePrompt payload serialization. Add equivalent coverage in src/api/providers/zoo-gateway.ts:258-261 and src/api/providers/zoo-gateway.ts:305-332, covering reasoning-delta emission and both completePrompt payload variants. Use the existing gateway streaming and completePrompt test helpers without changing production behavior.Source: Coding guidelines
src/api/providers/openai-compatible.ts (1)
221-234: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winMake temperature resolution consistent with
createMessage.In
createMessage, the temperature resolves withmodel.temperature ?? this.config.temperature ?? 0. Here incompletePrompt,model.temperatureis bypassed, which could cause it to ignore model-specific temperature overrides (such as those returned by a subclass'sgetModelimplementation).Consider updating the fallback to match
createMessage:♻️ Proposed fix
async completePrompt(prompt: string): Promise<string> { const model = this.getModel() const languageModel = this.getLanguageModel() const reasoningEffort = this.getReasoningEffort(model) const { text } = await generateText({ model: languageModel, prompt, maxOutputTokens: this.getMaxOutputTokens(), - temperature: model.info.supportsTemperature === false ? undefined : (this.config.temperature ?? 0), + temperature: + model.info.supportsTemperature === false + ? undefined + : (model.temperature ?? this.config.temperature ?? 0), ...(reasoningEffort && { providerOptions: { openaiCompatible: { reasoningEffort } }, }), })🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/openai-compatible.ts` around lines 221 - 234, Update the temperature expression in completePrompt to resolve model.temperature before this.config.temperature and the 0 default, while preserving the existing supportsTemperature check and undefined behavior for unsupported models. Match the resolution used by createMessage and use the model returned by getModel.src/api/providers/opencode-go.ts (1)
506-509: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider aligning
streamAnthropicMessagewith this temperature logic.The non-streaming Anthropic path now correctly checks
info.supportsTemperature !== false. Consider updatingstreamAnthropicMessage(around line 288) to use the exact same logic for consistency, in case a future Anthropic-format model explicitly disables temperature.💡 Proposed consistency update (outside this hunk)
private async *streamAnthropicMessage( // ... - temperature: this.supportsTemperature(modelId) ? (temperature ?? 1.0) : undefined, + temperature: info.supportsTemperature !== false && this.supportsTemperature(modelId) ? (temperature ?? 1.0) : undefined,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/opencode-go.ts` around lines 506 - 509, Update streamAnthropicMessage to apply the same temperature condition as the non-streaming Anthropic path: require both info.supportsTemperature !== false and this.supportsTemperature(modelId) before using the provided temperature or 1.0; otherwise pass undefined.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/api/providers/openai-compatible.ts`:
- Around line 221-234: Update the temperature expression in completePrompt to
resolve model.temperature before this.config.temperature and the 0 default,
while preserving the existing supportsTemperature check and undefined behavior
for unsupported models. Match the resolution used by createMessage and use the
model returned by getModel.
In `@src/api/providers/opencode-go.ts`:
- Around line 506-509: Update streamAnthropicMessage to apply the same
temperature condition as the non-streaming Anthropic path: require both
info.supportsTemperature !== false and this.supportsTemperature(modelId) before
using the provided temperature or 1.0; otherwise pass undefined.
In `@src/api/providers/vercel-ai-gateway.ts`:
- Around line 109-112: Expand package-local unit tests for
src/api/providers/vercel-ai-gateway.ts:109-112 to verify reasoning_content and
reasoning deltas yield reasoning chunks, and for
src/api/providers/vercel-ai-gateway.ts:143-170 to verify Kimi K3 and
optional-reasoning completePrompt payload serialization. Add equivalent coverage
in src/api/providers/zoo-gateway.ts:258-261 and
src/api/providers/zoo-gateway.ts:305-332, covering reasoning-delta emission and
both completePrompt payload variants. Use the existing gateway streaming and
completePrompt test helpers without changing production behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9c3e3961-676a-498e-badb-b742d77a7207
📒 Files selected for processing (16)
packages/types/src/__tests__/opencode-go.test.tspackages/types/src/providers/moonshot.tspackages/types/src/providers/opencode-go.tssrc/api/providers/__tests__/moonshot.spec.tssrc/api/providers/__tests__/opencode-go.spec.tssrc/api/providers/__tests__/vercel-ai-gateway.spec.tssrc/api/providers/__tests__/zoo-gateway.spec.tssrc/api/providers/fetchers/__tests__/opencode-go.spec.tssrc/api/providers/fetchers/__tests__/vercel-ai-gateway.spec.tssrc/api/providers/fetchers/vercel-ai-gateway.tssrc/api/providers/openai-compatible.tssrc/api/providers/opencode-go.tssrc/api/providers/vercel-ai-gateway.tssrc/api/providers/zoo-gateway.tssrc/api/transform/__tests__/ai-sdk.spec.tssrc/api/transform/ai-sdk.ts
navedmerchant
changed the title
github-actions
Bot
added
has-conflicts
github-actions
Bot
added
awaiting-review
- Honor requiredReasoningEffort in getOpenAiReasoning so Kimi K3 always sends reasoning_effort=max, matching the Roo provider precedent - Send resolved reasoning params in completePrompt and the non-streaming createMessage branch - Skip explicit cache_control breakpoints for Moonshot (automatic caching) via a useExplicitCacheBreakpoints hook - Rewrite the K3 moonshot specs for the OpenAI SDK handler path and add coverage for required-effort resolution and the non-streaming branch
github-actions
Bot
removed
the
has-conflicts
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
src/api/providers/opencode-go.ts (1)
136-144: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winPreserve the normal reasoning fallback when required metadata is incomplete.
If
requiredReasoningEffortis true butinfo.reasoningEffortis absent, this expression discardsparams.reasoningEffortand sends no reasoning effort. This diverges fromgetOpenAiReasoning()’s fallback behavior. Useinfo.reasoningEffort ?? params.reasoningEffort, or enforce and test the metadata invariant.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/opencode-go.ts` around lines 136 - 144, Update the reasoningEffort assignment in the provider model return flow to use info.reasoningEffort only when it is present, otherwise fall back to params.reasoningEffort even when requiredReasoningEffort is true. Preserve the existing getOpenAiReasoning() fallback behavior.src/api/providers/openai.ts (1)
312-323: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winForward per-call timeout/cancellation options to the SDK requests.
CompletePromptOptionsdefinesabortSignalandtimeoutMs, butcompletePromptonly uses them as type-level parameters. Pass them to the SDK call options so callers with stop/cancel behavior can actually abort the request.
src/api/controllers/openai.ts#L312-l: pass{ signal: options.abortSignal, timeout: options.timeoutMs }through the OpenAI request options, accounting for the Azure path if needed.src/api/controllers/opencode-go.ts#L491-l: pass the same options through both the Anthropic and OpenAI non-streaming completions.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/openai.ts` around lines 312 - 323, Forward per-call cancellation and timeout settings in completePrompt by adding options.abortSignal as the SDK signal and options.timeoutMs as the SDK timeout for the OpenAI request, including the Azure request path as needed. Apply the same signal and timeout forwarding to both Anthropic and OpenAI non-streaming completion calls in opencode-go’s corresponding completion flow.src/api/providers/moonshot.ts (1)
31-45: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winRetain a structural
maxTokensfor unknown Moonshot models.
getMoonshotModels()assigns a safe fallback for unknown model IDs, butresolveModelInfo()then clearsmaxTokensin the handler. SinceaddMaxTokensIfNeeded()only usesmodelInfo.maxTokenswhenmodelMaxTokensis absent, dynamically discovered Moonshot models without a user override send no usablemax_tokensto an API that requires it. Strip pricing, but keep the handler’s fallback from resolving toundefined.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/providers/moonshot.ts` around lines 31 - 45, Update MoonshotProvider.resolveModelInfo so unknown model IDs retain the fallback maxTokens from moonshotModels[moonshotDefaultModelId] instead of setting it to undefined. Continue clearing the pricing fields for unknown models, and preserve the existing known-model behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/api/transform/__tests__/reasoning.spec.ts`:
- Around line 550-594: Update the reasoning tests around getOpenAiReasoning so
they exercise the intended branches: in the “default settings” test, override
baseOptions.reasoningEffort with undefined; in the fallback test, configure
requiredWithoutEffort with a supported reasoning effort and provide a
user-selected reasoning effort in the options. Keep the existing assertions and
test intent unchanged.
---
Outside diff comments:
In `@src/api/providers/moonshot.ts`:
- Around line 31-45: Update MoonshotProvider.resolveModelInfo so unknown model
IDs retain the fallback maxTokens from moonshotModels[moonshotDefaultModelId]
instead of setting it to undefined. Continue clearing the pricing fields for
unknown models, and preserve the existing known-model behavior.
In `@src/api/providers/openai.ts`:
- Around line 312-323: Forward per-call cancellation and timeout settings in
completePrompt by adding options.abortSignal as the SDK signal and
options.timeoutMs as the SDK timeout for the OpenAI request, including the Azure
request path as needed. Apply the same signal and timeout forwarding to both
Anthropic and OpenAI non-streaming completion calls in opencode-go’s
corresponding completion flow.
In `@src/api/providers/opencode-go.ts`:
- Around line 136-144: Update the reasoningEffort assignment in the provider
model return flow to use info.reasoningEffort only when it is present, otherwise
fall back to params.reasoningEffort even when requiredReasoningEffort is true.
Preserve the existing getOpenAiReasoning() fallback behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: c318ec55-9e6a-4cc4-b2bd-3f6bac2afae2
📒 Files selected for processing (11)
README.mdpackages/types/src/providers/moonshot.tssrc/api/providers/__tests__/moonshot.spec.tssrc/api/providers/moonshot.tssrc/api/providers/openai-compatible.tssrc/api/providers/openai.tssrc/api/providers/opencode-go.tssrc/api/providers/vercel-ai-gateway.tssrc/api/providers/zoo-gateway.tssrc/api/transform/__tests__/reasoning.spec.tssrc/api/transform/reasoning.ts
🚧 Files skipped from review as they are similar to previous changes (5)
- packages/types/src/providers/moonshot.ts
- src/api/providers/tests/moonshot.spec.ts
- src/api/providers/openai-compatible.ts
- src/api/providers/vercel-ai-gateway.ts
- src/api/providers/zoo-gateway.ts
| it("should send the model's required effort even when the user disables reasoning", () => { | ||
| const requiredModel: ModelInfo = { | ||
| ...baseModel, | ||
| supportsReasoningEffort: ["max"], | ||
| requiredReasoningEffort: true, | ||
| reasoningEffort: "max", | ||
| } | ||
|
|
||
| const options = { | ||
| ...baseOptions, | ||
| model: requiredModel, | ||
| reasoningEffort: "disable" as const, | ||
| settings: { reasoningEffort: "disable", enableReasoningEffort: false } as ProviderSettings, | ||
| } | ||
|
|
||
| const result = getOpenAiReasoning(options) | ||
|
|
||
| expect(result).toEqual({ reasoning_effort: "max" }) | ||
| }) | ||
|
|
||
| it("should send the model's required effort with default settings", () => { | ||
| const requiredModel: ModelInfo = { | ||
| ...baseModel, | ||
| supportsReasoningEffort: ["max"], | ||
| requiredReasoningEffort: true, | ||
| reasoningEffort: "max", | ||
| } | ||
|
|
||
| const options = { ...baseOptions, model: requiredModel } | ||
|
|
||
| const result = getOpenAiReasoning(options) | ||
|
|
||
| expect(result).toEqual({ reasoning_effort: "max" }) | ||
| }) | ||
|
|
||
| it("should fall back to normal handling when requiredReasoningEffort has no model effort", () => { | ||
| const requiredWithoutEffort: ModelInfo = { | ||
| ...baseModel, | ||
| requiredReasoningEffort: true, | ||
| } | ||
|
|
||
| const result = getOpenAiReasoning({ ...baseOptions, model: requiredWithoutEffort, settings: {} }) | ||
|
|
||
| expect(result).toBeUndefined() | ||
| }) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Make the new tests exercise the branches they describe.
The “default settings” case still inherits reasoningEffort: "medium" from baseOptions, so it does not test an unset user selection. The fallback case has no supportsReasoningEffort, causing the normal path to return undefined regardless of fallback logic. Set the first case’s effort to undefined, and give the fallback model a supported effort plus a user-selected effort.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/api/transform/__tests__/reasoning.spec.ts` around lines 550 - 594, Update
the reasoning tests around getOpenAiReasoning so they exercise the intended
branches: in the “default settings” test, override baseOptions.reasoningEffort
with undefined; in the fallback test, configure requiredWithoutEffort with a
supported reasoning effort and provide a user-selected reasoning effort in the
options. Keep the existing assertions and test intent unchanged.
|
Closing this PR — the scope crept beyond the original intent (it grew to cover the generic OpenAI Compatible provider, Vercel AI Gateway, Zoo Gateway, and AI SDK message conversion on top of the core catalog additions). Superseded by a new, narrower PR that adds Kimi K3 to just the Moonshot and OpenCode Go providers. The merge-conflict resolution and the K3 correctness fixes from this branch (required |
2 participants
Related GitHub Issue
Closes #932
Description
Adds Kimi K3 support only where provider-specific model identifiers are confirmed:
kimi-k3to the direct Moonshot and curated OpenCode Go catalogs with provider-advertised context limits, 128K output, multimodal input, automatic prompt caching, required max reasoning, reasoning preservation, and confirmed pricing.moonshotai/kimi-k3metadata from Vercel AI Gateway; Zoo Gateway inherits the same normalized capabilities.reasoning_contentacross tool-call continuations, and avoid explicit cache breakpoints for automatic caching.moonshot.spec.tssuite.Intentionally does not add speculative static entries for Fireworks, Baseten, Bedrock, or Vertex because confirmed provider-specific IDs are unavailable. OpenRouter and other dynamic/user-supplied catalogs require no static addition.
Test Procedure
Validation completed for this change set:
@roo-code/typestests passed.Reviewers can verify the focused catalog, fetcher, provider, and converter specs modified by this PR and confirm Kimi K3 requests omit temperature, use max reasoning, preserve reasoning through tool calls, and omit explicit cache breakpoints.
Pre-Submission Checklist
Screenshots / Videos
Not applicable; this change has no UI impact.
Documentation Updates
Additional Notes
No changeset is included, per repository convention. The commit contains only the Kimi K3 implementation and focused tests.
Get in Touch
GitHub: @navedmerchant
Summary by CodeRabbit
temperaturefrom being sent when a model does not support it.