fix(inference/ollama): use prompt-guided tool calling so skills work on Ollama (#3098 sub-issue 3)#3229
Conversation
…on Ollama (tinyhumansai#3098) Ollama's `/v1/chat/completions` OpenAI-compat endpoint rejects the `tools` parameter with HTTP 400 "unsupported parameter: tools" for many models. The existing retry path in `compatible.rs` detects this and retries with `tools: None` — i.e. strips tools entirely. The model then replies in natural language but can't actually invoke any tool. Skills are markdown instruction documents the agent reads on demand; many skills tell the model to "use the file_read tool" or "run shell X". When tools are silently stripped, the model talks about doing the thing but doesn't do it — sub-issue 3 of tinyhumansai#3098: "skills not running." The fix: report `native_tool_calling = false` from the Ollama branch of `OpenAiCompatibleProvider::capabilities()`. The agent harness then embeds tool specs as text in the system prompt and parses tool calls out of the response (the `ToolsPayload::PromptGuided` path), which any chat model can follow — no upstream `tools` array, no 400, no silent retry-without-tools. Implementation: - Add `native_tool_calling: bool` field on `OpenAiCompatibleProvider` (defaults to `true`, preserving every existing call site). - Add `with_native_tool_calling(bool)` builder. - `capabilities()` returns the stored flag instead of hardcoded `true`. - `factory::make_ollama_provider` constructs the provider directly and calls `.with_native_tool_calling(false)`. LM Studio, OpenHuman backend, BYOK cloud providers (OpenAI/Anthropic/etc.), and direct `OpenAiCompatibleProvider::new` callers are unaffected — they keep the default `true`. Tests: - 3 new unit tests in `compatible_tests.rs` pin the builder behavior: default true, `with_native_tool_calling(false)` flips it, repeated calls are idempotent. - 2 new factory tests pin the integration: `ollama:<model>` reports `native_tool_calling=false`; `lmstudio:<model>` keeps it `true`. Scope: this fix benefits every Ollama user (desktop, all channels), not just Telegram. With PR tinyhumansai#3217, Telegram now reliably routes to Ollama when the user picks it — making this latent bug actively visible to those users. Closes sub-issue 3 of tinyhumansai#3098.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds a configurable native_tool_calling flag to OpenAiCompatibleProvider (default true) with a builder to toggle it; capabilities() and supports_native_tools() reflect the flag. make_ollama_provider now disables native tool calling for Ollama; tests cover the flag and factory behavior. ChangesNative Tool Calling Capability Configuration
🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
coderabbitai
Bot
added
rust-core
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/inference/provider/compatible.rs`:
- Around line 210-217: The supports_native_tools() accessor must reflect the new
native_tool_calling flag so both capability signals agree: update the
supports_native_tools(&self) -> bool implementation to return
self.native_tool_calling (or otherwise consult the same state used by
capabilities()) instead of always true; ensure with_native_tool_calling,
capabilities(), and supports_native_tools all use the same native_tool_calling
field so callers using the old API get the corrected behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b576c0c7-2895-40f5-a15a-e7271181f6a6
📒 Files selected for processing (4)
src/openhuman/inference/provider/compatible.rssrc/openhuman/inference/provider/compatible_tests.rssrc/openhuman/inference/provider/factory.rssrc/openhuman/inference/provider/factory_tests.rs
|
CI failure on the first attempt was a flake — confirmed by rerunning lanes 1 and 2 of The three tests that flaked:
Failure shape on all three: This PR's diff is scoped to |
…tools (tinyhumansai#3098) CodeRabbit review on PR tinyhumansai#3229 caught that `OpenAiCompatibleProvider` overrides `Provider::supports_native_tools()` with a hardcoded `true` (`compatible.rs:1961`), bypassing the new `native_tool_calling` field. The harness's actual gate for native-vs-prompt-guided tool dispatch lives at `traits.rs:415` and reads `supports_native_tools()`, not `capabilities().native_tool_calling` directly — so the original PR's opt-out flipped the capability signal but the agent harness still sent a `tools` array to Ollama and got HTTP 400. Update the override to return `self.native_tool_calling` so all three signals (`with_native_tool_calling(...)` builder, `capabilities()`, `supports_native_tools()`) agree. Cloud providers default to `true` exactly as before; only the Ollama factory branch now actually routes through the prompt-guided path the original PR intended. Regression test pins the invariant: after `with_native_tool_calling(false)`, both `supports_native_tools()` and `capabilities().native_tool_calling` must return `false`.
2 participants
Summary
native_tool_calling = false. The agent harness embeds tool specs in the system prompt as text and parses tool calls out of the response (ToolsPayload::PromptGuided), which works across the Ollama model zoo.OpenAiCompatibleProvider::with_native_tool_calling(bool)builder. Default staystrue— every other provider (OpenAI, OpenHuman backend, BYOK slugs, LM Studio) is unchanged.Problem
Sub-issue 3 of #3098 reports "Skills fail to execute" — most visibly when the user routes a channel (e.g. Telegram) through an Ollama model.
Trace:
file_readtool" or "runshellX".OpenAiCompatibleProvider::capabilities()previously hardcodednative_tool_calling: truefor every OpenAI-compat endpoint, including Ollama (compatible.rs:1248-1253).toolsparameter with HTTP 400 ("unsupported parameter: tools") for many models.compatible.rs:1748-1775) detects this and retries withtools: None— i.e. strips tools entirely.PR #3217 (sub-issue 1) makes this latent bug actively visible to Telegram users: Telegram now reliably routes to Ollama when
chat_provider = "ollama:…"is set, so every skill-bearing turn hits this path.Solution
Stop pretending Ollama supports native tool schemas. Build the Ollama provider with
native_tool_calling = falsefrom the start so the agent harness uses prompt-guided text tool specs — embedded in the system prompt, parsed back out of the response. Works on every Ollama model with no upstreamtoolsarray, no 400, no silent retry-without-tools.Concretely:
native_tool_calling: boolfield onOpenAiCompatibleProvider(defaults totrue, preserving every existing construction path).with_native_tool_calling(bool)builder method.capabilities()to return the stored flag instead of hardcodedtrue.factory::make_ollama_providernow constructs the provider directly (no helper indirection) and chains.with_native_tool_calling(false).LM Studio, OpenHuman backend, BYOK cloud providers (
openai:…,anthropic:…, etc.), and directOpenAiCompatibleProvider::newcallers are untouched — they keep thetruedefault. Only the explicit Ollama factory branch opts out.Submission Checklist
compatible_tests.rs(default true, override false, idempotent) + 2 factory tests (ollama_provider_opts_out_of_native_tool_calling,lmstudio_provider_keeps_native_tool_calling_enabled).native_tool_callingfield is read bycapabilities()(3 unit tests) and the Ollama factory branch is exercised by the new factory test; the LM Studio test is the regression guard.N/A: behavior fix on an existing capability (Ollama provider); no new feature row.## Related—N/A: no matrix row added or removed.capabilities(); no network calls.docs/RELEASE-MANUAL-SMOKE.md) —N/A: not a release-cut surface; behaves identically for cloud users.Closes #NNNin the## Relatedsection — see below (Telegram interface ignores local model selection + can't commit files or run skills via Telegram on macOS #3098 has 4 sub-issues; this PR closes sub-issue 3 only, issue stays open).Impact
toolsand got rejected.Related
/modeloverride inchannels/routes.rs:get_or_create_provider— it currently always rebuilds a cloud provider regardless of slug. Out of scope here.AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
fix/3098-ollama-prompt-guided-tools6c504c7edd00ae84f74446a9b7b119f54207b67cValidation Run
pnpm --filter openhuman-app format:check—N/A: Rust-only change.pnpm typecheck—N/A: no TS/TSX touched.cargo test --lib -p openhuman inference::provider→ 386 passed, 0 failed, 2 ignored;cargo test --lib -p openhuman channels:: agent::harness→ 1489 passed, 0 failed (regression suite). All five new tests pass.cargo fmt --checkclean;cargo check --libclean (pre-existing warnings only).N/A: app/src-tauri not touched.Validation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
toolsarray on outbound chat requests. The agent harness embeds tool specs in the system prompt as text and parses tool calls out of the response — the existing prompt-guided code path.Parity Contract
OpenAiCompatibleProvider::newcallers) inherits thetruedefault fornative_tool_calling. Behavior is byte-identical pre/post for those paths. Verified by the LM Studio regression test and by the unchangedcapabilities_reports_native_tool_callingtest.err_supports_no_tools_retrypath (compatible.rs:1748-1775) andis_native_tool_schema_unsupporteddetector (compatible.rs:780-795) are untouched — they remain as a safety net for any other OpenAI-compat endpoint that rejectstools.Duplicate / Superseded PR Handling
Summary by CodeRabbit
New Features
Bug Fixes
Tests