[fix] - harness console follows the Ollama→LM Studio fallback like /query and /health#651
Merged
Merged
Conversation
…/health
create_app read models.local_llm.{base_url,model} directly instead of
routing through llm.client.resolve_local_backend, so with fallback.enabled
true and the primary (Ollama :11434) down, gate.py's /query and /health
correctly switched to the fallback (LM Studio :1234) while the harness
console kept targeting the dead primary — the one gap in the graceful
LM Studio fallback story. The console now resolves the active backend the
same way (no network probe when fallback is disabled, the shipped default),
/api/status reports the RESOLVED provider/base_url/model instead of the raw
config primary, and an empty/unreadable config still degrades to the Ollama
default instead of failing app build.
Also make the unreachable-model error provider-neutral ("local model server
unreachable") — the old text said "is Ollama running?" even when base_url
pointed at LM Studio; details already carry the actual base_url.
Regression test drives fallback.enabled=true with a failing primary probe
and asserts /api/status reports the lmstudio backend.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
Self-sourcing helpers instead of an extra create_app local: _resolve_backend reads _llm_settings() itself and a _default_chat_client builds the non-injected client; the chat endpoint reads the cached settings at request time. Same behavior (the config read is lru-cached), same tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
Inline the lru-cached settings reads into the two tunable kwargs instead of a closure-local dict; per-request cost is a dict lookup, not a config re-parse (_get_config is @lru_cache(maxsize=8)). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
This was referenced Jul 23, 2026
Haha. Really a note to self might as well look into llama.cpp if ollama requires thought compared to lm studio or im overthinking what that last commit rant was about. Doesn't matter not but might be more difficult to change later when upgraded hardware makes perf increase noticeable
cgfixit
marked this pull request as ready for review
July 24, 2026 16:35
cgfixit
pushed a commit
that referenced
this pull request
Jul 26, 2026
…iddleware) The harness console is a state-changing loopback app (POST /api/soul, /api/model, /api/chat; readable /api/sessions history) that had no TrustedHostMiddleware, no CORS, no auth — while gate.py deliberately added TrustedHostMiddleware (PR #99) for the identical single-operator threat model, because CORS only governs response *readability*: it does not stop a DNS-rebinding page from driving a loopback server's state-changing POSTs server-side. Add the same defense with a loopback-only allow-list (127.0.0.1 / localhost / ::1) — deliberately tighter than gate.py's LAN-inclusive CORS list, since the harness refuses any non-loopback bind. Define the loopback allow-list once (_LOOPBACK_HOSTS) and enforce it at BOTH the bind (main) and the Host header (middleware). Test fixtures now set an allowed Host (the TestClient default "testserver" is correctly rejected); new test asserts a non-loopback Host -> 400 on both a GET and a POST. Stacked on #651 (both edit harness/server.py + tests/test_harness.py); merge #651 first. I6 unaffected — harness imports llm.client + starlette, never gate/graph/mcp. invariant-guard 28/0; full harness suite 28 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Phase 1's primary compat finding — the one real gap in the graceful LM Studio fallback story. The 8-minute scan verified the hot paths are already provider-generic (
llm/client.pyposts only OpenAI-body/chat/completions,utils/health.pyprobes genericGET /models,mcp_hybrid_server.pyhas zero LLM dependence,graph.py'soffline_best_effort/local_llmnodes go through the same client) and thatresolve_local_backend()+ theconfig.yamlfallbackblock already implement primary→fallback failover for gate/graph/health. Except the harness console:create_appreadmodels.local_llm.{base_url,model}directly, so withfallback.enabled: trueand Ollama down,/queryand/healthswitched to LM Studio while the console kept targeting the dead:11434.harness/server.py: route the console through the samellm.client.resolve_local_backend()(new_resolve_backendshim: no network probe when fallback is disabled — the shipped default — and empty/unreadable config still degrades to the Ollama default instead of failing app build)./api/statusnow reports the resolved provider/base_url/model instead of the raw config primary.harness/ollama.py: unreachable-error text was hardcoded"is Ollama running?"— now provider-neutral (detailsalready carry the actualbase_url)./api/statusmust report thelmstudiobackend. Proven to fail on the old code (revert-run) and pass on the fix.Invariant / Governance Impact: None.
harness/(out-of-band) now importsllm.client— a shared library module likeutils/, not one of the core three; I6 forbids imports betweengate.py/graph.py/mcp_hybrid_server.pyand the out-of-band packages, which is untouched.invariant-guard: 28 passed / 0 failed. No graph edge, gate, sanitizer, or soul path changed.Types of changes
Scope note: Out-of-band harness layer (Ollama/LM Studio compat).
Benefits / why
fallback.enabled: true, an LM-Studio-only (or Ollama-down) host now gets a working console, matching/query//healthbehavior — the pre-Ollama-migration workflow keeps working as a backup./api/statusstops lying about which backend chat actually talks to.fallback.enabled: false→ primary returned with no probe; all 26 pre-existing harness tests pass unchanged).Risks to monitor
llm/client.py) means a fallback selection sticks until restart even if the primary recovers — a deliberately unchanged, pre-existing property shared with gate.py;reset_local_backend_cache()exists for tests/reload. Flagged in the scan as an operability note, deliberately not "fixed" here (speculative TTL machinery under feature freeze).base_urlpointed at LM Studio,/healthwill (correctly) flag amodel:value that isn't in the server's/modelslist — LM Studio ids differ from Ollama tags; setfallback.modelper the config comment.Further comments
Verified:
ruffclean;pytest tests/test_harness.py→ 27 passed; revert-proof (oldserver.pyfails the new test, fix passes);invariant-guard28/0. Scan also confirmed three of #415's known migration items are already resolved on main (guardrails config → 11434 withintegration.pyoverriding fromGuardrailsConfig;terminal.htmlmessage already says Ollama;agentic/config.pyprovider list already retiredlmstudio) — ledger comment going to #415. Companion cleanup PR (orphanedmock_lmstudio.py+ alocalhost→127.0.0.1config consistency nit) follows separately.🤖 Generated with Claude Code
https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
Generated by Claude Code