Skip to content

[fix] - harness console follows the Ollama→LM Studio fallback like /query and /health#651

Merged
cgfixit merged 10 commits into
mainfrom
claude/cyclaw-optimize-harness-fallback
Jul 24, 2026
Merged

[fix] - harness console follows the Ollama→LM Studio fallback like /query and /health#651
cgfixit merged 10 commits into
mainfrom
claude/cyclaw-optimize-harness-fallback

Conversation

@cgfixit

@cgfixit cgfixit commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Proposed changes

Phase 1's primary compat finding — the one real gap in the graceful LM Studio fallback story. The 8-minute scan verified the hot paths are already provider-generic (llm/client.py posts only OpenAI-body /chat/completions, utils/health.py probes generic GET /models, mcp_hybrid_server.py has zero LLM dependence, graph.py's offline_best_effort/local_llm nodes go through the same client) and that resolve_local_backend() + the config.yaml fallback block already implement primary→fallback failover for gate/graph/health. Except the harness console: create_app read models.local_llm.{base_url,model} directly, so with fallback.enabled: true and Ollama down, /query and /health switched to LM Studio while the console kept targeting the dead :11434.

  • harness/server.py: route the console through the same llm.client.resolve_local_backend() (new _resolve_backend shim: no network probe when fallback is disabled — the shipped default — and empty/unreadable config still degrades to the Ollama default instead of failing app build). /api/status now reports the resolved provider/base_url/model instead of the raw config primary.
  • harness/ollama.py: unreachable-error text was hardcoded "is Ollama running?" — now provider-neutral (details already carry the actual base_url).
  • Regression test: fallback enabled + failing primary probe → /api/status must report the lmstudio backend. Proven to fail on the old code (revert-run) and pass on the fix.

Invariant / Governance Impact: None. harness/ (out-of-band) now imports llm.client — a shared library module like utils/, not one of the core three; I6 forbids imports between gate.py/graph.py/mcp_hybrid_server.py and the out-of-band packages, which is untouched. invariant-guard: 28 passed / 0 failed. No graph edge, gate, sanitizer, or soul path changed.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)

Scope note: Out-of-band harness layer (Ollama/LM Studio compat).

Benefits / why

  • Completes the operator's fallback contract: with fallback.enabled: true, an LM-Studio-only (or Ollama-down) host now gets a working console, matching /query//health behavior — the pre-Ollama-migration workflow keeps working as a backup.
  • /api/status stops lying about which backend chat actually talks to.
  • Zero behavior change for the shipped default (fallback.enabled: false → primary returned with no probe; all 26 pre-existing harness tests pass unchanged).

Risks to monitor

  • The resolver's process-lifetime cache (documented in llm/client.py) means a fallback selection sticks until restart even if the primary recovers — a deliberately unchanged, pre-existing property shared with gate.py; reset_local_backend_cache() exists for tests/reload. Flagged in the scan as an operability note, deliberately not "fixed" here (speculative TTL machinery under feature freeze).
  • Note for enablers: with base_url pointed at LM Studio, /health will (correctly) flag a model: value that isn't in the server's /models list — LM Studio ids differ from Ollama tags; set fallback.model per the config comment.

Further comments

Verified: ruff clean; pytest tests/test_harness.py27 passed; revert-proof (old server.py fails the new test, fix passes); invariant-guard 28/0. Scan also confirmed three of #415's known migration items are already resolved on main (guardrails config → 11434 with integration.py overriding from GuardrailsConfig; terminal.html message already says Ollama; agentic/config.py provider list already retired lmstudio) — ledger comment going to #415. Companion cleanup PR (orphaned mock_lmstudio.py + a localhost127.0.0.1 config consistency nit) follows separately.

🤖 Generated with Claude Code

https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT


Generated by Claude Code

claude added 3 commits July 23, 2026 04:25
…/health

create_app read models.local_llm.{base_url,model} directly instead of
routing through llm.client.resolve_local_backend, so with fallback.enabled
true and the primary (Ollama :11434) down, gate.py's /query and /health
correctly switched to the fallback (LM Studio :1234) while the harness
console kept targeting the dead primary — the one gap in the graceful
LM Studio fallback story. The console now resolves the active backend the
same way (no network probe when fallback is disabled, the shipped default),
/api/status reports the RESOLVED provider/base_url/model instead of the raw
config primary, and an empty/unreadable config still degrades to the Ollama
default instead of failing app build.

Also make the unreachable-model error provider-neutral ("local model server
unreachable") — the old text said "is Ollama running?" even when base_url
pointed at LM Studio; details already carry the actual base_url.

Regression test drives fallback.enabled=true with a failing primary probe
and asserts /api/status reports the lmstudio backend.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
Self-sourcing helpers instead of an extra create_app local: _resolve_backend
reads _llm_settings() itself and a _default_chat_client builds the
non-injected client; the chat endpoint reads the cached settings at request
time. Same behavior (the config read is lru-cached), same tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
Inline the lru-cached settings reads into the two tunable kwargs instead
of a closure-local dict; per-request cost is a dict lookup, not a config
re-parse (_get_config is @lru_cache(maxsize=8)).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
cgfixit added 7 commits July 24, 2026 11:35
Haha. Really a note to self might as well look into llama.cpp if ollama requires thought compared to lm studio or im overthinking what that last commit rant was about. Doesn't matter not but might be more difficult to change later when upgraded hardware makes perf increase noticeable
@cgfixit
cgfixit marked this pull request as ready for review July 24, 2026 16:35
@cgfixit
cgfixit merged commit 923bddc into main Jul 24, 2026
39 of 40 checks passed
@cgfixit
cgfixit deleted the claude/cyclaw-optimize-harness-fallback branch July 24, 2026 16:35
cgfixit pushed a commit that referenced this pull request Jul 26, 2026
…iddleware)

The harness console is a state-changing loopback app (POST /api/soul,
/api/model, /api/chat; readable /api/sessions history) that had no
TrustedHostMiddleware, no CORS, no auth — while gate.py deliberately added
TrustedHostMiddleware (PR #99) for the identical single-operator threat
model, because CORS only governs response *readability*: it does not stop a
DNS-rebinding page from driving a loopback server's state-changing POSTs
server-side. Add the same defense with a loopback-only allow-list
(127.0.0.1 / localhost / ::1) — deliberately tighter than gate.py's
LAN-inclusive CORS list, since the harness refuses any non-loopback bind.

Define the loopback allow-list once (_LOOPBACK_HOSTS) and enforce it at BOTH
the bind (main) and the Host header (middleware). Test fixtures now set an
allowed Host (the TestClient default "testserver" is correctly rejected);
new test asserts a non-loopback Host -> 400 on both a GET and a POST.

Stacked on #651 (both edit harness/server.py + tests/test_harness.py);
merge #651 first. I6 unaffected — harness imports llm.client + starlette,
never gate/graph/mcp. invariant-guard 28/0; full harness suite 28 passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138hmScu3tFU4v1jzvFUwVT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants