Add hardened OpenAI-compatible LLM provider

[Chris0Jeky]

Refs #1306.

This is the clean replacement for parked PR #1505. It leaves the parked PR and its discussion untouched while replaying the useful provider work onto current main and addressing the documented review findings.

Summary

• add a distinct, config-gated OpenAICompatible provider for public OpenAI Chat Completions endpoints
• support buffered completion, structured extraction, board context, real upstream SSE, bounded response parsing, optional non-secret gateway headers, and explicit buffered-stream degradation
• preserve the landed direct-only telemetry controls and add fixed-origin egress plus dispatch tracking for the compatible registered client
• settle quota reservations from proven dispatch state, keep Polly and companion circuit states independent and generation-safe, and persist only sanitized error placeholders
• document the exact localhost development/test contract, provider inventories, deployment configuration, and verification evidence

Verification

• focused Application: 136 passed, 0 failed, 0 skipped
• focused API: 80 passed, 0 failed, 0 skipped
• full backend: 7,660 passed, 5 intentional skips, 0 failed
  • Domain 1,636
  • Application 3,678
  • API 2,189 + 4 skips
  • CLI 100
  • Architecture 22 + 1 skip
  • Integration 35
• docs governance: passed
• golden-principles governance: passed
• GitHub-operations governance: passed
• appsettings JSON parse: passed
• Docker Compose interpolation/config validation: passed with local non-secret validation placeholders
• git diff --check origin/main..HEAD: passed
• every PR commit carries a Signed-off-by trailer

Human validation still required

A maintainer-supplied compatible-provider key and a visibly incremental stream through the real Taskdeck UI remain required before issue closure or a claim of live-provider UX readiness. Deterministic loopback transport tests do not satisfy either gate.

This PR intentionally uses Refs #1306, not a closing keyword, until that human gate is complete.

[Chris0Jeky]

Exact-head review request

Head: 1f529604aabe80a422f38c930658f151e860d915
Base: 2daeee9449cbdc454436df2a7acc702d3ecf5034

Local gates are green: focused Application 136/136, focused API 80/80, full serialized backend 7,660 passed / 5 intentional skips / 0 failed, docs/golden/GitHub-operations governance, compose/config parsing, diff, and DCO.

The maintainer vendor-key and visibly incremental real-UI stream remain explicit human gates; this PR does not close #1306.

@codex review exact head 1f52960, please.

[Chris0Jeky]

Hosted DCO triage

The advisory DCO failure is understood and reproducible. Commits 36e9efda, 83a3d390, and b3c5ca67 each contain an auto-generated # Conflicts: block after an otherwise identity-matching Signed-off-by line. The pinned KineticCafe/actions-dco v3 strict parser accepts trailers only in the final trailer block, so those three commit objects fail while the other four pass.

I will not force-push/rewrite this published branch. This PR remains open only long enough to complete its independent/Codex finding pass. After the queued mainline changes land, I will publish a fresh signed successor from current origin/main, carry forward all valid findings, re-run exact-head gates, and only then close this PR as superseded. Do not merge #1537. The parked #1505 remains untouched until the successor is proven.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 1f529604aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Chris0Jeky]

Fresh adversarial exact-head review — findings

Reviewed head: 1f529604aabe80a422f38c930658f151e860d915
Base: 2daeee9449cbdc454436df2a7acc702d3ecf5034

I inspected the full replacement diff, the PR conversation/reviews/threads, all parked #1505 review discussion, and the #1306 acceptance remainder. I also reran focused exact-tree tests: Application 163/163, API 81/81, Architecture 2/2; git diff --check is clean.

MEDIUM — preserve the existing binary constructor contract for EgressEnvelopeHandler

backend/src/Taskdeck.Application/Services/EgressEnvelopeHandler.cs:22-26 replaces the existing public three-parameter constructor

EgressEnvelopeHandler(
    IEgressRegistry,
    ILogger<EgressEnvelopeHandler>?,
    string?)

with a four-parameter constructor whose new followRedirects argument is optional. That is source-compatible but not binary-compatible: optional arguments are supplied by the caller at compile time, so an already-compiled consumer still references the exact three-parameter .ctor, which no longer exists and will fail with MissingMethodException.

Keep the original exact three-parameter overload and delegate it to a new four-parameter overload (or another internal construction path), then add an API-compatibility/reflection regression for the original signature. This is the same public-constructor compatibility standard the PR correctly applies to LlmTokenEvent.

LOW — the carried Testing Guide anchor fix is still missing

docs/TESTING_GUIDE.md:540 and :1943 still link to #current-verified-totals-2026-04-25, but the live heading at line 13 is Current Verified Totals (2026-05-16), so both links are broken. This was explicitly recorded in the #1306 acceptance remainder after #1505 and should be corrected in the clean successor that carries those findings.

No CRITICAL or HIGH findings in this pass. The known hosted DCO failure already has a sound no-rewrite supersession plan on this PR; do not merge #1537.

[Chris0Jeky]

Focused quota/cancellation follow-up — additional findings

Exact head remains 1f529604aabe80a422f38c930658f151e860d915. A separate focused pass over the quota, cancellation, and persistence seam found two additional defects:

HIGH — oversized terminal events discard authoritative usage and the persisted terminal outcome

OpenAiCompatibleLlmProvider.StreamCoreAsync permits a configured compatible response budget up to 4 MiB and can emit a valid buffered/non-SSE response as one terminal LlmTokenEvent carrying content, provider/model, and authoritative TokensUsed. In ChatService.StreamResponseAsync (ChatService.cs:670-681), the 1 MiB safety-cap branch executes before copying that terminal metadata. It yields a local error and breaks, but leaves tokensUsed unset and never marks persistEmptyTerminalOutcome.

For example, with an allowed 2–4 MiB provider response budget, a 1.1 MiB non-SSE response carrying total_tokens: 100000 produces a visible safety error, persists no assistant/error placeholder on reload, and commits only the reservation estimate (normally 2,000) in the finally block instead of the already-known authoritative usage. That combines reload-visible outcome loss with a repeatable quota undercharge at an allowed configuration.

Capture terminal provider/model/usage before applying the content cap, persist the sanitized safety placeholder/degraded outcome, and commit the authoritative count. Add a regression for an oversized terminal event with nonzero usage and no prior content.

MEDIUM — authoritative SSE usage is forgotten if a later event/read fails

OpenAiCompatibleLlmProvider.cs:461-462 stores a positive usage-only SSE chunk in tokensUsedSeen, but the subsequent parsed-error paths at lines 455-458 and 493-497 call TerminalError(parsed.Error) without that count. The outer post-header exception conversion at lines 301-319 likewise creates a usage-less terminal event even when StreamCoreAsync had already observed usage.

Thus a stream that reports total_tokens: 5000 and then emits an upstream error event or suffers a socket/read failure before [DONE] is persisted as an error but quota settles to the reservation estimate (for example 2,000), discarding authoritative usage already received. Thread tokensUsedSeen through every terminal-error path (including post-usage exceptions) and add usage-then-error/read-failure regressions.

The focused pass otherwise confirms the repaired pre-dispatch/post-dispatch cancellation and error-only settlement cases, including sanitized empty terminal persistence.

[Chris0Jeky]

Circuit-state follow-up — additional MEDIUM finding

Exact head remains 1f529604aabe80a422f38c930658f151e860d915.

MEDIUM — a pre-dispatch Polly rejection can be counted as a new companion failure and extend the outage

The compatible provider's generic buffered catch (OpenAiCompatibleLlmProvider.cs:204-212) classifies every unexpected exception as a transport failure and calls RecordProviderFailure. That includes Polly's BrokenCircuitException<HttpResponseMessage>, even though the registered handler ordering means the open Polly policy rejects before the dispatch tracker/transport is reached.

This is reachable when the two independent states diverge through cancellation:

1. A threshold-crossing 5xx makes Polly Open before the response is returned.
2. The caller cancels while the provider reads that 5xx body. The caller-cancellation catch rethrows, and AbandonProviderRequest is deliberately a no-op for the ordinary companion lease (CircuitBreakerStateTracker.cs:204-210), leaving Polly Open while the companion state remains Closed.
3. A later request passes companion TryEnterProviderRequest, but Polly rejects it pre-dispatch. The generic catch records that rejection as a new companion transport failure.
4. At threshold 1, a request made just before Polly's original cooldown expires opens the companion circuit for a fresh full cooldown, nearly doubling the effective outage without another upstream attempt.

Do not feed a pre-dispatch BrokenCircuitException into the companion failure counter. Preserve half-open lease cleanup if that state can be reached, and add a registered-pipeline regression covering 5xx → cancellation after headers → later Polly rejection, asserting no fresh companion cooldown is started.