fix(memory): thread BYO embedding credential — stop Cohere empty-key 401 flood (#3354)#3358
Conversation
…sai#3354) Cohere's hosted /v2/embed always needs a bearer; with an empty key it returned 401 "no api key supplied" on every call, and since 401 is not retryable each embed attempt reported an error. The memory pipeline re-embeds per document, flooding Sentry (TAURI-RUST-52S, 8.7k events from one misconfigured user). Fast-fail when the key is empty/whitespace with a classifiable "API key not set" message so no request is fired and the error is demoted to a breadcrumb. Scoped to Cohere — the OpenAI-compatible provider legitimately supports keyless local/custom endpoints.
…eyMissing (tinyhumansai#3354) Defense-in-depth for TAURI-RUST-52S: demote any residual Cohere 401 of this shape (older clients without the call-site guard, or a present-but-rejected key) so the flood stays out of Sentry. The existing arm only matched 'api key not set' / 'missing api key' / 'no api key is configured'; Cohere's exact body wording fell through.
📝 WalkthroughWalkthroughAdds an early empty/whitespace API-key guard for Cohere embeds, extends the observability classifier to match Cohere's "no api key supplied" 401 text, and threads resolved embedding API keys into memory factory creation and callers so keyed embedding providers receive BYO credentials. ChangesCohere embedding API key guard and error classification
Memory factory and runtime wiring for embedding API keys
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsStopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a Comment |
…tinyhumansai#3354) The memory pipeline built its embedder via the keyless `create_embedding_provider`, so the user's configured API key was never passed for any provider. A user who selected Cohere — even with a valid key — sent an empty `Bearer ` and got a guaranteed 401 "no api key supplied" on every embed (TAURI-RUST-52S), and the same gap silently broke BYO OpenAI / Voyage memory embeddings. Resolve the stored credential at the three call sites that hold the full `Config` (runtime_node, agent session builder, channels startup) via `resolve_api_key`, thread it through `create_memory_with_local_ai` / `create_memory_full`, and build the provider with `create_embedding_provider_with_credentials`. cloud/managed/ollama/none ignore the key; keyed providers now authenticate. The key is never logged. `create_memory` (tests + migration, no Config in scope) passes an empty key, preserving prior behavior.
…#3354) Lock the round-trip the memory factory now depends on: a stored `embeddings:<slug>` key resolves back, an unconfigured provider stays empty (no cross-bleed), and `custom:<url>` normalizes to the `custom` slug.
Summary
CohereEmbedding::embed()on an empty key (no wasted request), and classify Cohere's"no api key supplied"401 asApiKeyMissingso any residual event is demoted.Problem
A user selected Cohere as their embedding provider. Every memory embed produced
401 "no api key supplied"; not retryable, so each call reported an error and the per-document memory pipeline flooded Sentry.The real cause:
memory_store/factories.rsbuilt the embedder viacreate_embedding_provider(provider, model, dims)— the keyless factory. The user's configured key (stored underembeddings:<slug>) was never resolved or passed, for any provider. The RPC "test embed" path (embeddings/rpc.rs) did it correctly viaresolve_api_key+create_embedding_provider_with_credentials; the memory pipeline did not. So even a correctly-keyed Cohere setup got empty-bearer 401s.The Sentry event escaped demotion because
expected_error_kind()only matched"api key not set"/"missing api key"/"no api key is configured"— Cohere's"no api key supplied"wording matched none.Solution
Real fix — thread the credential (
memory_store/factories.rs+ 3 call sites):create_memory_full/create_memory_with_local_aitake anembedding_api_key: &strand build viacreate_embedding_provider_with_credentials.Config—runtime_node::ops,agent::harness::session::builder,channels::runtime::startup— resolve it withresolve_api_key(config, &config.memory.embedding_provider).create_memory(tests + migration, noConfigin scope) passes"", preserving prior behavior. The key is never logged.Safety net (defense-in-depth):
embeddings/cohere.rs: bail before the request when the key is empty/whitespace, with a classifiable"API key not set"message — covers genuinely-unconfigured setups without a wasted round-trip or a Sentry event.core/observability.rs: add"no api key supplied"to theApiKeyMissingarm for any residual 401 (older clients without the guard, or a present-but-rejected key).Sibling: #2898 (429 embed-flood fix, same class).
Submission Checklist
resolve_api_keycredential round-trip + custom-slug normalization; empty/whitespace Cohere key guard; verbatim Cohere 401-body classification; existing over-suppression guard still greencreate_memory_fullN/A: behaviour-only change(no feature row added/removed/renamed)## Related—N/A: no matrix feature touchedN/A: no release-cut surface touchedCloses #NNNin## RelatedImpact
Related
AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
fix/3354-cohere-empty-key-flood9eabd86e7Validation Run
pnpm --filter openhuman-app format:check— noapp/files changedpnpm typecheck— no TypeScript changedembeddings::rpc::tests(2 ok),embeddings::cohere(7 ok),memory_store::factories(16 ok),core::observability::tests::classif*(46 ok)cargo fmt --check+cargo check(exit 0)app/src-taurifiles changedValidation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
Parity Contract
create_memory(tests/migration) keeps empty-key path; correctly-keyed providers' 429 backoff tests still pass.create_embedding_provider_with_credentialscovers the same provider arms as the keyless factory;custom:<url>prefix self-stripped (custom_endpoint stays None).Duplicate / Superseded PR Handling
Summary by CodeRabbit
Bug Fixes
Improvements
Tests