Skip to content

fix(core): price OpenAI prompt-cache input at the documented cached rate (#13104) - #13160

Open
santhiprakash wants to merge 2 commits into
continuedev:mainfrom
santhiprakash:fix/13104-openai-cached-input-tokens
Open

fix(core): price OpenAI prompt-cache input at the documented cached rate (#13104)#13160
santhiprakash wants to merge 2 commits into
continuedev:mainfrom
santhiprakash:fix/13104-openai-cached-input-tokens

Conversation

@santhiprakash

Copy link
Copy Markdown

Description

calculateOpenAICost() in core/llm/utils/calculateRequestCost.ts bills every prompt token at the standard input rate and ignores usage.promptTokensDetails.cached_tokens, so any OpenAI request that hits the prompt cache (the common case for stable, long system prompts) is over-reported by the cache-read discount. The Anthropic branch in the same file already handles cache tokens; the OpenAI branch does not.

This PR:

  1. Adds cachedInput to the OpenAI pricing table for the model families where OpenAI documents a prompt-cache rate (gpt-4o, gpt-4o-mini).
  2. Splits usage.promptTokens into uncached and cached portions and bills each at its own rate, mirroring the Anthropic branch.
  3. Clamps cachedTokens to promptTokens to defend against malformed usage payloads that report more cached tokens than total prompt tokens.
  4. Keeps the original behavior for models without a documented cachedInput rate: the cached portion is treated as uncached and billed at the standard input rate (the API would never return a cached_tokens value for them in the first place).

Tests

Added regression cases in core/llm/utils/calculateRequestCost.vitest.ts:

  • gpt-4o with 700/1000 prompt tokens cached
  • gpt-4o with all 1000 prompt tokens cached
  • gpt-4o with explicit zero cached tokens (no cache rows in breakdown)
  • gpt-4o-mini with 800/1000 prompt tokens cached
  • gpt-4 with cached tokens reported but no cachedInput rate defined (full prompt billed at the standard input rate)
  • gpt-4o with cached tokens > prompt tokens (clamped to promptTokens)

The fix was verified by extracting calculateOpenAICost into a standalone Node script and running 11 cases against hand-computed expected costs (all pass). I could not run the vitest suite in this environment (no node_modules at the workspace root); the upstream vitest suite should be re-run by the maintainer.

Checklist

I have read the CLA Document and I hereby sign the CLA

🤖 Generated with Hermes Agent

…ate (continuedev#13104)

calculateOpenAICost() previously billed every prompt token at the standard
input rate, ignoring usage.promptTokensDetails.cached_tokens. For OpenAI
requests that hit the prompt cache (gpt-4o / gpt-4o-mini with stable system
prompts), this over-reported costs by the cache-read discount.

Add cachedInput to the pricing table for the families that document a
cache rate (gpt-4o, gpt-4o-mini), split prompt tokens into uncached and
cached portions, and bill each at its own rate. Models without a
documented cachedInput rate keep their existing behavior — cachedTokens
is ignored and the full prompt is billed at the standard input rate.

Includes regression tests covering: gpt-4o with partial cache, fully
cached input, zero cached tokens, gpt-4o-mini, gpt-4 (no rate defined),
and clamping cachedTokens when the API reports more cached than total
prompt tokens.

Verified by extracting calculateOpenAICost into a standalone Node script
and running 11 cases against hand-computed expected costs (all pass).
@santhiprakash
santhiprakash requested a review from a team as a code owner August 18, 2026 20:23
@santhiprakash
santhiprakash requested review from sestinj and removed request for a team August 18, 2026 20:23
…nuedev#13104)

Fix the prettier-check failure on PR continuedev#13160 introduced by the previous
commit's hand-formatted long lines.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant