fix(core): price OpenAI prompt-cache input at the documented cached rate (#13104) - #13160
Open
santhiprakash wants to merge 2 commits into
Open
fix(core): price OpenAI prompt-cache input at the documented cached rate (#13104)#13160santhiprakash wants to merge 2 commits into
santhiprakash wants to merge 2 commits into
Conversation
…ate (continuedev#13104) calculateOpenAICost() previously billed every prompt token at the standard input rate, ignoring usage.promptTokensDetails.cached_tokens. For OpenAI requests that hit the prompt cache (gpt-4o / gpt-4o-mini with stable system prompts), this over-reported costs by the cache-read discount. Add cachedInput to the pricing table for the families that document a cache rate (gpt-4o, gpt-4o-mini), split prompt tokens into uncached and cached portions, and bill each at its own rate. Models without a documented cachedInput rate keep their existing behavior — cachedTokens is ignored and the full prompt is billed at the standard input rate. Includes regression tests covering: gpt-4o with partial cache, fully cached input, zero cached tokens, gpt-4o-mini, gpt-4 (no rate defined), and clamping cachedTokens when the API reports more cached than total prompt tokens. Verified by extracting calculateOpenAICost into a standalone Node script and running 11 cases against hand-computed expected costs (all pass).
…nuedev#13104) Fix the prettier-check failure on PR continuedev#13160 introduced by the previous commit's hand-formatted long lines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
calculateOpenAICost()incore/llm/utils/calculateRequestCost.tsbills every prompt token at the standard input rate and ignoresusage.promptTokensDetails.cached_tokens, so any OpenAI request that hits the prompt cache (the common case for stable, long system prompts) is over-reported by the cache-read discount. The Anthropic branch in the same file already handles cache tokens; the OpenAI branch does not.This PR:
cachedInputto the OpenAI pricing table for the model families where OpenAI documents a prompt-cache rate (gpt-4o,gpt-4o-mini).usage.promptTokensinto uncached and cached portions and bills each at its own rate, mirroring the Anthropic branch.cachedTokenstopromptTokensto defend against malformed usage payloads that report more cached tokens than total prompt tokens.cachedInputrate: the cached portion is treated as uncached and billed at the standard input rate (the API would never return acached_tokensvalue for them in the first place).Tests
Added regression cases in
core/llm/utils/calculateRequestCost.vitest.ts:gpt-4owith 700/1000 prompt tokens cachedgpt-4owith all 1000 prompt tokens cachedgpt-4owith explicit zero cached tokens (no cache rows in breakdown)gpt-4o-miniwith 800/1000 prompt tokens cachedgpt-4with cached tokens reported but nocachedInputrate defined (full prompt billed at the standard input rate)gpt-4owith cached tokens > prompt tokens (clamped topromptTokens)The fix was verified by extracting
calculateOpenAICostinto a standalone Node script and running 11 cases against hand-computed expected costs (all pass). I could not run the vitest suite in this environment (nonode_modulesat the workspace root); the upstream vitest suite should be re-run by the maintainer.Checklist
I have read the CLA Document and I hereby sign the CLA
🤖 Generated with Hermes Agent