fix(kiro): estimate chat-file input tokens from every human turn, not a 500-char slice - #934
Open
ozymandiashh wants to merge 1 commit into
Open
Conversation
ozymandiashh
marked this pull request as draft
August 5, 2026 10:04
Collaborator
Author
|
Converted to draft: this had two independent reviews but a third never landed. Re-running it; will mark ready once complete. |
ozymandiashh
force-pushed
the
fix/port-kiro
branch
3 times, most recently
from
August 5, 2026 13:39
a11876c to
35e4e42
Compare
ozymandiashh
marked this pull request as ready for review
August 5, 2026 13:39
ozymandiashh
marked this pull request as draft
August 5, 2026 14:13
… invalidate cached history decodeKiroChatFile estimated input tokens from pendingUserMessage - the last human turn sliced to 500 chars - while output summed every bot char, so any multi-turn session or final prompt over 500 chars under-reported input tokens (and therefore costUSD) severalfold. The estimate is now the sum of every human turn's full character count, with the 500-char cap kept for the display userMessage only. That closes most of the gap but does NOT reach parity with the modern-execution, CLI-session and V2 arms: those count tool and system records as input - their code comments state tool results are fed back to the model - while the chat arm still counts only human records. Tool content demonstrably exists in the format (the G2 fixture carries a tool record), so the chat arm still under-reports; it just under-reports far less than before. The blast radius is the chat-file arm alone: the IDE-file dispatcher routes any record carrying a chat array plus metadata to decodeKiroChatFile, so this is chat-shaped Kiro IDE files, not every Kiro prompt. The identity-preamble exclusion now trims leading whitespace before its startsWith match. Pre-fix, a near miss (a leading newline, a BOM, a wrapper) was nearly harmless, because the preamble only mattered if it happened to be the last human turn; post-fix, every unmatched system-injected human record adds its FULL length to input, and preambles are large - a missed match is a silent multi-thousand-token inflation on every affected session. Leading whitespace tolerance is cheap (a genuine prompt never starts with whitespace plus an identity tag) and the failure asymmetry favours exclusion: a false negative inflates tokens, a false positive only skips a preamble. A renamed preamble remains a residual risk, noted in the near-miss regression test. Cached history is affected, which is what a user actually sees. session-cache serves unchanged files without invoking the provider parser, so bump kiro's PROVIDER_PARSE_VERSIONS fingerprint (ide-parsing-v1 -> v2) to force one re-parse of every already-cached kiro session; without it the pre-fix token and cost numbers would be served forever. The daily rollup ALSO needs invalidating for this fix to be fully visible: days finalized before the fix keep their pre-fix kiro cost in the daily cache, and ensureCacheHydrated re-derives them only on a version bump, a savings-config change, a timezone change, or an incomplete cache — the session-cache re-parse alone leaves finalized day totals untouched. So this commit bumps BOTH layers: the session-cache PROVIDER_PARSE_VERSIONS fingerprint above forces the one re-parse of every already-cached kiro session, and DAILY_CACHE_VERSION (15 -> 17, MIN_SUPPORTED_VERSION raised with it; 16 is skipped because main already claimed it for the codex structural-discovery fix, and claiming 16 here would load a main-built v16 cache as current and complete, so the invalidation would never fire) forces the daily rollup's one-time re-derivation, so finalized day totals are rebuilt under the corrected estimate. The re-derive reaches every day whose kiro chat files still exist; sourceless days carry forward with their pre-fix totals under the v14 NEVER-LOSE rule (a carry-forward, not a refresh — nothing can reconstruct them once the files are gone). Update the G2 parity golden: A1 was pinned at 125 tokens for a 3000-char prompt (the 500-char slice / 4); the corrected value is 750 (3000 / 4), with a comment marking 125 as a pre-fix value so it is not restored. 3000, 2400 and 1000+1000 are all exact multiples of four, so add G2b pinning the estimator's rounding with an odd length (3001 chars -> 751 tokens; round and floor would both give 750). Add a money-path regression test (2400-char prompt -> 600 tokens, userMessage still 500-capped for display), a multi-turn accumulation test (an identical resubmitted prompt counts again - a real second model input; identity messages stay excluded), a near-miss identity test (leading-newline and BOM preambles stay excluded), extend the kiro cache-invalidation test to pin the v1 -> v2 fingerprint bump, and add a daily-cache regression test seeding a complete pre-fix v15 cache (unchanged savings hash and timezone, so nothing but the version bump can invalidate it) and proving the bump forces the re-derive that lands the corrected kiro cost — while the v15 file is never rewritten.
ozymandiashh
force-pushed
the
fix/port-kiro
branch
from
August 5, 2026 14:20
35e4e42 to
bc5a85d
Compare
ozymandiashh
marked this pull request as ready for review
August 5, 2026 14:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports upstream
6c4645a.decodeKiroChatFileestimated input tokens frompendingUserMessage— the last human turn, sliced to 500 characters — while output summed every bot character. So any multi-turn session, or any final prompt over 500 characters, under-reported input tokens and thereforecostUSD, severalfold in the worst case.It now accumulates every human turn's full character count, matching the modern-execution, ws-session, CLI-session and V2 arms which already did. The 500-char cap stays for the display
userMessageonly.Scope
The chat-file arm alone. The IDE-file dispatcher routes any record carrying a
chatarray plus metadata todecodeKiroChatFile, so this is chat-shaped Kiro IDE files — not every Kiro prompt. Identity preambles and tool content are excluded from the sum, so "full prompt" means the sum of human message characters rather than everything in the file.The parity golden was defending the bug
kiro-golden.test.ts's G2 case was titled "A1 input tokens derived from truncated prompt" — it pinned the buggy value, because the corpus was captured before upstream's fix. Fix the decoder and it goes red.The pin moves from 125 to 750, derived from the fixture rather than copied from the new output: the prompt is 3000 characters, the estimator is
ceil(chars / 4), so 3000/4 = 750 where 125 was 500/4. A comment marks 125 as a pre-fix value so nobody restores it. No other kiro golden encoded the same assumption — checked.Cache invalidation, and what this PR deliberately does not do
The user-visible part is cached history. Without invalidation, already-parsed sessions keep pre-fix tokens and cost forever while new ones use corrected maths — the report silently disagrees with itself.
The kiro provider parse fingerprint is bumped here, which discards the provider section and forces a re-parse of unchanged chat files. Verified independently: that check runs per provider section at scan time and has nothing to do with the daily cache.
The daily rollup also needs invalidating for the fix to be fully visible — days already finalized are returned as-is unless a savings-config change, a TZ change, a completeness flag or a gap fires a re-derivation. This PR does not bump that constant: a single bump in the release covers every parser-behaviour change landing with it, and #926 carries it. Three PRs editing the same line would conflict and leave you asking which "16" is which.
If this and #926 land apart, the effect is precise: the session cache re-parses so new numbers are correct, but a finalized daily rollup keeps pre-fix kiro cost until something else forces re-hydration.
Verification
Core 509, kiro 77, cache suites 67 — including a regression test that pins 125 under the old fingerprint and 750 under the new one.