Skip to content

bug(runtime): file tools lose numeric precision, accept overlapping edits, and misread PNGs #5589

Description

@testikun

What happened

Three Runtime file-tool defects were reproduced through normal Desktop tasks with a real gpt-5.6-terra model and confirmed by the files and committed RuntimeEvents:

Tool Trigger Actual Expected
FormatJson Valid JSON with a 64-bit integer, precise decimal, or large exponent Silently changes the values, including a number becoming null Preserve numeric values, or refuse before modifying the file
Edit Two exact matches overlap Treats the input as unique and replaces the first span Reject ambiguous input and leave the file unchanged
Read A valid PNG has no image filename suffix Returns a successful UTF-8 text page containing binary garbage Recognize supported image bytes and return an image

How to reproduce

1. FormatJson changes values while formatting

Create numbers.json:

{"account_id":9223372036854775807,"amount":0.123456789012345678901,"large":1e400}

Call FormatJson({"path":"numbers.json","sort_keys":false}) and read the file back. The operation succeeds, but the file now contains:

{
  "account_id": 9223372036854776000,
  "amount": 0.12345678901234568,
  "large": null
}

sort_keys:true has the same problem. The three numeric categories each reproduce independently; ordinary small numbers are a passing control.

2. Edit misses overlapping exact matches

Create overlap.txt with three identical lines:

retry();
retry();
retry();

Call:

{"path":"overlap.txt","old_string":"retry();\nretry();","new_string":"MARKER();"}

The match can start on line 1 or line 2, but Edit succeeds and writes MARKER(); followed by one retry(); line. Minimal single-line examples are banana/ana and AAAA/AAA.

For models that default to ApplyPatch, disable the model's ApplyPatch setting to expose ordinary Edit. The recorded successful run used applyPatch:false and invoked Edit exactly once.

3. Read misclassifies an extensionless PNG

Copy any valid PNG to an extensionless filename without changing its bytes. Call Read on the .png path and then its extensionless twin. The .png result is an image; the twin is JSON text containing the PNG header and replacement characters. A .bin suffix also reproduces.

The recorded fixture is a 320×180 red/blue PNG. Both names contain the same 1,148 bytes (SHA-256 f23162dba72a5a394a453e7d5116d9c5580645b36e9c477eb5d4480ada4ab59e). The model sees the first image's colors and reports that the second tool returned text garbage. This occurs before provider serialization and is separate from #5277; this comparison used the native OpenAI Chat adapter with vision enabled.

Environment

  • Baseline: 420c221862b7d69a4e05a4bbd787aafa4ba0f525; the relevant Runtime code is unchanged at current 8be311535.
  • Maka Desktop development build, macOS arm64, Electron 43.4.1.
  • Real gpt-5.6-terra through a local provider, using an isolated profile and temporary files.
  • Runtime implementation was unmodified during reproduction. Standard build and Runtime typecheck passed. The focused baseline regressions have 8 expected defect failures and 2 passing image controls.

Logs, screenshots, or additional context

These are direct screenshots of the actual app and model/tool execution; no UI text or event output was fabricated.

FormatJson:

FormatJson changes numeric values

Edit:

Edit accepts overlapping matches

Read:

Read classifies identical PNG bytes differently

Likely causes:

  • The FormatJson branches in filesystem-executor.ts and filesystem-worker/operations.ts round-trip numbers through JSON.parse/JSON.stringify.
  • edit-replace.ts advances occurrence counting by the full needle length, skipping overlapping starts.
  • image-file.ts classifies by extension before workspace-executor.ts and the filesystem worker choose an image or UTF-8 read.

Acceptance: preserve numeric values with either sorting setting, reject all ambiguous Edit matches without writing, and preserve image semantics under opaque filenames while keeping existing image limits and normal text reads.

AI disclosure: OpenAI Codex prepared and submitted this report, screenshots, and the take claim at @testikun's request. @testikun is the human contributor of record.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions