fix(tools): raise a graceful ToolError when a memory file is not valid UTF-8#1693
Open
WalkingDreams798 wants to merge 1 commit into
Open
fix(tools): raise a graceful ToolError when a memory file is not valid UTF-8#1693WalkingDreams798 wants to merge 1 commit into
WalkingDreams798 wants to merge 1 commit into
Conversation
…d UTF-8 `view` / `str_replace` / `insert` read memory files via `_read_file_content`, which decodes strict UTF-8 and only caught `FileNotFoundError`. A memory file containing non-UTF-8 bytes therefore raised an uncaught `UnicodeDecodeError` out of the tool (platform-independent, since the encoding is hard-coded to utf-8), instead of a normalized `ToolError` like every other failure mode. Catch `UnicodeDecodeError` in both the sync and async readers and surface it as a `ToolError`. Added sync + async regression tests reading a binary file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The local-filesystem memory tool reads files through
_read_file_content(and its async twin_async_read_file_content), which decodes strict UTF-8 and only catchesFileNotFoundError:A memory file containing non-UTF-8 bytes therefore raised an uncaught
UnicodeDecodeErrorout ofview/str_replace/insert, instead of the normalizedToolErrorused for every other failure mode. Because the encoding is hard-coded toutf-8, this reproduces on all platforms.Repro
Change
Catch
UnicodeDecodeErrorin both the sync and async readers and surface it as aToolError("… is not a valid UTF-8 text file …"). Added sync + async regression tests that write a binary file and assert the gracefulToolError.Verification
uv run pytest -k non_utf8 tests/lib/tools/memory_tools/test_filesystem.py→ 2 passeduv run ruff check→ all checks passeduv run pyright(strict) → 0 errors, 0 warnings