Skip to content

feat(transcript): externalize images from Codex transcripts#1591

Merged
suhaanthayyil merged 2 commits into
feat-image-externalizefrom
feat-image-externalize-codex
Jul 1, 2026
Merged

feat(transcript): externalize images from Codex transcripts#1591
suhaanthayyil merged 2 commits into
feat-image-externalizefrom
feat-image-externalize-codex

Conversation

@suhaanthayyil

@suhaanthayyil suhaanthayyil commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

https://entire.io/gh/entireio/cli/trails/717

Stacked on #1589 (base branch feat-image-externalize) — this PR is only the Codex delta.

What: Extend image externalization to Codex. Codex rollout JSONL embeds images as base64 data-URIs (data:image/…;base64,…) — in input_image image_url values, user messages, and function_call_output — and inlines multi-MB generated images the same way. This lifts them into per-session checkpoint assets/ blobs with a placeholder, and reinjects byte-exactly on restore, exactly like the Claude Code codec.

Why / how it helps: Codex is a bigger win than Claude here. Real sessions on disk carry 100+ MB of inline image base64; OpenAI's own issue #22603 reports image-heavy rollouts taking minutes to open. Externalizing shrinks full.jsonl, cuts what's pushed to the platform, dedupes identical images, and keeps restore lossless. Also: without externalization, an inline Codex input_image base64 (which redaction does not skip, since the enclosing type is input_image, not image) can be mangled by the high-entropy redaction pass — extracting before redaction preserves it.

How:

  • Refactor the codec into a shared extraction engine + reinject routine that take a per-agent collector; codecs now differ only in how they find images.
  • New codexCodec keyed on the data:image/…;base64,… marker (not a specific field), so input, generated, and tool-output images are covered uniformly. The bare base64 value is swapped (the tiny data:…;base64, prefix stays inline), so the swap is format-agnostic and byte-exact — the same value-swap contract as Claude.
  • Detect the media type from the decoded magic bytes (declared type as fallback): real Codex data-URIs mislabel some PNGs as image/jpeg, so the asset filename/manifest should reflect the actual content. Metadata only — the transcript's declared type is untouched, so the round trip stays byte-exact.
  • Register agent.AgentTypeCodex in the codec map. No changes to condensation/persistence/finalize/settings — that plumbing is agent-agnostic; reuses the same opt-in redaction.externalize_images flag.

Codex's SanitizePortableTranscript (which runs before full.jsonl is chunked) only drops compaction lines and strips reasoning.encrypted_content; image-bearing message and tool-output lines pass through, so the placeholder survives and the byte-exact contract holds against the stored bytes.

Testing:

  • mise run lint — 0 issues; mise run test:ci — unit + integration + Vogon canary green on the committed tree.
  • Codec units: data-URI round trip; function_call_output embedded data-URI; multiple images in one message (the real Codex case); dedup; text-only no-op; tiny data-URI left inline; media-type detection (jpeg vs png); extract-before-redact ordering (secret redacted, image reinjects). Claude codec unchanged and still passing.
  • Real-data: ran the codec over real Codex rollouts — 176 images across 12 sessions, every asset valid PNG/JPEG magic, extract→reinject byte-exact, up to ~119 MB saved in one session.
  • Integration (real hook flow): a Codex session (rollout with a data-URI image) → post-commit condensation → stop finalize → assert the real entire/checkpoints/v1 full.jsonl carries the placeholder (base64 gone), the asset blob + manifest.json are written, and ReadSessionContent reinjects byte-exact.

Risk / rollback: Off by default (opt-in flag), no-op for agents without a codec, reinject is a no-op without placeholders. Revert by reverting the two commits on this branch. Incidental large data-URIs embedded in fetched tool output are also externalized — harmless (still bloat, still lossless).

Follow-ups: Cursor/Copilot/others; platform-side web rendering of the stored asset (shared with #1589).

Codex rollout JSONL embeds images as base64 data-URIs (data:image/…;base64,…)
in input_image image_url values, user messages, and function_call_output — and
inlines multi-MB generated images the same way, which bloats the transcript
badly (real sessions here carry 100+ MB of inline image base64). Externalize
them into checkpoint assets like Claude Code.

- Refactor the codec into a shared extraction engine + reinject routine that
  take a per-agent collector, so codecs differ only in how they find images.
- Add codexCodec keyed on the data-URI marker (not a specific field), covering
  input, generated, and tool-output images uniformly. The bare base64 value is
  swapped (the tiny data:…;base64, prefix stays inline), so the swap is
  format-agnostic and byte-exact.
- Detect the media type from the decoded magic bytes (declared type is the
  fallback): real Codex data-URIs mislabel some PNGs as image/jpeg, and the
  asset filename/manifest should reflect the actual content. Metadata only — the
  transcript's declared type is untouched, so the round trip stays byte-exact.

Verified byte-exact extract/reinject over 176 images across real Codex rollouts
(valid PNG/JPEG magic on every asset), saving up to ~119 MB in one session.
Drive the real Codex hook binary (user-prompt-submit → apply_patch
post-tool-use → mid-turn commit condensation → stop finalize) on a rollout
transcript with an inline data-URI image, externalization enabled via
settings.local.json. Assert the actual entire/checkpoints/v1 full.jsonl stores
a placeholder (base64 gone, inside the preserved data-URI) that survives Codex's
SanitizePortableTranscript, writes the asset blob + manifest, and that
ReadSessionContent reinjects the image.
Copilot AI review requested due to automatic review settings July 1, 2026 16:24
@suhaanthayyil suhaanthayyil requested a review from a team as a code owner July 1, 2026 16:24
@suhaanthayyil suhaanthayyil merged commit 7d51237 into feat-image-externalize Jul 1, 2026
9 checks passed
@suhaanthayyil suhaanthayyil deleted the feat-image-externalize-codex branch July 1, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the transcript image externalization pipeline to OpenAI Codex rollouts by introducing a Codex-specific collector and refactoring the existing Claude-only codec into a shared extraction/reinjection engine. This reduces checkpoint transcript bloat (large inline data:image/...;base64,... payloads) while preserving a byte-exact restore contract via placeholder swapping and reinjection.

Changes:

  • Refactors image extraction/reinjection into shared helpers and introduces a Codex codec that detects image data-URIs anywhere in decoded JSON leaves.
  • Adds media-type detection from magic bytes (with declared type as fallback) to improve asset metadata/filenames without affecting transcript round-tripping.
  • Adds Codex-focused unit tests and an integration test that exercises the real Codex hook flow and validates persisted placeholders + byte-exact reinjection.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
cmd/entire/cli/transcript/imageextract/imageextract.go Adds Codex support via a data-URI collector and refactors into shared extract/reinject routines with content-based media-type detection.
cmd/entire/cli/transcript/imageextract/imageextract_test.go Updates shared tests to reflect Codex now having a codec and uses media-type constants.
cmd/entire/cli/transcript/imageextract/codex_test.go Adds Codex-specific unit tests for round-trip, tool-output embedding, multi-image handling, dedup, thresholds, and redact-ordering behavior.
cmd/entire/cli/integration_test/codex_image_externalize_test.go Adds an end-to-end integration test validating persisted placeholders/assets and byte-exact restore through real Codex hook execution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants