[codex] improve bounded native cognition extraction#9
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 08ee8a022d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if force or not checkpoint or checkpoint[0] != stat.st_ino or checkpoint[1] != stat.st_size: | ||
| pending.append((path, force)) |
There was a problem hiding this comment.
When --reextract-days covers more than --max-files recent files, this force condition keeps already-checkpointed recent files in pending forever, so pending[:max_files] selects the same newest batch on every run. With the default budget of 40, older files inside the re-extraction window are never revisited unless the operator sets --max-files above the entire window size, unlike normal incremental imports that progress after checkpointing.
Useful? React with 👍 / 👎.
1 participant
What changed\n\n- harden bounded Hermes/native-history parsing so malformed nested records cannot abort a batch\n- extract high-signal later sentences and Markdown bullets into eight reviewable memory types\n- add aggregate-only error/type metrics, idempotent bounded re-extraction, and isolated temporary-database backtesting\n- document the manual quality gate before production re-extraction and daily crystallization\n\n## Why\n\nThe previous extractor could stop on nested non-string role values and inspected only the first sentence of each bounded message. That missed verified decisions, root causes, project states, and open loops commonly written in later bullets.\n\n## Safety boundaries\n\n- native reads remain capped at 64 KB and 12 messages\n- automatic extraction remains capped at three proposals per session and 800 characters per summary\n- proposals still pass daily sensitivity, type, capacity, and deduplication gates\n- the public diff contains no local paths, real memory/session IDs, databases, transcripts, or deployment records\n\n## Validation\n\n- 155 tests passed\n- \n- public safety check passed\n- \n- added-line scan found no personal paths or real memory/session IDs