[aakhter]

Saved Codex conversations currently require manual JSONL parsing to retrieve a thread and its children. Add codeman history read with file/native-ID/root selectors and topic/date filters that preserve complete matching turns.

The command runs locally without a Codeman server. It bounds scans and output separately, checks native identity and file consistency, and reports incomplete evidence or unproven ancestry explicitly. It adds no HTTP endpoints. Includes CLI documentation, skill routing before the managed-session guard, the synchronized plugin reference, and a minor changeset.

Validation:

• 30 focused history and plugin tests passed, including ambiguous IDs, descendants, filters, conflicting metadata, UTF8, file changes and budget limits.
• Typecheck, lint, formatting, lockfile, plugin parity and diff checks passed.
• Actual CLI smoke with isolated HOME and no CODEMAN environment returned a complete matching turn without a server; output overflow exited with an error.

Codex JSONL only. Explicit path parent directories must be trusted; this is not a filesystem sandbox. Store completeness does not imply every historical conversation was retained.

[Ark0N]

Thanks, Aamer, for this. It adds an offline codeman history read command that reads saved Codex rollouts by file, native id, or a root conversation with its descendants, with whole-turn topic and date filters and hard read and output budgets. The care around identity, ancestry and fail-closed output shows, and I ran it against all 32 Codex rollouts on my machine without a server: every conversation came back with the right user and assistant turns and no injected context.

Two things need to change before this can merge.

1. Forked threads are refused, and one forked child blocks the whole family (src/local-history.ts:303)

readMessages refuses any later session_meta whose id differs from the file's own. Codex writes exactly that for a fork: in codex-rs/core/src/agent/control/spawn.rs the fork filter keeps RolloutItem::SessionMeta, and record_initial_history in codex-rs/core/src/session/mod.rs persists the copied prefix for ForkPersistence::Copied, which thread_manager.rs uses. So a forked thread's rollout holds its own session_meta followed by the parent's. With a file of that shape, --session-id <child> fails, and --root-session-id <parent> fails for the entire family. test/local-history.test.ts:158 pins the refusal as intended behaviour. Please treat a later session_meta whose id is a declared ancestor (parent, forked_from_id, or the shared session_id) as the start of an inherited prefix, and either drop that prefix (it is already in the ancestor's own file) or attribute it to that ancestor, keep refusing an unrelated id, and add a fixture shaped like a real forked rollout. If you have a store with sub-agents spawned in paginated history mode, please check that path too.

2. The id selectors stop working on larger stores (src/local-history.ts:141)

Each lookup reads every rollout's session_meta line first, and that line carries Codex's base instructions (about 19 KB on every rollout here), so each file costs about 20 KiB of the shared scan budget. With real-sized metadata a lookup succeeds at 800 rollouts and fails at 840 on the default 16 MiB, and fails at 3,300 even at the 64 MiB maximum. The 8192-entry cap (maxEntries) has no CLI flag in src/cli-history.ts. Codex never prunes its store and sub-agent threads add up quickly, so past that point --session-id and --root-session-id cannot work at any setting. Please decouple the metadata cost from the transcript budget, and make the remaining caps raisable.

Smaller things, fine in the same revision:

• src/local-history.ts:155, :162, :173: one unrelated file in the store (invalid UTF-8 in its head, larger than 128 KiB without session_meta, or changing while its head is read) aborts every id lookup. Counting it in unattributedFiles with complete: false would match how small files without metadata are already treated.
• src/local-history.ts:277: a rollout that a running Codex pane is still appending to fails with "changed after attribution", and so does a family with a running child. Reading a snapshot up to the last complete line would avoid that.
• docs/wiki/Codeman-Skill.md:126 still says the skill refuses to act outside a Codeman session. It needs to match the new SKILL.md.
• src/local-history.ts:228: this is a second parser of the Codex rollout format next to src/codex-transcript.ts, with a different injected-context rule (four fixed prefixes versus any #/< user row). Please share the shape helpers and CODEX_HOME resolution with that module, or at least the injected-context rule.
• src/local-history.ts:181: a sessions directory that is itself a symlink is refused, while the existing scanner follows it.
• Both new files need an @fileoverview block (CLAUDE.md: all files have one).
• docs/offline-history.md repeats skills/codeman/reference/history.md and nothing links to it. A README or wiki entry next to the codeman agent docs would serve users better.
• User records all get phase: "progress" (src/local-history.ts:235), date bounds are UTC dates without saying so (:348), --json silently overrides --format text (src/cli-history.ts:29), and --format text plus the default CODEX_HOME root have no test.

Two design points are still open on my side and are not yours to resolve: whether the codeman skill should advertise this offline reader in its description and relax its "refuse to act outside a Codeman session" guard (the skill is often installed user-level, so its description fires in unrelated sessions), and whether a Codex-only reader should take the top-level history verb, which becomes stable CLI surface once released. The skill-file changes in this PR depend on the first.

Once the two numbered items are addressed this is close to merge.

[aakhter]

Thanks for the careful review, and for running it against your store. Everything below is pushed.

1. Forked threads. Agreed, and the refusal test pinned the wrong behaviour. A later session_meta whose id is a declared ancestor (parent_thread_id, forked_from_id or the shared root session_id, and transitively the ancestors those declare, for a parent that was itself a fork) now opens an inherited prefix, and the prefix is dropped since it already lives in the ancestor's own file. Any other id still refuses the file. The boundary follows Codex:

• Paginated: records with ordinal < subagent_history_start_ordinal are inherited, the same rule thread_history_materialization.rs uses. A paginated thread with a history_base has nothing inherited in its file. This is the paginated sub-agent path you mentioned; my store has 32 of them and every one now reads.
• Legacy: the copy ends at the thread_settings_applied event record_initial_history appends after it. A legacy copy keeps the parent's own thread_settings_applied events too (spawn.rs only strips them for paginated destinations), so with more than one candidate the cut is the last one whose preceding messages are still a prefix of the parent's own messages. Without the parent's file (--file), the cut falls at the last candidate, so no parent text is attributed to the child, and the result says unverifiedPrefixes: 1, complete: false.

A copy that never reaches its end (initialisation died) is reported as incompletePrefixes. Fixtures in test/fixtures/codex-rollout.ts follow the structure of real rollouts (key order from SessionMeta, a ~19 KiB base_instructions, the event types around the boundary); no conversation content was copied. Against my store (2,916 rollouts, 44 forks, 13 legacy and 31 paginated), every forked child and every family now reads with complete: true, and none repeats the parent's turns. A /fork without a spawn parent now counts as a descendant of its forked_from_id. Tests: the forked and spawned threads block in test/local-history.test.ts.

2. Large stores. Attribution now has its own budget, --metadata-bytes (default 1 GiB, max 16 GiB), separate from --scan-bytes (the selected transcripts, default 16 MiB, max raised to 256 MiB). The head is read in 32 KiB steps and stops at the first session_meta, so a real rollout costs one step, not the 128 KiB window. --max-entries is now a flag (default 65536, max 1048576). Directory depth stays fixed at 8, because Codex writes YYYY/MM/DD, so it never grows with the store. The test writes 3,500 rollouts with real-sized metadata and looks one up at the defaults; the old code fails that at any setting. On my store the old code failed every id lookup; now one takes about 0.9 s.

Smaller items:

• Unrelated bad files (:155, :162, :173): invalid UTF-8 in the head, no session_meta within 128 KiB, and a change during the head read all count in unattributedFiles with complete: false. Named with --file, they are still errors. A store directory that cannot be read counts in a new unreadableDirectories instead of aborting. When the id is not found, the error now says how many files could not be attributed.
• Live rollouts (:277): a read takes a snapshot. The device and inode must match the attributed file, the size may only grow, and everything up to the last complete line is read. A record cut mid-write counts as partialRecords. A file that was replaced or truncated still fails. Tests in test/local-history-live.test.ts cover a growing rollout, a family with a running child, an unrelated file shrinking mid-read, and the replaced and truncated cases.
• docs/wiki/Codeman-Skill.md: now matches the current SKILL.md, and the reference table lists reference/history.md. I left the skill description, its guard and the history verb as they are. Those are your two open design points, and I will change the skill files to follow whatever you decide.
• Second parser (:228): the store location, the three message shapes and the injected-context rule are now in src/codex-rollout-format.ts, used by both codex-transcript.ts and local-history.ts. The rules did differ, and the Past Sessions one (any # or < on a response_item user row) is the right one. On my store the four fixed prefixes let 1,355 <recommended_plugins> rows through as user turns, plus <subagent_notification>, <codex_internal_context>, <skill> and <image> wrappers. Real prompts that start with # all have a user_message or UserMessage twin, and events are never filtered. codex-transcript.ts behaves exactly as before, and its 18 tests pass unchanged.
• Symlinked sessions (:181): the root is followed as codex-transcript.ts follows it (readdir on the root follows the link), and links below it are still skipped. Same checks as the scanner, which has no others.
• @fileoverview: on local-history.ts, cli-history.ts, the new shared module and the new test files.
• docs/offline-history.md is gone. There is now a README section after codeman agent, a wiki section in Driving Codeman From An Agent (linked from Home and from the skill page), and the full reference stays in skills/codeman/reference/history.md.
• Phase: user records have phase: null. Assistant records keep Codex's own commentary or final_answer, and null when none was recorded. Date bounds are UTC calendar dates, now computed from the parsed timestamp (one with an offset counts on its UTC date), and the help text and docs say UTC. --json with --format text is now an error, and there are tests for --format text output and for the default $CODEX_HOME/sessions and ~/.codex/sessions roots.

[Ark0N]

Thanks, Aamer, for the second round. The PR adds an offline codeman history read that reads saved Codex rollouts by file, native id, or a root conversation with its descendants, with whole-turn topic and UTC date filters and separate, raisable read and output budgets.

Both numbered items from the first review are resolved. I checked the fork handling against codex-rs: record_initial_history pushes the thread_settings_applied event after a ForkPersistence::Copied prefix, paginated sub-agents drop that event from the copy and declare subagent_history_start_ordinal, and your ordinal < start and incomplete-prefix rules match thread_history_materialization.rs and rollout/src/ordinal.rs. On the Codex store here (32 rollouts), all 29 with conversation text read, and all 58 --session-id and --root-session-id lookups came back with complete: true. Typecheck, lint, format and the touched test files pass.

Smaller things, fine in the same revision:

1. src/local-history.ts:390: only *.jsonl files are attributed. Codex's local_thread_store_compression feature (off by default, under development) rewrites rollouts older than seven days to *.jsonl.zst in place, and Codex's own readers accept both. Such a file is skipped without being counted, so a compressed family member is missing while the result says complete: true, and --session-id on a compressed thread reports it missing. Please count *.jsonl.zst files in unattributedFiles (or a counter of their own) so complete turns false and the not-found error names them.
2. src/local-history.ts:682 (with :687): when a legacy copy has more than one thread_settings_applied candidate, the boundary proof reads the parent again through ancestorMessages, so a family read charges the parent to --scan-bytes twice. For a 21,245-byte parent and a 42,263-byte child the read reported 84,753 bytes, and --scan-bytes 63508 (the two files together) failed. Please reuse the parent's own read, or its messages, instead of reading it a second time.
3. src/local-history.ts:554: if the parent's file is in the store but its read fails (my probe: one invalid UTF-8 byte appended to its body), --session-id <child> fails with History source contains invalid UTF8 although the child's file is intact. The module already has a rule for a prefix it cannot prove (cut at the last candidate, unverifiedPrefixes: 1, complete: false). Please fall back to that rule for any error from the parent read other than a budget error.
4. src/cli-history.ts:89: --max-bytes is validated inside formatLocalHistory (src/local-history.ts:733), after the whole store has been scanned. Please validate it before readLocalHistory, next to the other budgets.

Two design points are still open on my side, and merging waits on them:

• Whether the codeman skill advertises the offline reader in its description and narrows its "refuse to act outside a Codeman session" guard. The PR as it stands takes the yes side in skills/codeman/SKILL.md (description and section 0), its mirror under plugins/codeman/, and the guard sentence in docs/wiki/Codeman-Skill.md. Those hunks are separate from the reader itself.
• Whether a Codex-only reader takes the top-level history verb. Under docs/versioning-policy.md a command name and its documented flags become stable surface once released, unless the docs label the feature experimental.

Nothing is needed from you on those two. The four items above fit the next push; after that, the two design points are what stands between this PR and merge.

[aakhter]

Thanks for the third round. All four items are in 17b2edc:

1. Compressed rollouts. The store scan now counts *.jsonl.zst in a new scan.compressedFiles counter, so complete is false. The match is codex-rs's own: is_compressed_rollout_path (a name ending in .jsonl.zst). A compressed file whose plain .jsonl sibling exists is skipped, the same way should_skip_compressed_sibling skips it, because the plain file is read or counted on its own. A lookup that misses names the compressed file whose rollout-<timestamp>-<id> name carries the wanted id. The name is only a hint, never identity. When no compressed name matches, the error gives the count. A Zstandard file passed to --file, or stored under a .jsonl name, is recognised by its magic number and refused or counted as unattributed, instead of being parsed as text. One related gap I left out of this PR: the Past Sessions scanner (codex-transcript.ts, entry.name.endsWith('.jsonl')) also skips compressed rollouts without saying so, so those threads are missing from the list. It shares no file-matching code with this reader, so fixing it would be a separate change.

2. Parent read twice. Each selected rollout is now parsed once, into the views the reader needs: its raw messages for a child's proof, and its own messages plus any legacy prefix. A family read reuses the parent's parse for the proof. The test has the same shape as your numbers: with --scan-bytes equal to the two files' sizes, the family read succeeds and bytesRead equals their sum, and one byte less fails. The same double read also counted the parent's malformed and inherited records twice. Those are now counted once, with a test.

3. Unreadable parent. If the parent's read fails for any reason other than a budget, the child falls back to the unproven-prefix rule (cut at the last candidate, unverifiedPrefixes: 1, complete: false). Two tests cover it: your probe (an invalid UTF-8 byte appended to the parent), and a parent replaced between attribution and the proof read. A budget error during the parent read still fails the read (test). A read of the parent itself still fails on its own bad bytes.

4. --max-bytes is now validated before readLocalHistory, next to the other budgets. The test passes an invalid value with a sessions root that does not exist and gets the --max-bytes error, not "Sessions root not found". While there, I made the budget flags accept plain decimal integers only, so 0x10, 1e3 and 5 are refused instead of being read as numbers.

The skill reference (and its plugin mirror), the README line, the wiki page and the changeset describe the new counter and the single read.

I have not touched the two design points: the skill description and guard, and the top-level history verb. Both are your call.

[Ark0N]

Thanks, Aamer, for the fourth round. The PR adds an offline codeman history read that reads saved Codex rollouts by file, native id, or a root conversation with its sub-agents and forks, with whole-turn topic and UTC date filters and separate, raisable read and output budgets.

All four items from the last review are in 17b2edc, and each is pinned by a test:

1. Compressed rollouts: *.jsonl.zst files are counted in scan.compressedFiles, so complete turns false; a file with a plain .jsonl sibling is skipped as Codex's own discovery skips it, a lookup that misses names the compressed file carrying the wanted id, and a Zstandard file passed to --file is refused by its magic number.
2. One parent read: each selected rollout is parsed once (src/local-history.ts:759), so a family read with --scan-bytes equal to the two files' sizes succeeds with bytesRead equal to their sum, and one byte less fails.
3. Unreadable parent: a parent with an invalid UTF-8 byte, or one replaced after attribution, leaves the child readable with unverifiedPrefixes: 1 and complete: false, while a budget error during that read still fails it.
4. --max-bytes is validated before the store is touched (src/cli-history.ts:89), and the budget flags now take plain decimal integers only.

I also ran it against the Codex store here from outside a Codeman session: all 58 --session-id and --root-session-id lookups on the 29 rollouts with conversation text came back with complete: true, an output overflow exits 1, and --json --format text exits 1. Typecheck, lint, format, the plugin mirror check and validation, and the touched and neighbouring test files pass.

Two design points are still open on my side, and merging waits on them:

• Whether the codeman skill advertises the offline reader in its description and narrows its "refuse to act outside a Codeman session" guard. That covers the frontmatter description, the routing paragraph before section 0 and the section 0 guard sentence in skills/codeman/SKILL.md, their mirror under plugins/codeman/, and the self-gate bullet in docs/wiki/Codeman-Skill.md:126.
• Whether a Codex-only reader takes the top-level history verb. Nothing in the README, the wiki or the skill reference labels the command experimental, so under docs/versioning-policy.md the name and its documented flags become stable CLI surface once released.

Nothing is needed from you on those two.

[Ark0N]

Merged in 1.46.0 (https://github.com/Ark0N/Codeman/releases/tag/codeman%401.46.0). Thank you for four careful rounds. Both design points are settled the way you built it: codeman history is a top-level verb and stable CLI surface from this release, and the codeman skill advertises the offline reader and routes to it before its in-session guard. The README, the plugin README and the plugin and marketplace descriptions now say so too (they still claimed the skill refuses everything outside a session).