Skip to content

feat: compact held backlog summaries - #18

Merged
Profreshor merged 2 commits into
mainfrom
fm/helm-idle-wake-and-digest
Nov 12, 2026
Merged

Profreshor merged 2 commits into
mainfrom
fm/helm-idle-wake-and-digest

Conversation

@Profreshor

@Profreshor Profreshor commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Intent

Regarding your wastes list, I would say go for all three. Since we're using a custom fork of firstmate, we need to be mindful of the upstream changes, especially when inefficiencies are patched and try to stay as current as possible. I do not want to optimize for the sake of optimization, rather I would like to see tangible refinements.

Context: the "wastes list" is three costs seen at the firstmate primary session on 2026-11-12: (1) idle supervision cycle-boundary wake-ups reach the primary, which re-reads its whole context to find nothing queued; (2) after a no-mistakes worker commits, the primary is woken only to tell it to start its validation run; (3) the context re-emit digest after a compaction is about 125 KB and the away return brief prints every held backlog item in full. The fork was synced with upstream on 2026-11-12 (Profreshor/firstmate PR 16); the sync report's section "The three wastes" found all three still open for a Claude-run primary and names the exact code owners: for (1), the Claude park boundary in bin/fm-supervision-host.sh (boundary_exit, default 27000 seconds under the 28800-second Stop hook registration), delivered as an actionable host close by bin/fm-claude-stop-autoarm.sh, with upstream's Codex-only silent park (c61646f) as the only existing silent boundary; for (2), the first committed done: handoff in fm_dod_block in bin/fm-dod-lib.sh; for (3), bin/fm-session-start.sh --reemit, which still emits the fleet, every task's metadata and bounded status tail, the full preference and learning files, and a compact backlog view that keeps every held, in-flight, and blocked row with its full reason, and bin/fm-afk-return.sh, which renders the entire held listing with hold reasons.

What Changed

  • Add a shared formatter that condenses held backlog entries to their ID, title, hold kind, and hold-until date while linking to the full-reason command.
  • Use the compact held-entry format in AFK return briefs and fm-session-start --reemit; retain full held details during normal startup.
  • Document and test the compact re-emit and return-brief behavior.

Re-emit size

Measured on a synthetic fixture home with real tasks-axi 0.2.6: 44 held items, each with a 320-character reason and a 95-character title, plus 6 in flight, 4 blocked, 20 ready, and about 71 KB of preference and learning filler.
The before figure comes from the pre-change scripts (git archive of the base), the after figure from this branch, both with fm-session-start.sh --reemit.

Fixture --reemit Before After
Whole output 110757 bytes 91698 bytes
Held group 24653 bytes 5576 bytes

The operating home's own held listing (44 items, read-only) measured 23903 bytes in full form and 5672 bytes in the new form.
Most of what remains in that home's re-emit is its curated preference and learning files, which this change does not shorten.

Scope: this PR is only the held-list part of the third waste.
The second waste ships in a sibling PR, and the first waste was measured and moved to a separate diagnosis task.

No activation is needed: the change takes effect when the home updates to it.

Risk Assessment

✅ Low: The bounded held-list compaction is correctly shared by re-emit and AFK-return paths, the fixer-round helper is now zero-argument as required, and no material correctness, security, scope, or test-quality defect was substantiated.

Testing

In two disposable marked lab homes, I seeded real held tasks and drove the re-emit and away-return CLIs. Both rendered concise held rows plus the full-reasons command, while omitting a deliberately oversized sensitive reason; both labs were removed afterward.

  • Live validation: ✅ go - 2 of 2 scenarios driven live against the product
Scenario Result Live Evidence
Context re-emit compacts a held task ✅ pass live held-list-reemit-live.txt
Away return does not leak full held reasons ✅ pass live held-list-away-return-live.txt
Evidence: Live --reemit digest with a long held reason omitted

Source: Live --reemit digest with a long held reason omitted

RE-EMIT CUE: if this output ends with "Output too large ... Full output saved to <path>",
read that whole file now, before acting. Otherwise read data/captain.md, data/captain-shared.md,
and data/learnings.md under /tmp/fm-lab.BxO5e0/data now if any of them is missing below.

================================================================================
SESSION START (CONTEXT RE-EMIT) - /tmp/fm-lab.BxO5e0
================================================================================
This session already took the helm at its own startup and has only lost its
context. Lock ownership is re-verified and the durable records below are
reprinted, but the sweeps startup already reconciled - project clone refresh,
secondmate convergence and liveness, pending remote handoff
retry, X-mode artifact writes, and stale Herdr child cleanup - are NOT repeated.
Queued wakes ARE still drained: they arrived after startup and are this turn work.

LOCK
--------------------------------------------------------------------------------
lock acquired: harness pid 2962076

HOME ROLE
--------------------------------------------------------------------------------
fm-role: user (default)

BOOTSTRAP
--------------------------------------------------------------------------------
BOOTSTRAP_INFO: lavish-axi >=0.1.80 enables confirmed board replies; this older compatible version retains the legacy reply path, but upgrade to prevent handing back a board before its reply is accepted

WAKE QUEUE
--------------------------------------------------------------------------------
(no queued wakes)
================================================================================
SUPERVISION OPERATING INSTRUCTIONS - primary harness: codex
================================================================================
Current state:
- Lock: held by this session; this session owns normal supervision unless away mode says otherwise.
- Away/quiet mode: inactive.
- X mode: inactive; use the default watcher cadence.
- Ordinary wake: take the next foreground bin/fm-watch-checkpoint.sh checkpoint as directed below.

Mode: Codex foreground checkpoint.

When this session owns supervision and away mode is not active:
1. Drain first with `bin/fm-wake-drain.sh`.
   After handling all emitted wakes and reconciling open decisions and unread status lines, run the exact `--ack-through` command printed as `WAKE_ACK_REQUIRED`; until then the work remains durable for idempotent re-handling after interruption.
2. Source `/tmp/fm-lab.BxO5e0/config/x-mode.env` first when Relay is active.
3. First cycle: run one foreground watcher checkpoint with `bin/fm-watch-checkpoint.sh --seconds "${FM_CODEX_WATCH_CHECKPOINT:-180}"`.
4. Ordinary wake: if the command prints `signal:`, `stale:`, `check:`, or `heartbeat`, drain queued wakes, handle that wake, then start the next checkpoint.
5. If the command prints `checkpoint:` or exits 124 with no wake, drain queued wakes anyway, process any queued user message now visible to Codex, then start the next checkpoint.
6. Never use shell `&` or Codex background tasks for firstmate watcher supervision.
7. Do not run `bin/fm-watch-arm.sh` as Codex's normal supervision command.
   If it is ever shelled anyway, a backgrounded, piped, or bundled anti-pattern is denied automatically by the PreToolUse seatbelt (`bin/fm-arm-pretool-check.sh`) registered in `.codex/hooks.json`.
8. Failure or missing cycle only: drain queued wakes, inspect the failure, then start a fresh foreground checkpoint.

Codex cannot reason while a foreground tool call is running.
The bounded checkpoint returns control regularly so user messages and queued wakes can be handled without relying on background-task wake semantics.


================================================================================
READ-ONCE CONTRACT
================================================================================
Everything below is printed in full for this session start: every state/*.meta,
a compact data/backlog.md listing, a bounded tail of every state/*.status,
data/projects.md, data/secondmates.md, data/captain.md, data/captain-shared.md,
and data/learnings.md.
Do NOT re-read any of them after reading this digest, and do NOT bulk-read
data/backlog.md or state/*.status: re-reading everything defeats the entire
point of this command.

Go to a source directly only when:
  - this digest flagged it ABSENT (then rebuild or create it per AGENTS.md),
  - its contents looked unparseable or corrupt,
  - an individual full status log is needed for older wake-event history, or a
    status line was capped and its tail matters (each task's full log path is
    printed with its tail),
  - a full task body is needed (bin/fm-tasks-axi.sh show <id> --full, or data/backlog.md),
  - the backlog listing disclosed omitted queued items and this turn needs them,
  - the NETWORK CHECKS section reported its checks still IN PROGRESS and this
    turn needs their verdict (bin/fm-startup-network.sh report),
  - or a STARTUP TRUNCATED banner named the stage that would have printed it, in
    which case that stage's sources were never emitted and must be reconciled,
  - or a context re-emit opened with a read cue because the
    harness cut the digest short.

================================================================================
FLEET STATE
================================================================================

data/backlog.md
--------------------------------------------------------------------------------
compact backlog listing (tasks-axi; done rows omitted; every in-flight and blocked row shown in full; held rows one line each; ready queued bounded to 20; task bodies omitted)

in flight:
count: 0
tasks: 0 in_flight tasks in this backlog

held (captain- or time-gated; an in-flight item that is also held appears in both groups):
1 held item(s), one line each (id | title | hold kind | until); every full hold reason: bin/fm-tasks-axi.sh list --state held --fields hold_kind,hold_reason,hold_until
  held-lengthy | Approve the deliberately long deployment exception | captain | until -

blocked queued:
count: 0
tasks: 0 blocked queued tasks in this backlog

ready queued (dispatchable now):
count: 0
ready: 0 unblocked queued tasks
ready_public_followups: 0 delivery-ready obligations
Full task bodies remain available on demand: bin/fm-tasks-axi.sh show <id> --full when compatible tasks-axi is available, or data/backlog.md.

Work under way (state/*.meta)
--------------------------------------------------------------------------------
(none)

Orphan status logs (state/*.status without matching .meta)
--------------------------------------------------------------------------------
(none)

AFK
--------------------------------------------------------------------------------
absent

================================================================================
NETWORK CHECKS
================================================================================
completed off the startup path in 1s: GitHub authentication.
(silent - no problems found)

================================================================================
CONTEXT
================================================================================

data/projects.md
--------------------------------------------------------------------------------
ABSENT

data/secondmates.md
--------------------------------------------------------------------------------
ABSENT

data/captain.md
--------------------------------------------------------------------------------
ABSENT

data/captain-shared.md (shared, main-authoritative, read-only in secondmate homes)
--------------------------------------------------------------------------------
ABSENT

data/learnings.md
--------------------------------------------------------------------------------
ABSENT

================================================================================
NEXT STEP
================================================================================
Follow the supervision operating instructions block above for harness 'codex'.
This script never starts supervision itself.

The digest above is complete for this session start. The READ-ONCE CONTRACT
section near the top of it governs what may still be read from disk.
Evidence: Live away-return brief with a long held reason omitted

Source: Live away-return brief with a long held reason omitted

fm-afk-launch: away-posture record archived at /tmp/fm-lab.BxO5e0/state/afk-contracts/1794506587.afk-contract
fm-afk-launch: away mode stopped; no daemon terminal was running, .afk cleared, and the posture record archived
=== Return brief (away 2026-11-12T18:03:07Z -> 2026-11-12T18:03:09Z, 2s) ===
Supervisor health:
  - GAP: the watcher beat was 999999s old at return (grace 300s)
Your instructions:
  your words at entry:
    Return after reviewing the held deployment exception.
  the away session took no action under them.
Waiting on you:
  held in the backlog:
    1 held item(s), one line each (id | title | hold kind | until); every full hold reason: bin/fm-tasks-axi.sh list --state held --fields hold_kind,hold_reason,hold_until
      held-lengthy | Approve the deliberately long deployment exception | captain | until -
Tried and failed, or could not be fixed:
  (nothing)
Landed, cleanup due:
  (nothing)
Handled while away:
  0 outcome(s) handled by the away session (0 routine, 0 escalated above)
  (no routine outcomes recorded in the store for this window)
Cost: 0 supervision outcome(s) recorded (0 routine, 0 captain); 0 task(s) live at return.
catch-up health: GAP: the watcher beat was 999999s old at return (grace 300s)
fm-afk-return: catch-up clear; ordinary captain work may proceed
away_return_exit=0

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed ✅
  • 🚨 bin/fm-session-start.sh:527 - Intent conformance: the required goal says to “go for all three,” including (1) preventing the Claude 27,000-second park-boundary wake from reaching the primary (the stated owners are bin/fm-supervision-host.sh and bin/fm-claude-stop-autoarm.sh) and (2) starting validation from the committed fm_dod_block handoff in bin/fm-dod-lib.sh. This range changes none of those owners; its changed re-emit path only compacts held rows. It also leaves the other named re-emit bulk (fleet, task metadata/status tails, and preference/learning files) unchanged. Confirm this is intentionally a partial delivery, or extend the change to the required three refinements.
  • ⚠️ bin/fm-hold-reason-lib.sh:86 - Simplification: fm_hold_compact_stream adds a configurable &lt;full-reasons command&gt; argument, but both callers pass the identical literal. No stated requirement needs this option; make the helper zero-argument (with the command fixed internally) to remove the speculative parameter.

🔧 Fix applied.
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 2 of 2 scenarios driven live against the product
Scenario Result Live Evidence
Context re-emit compacts a held task ✅ pass live held-list-reemit-live.txt
Away return does not leak full held reasons ✅ pass live held-list-away-return-live.txt
  • FM_HOME=<marked disposable lab> bin/fm-tasks-axi.sh add/hold ...
  • FM_HOME=<marked disposable lab> bin/fm-session-start.sh --reemit --source codex
  • FM_HOME=<marked disposable lab> bin/fm-afk-contract.sh enter ... && bin/fm-afk-return.sh begin
  • Output assertions confirming compact rows and absence of the deliberately oversized hold reason
  • tests/fm-session-start.test.sh &amp;&amp; tests/fm-afk-return.test.sh (started as targeted regression coverage; live artifacts above are the acceptance evidence)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…turn brief

A context re-emit follows a startup that already listed every held row with
its full reason, and the away return brief printed every held reason too.
Both now print each held item as one line (id, title, hold kind, until date)
with the count and the command that prints every full reason. In-flight and
blocked rows, open decisions, and the first startup listing stay complete.
tasks-axi lists a hold whose until date has arrived as ready work, so no
due hold is hidden.

On a 44-held-item fixture the re-emit drops from 110757 to 91698 bytes; its
held group drops from 24653 to 5576 bytes.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T18:11:00.907063Z e2310cb PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Profreshor
Profreshor merged commit dab48d0 into main Nov 12, 2026
67 of 71 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant