Repository navigation
doctor: list legacy state (dedicated worktree corpora, unattributed config entries) - #1418
hussain-alsaibai wants to merge 5 commits into
Conversation
c4e0e56 to
7aa2223
Compare
zzet
left a comment
There was a problem hiding this comment.
Changes requested for three P2 gaps in the new legacy-state collector.
-
P2 — Use a daemon-wide census before classifying global config. collectLegacyState:106–121 obtains
list_checkoutsthrough the current-root MCP relay, whose catalog is filtered to that session's workspace. It then compares all global config entries and treats an absent row as no active intent. With healthy tracked repositories in separate workspaces, doctor run from one reports the other as intentless and misses its dedicated corpora. An untracked CWD also fails the relay's preflight. Please use an administrative read-only unbound inventory client, or explicitly honor coverage and never infer absence outside it. Add collector tests for separate workspaces and invocation outside a tracked repo. -
P2 — Disable structural truncation for this machine-consumed census. The same request passes an empty argument map, so
list_checkoutsdefaults to a 40,000-byte budget. Its response budgeter can drop nested checkout/graph rows and whole families; unmarshalling intoFamiliesOverviewignores truncation metadata. The collector still declares the census checked and infers absence. A large worktree inventory can therefore produce false intentless warnings or omit the legacy corpora the command should diagnose. Please request documentedmax_bytes:0, validate completeness before classification, and add an oversized collector fixture with a relevant corpus near the truncated tail. A fields projection can reduce unrelated evidence without dropping required rows. -
P2 — Redact family and prefix identity as well as root paths. dedicatedWorktreeCorpora:169–172 redacts only the root, but copies
DisplayRemoteandRepoPrefixverbatim. JSON exposes both; human rendering also prints the raw prefix. Thus--redactreports can still disclose repository identity, private remotes, and local family paths. Issue #1083 explicitly requires prefix/path redaction. Please use consistent pseudonyms for family/prefix identifiers in both formats and add actual redactor/collector golden tests with raw identifiers, rather than prepopulating already-redacted strings.
Evidence distinction: a focused stdlib helper harness copied the production scope/classifier/redactor helpers and reproduced the scope false positive and identifying-field leak. Its passing assertions confirm those bug behaviors; it is not an end-to-end daemon test. The truncation finding is source-traced through the existing structural budgeter and its oversized fixtures, not a live large-store reproduction.
7aa2223 to
4a31f7a
Compare
Review of zzet#1418 found three ways the collector could report a repository as intentless when it had simply not been looked at. All three are the same shape: partial evidence read as complete evidence. 1. The census was workspace-scoped. `requireCheckoutTool(root, ...)` answered "families visible to THIS session's workspace", so a doctor run inside one tracked repository never saw the healthy repositories tracked in another workspace, and classification read their absence as "no active intent". An untracked cwd failed the relay preflight outright. The census now dials an unbound administrative session, which the tool documents as the daemon-wide inventory -- the only read whose absence means what it appears to mean. `collectLegacyState` no longer takes a root it did not use. 2. The census was structurally truncatable. An empty argument map let `list_checkouts` apply its 40,000-byte default; the budgeter drops nested checkout/graph rows and whole families when a large inventory overflows, and unmarshalling into FamiliesOverview ignores the truncation metadata. The request now sends `max_bytes: 0` and fails closed when the response still carries `_truncated_by_budget`. 3. --redact only rewrote the root. `display_remote` and `repo_prefix` were copied verbatim into the JSON, and the human rendering printed the raw prefix, so a redacted report could still name a private remote. Both now go through a stable per-run pseudonym. Tests cover each arm via new seams (`doctorCensusTool`, `doctorLegacyEnv`, `doctorConfiguredPaths`) so the collector runs without a live daemon, plus a separate-workspaces fixture and a raw-identifier redaction golden. NOT COMPILED: this container has no C compiler, and the package imports CGO tree-sitter parsers, so `go test ./cmd/gortex` cannot run here. The fetch / truncation / classification control flow was verified by porting it to a CGO-free harness that passed 10/10 and went red (4 failures) when the old behaviour was restored. `gofmt -e` parses all four files. It needs a real `go test ./cmd/gortex` on a CGO-enabled runner before this is pushed.
|
Addressed all three P2 findings in P2-1 — daemon-wide census. P2-2 — structural truncation. The census requests P2-3 — identity redaction. Verification status — please read. I could not run |
zzet
left a comment
There was a problem hiding this comment.
Reviewed commit: 4a31f7a3e663951f48da80beac3d5cbca5dfc076.
[P2] Redact every catalog/config root — cmd/gortex/doctor_legacy.go:262,284,294. Passing roots through doctorPath does not satisfy this section's redaction contract: doctor_redact.go:68–72 leaves roots outside the current checkout and home unchanged, and home-relative roots retain other checkout names. A complete census can therefore disclose unrelated repository paths in both JSON and human output despite --redact. A focused regression using the real collector and redactor reproduced both output leaks; all 14 existing Legacy test roots passed. Use consistent aliases for every catalog/config root, preserving correlations without exposing the original paths, and cover sibling-home and outside-home roots in both formats.
[P2] Restrict unattributed classification to manual-config-only intents — cmd/gortex/doctor_legacy.go:291–299,308. The current exclusion checks only cli_track/mcp_track, so a configured checkout with project_membership alone, or mixed with manual_config, is reported as unattributed and can receive the auto-indexed heuristic and remediation. That is known provenance, whereas the stated classification requires only manual_config. Add an explicit manual-config-only intent-set guard and project-only/mixed-intent negatives. This second finding is based on source tracing; no additional runtime reproduction is claimed.
|
Addressed both P2 findings in P2 - Redact every catalog/config root. The census is daemon-wide, so a root can live in a sibling home or outside home entirely, and The census now gets its own root redactor: P2 - Restrict unattributed classification to manual-config-only intents. Tests. Verification note. The cgo tree-sitter grammars do not build on this host (198 build-constraint exclusions from the |
zzet
left a comment
There was a problem hiding this comment.
The previous off-CWD/sibling-home root-redaction issue and the project_membership-only/mixed false-positive issue are fixed in this revision. One P2 completeness issue remains.
[P2] Classify the manual-config source-kind set, rather than counting intent rows
At cmd/gortex/doctor_legacy.go:348–351, isManualConfigOnly requires exactly one entry. However, CheckoutOverview.Intents contains one entry per active tracking row, not one per distinct source kind (internal/indexer/checkout_overview.go:338–350). The catalog permits separate rows with the same kind and different locators: its upsert key is (checkout_id, source_kind, source_locator) (internal/graph/store_sqlite/catalog.go:1149–1155), and the intent validator accepts those distinct locators.
A configured checkout with active (manual_config, /old/root) and (manual_config, /new/root) rows therefore yields ["manual_config", "manual_config"]. There is no CLI, MCP or project attribution, but the new predicate returns false and the collector suppresses its unattributed warning/count. The source path also permits retained locators: recordCheckout preserves an existing checkout ID, then upserts SourceLocator: root; config seeding calls it with TrackSourceConfig, without revoking the prior locator in that path.
Require a nonempty list in which all kinds are manual_config, allowing duplicate rows. Keep excluding project_membership, cli_track, mcp_track, and mixed manual/attributed rows. Add regressions for two manual-config rows and for mixed rows so the previous fix stays intact.
This finding is derived from the catalog/projection/classifier source invariants. No current-head native reproduction or end-to-end relocation run is claimed.
e1185a5 to
7861744
Compare
Review of zzet#1418 found three ways the collector could report a repository as intentless when it had simply not been looked at. All three are the same shape: partial evidence read as complete evidence. 1. The census was workspace-scoped. `requireCheckoutTool(root, ...)` answered "families visible to THIS session's workspace", so a doctor run inside one tracked repository never saw the healthy repositories tracked in another workspace, and classification read their absence as "no active intent". An untracked cwd failed the relay preflight outright. The census now dials an unbound administrative session, which the tool documents as the daemon-wide inventory -- the only read whose absence means what it appears to mean. `collectLegacyState` no longer takes a root it did not use. 2. The census was structurally truncatable. An empty argument map let `list_checkouts` apply its 40,000-byte default; the budgeter drops nested checkout/graph rows and whole families when a large inventory overflows, and unmarshalling into FamiliesOverview ignores the truncation metadata. The request now sends `max_bytes: 0` and fails closed when the response still carries `_truncated_by_budget`. 3. --redact only rewrote the root. `display_remote` and `repo_prefix` were copied verbatim into the JSON, and the human rendering printed the raw prefix, so a redacted report could still name a private remote. Both now go through a stable per-run pseudonym. Tests cover each arm via new seams (`doctorCensusTool`, `doctorLegacyEnv`, `doctorConfiguredPaths`) so the collector runs without a live daemon, plus a separate-workspaces fixture and a raw-identifier redaction golden. NOT COMPILED: this container has no C compiler, and the package imports CGO tree-sitter parsers, so `go test ./cmd/gortex` cannot run here. The fetch / truncation / classification control flow was verified by porting it to a CGO-free harness that passed 10/10 and went red (4 failures) when the old behaviour was restored. `gofmt -e` parses all four files. It needs a real `go test ./cmd/gortex` on a CGO-enabled runner before this is pushed.
|
Rebased onto the current The rebase was clean and the change is unchanged:
For the record, the six files are: |
|
Addressed in You are right, and the distinction is the whole point: Confirmed in the source before changing anything:
So Regressions added
That brings the file to twelve VerificationThe cgo tree-sitter grammars still do not build on this host, so I checked the predicate against the old one over nine intent sets in an isolated harness: the new one is correct on all nine, and the old one is wrong on exactly two - the duplicate-row cases you identified: |
zzet
left a comment
There was a problem hiding this comment.
Reviewed commit: 81fbce5c652c1cf0b612a9dec26fad863cde447a.
The duplicate-manual-row warning and earlier root/provenance-redaction findings are fixed. One P2 completeness issue remains: a global-config read or parse failure is treated as an empty configured-repository list, allowing the new section to report a completed check with no legacy state. Please propagate that failure as incomplete/error and cover both JSON and human output.
This finding is source-derived, not runtime-reproduced. No new native test or end-to-end daemon run is claimed.
| path = p | ||
| } | ||
| gc, err := config.LoadGlobal(path) | ||
| if err != nil || gc == nil { |
There was a problem hiding this comment.
[P2] Preserve global-config load failures as an incomplete inventory
This branch returns nil for every config.LoadGlobal error, indistinguishable from a successfully read configuration with no repositories. With a running daemon and a valid exhaustive checkout census, a malformed config.yaml (for example, repos: [) or an unreadable config therefore reaches classifyConfiguredRepos as an empty list after Checked has already been set to true. When no dedicated worktree corpora are present, JSON reports legacy_state.checked: true with no findings and the human section prints "no legacy state found".
The other doctor sections do not report this global-config error: their probes cover daemon/binary wiring and agent integration, and the exit calculation considers runtime blockers. Thus the configured-entry inventory was not examined, but the new section presents it as complete.
Please return/propagate the config-load error and mark this inventory incomplete/error instead of empty success; warning-only exit semantics can remain unchanged. Add actual-reader/collector regressions for malformed YAML and a read failure, checking both JSON state and human output. Preserve the legitimate missing-file case, which LoadGlobal already treats as an empty config.
Source-derived finding; no runtime reproduction claimed.
…ntries) Older versions left state on disk that nothing reported: a corpus per linked worktree, and config entries the daemon wrote for itself. Neither is visible today, and both cost disk and memory constantly. Add a "Legacy state" section to `gortex doctor` and a `legacy_state` key in `--json`. When the daemon answers, it reads the existing `list_checkouts` relay for the catalog and config.yaml for the configured repo list; when the daemon is down it prints one line and never opens the store. - Dedicated worktree corpora: every linked worktree (AdminName non-empty) with effective_mode dedicated and its own GraphID, with family, prefix, graph state, live coordinator and intents. A corpus that is also the family primary is called out, since removing it is not a simple cleanup. - Unattributed config entries: configured repos whose only intent is manual_config. "Likely auto-indexed" when the intent's UUIDv7 timestamp is more than 10 minutes after the checkout's — a real entry gets both in one boot, a leaked one on two. Both ids are already UUIDv7, so no schema step is needed. - Intent-less config entries: configured repos with no active intent, or no checkout row at all. Wording says "unattributed", never "leaked", and every kind names the fix the user can run today (`gortex untrack` / `gortex track`). Findings are warnings, so the exit status is unchanged, and doctor still writes zero bytes. `list_checkouts` gains additive `intent_details` (intent id, source kind, created_at) alongside the existing `intents` list, which every current consumer keeps reading. Tests cover the classification fixture the issue specifies (real intents, an offline-track record, a simulated leak, an intent-less entry), the UUIDv7 timestamp decode, the worktree/primary/automatic discrimination, both renderings, and the skipped-daemon line.
Review of zzet#1418 found three ways the collector could report a repository as intentless when it had simply not been looked at. All three are the same shape: partial evidence read as complete evidence. 1. The census was workspace-scoped. `requireCheckoutTool(root, ...)` answered "families visible to THIS session's workspace", so a doctor run inside one tracked repository never saw the healthy repositories tracked in another workspace, and classification read their absence as "no active intent". An untracked cwd failed the relay preflight outright. The census now dials an unbound administrative session, which the tool documents as the daemon-wide inventory -- the only read whose absence means what it appears to mean. `collectLegacyState` no longer takes a root it did not use. 2. The census was structurally truncatable. An empty argument map let `list_checkouts` apply its 40,000-byte default; the budgeter drops nested checkout/graph rows and whole families when a large inventory overflows, and unmarshalling into FamiliesOverview ignores the truncation metadata. The request now sends `max_bytes: 0` and fails closed when the response still carries `_truncated_by_budget`. 3. --redact only rewrote the root. `display_remote` and `repo_prefix` were copied verbatim into the JSON, and the human rendering printed the raw prefix, so a redacted report could still name a private remote. Both now go through a stable per-run pseudonym. Tests cover each arm via new seams (`doctorCensusTool`, `doctorLegacyEnv`, `doctorConfiguredPaths`) so the collector runs without a live daemon, plus a separate-workspaces fixture and a raw-identifier redaction golden. NOT COMPILED: this container has no C compiler, and the package imports CGO tree-sitter parsers, so `go test ./cmd/gortex` cannot run here. The fetch / truncation / classification control flow was verified by porting it to a CGO-free harness that passed 10/10 and went red (4 failures) when the old behaviour was restored. `gofmt -e` parses all four files. It needs a real `go test ./cmd/gortex` on a CGO-enabled runner before this is pushed.
…anual-config-only intents
a867221 to
2df5cb4
Compare
|
Addressed in You are right, and the fix is smaller than the finding sounds because data, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
// Absent GlobalConfig is not an error - return empty struct.
return gc, nil
}
return nil, fmt.Errorf("reading global config: %w", err)
}
if err := yaml.Unmarshal(data, gc); err != nil {
return nil, fmt.Errorf("parsing global config: %w", err)
}A missing file returns
Regression
That brings the file to thirteen VerificationAs before, the cgo tree-sitter grammars do not build on this host, so I confirmed the behaviour change in an isolated harness running the old and new collector side by side. The old one claims a completed check: and the new one reports the failure in both formats: One note on my own verification: stubbing the four failing grammars let the package compile far enough for |
What
Adds a Legacy state section to
gortex doctorand alegacy_statekey to--json, closing #1083. Read-only listing only — no removal, no schema step.How
AdminName != "") witheffective_mode: dedicatedand their ownGraphID. The family's main checkout and every automatic checkout are skipped, so a 20-worktree fixture yields 0 lines for the automatic ones. A corpus that is also the family primary gets its own note.manual_config. Acli_track/mcp_trackintent is never flagged.The auto-indexed heuristic needs no schema change
A leaked entry gets its checkout row on one boot and its
manual_configintent on a later one.CheckoutIDandIntentIDare bothuuid.NewV7(), so both creation times are already in the ids. The section decodes the 48-bit millisecond prefix of each and flags the entry when the gap exceeds 10 minutes. A real entry gets both within seconds; the offline-tracked false positive has the same timestamp and is not flagged.To expose the intent side,
list_checkoutsgains an additiveintent_detailsarray (intent id, source kind, created_at). The existingintentssource-kind list is unchanged, so every current consumer keeps working.Done-when coverage
cli_track/mcp_trackentries unflaggged; offline-track (manual_config, same timestamps) flagged unattributed only; simulated leak (T → T+1h) flagged unattributed and likely auto-indexed; intent-less entry flagged; 0 recorded-intent entries flagged —TestLegacyStateClassificationTestDedicatedWorktreeCorporalegacy_stateJSON incl. the "daemon not running" line —TestPrintDoctorLegacyFindings,TestPrintDoctorLegacyDaemonDown,TestLegacyStateJSONShapedoctorwrites 0 bytes: the new section issues no write call at all, and readsconfig.yamlthroughconfig.LoadGlobal, which is read-only.doctorExitstill only readsruntimeblockers.--redactapplies to every path in the section (all roots go throughdoctorPath).Not covered here, because it needs the release benchmark host with the real 20-worktree fixture: the ≤1 s wall-time check (
with vs. withouton the upgraded store), and the end-to-end count against the v0.64.10-shape fixture manifest. The section adds no new daemon round trip — it reuses thelist_checkoutscallgortex repos familiesalready makes — so the only cost is one relay on a command that already performed a handshake.Test plan
./internal/indexer/non-test code builds; its test binary exceeds this machine's 7 GiB RAM (compile: signal: killed), which also reproduces on the untouched tree, so it is an environment limit here rather than a regression from this change. CI runs it normally.