Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
75e0411
docs(design): repository-artifact and dependency datamodel for schema v2
lamwassi Nov 20, 2026
cf2cab1
feat(artifacts): emit repository artifacts and go.mod dependencies at L1
lamwassi Nov 20, 2026
144cbcb
docs(design): schema v2 id format for Go callables and types
lamwassi Nov 20, 2026
0f68195
feat(ids): Go callable and type ids use the bare member name (TS/Pyth…
lamwassi Nov 20, 2026
7ed66ce
test(v2emit): drop unused constants after the id-format change
lamwassi Nov 20, 2026
210a511
scripts: add end-to-end eval (analysis.json -> Neo4j projection -> load)
lamwassi Nov 20, 2026
e605867
scripts(eval_e2e): fix bash 3.2 unbound array and build outside the p…
lamwassi Nov 20, 2026
85e2ed1
scripts(eval_e2e): add --json-only to write and check analysis.json only
lamwassi Nov 20, 2026
000ebad
docs(design): cross-file method ids use the declaring file
lamwassi Nov 20, 2026
dd9349b
feat(ids): cross-file methods get ids under their declaring file
lamwassi Nov 20, 2026
19c862a
fix(neo4j): purge every changed module before upserting any
lamwassi Nov 20, 2026
7acabb6
docs(design): add application.name alongside application_name
lamwassi Nov 20, 2026
60ce063
feat(schema): add application.name alongside application_name
lamwassi Nov 20, 2026
6290919
docs(design): start_line and end_line on Go span-bearing models
lamwassi Nov 21, 2026
eb276e1
feat(schema): start_line and end_line on Go span-bearing models
lamwassi Nov 21, 2026
4f95fed
feat(schema): start_line and end_line on the JSON module
lamwassi Nov 21, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 40 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,36 @@ v2 vocabulary (parity clause: add at the leaves, never rename shared names). The
committed spec is the full transcript: `docs/design/specs/v2-l1-emission.md`.

### Identity & positioning (Group A — reused verbatim by L3/L4)
- **`application.name`** [user, 2026-11-20; spec `docs/design/specs/v2-application-name-field.md`]: an additive
field with the same value as `application_name`. `application_name` is kept unchanged (renaming a shared
field is forbidden), and `:GoApplication.name` carries the same value in Neo4j.
- **`application.id`** = `can://<app>` (app outermost, so a polyglot app shares one prefix),
where `<app>` is the **`--app-name`** value, defaulting to the input directory's base name
(per the CLI contract; the SDK's Neo4j backend must use the same anchor). Module ids are
`can://<app>/go/<file>`; externals are language-neutral `can://<app>/@external/<import-path>/<sig>`.
`can://<app>/go/<file>`; externals are language-neutral `can://<app>/@external/<import-path>/<Type>/<Member>` (see § Id member segments).
L3/L4 build ids on this unchanged.
- **Callable signature** = existing `signatureOf()` output as the last `can://` path
segment; receiver folded in for methods. One canonicalizer, unchanged from v1.
- **Id member segments (spec `docs/design/specs/v2-go-ids.md`, issue #46)**: a type is
`<module>/<Type>`; a method is `<module>/<Type>/<Method>`; a package-level function is
`<module>/<Name>` (a method declared in another file uses that file as `<module>`, see
§ Cross-file method ids below); a closure is `<enclosing>/closure_N`. The package import path is dropped:
the member segment is the signature's part after its last dot (Go names contain no dots).
Follows `codeanalyzer-typescript` `idFromSig` and `codeanalyzer-python` `child_id`
[user, 2026-11-20]. Externals: `<app>/@external/<import-path>/<Type>/<Member>` (import path
whole) [user, 2026-11-20].
- **Callable and type `signature`** = the full `signatureOf()` output (import-qualified), a
JSON field on both (types gained it, additive, for the Neo4j `:GoType.signature`) [user,
2026-11-20]. The id carries only the bare member name, so this field is where the package
path lives. One canonicalizer, unchanged from v1.
- **Cross-file method ids** [user, 2026-11-20]: a method declared in another file than its receiver
type gets `<declaring-module>/<Type>/<Method>`, e.g. `…/handlers.go/frontendServer/logoutHandler`.
The node stays in the receiver type's `callables{}`. `source_file` is kept for text recovery.
Neo4j adds `GO_DECLARES` from the declaring `:GoModule` to the method, beside `GO_HAS_METHOD`.
Spec: `docs/design/specs/v2-go-ids.md` § Amendment.
- **Generic types**: `signatureOf()` carries no type parameters, so a generic type or method
ids as `<module>/<Set>/<Add>`, with no `[T]` in the id.
- **`start_line` / `end_line`** [user, 2026-11-21; spec `docs/design/specs/v2-span-lines.md`]: top-level keys next
to `span` on `Module` (added 2026-11-21 so the JSON and Neo4j modules match), `Import`, `Parameter`, `Callable`, `Type`, `Field` and `BodyNode`, equal to `span.start[0]` and
`span.end[0]`. No `code_start_line`. Neo4j nodes carry the same two properties beside `span_json`.
- **`span.bytes`** = **UTF-8 byte offsets** into `module.source` (Go strings are
natively UTF-8; avoids the UTF-16/rune slicing mismatch TS hit in cants#179).

Expand Down Expand Up @@ -64,8 +87,9 @@ committed spec is the full transcript: `docs/design/specs/v2-l1-emission.md`.
`(*strings.Builder).WriteString`) is no longer dropped: the call node's `callee` and the
`call_graph` edge `dst` are the callee's `@external` id. Builtins, type conversions
(no longer minted as `<Type>.__new__`) and calls through function values stay `null`.
- **Id** = `can://<app>/@external/<import-path>/<sig>` where `<sig>` is the callee signature
minus the `<import-path>.` prefix (`Println`, `Builder.WriteString`); it may also name a
- **Id** = `can://<app>/@external/<import-path>/<Type>/<Member>` where `<sig>` is the callee signature
minus the `<import-path>.` prefix; its dots become segments (`Println` → `…/fmt/Println`,
`Builder.WriteString` → `…/strings/Builder/WriteString`); it may also name a
library *interface* method (`@external/io/Writer.Write`). Methods on generic library types
record the declared (origin) signature, not the instance. `@external` is
language-neutral (not `go/`), so a polyglot app shares one external namespace.
Expand Down Expand Up @@ -129,6 +153,17 @@ Reuses Group A ids/spans and the L3 vocabulary unchanged; every L4 fact is an ad
- A Go-specific `select` cfg kind was not added (`select` is a `switch` with `switch_case`
edges).

### Repository-artifact decisions (spec `docs/design/specs/v2-artifacts.md`, branch `feat/artifacts`)
Made with the user in the 2026-11-19 design loop, anchored on `codeanalyzer-python`'s `PyArtifact` / `PyDependency`.
- **Placement:** `application.artifacts{<path>: artifact}` (map); `application.dependencies[]` (list).
- **Scope:** all non-`.go` project files [user, 2026-11-19]. Binaries are metadata only (`source:""`) [user].
- **Artifact id:** `can://<app>/artifact/<path>`, app-scoped (Python parity) [user]. The Go projector's `:Artifact` flips to `CanNode=true`.
- **Package id:** version-less `pkg:golang/<module-path>` [user]; version lives on the dependency edge.
- **Source text:** only with `--artifact-text`, default off [user]. `config_keys` deferred [user].
- **Dependencies:** every `go.mod` require; `direct=false` on `// indirect`; `kind:"runtime"`; `go.sum` supplies `locked_version` only, `go.sum`-only entries excluded [user].
- **`provides_imports`:** longest-prefix match against `go.mod` module paths [user].
- **Level:** L1, every level, not gated [user].

## Neo4j-projection decisions

The graph projection (`--emit neo4j`) is a **second projection of the same v2 envelope**,
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ Flags:
-f, --format string Output format: json|msgpack (default "json")
--graph-field-depth int Field-path depth bound for L3 dataflow nodes (k limit; applies at -a 3 and -a 4) (default 3)
-h, --help help for cango
--artifact-text Capture each text artifact's whole source in analysis.json (default: metadata only)
-i, --input string Project root to analyze (required)
-j, --jobs int Worker parallelism (default: CPU cores)
--library-models string JSON file of library stand-in flows overriding the all-to-all default (-a 4 and --emit neo4j only)
Expand Down
4 changes: 4 additions & 0 deletions cmd/codeanalyzer/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ func rootCmd() *cobra.Command {
libraryModels string
targetFiles []string
skipTests bool
artifactText bool
eager bool
cacheDir string
jobs int
Expand Down Expand Up @@ -172,6 +173,7 @@ via CLDK(language="go").analysis(project_path=...).`,
AnalyzerVersion: version,
TargetFiles: targetFiles,
SkipTests: skipTests,
ArtifactText: artifactText,
Eager: eager,
CacheDir: cacheDir,
Jobs: jobs,
Expand Down Expand Up @@ -238,6 +240,8 @@ via CLDK(language="go").analysis(project_path=...).`,
f.StringSliceVarP(&targetFiles, "target-files", "t", nil,
"Restrict analysis to specific files (incremental mode)")
f.BoolVar(&skipTests, "skip-tests", true, "Skip *_test.go files")
f.BoolVar(&artifactText, "artifact-text", false,
"Capture each text artifact's whole source in analysis.json (default: metadata only)")
f.BoolVar(&eager, "eager", false, "Force clean rebuild (ignore cache)")
f.StringVarP(&cacheDir, "cache-dir", "c", "", "Cache directory (default: ~/.cldk/go-cache)")
f.IntVarP(&jobs, "jobs", "j", 0, "Worker parallelism (default: CPU cores)")
Expand Down
29 changes: 29 additions & 0 deletions docs/design/specs/v2-application-name-field.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Spec: `application.name` alongside `application_name` (codeanalyzer-go)

## Contract-Impact Triage

**Does this change schema v2 output?** Yes, additively. The v2 `application` node gains a
`name` field. No existing field is renamed or removed.

| Change type | Analyzers | SDKs | Docs |
| --- | --- | --- | --- |
| Additive schema field | **codeanalyzer-go** (schema, emit, Neo4j projector) | none now (python-sdk and codeanalyzer-python keep reading `application_name`) | `CLAUDE.md` § Schema decisions, this spec |

## Design-loop decisions

Made with the user on 2026-11-20.

1. **Field.** `application.name` is added to the v2 application node in `analysis.json`. Its
value is the same as `application_name`.
2. **Keep `application_name`.** Renaming it was rejected. The parity rule forbids renaming shared
fields, and `codeanalyzer-python` and `python-sdk` read `application_name`.
3. **Neo4j.** The `:GoApplication` node gets a `name` property with the same value as
`application_name`, so the two projections keep matching.
4. **Tracking.** One issue in `codeanalyzer-go`, one checklist. A PR on `fix-app-name`.
5. **Release.** One PR, one release. The python-sdk can read `name` later; nothing is required
of it now.

## Out of scope

- Removing or deprecating `application_name`. That is a later, separate schema change.
- Changing the python-sdk or codeanalyzer-python readers.
155 changes: 155 additions & 0 deletions docs/design/specs/v2-artifacts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Spec: repository artifacts and dependencies (codeanalyzer-go)

> **Cross-repo spec.** Canonical home is `codellm-devkit/.github → docs/design/specs/`,
> beside the coordinating parent issue. Staged locally on `feat/artifacts` until that repo
> is available (same precedent as `v2-l1-emission.md`). This is the design transcript of the
> datamodel loop, linked from the tracking record, not pasted into it.

## Contract-Impact Triage

**Does this change schema v2 output?** Yes. It adds two application-scoped fields
(`artifacts`, `dependencies`), new id shapes for artifact and package nodes, one new CLI flag
(`--artifact-text`), and populates the already-declared Neo4j neutral layer (`:Artifact`,
`:Package`, `HAS_ARTIFACT`, `DECLARES_DEPENDENCY`, `LOCKS`).

| Change type | Analyzers | SDKs | Docs |
| --- | --- | --- | --- |
| Schema v2 evolution (additive) | **codeanalyzer-go** (emit + projector); codeanalyzer-python: none (parity adopted, no id change) | **python-sdk** (Go backend reconstruction of `:Artifact` / `:Package`) | `CLAUDE.md` § Schema decisions, this spec |

Anchor: `codeanalyzer-python` already emits this layer (`PyArtifact`, `PyDependency`,
`PyConfigKey`, `application.artifacts`, `application.dependencies`,
`codeanalyzer/schema/py_schema.py`), and `python-sdk/cldk/analysis/commons/artifacts.py`
rebuilds it from Neo4j, with the neutral labels shared across analyzers.

## Problem

The v2 schema and projector declare the neutral repository-artifact layer, but the Go
analyzer never populates it. `go.mod` is read only for `go mod download`. Against the Go
analysis of `microservices-demo/src/frontend`, `MATCH (n:Artifact)` returns 0 rows.

## Design-loop decisions (the transcript)

Each decision was made with the user, one question per decision, in the order below.

### Application placement
- **`application.artifacts`** is a **map keyed by repo-relative path**, as Python does. Stays
reachable from `application` by containment (keystone rule).
- **`application.dependencies`** is a **list** (Python parity).

### Artifact scope and fields
- **Scope: all non-`.go` project files** (user decision, broader than the recommended manifests-only
scope). `.go` files are modules, not artifacts. Directories `.git/` and `vendor/` are excluded
(implementation default, see Open items).
- **Artifact fields** (Python `PyArtifact` parity):
| field | type | notes |
| --- | --- | --- |
| `id` | string | `can://<app>/artifact/<path>` |
| `kind` | string | `"artifact"` |
| `path` | string | repo-relative POSIX path (also the map key) |
| `format` | string | `toml\|yaml\|json\|ini\|properties\|requirements\|dockerfile\|text\|binary`, plus Go's `gomod` / `gosum` (Open items) |
| `roles` | string[] | `dependency-manifest` for `go.mod`, `lockfile` for `go.sum`; others only when recognized |
| `size_bytes` | int | always |
| `sha256` | string | full file hash, always |
| `source` | string | whole file text **only** with `--artifact-text`; otherwise `""` |
| `extraction` | string | `none` \| `partial` \| `full`; `full` when `source` is set, else `none` |
- **Binary files**: metadata only. `format:"binary"`, `size_bytes` and `sha256` set,
`source:""` (user decision).
- **`--artifact-text`** (default **off**): when on, `source` carries the whole file text
(user decision). Off keeps default output to metadata.
- **`config_keys`** is **deferred** (user decision): not emitted in this change. No
`:ConfigKey` rows are written.

### Artifact id
- **`can://<app>/artifact/<path>`** (Python parity, user decision). App-scoped, so the
projector's `can://<namespace>/<app>/` prefix scoping covers artifact nodes. This changes the
Go projector's `:Artifact` flag to `CanNode=true`; the current Go schema comment ("neutral,
not a can:// id") is superseded.

### Package and dependencies
- **Package id: version-less `pkg:golang/<module-path>`** (user decision), for example
`pkg:golang/github.com/gorilla/mux`. Mirrors Python's `pkg:pypi/<name>`. The version lives
on the edge and the record, so every dependent app converges on one `:Package`.
- **Dependency records come from every `go.mod` `require`** (user decision). Direct and
indirect entries alike.
| field | source |
| --- | --- |
| `name` | module path |
| `ecosystem` | `"golang"` |
| `spec` | version in `go.mod` (for example `v1.8.0`) |
| `kind` | `"runtime"` for every entry (`go.mod` has no dev/optional split) |
| `direct` | `false` when the line carries `// indirect`, else `true` |
| `locked_version` | version pinned in `go.sum` for that module, when present |
| `declared_in` | the `go.mod` artifact id |
| `provides_imports` | import paths in the project that resolve to this module (see below) |
| `prov` | `["declared"]`, plus `"lockfile"` when `go.sum` supplies `locked_version` |
- **`go.sum`-only entries are not dependencies** (user decision, the non-recommended option
would have included them; the recommended default is not to, because `go.sum` retains
entries for pruned modules). `go.sum` contributes only `locked_version` and `prov`.
- **`provides_imports`**: populated by longest-prefix match of each project import path
against the `go.mod` module paths (user decision, Python parity). Import paths that match
no requirement are **not** recorded in this change (see Open items).

### Analysis level
- **L1, every level** (user decision). Parse-only facts, identical at `-a 1` through `-a 4`.
Not gated.

### Neo4j projection (the neutral layer, unchanged labels)
- `:Artifact {id, path, format}` (keyed by `id`, `CanNode=true`), under `HAS_ARTIFACT`
from `:GoApplication`.
- `:Package {id, ecosystem, name}` keyed by `id`.
- `DECLARES_DEPENDENCY` `(:Artifact)→(:Package)`, `_k = kind` (`runtime`), carrying `spec`,
`direct`, `prov` as properties.
- `LOCKS` `(:Artifact)→(:Package)` from `go.sum`, carrying `locked_version`.
- No `:ConfigKey` rows and no `DEFINES_CONFIG` rows in this change.

## Open items (flagged for review, not asked)

These are defaults I chose while writing the spec. Confirm them during review:

1. **`format` values for go.mod and go.sum.** I used `gomod` and `gosum`, because the Python
enum has no Go case. The alternative is `text`.
2. **Excluded directories.** `.git/` and `vendor/` are skipped. `vendor/` is third-party
source, not project configuration.
3. **Imports with no matching requirement.** Not recorded. They are `unresolved_imports` in
Python. That field is deferred with the repository-provenance field.
4. **`locked_version` when `go.sum` has several lines for one module** (the `h1:` and
`/go.mod` hashes). Take the version common to both lines; error if they disagree.

## Resolution (implementation, branch `feat/artifacts`)

The open items were resolved with the defaults below. Two are deviations from the wording
above, marked **(deviation)**:

1. **`format`**: `gomod` for `go.mod`, `gosum` for `go.sum`, adopted.
2. **Excluded directories**: `.git/` and `vendor/` are skipped at any depth. The tool's own
outputs are also skipped when they sit in the project: `analysis.json` and `graph.cypher`
in the `-o` directory, and the `-c` cache subtree. Without this, a run that writes into the
project would capture the previous run's output.
3. **Unrecorded imports**: an import matching no `go.mod` requirement is not recorded.
4. **Locked version (deviation)**: a requirement is locked only when go.sum has **both** the
`h1:` and the `/go.mod` line for the exact required version. A version with only one line
is left unlocked (`locked_version` absent, `prov` without `lockfile`), with no error. A
disagreement cannot arise, since the lookup is keyed by the required version.
5. **Artifact id root (deviation)**: the code's grammar is `can://<namespace>/<app>/…`, the
same root every other node uses, so an artifact id is
`can://<namespace>/<app>/artifact/<path>`. The `can://<app>/artifact/<path>` form in the
decisions above is shorthand for that root.

Also decided at implementation: `source` is omitted from JSON when empty (`omitempty`), not
written as `""`; only the root `go.mod` contributes dependencies (nested modules are artifacts
only); `dependencies` is absent when there is no root `go.mod`.

## Consumers

- **codeanalyzer-go**: emits the fields; the projector writes the rows above. Tests: a fixture
with `go.mod` and `go.sum`, a projector test for the new rows, and a `--artifact-text`
on/off test.
- **python-sdk**: the Go backend reads `:Artifact` / `:Package` through the shared
`cldk/analysis/commons/artifacts.py` reconstructors. The Go backend needs its artifact getters
added. Scope of that rung is decided at decomposition.

## Out of scope (deferred)

- Config keys and `DEFINES_CONFIG` (user decision).
- `unresolved_imports`, repository provenance (`repository`), `config_uses`.
- Codeanalyzer-python: no change.
Loading
Loading