Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,17 @@ Reuses Group A ids/spans and the L3 vocabulary unchanged; every L4 fact is an ad
- A Go-specific `select` cfg kind was not added (`select` is a `switch` with `switch_case`
edges).

### Repository-artifact decisions (spec `docs/design/specs/v2-artifacts.md`, branch `feat/artifacts`)
Made with the user in the 2026-11-19 design loop, anchored on `codeanalyzer-python`'s `PyArtifact` / `PyDependency`.
- **Placement:** `application.artifacts{<path>: artifact}` (map); `application.dependencies[]` (list).
- **Scope:** all non-`.go` project files [user, 2026-11-19]. Binaries are metadata only (`source:""`) [user].
- **Artifact id:** `can://<app>/artifact/<path>`, app-scoped (Python parity) [user]. The Go projector's `:Artifact` flips to `CanNode=true`.
- **Package id:** version-less `pkg:golang/<module-path>` [user]; version lives on the dependency edge.
- **Source text:** only with `--artifact-text`, default off [user]. `config_keys` deferred [user].
- **Dependencies:** every `go.mod` require; `direct=false` on `// indirect`; `kind:"runtime"`; `go.sum` supplies `locked_version` only, `go.sum`-only entries excluded [user].
- **`provides_imports`:** longest-prefix match against `go.mod` module paths [user].
- **Level:** L1, every level, not gated [user].

## Neo4j-projection decisions

The graph projection (`--emit neo4j`) is a **second projection of the same v2 envelope**,
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ Flags:
-f, --format string Output format: json|msgpack (default "json")
--graph-field-depth int Field-path depth bound for L3 dataflow nodes (k limit; applies at -a 3 and -a 4) (default 3)
-h, --help help for cango
--artifact-text Capture each text artifact's whole source in analysis.json (default: metadata only)
-i, --input string Project root to analyze (required)
-j, --jobs int Worker parallelism (default: CPU cores)
--library-models string JSON file of library stand-in flows overriding the all-to-all default (-a 4 and --emit neo4j only)
Expand Down
4 changes: 4 additions & 0 deletions cmd/codeanalyzer/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ func rootCmd() *cobra.Command {
libraryModels string
targetFiles []string
skipTests bool
artifactText bool
eager bool
cacheDir string
jobs int
Expand Down Expand Up @@ -172,6 +173,7 @@ via CLDK(language="go").analysis(project_path=...).`,
AnalyzerVersion: version,
TargetFiles: targetFiles,
SkipTests: skipTests,
ArtifactText: artifactText,
Eager: eager,
CacheDir: cacheDir,
Jobs: jobs,
Expand Down Expand Up @@ -238,6 +240,8 @@ via CLDK(language="go").analysis(project_path=...).`,
f.StringSliceVarP(&targetFiles, "target-files", "t", nil,
"Restrict analysis to specific files (incremental mode)")
f.BoolVar(&skipTests, "skip-tests", true, "Skip *_test.go files")
f.BoolVar(&artifactText, "artifact-text", false,
"Capture each text artifact's whole source in analysis.json (default: metadata only)")
f.BoolVar(&eager, "eager", false, "Force clean rebuild (ignore cache)")
f.StringVarP(&cacheDir, "cache-dir", "c", "", "Cache directory (default: ~/.cldk/go-cache)")
f.IntVarP(&jobs, "jobs", "j", 0, "Worker parallelism (default: CPU cores)")
Expand Down
155 changes: 155 additions & 0 deletions docs/design/specs/v2-artifacts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
# Spec: repository artifacts and dependencies (codeanalyzer-go)

> **Cross-repo spec.** Canonical home is `codellm-devkit/.github → docs/design/specs/`,
> beside the coordinating parent issue. Staged locally on `feat/artifacts` until that repo
> is available (same precedent as `v2-l1-emission.md`). This is the design transcript of the
> datamodel loop, linked from the tracking record, not pasted into it.

## Contract-Impact Triage

**Does this change schema v2 output?** Yes. It adds two application-scoped fields
(`artifacts`, `dependencies`), new id shapes for artifact and package nodes, one new CLI flag
(`--artifact-text`), and populates the already-declared Neo4j neutral layer (`:Artifact`,
`:Package`, `HAS_ARTIFACT`, `DECLARES_DEPENDENCY`, `LOCKS`).

| Change type | Analyzers | SDKs | Docs |
| --- | --- | --- | --- |
| Schema v2 evolution (additive) | **codeanalyzer-go** (emit + projector); codeanalyzer-python: none (parity adopted, no id change) | **python-sdk** (Go backend reconstruction of `:Artifact` / `:Package`) | `CLAUDE.md` § Schema decisions, this spec |

Anchor: `codeanalyzer-python` already emits this layer (`PyArtifact`, `PyDependency`,
`PyConfigKey`, `application.artifacts`, `application.dependencies`,
`codeanalyzer/schema/py_schema.py`), and `python-sdk/cldk/analysis/commons/artifacts.py`
rebuilds it from Neo4j, with the neutral labels shared across analyzers.

## Problem

The v2 schema and projector declare the neutral repository-artifact layer, but the Go
analyzer never populates it. `go.mod` is read only for `go mod download`. Against the Go
analysis of `microservices-demo/src/frontend`, `MATCH (n:Artifact)` returns 0 rows.

## Design-loop decisions (the transcript)

Each decision was made with the user, one question per decision, in the order below.

### Application placement
- **`application.artifacts`** is a **map keyed by repo-relative path**, as Python does. Stays
reachable from `application` by containment (keystone rule).
- **`application.dependencies`** is a **list** (Python parity).

### Artifact scope and fields
- **Scope: all non-`.go` project files** (user decision, broader than the recommended manifests-only
scope). `.go` files are modules, not artifacts. Directories `.git/` and `vendor/` are excluded
(implementation default, see Open items).
- **Artifact fields** (Python `PyArtifact` parity):
| field | type | notes |
| --- | --- | --- |
| `id` | string | `can://<app>/artifact/<path>` |
| `kind` | string | `"artifact"` |
| `path` | string | repo-relative POSIX path (also the map key) |
| `format` | string | `toml\|yaml\|json\|ini\|properties\|requirements\|dockerfile\|text\|binary`, plus Go's `gomod` / `gosum` (Open items) |
| `roles` | string[] | `dependency-manifest` for `go.mod`, `lockfile` for `go.sum`; others only when recognized |
| `size_bytes` | int | always |
| `sha256` | string | full file hash, always |
| `source` | string | whole file text **only** with `--artifact-text`; otherwise `""` |
| `extraction` | string | `none` \| `partial` \| `full`; `full` when `source` is set, else `none` |
- **Binary files**: metadata only. `format:"binary"`, `size_bytes` and `sha256` set,
`source:""` (user decision).
- **`--artifact-text`** (default **off**): when on, `source` carries the whole file text
(user decision). Off keeps default output to metadata.
- **`config_keys`** is **deferred** (user decision): not emitted in this change. No
`:ConfigKey` rows are written.

### Artifact id
- **`can://<app>/artifact/<path>`** (Python parity, user decision). App-scoped, so the
projector's `can://<namespace>/<app>/` prefix scoping covers artifact nodes. This changes the
Go projector's `:Artifact` flag to `CanNode=true`; the current Go schema comment ("neutral,
not a can:// id") is superseded.

### Package and dependencies
- **Package id: version-less `pkg:golang/<module-path>`** (user decision), for example
`pkg:golang/github.com/gorilla/mux`. Mirrors Python's `pkg:pypi/<name>`. The version lives
on the edge and the record, so every dependent app converges on one `:Package`.
- **Dependency records come from every `go.mod` `require`** (user decision). Direct and
indirect entries alike.
| field | source |
| --- | --- |
| `name` | module path |
| `ecosystem` | `"golang"` |
| `spec` | version in `go.mod` (for example `v1.8.0`) |
| `kind` | `"runtime"` for every entry (`go.mod` has no dev/optional split) |
| `direct` | `false` when the line carries `// indirect`, else `true` |
| `locked_version` | version pinned in `go.sum` for that module, when present |
| `declared_in` | the `go.mod` artifact id |
| `provides_imports` | import paths in the project that resolve to this module (see below) |
| `prov` | `["declared"]`, plus `"lockfile"` when `go.sum` supplies `locked_version` |
- **`go.sum`-only entries are not dependencies** (user decision, the non-recommended option
would have included them; the recommended default is not to, because `go.sum` retains
entries for pruned modules). `go.sum` contributes only `locked_version` and `prov`.
- **`provides_imports`**: populated by longest-prefix match of each project import path
against the `go.mod` module paths (user decision, Python parity). Import paths that match
no requirement are **not** recorded in this change (see Open items).

### Analysis level
- **L1, every level** (user decision). Parse-only facts, identical at `-a 1` through `-a 4`.
Not gated.

### Neo4j projection (the neutral layer, unchanged labels)
- `:Artifact {id, path, format}` (keyed by `id`, `CanNode=true`), under `HAS_ARTIFACT`
from `:GoApplication`.
- `:Package {id, ecosystem, name}` keyed by `id`.
- `DECLARES_DEPENDENCY` `(:Artifact)→(:Package)`, `_k = kind` (`runtime`), carrying `spec`,
`direct`, `prov` as properties.
- `LOCKS` `(:Artifact)→(:Package)` from `go.sum`, carrying `locked_version`.
- No `:ConfigKey` rows and no `DEFINES_CONFIG` rows in this change.

## Open items (flagged for review, not asked)

These are defaults I chose while writing the spec. Confirm them during review:

1. **`format` values for go.mod and go.sum.** I used `gomod` and `gosum`, because the Python
enum has no Go case. The alternative is `text`.
2. **Excluded directories.** `.git/` and `vendor/` are skipped. `vendor/` is third-party
source, not project configuration.
3. **Imports with no matching requirement.** Not recorded. They are `unresolved_imports` in
Python. That field is deferred with the repository-provenance field.
4. **`locked_version` when `go.sum` has several lines for one module** (the `h1:` and
`/go.mod` hashes). Take the version common to both lines; error if they disagree.

## Resolution (implementation, branch `feat/artifacts`)

The open items were resolved with the defaults below. Two are deviations from the wording
above, marked **(deviation)**:

1. **`format`**: `gomod` for `go.mod`, `gosum` for `go.sum`, adopted.
2. **Excluded directories**: `.git/` and `vendor/` are skipped at any depth. The tool's own
outputs are also skipped when they sit in the project: `analysis.json` and `graph.cypher`
in the `-o` directory, and the `-c` cache subtree. Without this, a run that writes into the
project would capture the previous run's output.
3. **Unrecorded imports**: an import matching no `go.mod` requirement is not recorded.
4. **Locked version (deviation)**: a requirement is locked only when go.sum has **both** the
`h1:` and the `/go.mod` line for the exact required version. A version with only one line
is left unlocked (`locked_version` absent, `prov` without `lockfile`), with no error. A
disagreement cannot arise, since the lookup is keyed by the required version.
5. **Artifact id root (deviation)**: the code's grammar is `can://<namespace>/<app>/…`, the
same root every other node uses, so an artifact id is
`can://<namespace>/<app>/artifact/<path>`. The `can://<app>/artifact/<path>` form in the
decisions above is shorthand for that root.

Also decided at implementation: `source` is omitted from JSON when empty (`omitempty`), not
written as `""`; only the root `go.mod` contributes dependencies (nested modules are artifacts
only); `dependencies` is absent when there is no root `go.mod`.

## Consumers

- **codeanalyzer-go**: emits the fields; the projector writes the rows above. Tests: a fixture
with `go.mod` and `go.sum`, a projector test for the new rows, and a `--artifact-text`
on/off test.
- **python-sdk**: the Go backend reads `:Artifact` / `:Package` through the shared
`cldk/analysis/commons/artifacts.py` reconstructors. The Go backend needs its artifact getters
added. Scope of that rung is decided at decomposition.

## Out of scope (deferred)

- Config keys and `DEFINES_CONFIG` (user decision).
- `unresolved_imports`, repository provenance (`repository`), `config_uses`.
- Codeanalyzer-python: no change.
20 changes: 20 additions & 0 deletions internal/core/analyzer.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import (
"github.com/codellm-devkit/codeanalyzer-go/internal/analysis"
"github.com/codellm-devkit/codeanalyzer-go/internal/dataflow"
"github.com/codellm-devkit/codeanalyzer-go/internal/options"
"github.com/codellm-devkit/codeanalyzer-go/internal/repository"
"github.com/codellm-devkit/codeanalyzer-go/internal/schema"
"github.com/codellm-devkit/codeanalyzer-go/internal/sdg"
"github.com/codellm-devkit/codeanalyzer-go/internal/semantic_analysis"
Expand Down Expand Up @@ -93,6 +94,25 @@ func New(opts options.AnalysisOptions) *Analyzer {

// Analyze runs the full analysis pipeline and returns a GoApplication.
func (a *Analyzer) Analyze() (*schema.GoApplication, error) {
app, err := a.analyze()
if err != nil {
return nil, err
}
// The repository layer reads files outside the Go source, so it is computed
// on every run, after the (possibly cached) application is ready.
facts, err := repository.Facts(a.opts.InputPath, app, repository.Options{
CaptureText: a.opts.ArtifactText,
OutputDir: a.opts.OutputDir,
CacheDir: a.opts.CacheDir,
})
if err != nil {
return nil, fmt.Errorf("repository layer: %w", err)
}
app.Repository = facts
return app, nil
}

func (a *Analyzer) analyze() (*schema.GoApplication, error) {
if _, err := resolveIdentity(a.opts); err != nil {
return nil, err
}
Expand Down
109 changes: 109 additions & 0 deletions internal/core/repository_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
package core_test

import (
"encoding/json"
"os"
"path/filepath"
"testing"

"github.com/codellm-devkit/codeanalyzer-go/internal/core"
"github.com/codellm-devkit/codeanalyzer-go/internal/options"
v2 "github.com/codellm-devkit/codeanalyzer-go/internal/schema/v2"
"github.com/codellm-devkit/codeanalyzer-go/internal/syntactic_analysis/v2emit"
)

// repoProject is a Go module with no requires (so no network materialization),
// one .go file, and two non-.go files.
func repoProject(t *testing.T) string {
t.Helper()
dir := t.TempDir()
files := map[string]string{
"go.mod": "module example.com/repoapp\n\ngo 1.25.0\n",
"go.sum": "",
"main.go": "package main\n\nfunc main() {}\n",
"README.md": "# repo app\n",
}
for rel, content := range files {
if err := os.WriteFile(filepath.Join(dir, rel), []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
return dir
}

// emitJSON runs Analyze at level 1 and returns the v2 analysis.json bytes.
func emitJSON(t *testing.T, dir string, artifactText bool) []byte {
t.Helper()
opts := options.AnalysisOptions{
InputPath: dir, AppName: "repoapp", Level: options.LevelSymbolTable,
SkipTests: true, CacheDir: t.TempDir(), OutputDir: t.TempDir(), ArtifactText: artifactText,
}
app, err := core.New(opts).Analyze()
if err != nil {
t.Fatal(err)
}
identity, err := v2.ResolveIdentity("repoapp", nil)
if err != nil {
t.Fatal(err)
}
payload := v2emit.Emit(app, identity, dir, int(opts.Level), "test", 0, nil, nil)
out, err := json.Marshal(payload)
if err != nil {
t.Fatal(err)
}
return out
}

func TestAnalyze_ArtifactsDefaultMetadataOnly(t *testing.T) {
dir := repoProject(t)
var doc struct {
Application struct {
Artifacts map[string]v2.Artifact `json:"artifacts"`
} `json:"application"`
}
if err := json.Unmarshal(emitJSON(t, dir, false), &doc); err != nil {
t.Fatal(err)
}
art, ok := doc.Application.Artifacts["README.md"]
if !ok {
t.Fatalf("README.md missing from artifacts: %v", doc.Application.Artifacts)
}
if art.ID != "can://repoapp/repoapp/artifact/README.md" {
t.Errorf("artifact id = %q", art.ID)
}
if art.Format != "text" || art.Extraction != "none" || art.Source != "" {
t.Errorf("default artifact should be metadata only: %+v", art)
}
if _, ok := doc.Application.Artifacts["main.go"]; ok {
t.Errorf("a .go file became an artifact")
}
}

func TestAnalyze_ArtifactTextCapturesSource(t *testing.T) {
dir := repoProject(t)
var doc struct {
Application struct {
Artifacts map[string]v2.Artifact `json:"artifacts"`
} `json:"application"`
}
if err := json.Unmarshal(emitJSON(t, dir, true), &doc); err != nil {
t.Fatal(err)
}
art := doc.Application.Artifacts["README.md"]
if art.Source != "# repo app\n" || art.Extraction != "full" {
t.Errorf("--artifact-text: source/extraction = %q/%q", art.Source, art.Extraction)
}
mod := doc.Application.Artifacts["go.mod"]
if mod.Format != "gomod" || len(mod.Roles) != 1 || mod.Roles[0] != "dependency-manifest" {
t.Errorf("go.mod artifact = %+v", mod)
}
}

func TestAnalyze_ArtifactsReproducibleAcrossRuns(t *testing.T) {
dir := repoProject(t)
a := emitJSON(t, dir, false)
b := emitJSON(t, dir, false)
if string(a) != string(b) {
t.Errorf("two runs over the same project differ")
}
}
Loading
Loading