Skip to content

[AI-3439] The router reads and writes the agent directory - #2167

Draft
thesyncim wants to merge 6 commits into
mainfrom
agents/directory-sync
Draft

thesyncim wants to merge 6 commits into
mainfrom
agents/directory-sync

Conversation

@thesyncim

@thesyncim thesyncim commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Linear: AI-3439. Dashboard counterpart: GetStream/volt-dashboard#1337 (AI-3440). Follow-up for the CLI and the other SDKs: AI-3441.

Why

An agent kept in a repository is a directory: agent.yaml, instructions.md, guardrail.md, skills/, knowledge/, simulations/ and ui/. Until now, every client read that directory itself and sent the router fields it had mapped. The Go SDK, the JS SDK, the CLI and the other SDKs each have their own parser. They drift: the JS parser already refuses keys the Go one accepts, and the CLI has a third parser. The dashboard couldn't read or write the format at all without becoming one more parser.

This makes the router the reader and writer of the format. A client sends the directory's files as they are. The router parses them, validates them against the schema and stores the result, and it can hand an agent's directory back. There is one set of rules and one set of error messages, and the CLI, the SDKs and the dashboard can all sync the same way.

flowchart LR
  subgraph before["Before"]
    direction LR
    G1[Go SDK parser] --> M1["POST /v1/agents/sync<br/>mapped fields"]
    J1[JS SDK parser] --> M1
    C1[CLI parser] --> M1
  end
  subgraph after["After"]
    direction LR
    G2[Go SDK] -->|files| F["POST /v1/agents/sync/files"]
    J2[JS SDK] -->|files| F
    D2[Dashboard] -->|files| F
    F --> A[agentdir<br/>parse · validate]
    A --> S[(config)]
    S --> R[agentdir<br/>render] --> E["GET /v1/agents/configs/{id}/directory"]
  end
Loading

What it adds

POST /v1/agents/sync/files: sync a directory from its files

{
  "name": "front-desk",              // optional when agent.yaml names it
  "files": [                         // at most 1024
    { "path": "agent.yaml", "content": "name: front-desk\nllm: openai/gpt-5.6\n..." },
    { "path": "instructions.md", "content": "You answer the phone for ..." },
    { "path": "skills/booking/SKILL.md", "content": "---\nname: booking\n..." }
  ],
  "dry_run": true,          // answer with what it would store and change; write nothing
  "check_changes": true,    // 409 unsynced_changes if the dashboard edited it since the last sync
  "base_change": "…"        // go ahead over those edits
}
  • The response is the same SyncAgentResult as /v1/agents/sync, plus changes: the skills, knowledge documents, knowledge URLs and simulations the sync creates, updates or deletes ([{resource_type, name, action}]). Dry runs return it too.
  • The router hashes the files it used, so syncing the same directory again writes nothing.
  • A dry run is refused wherever the real sync would be, including the knowledge credit and spend checks. Its doc names the two things that can't be known without doing the work: whether the provider accepts the passages, and whether a URL can be read.
  • Files outside the format are left out, with one summary warning.
  • Settings set in code next to a directory win over the files, as they did in the SDKs: instructions, guardrail, skills, harness, subagent, sandbox, tags, llm, stt, tts, sts, voice, greeting and video.
  • It runs through the same sync core as /v1/agents/sync, so connectors, left-out connectors, knowledge, simulations, warnings and conflicts behave the same.

Schema validation at the router, with positions

The schema is checked at every level: unknown keys, retired keys and wrong types. Problems come back together, each with its file, line and column:

{
  "error": {
    "type": "invalid_request",
    "code": "invalid_agent_directory",
    "message": "agent.yaml:3:1: \"voise\" is not something agent.yaml says; …",
    "problems": [
      { "file": "agent.yaml", "line": 3, "column": 1,  "message": "\"voise\" is not something agent.yaml says; name, description, mode, …, ui, app are" },
      { "file": "agent.yaml", "line": 4, "column": 10, "message": "plugins: no longer read: plugins were replaced by connectors, so name the provider under connectors" },
      { "file": "agent.yaml", "line": 6, "column": 3,  "message": "greeting: \"txt\" is not something greeting says; text, mode are" }
    ]
  }
}
  • Enum, pattern and range checks reuse the constraints the API already declares. The request built from the files is validated against the same schema, and each error is mapped back to its line, so the constraints aren't written down twice.
  • Checks that depend on stored state (a skill that needs a sandbox, floor_control on this router, channels, region and the like) are collected and located the same way.
  • The list is capped: problems holds at most 100 entries, and problems_omitted counts the rest.

A 409 unsynced_changes on either sync route now says what the sync would write over: conflicts: [{resource_type, name, removed?}], plus a last_change to send back as base_change. Clients no longer have to parse the message.

GET /v1/agents/configs/{id}/directory: the agent as files

It returns {name, files, not_exported}.

  • Canonical form: keys in the SDKs' order, defaults left out, and connectors that only name themselves written as bare names. Syncing the result back changes nothing.
  • not_exported lists what the files can't express for this agent, such as a binding's timeout or policy, knowledge that isn't text, or files over the limits.
  • Settings with no agent.yaml key (llm_fallbacks, review, episode_cards) are neither written nor touched by a sync.

Package layout

  • acceleration/internal/agentdir:
    • Parse, Hash (length-prefixed parts) and the limits (MaxFiles, MaxBytes, per-file and UI caps);
    • a YAML reader that keeps positions;
    • the agent.yaml grammar;
    • skills, knowledge, urls.yaml and simulations;
    • the ui: compiler;
    • Render.
  • acceleration/internal/api/directory.go: turns the parsed files into a sync request, resolves connectors on the server by id or unique label (scoped to the tenant), runs the schema check, and serves the two operations.
  • Migration 20270105090000_agent_config_ui_source.sql: a nullable agent_configs.ui_source column, so an export can write ui: back.

SDKs

  • Go and JS sync by sending files to /sync/files, along with the settings set in code, which win as before.
    • They send only what the format names: agent.yaml, instructions.md, guardrail.md, skills/, knowledge/, simulations/, ui/, and the UI paths agent.yaml names.
    • Files are read at sync time.
    • Symlinks out of the directory are refused, special files are skipped, and the size is checked before sending.
  • Their local parsers stay for now. Load and loadFolder fill and refuse exactly as on main, for code that reads them. The router is what validates and stores. Removing the SDK parsers is AI-3441.
  • .agent_sync has one format in both SDKs: {hash, sync_hash, synced_at}.
  • The other SDKs' generated clients are regenerated for the new operations and fields. Only generated files change, with no behaviour change. They keep using /v1/agents/sync until AI-3441.

Not breaking

  • POST /v1/agents/sync and SyncAgentRequest are byte-identical in the spec, and the existing SyncSuite passes unchanged. Published SDKs and the CLI keep working.
  • The spec diff only adds things: two operations, their schemas, and new fields on ErrorDetail and SyncAgentResult.
  • /v1/agents/sync now validates skills, sources and URLs before it writes anything, and refuses duplicate skill names up front. Before, it could fail partway through.
  • An existing .agent_sync stamp causes one extra sync after upgrading. That sync writes nothing.
  • Deploy order: the new SDKs call /sync/files, so the router has to ship first.

Validation

  • Router:
    • go build ./... and go vet ./... pass, and go run ./cmd/openapi leaves no diff.
    • go test -tags integration -p 1 passes for ./internal/api (the whole package, including the spec staleness test), ./internal/agentdir, ./internal/store and ./internal/knowledge/urls.
  • Go SDK: go vet ./... and go vet -tags e2e . pass, go test -race ./... passes, and go generate leaves no diff.
  • JS SDK: npm test passes 785/785, and npm run types -- --check reports the types in step with the spec.
  • Other SDKs:
    • Rust, Ruby and PHP: their generator checks and tests pass.
    • .NET: the check passes. Two timing tests fail, and at least one of them also fails on main.
    • Swift: in step.
    • Python and Kotlin: regenerated, but their tests weren't run locally.
    • Dart is not regenerated. Its Flutter image can't be pulled here, so its generated-client check will fail until someone runs dart run tool/generate.dart.
  • Live, against a local router built from this branch:
    • A rich agent goes export → sync → export with no config diff, and the next sync is unchanged.
    • Edits made through PATCH are not reverted by an export, and what can't be written shows up in not_exported.
    • A dry run writes nothing, and returns the same 409, conflicts and changes as the real sync.
    • Path traversal, absolute paths and duplicates are refused.
    • Problems across four files come back in one response, with positions.
  • Two independent review passes. Their findings were fixed with regression tests:
    • an export reverting a dashboard UI edit;
    • a duplicate skill name failing after a partial write;
    • skill text lost on a round trip;
    • the SDKs uploading unrelated files;
    • a dry run not checking knowledge the way the real sync does;
    • unbounded errors.

…ory format

Parse reads a directory from the files a client sent: agent.yaml,
instructions.md, guardrail.md, skills, knowledge, knowledge/urls.yaml,
simulations and the UI files agent.yaml names. It decides which of the
files are the directory, so every client's send of one folder reads and
hashes the same. Unknown and retired keys and wrong types are refused,
every problem at once, each with its file, line and column. Render
writes a directory back as canonical files a sync reads again, and says
what it could not write. The grammar and the ui: compiler are the Go
SDK's, ported with its tests and the JavaScript SDK's refusals.
POST /v1/agents/sync/files takes the files of an agent's folder and
reads them with agentdir, so every client's directory means the same
thing. A directory that does not read is a 400 invalid_agent_directory
whose problems name the file, line and column of each: what does not
parse, what the schema refuses and what the router cannot run, together
and bounded. The router finds a connector's connection among the app's
own by id or label, keeps the left-out behaviour, compiles ui: into
ui_instructions and takes the hash from the files it used. What a
program sets in code beside the directory goes in the same request and
wins over the files. dry_run answers what would be stored, refuses what
the sync would, the money for its knowledge included, and writes
nothing. It shares its implementation with POST /v1/agents/sync, whose
request is unchanged.

GET /v1/agents/configs/{id}/directory writes a stored agent back as its
canonical files, with what a directory cannot say in not_exported.
agent_configs.ui_source keeps what ui_instructions was compiled from
until somebody writes others, so the ui key and its files are exported.

A sync checks everything a write would refuse before the first write,
two skills of one name included. Its answer lists the skills, knowledge
documents, knowledge urls and simulations it creates, updates or deletes
as changes, and an unsynced_changes refusal carries what it would write
over as conflicts, with the last_change to send back as base_change.
…reads

Sync reads the files the format names when it syncs, and no other file
of the folder, and sends them to POST /v1/agents/sync/files with what
the code set beside the directory, which still wins: the instructions,
the guardrail, the harness's subagent, sandbox and skills, and the cost
tracking labels. The router decides what agent.yaml means and which
connections it names. Load still parses the directory for the program
holding it.

A link out of the directory is refused, what is not a regular file of
text is left behind, and a directory over what a sync sends is refused
by name. .agent_sync keeps a hash of what was sent and the sync_hash the
router stored it under, in the format the JavaScript SDK writes, and a
sync is skipped only while the stored config still has that hash and
left no connector out.
…reads

sync reads the files the format names when it syncs, and no other file
of the folder, and sends a folder from loadFolder to
POST /v1/agents/sync/files with what the code set beside it, which still
wins: instructions, guardrail, harness, skills, the pipeline's models
and the cost labels. The router reads the files, finds the connections
and compiles ui:. loadFolder still parses the directory for the program
holding it.

A directory over what a sync sends is refused by name. .agent_sync keeps
a hash of what was sent and the sync_hash the router stored it under, in
the format the Go SDK writes, and a sync is skipped only while the
stored config still has that hash and left no connector out. A folder
built by hand without files, and an agent written in code, sync as
before.
The sdk skill's note says what the other SDKs and the getstream CLI
should do to send a directory's files; the generative-ui and connectors
skills say where ui: is compiled and connections are found for a sync
of files. Removing the SDKs' own parsers is AI-3441.
POST /v1/agents/sync/files and GET /v1/agents/configs/{id}/directory, the
changes a sync answers with, and the problems, conflicts and last_change
an error carries, as each SDK's generator writes them. Generated files
only.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant