Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 37 additions & 18 deletions docs/docs/design/validators.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
# Validators

Status: **compile-time checks done** (core#32–40, target types core#105) ·
**not run** — codegen / runtime enforcement is phase 4 (design below)
Status: **compile-time checks done** (core#32–40, target types core#105,
aliases core#109) · **run in Rust** (phase 4b–4c, comline-rust#52) ·
TypeScript not yet (4d)

A **validator** is a named, parameterised check attached to a
[struct](../guide/idl/structure.md) or [error](../guide/idl/error.md) field. The
declaration, its `validate` block, and the `@validators` field annotation parse,
freeze into structured IR, and are checked at `comline build` time — unknown
validator, bad keyword arg, undeclared `params.*` reference all fail the build.
Nothing *runs* them against data yet.
[struct](../guide/idl/structure.md) or [error](../guide/idl/error.md) field, or
to a `type` alias. The declaration, its `validate` block, and the `@validators`
annotation parse, freeze into structured IR, and are checked at `comline build`
time — unknown validator, bad keyword arg, undeclared `params.*` reference all
fail the build. Generated Rust runs them; the user guide's
[validators page](../guide/idl/validator.md#in-generated-code) describes what
it emits.

## Shape

Expand Down Expand Up @@ -95,7 +98,8 @@ struct Message {
| 3c — structured `validate` block, check `params.*` refs | ✅ core#39 |
| 3d — check keyword-arg literal **types** against the property types | ✅ core#40 |
| 3e — `for` target types; check fields and `value.*` against them, imported validators included | ✅ core#105 |
| 4 — runtime enforcement: generators emit the checks | drafted ([below](#phase-4-runtime-enforcement-design)); 4a ✅ core#108, 4b–4d not started |
| 3f — `@validators` (and `@deprecated`) on a `type` alias | ✅ core#109 |
| 4 — runtime enforcement: generators emit the checks | 4a ✅ core#108; 4b–4c ✅ for Rust (comline-rust#52, runtime#12, generation#57, cli#71); 4d (TypeScript) not started |

## Constraint: stays non-Turing-complete

Expand All @@ -106,14 +110,29 @@ bounds the answers to the questions below.

## Phase 4 — runtime enforcement (design)

### Where codegen is today

`comline generate` (the `rust` target,
`core/core/src/codelib_gen/rust.rs`) emits plain data: `#[derive(Serialize,
Deserialize)] pub struct`s, `pub trait`s for protocols, C-like `enum`s. No
`validate`, no decode hook, no runtime crate — the `runtime` repo is dormant.
`@validators` is frozen onto the field IR and then ignored by every generator.
Phase 4 is greenfield: nothing to retrofit, only something to add.
### What shipped (4b–4c, Rust)

The design below held, with these differences:

- **Where `ValidationError` lives.** In comline-runtime's `validation` module
(`alloc` layer), not a separate `comline_rt` crate. A package with
validators but no protocol now depends on the runtime too.
- **`field` is a path, not a name.** `ValidationError.field` is a `String`:
`handle`, `friends[1]`, `author.handle`. `{value.name}` renders the same
path, so a message says which element or nested value failed.
- **A trait, not inherent methods.** Generated types implement
`Validate`, whose `validate()` returns `ValidationErrors` and
`validate_first()` a `ValidationError`. The trait lets one type's check
recurse into another's, across crates.
- **Recursion.** `validate()` checks nested structs and array elements, not
just the type's own fields.
- **Resolution lives in the codegen kernel.** `comline_codegen::validation`
works out each value's checks (the alias chain's, innermost first, then the
field's own) and binds every property, with constants and `u32::MAX`-style
bounds resolved. A generator only lowers the result.
- **Params.** A params struct borrows (`&'a str`), which `#[serde(try_from)]`
handles badly, so the dispatcher runs the check after decoding instead and
refuses the call the way it refuses undecodable params.

### What "run" means

Expand Down Expand Up @@ -228,8 +247,8 @@ left to right, parenthesising each comparison.
| Sub-phase | Work |
|---|---|
| 4a | **Done (core#108).** Freeze the `validate` condition as a structured expression tree (keep the text + `references` the checks use). |
| 4b | `rust` generator emits `validate()` (collect-all) and `validate_first()` (fail-fast); stand up the `comline_rt` support crate with `ValidationError` / `ValidationErrors`; implement the `value.*` accessor set. |
| 4c | `comline.toml` `validate_on_decode` knob (`off` / `collect` / `first`) — `#[serde(try_from)]` shim in the `rust` target; default `off`. [Sketch below](#auto-validation-on-decode-4c). |
| 4b | **Done for Rust.** `rust` generator emits `validate()` (collect-all) and `validate_first()` (fail-fast); stand up the `comline_rt` support crate with `ValidationError` / `ValidationErrors`; implement the `value.*` accessor set. |
| 4c | **Done for Rust.** `comline.toml` `validate_on_decode` knob (`off` / `collect` / `first`) — `#[serde(try_from)]` shim in the `rust` target; default `off`. [Sketch below](#auto-validation-on-decode-4c). |
| 4d | The other generators (python / typescript / luau) once they exist; imported / cross-schema validators (needs the imported schema's IR at generate time, like multi-version `generate`). |

### Auto-validation on decode (4c)
Expand Down
21 changes: 21 additions & 0 deletions docs/docs/guide/idl/types.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,27 @@ or fixed `Type[10]`), and unions (`union(TypeA TypeB)`).
`string` generates into, so you see it in generated Rust, never in a `.ids`
file. Writing it gets the same hint.

## Type aliases

`type NAME = TYPE` gives a type a second name:

```ids
type UserId = u64
type Handle = string

struct User {
id: UserId
handle: Handle
}
```

An alias is transparent: `UserId` is a `u64` everywhere, in the wire format and
in generated code (`pub type UserId = u64;`, `export type UserId = number;`).
An alias can name another alias.

An alias can carry [`@validators`](validator.md#on-a-type-alias), which then
check every value of it, and `@deprecated`, which warns wherever it's used.

## In generated code

| Comline | Rust | TypeScript |
Expand Down
93 changes: 82 additions & 11 deletions docs/docs/guide/idl/validator.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Validators

!!! note "Checked, not run"
`comline build` verifies validators — an unknown validator name, a validator
put on a field of a type it doesn't apply to, a keyword argument that isn't a
property or whose literal type is wrong, or an undeclared `params.*`
reference all fail the build. Running them against data
(in generated code) is not done yet — see the
[design note](../../design/validators.md).
!!! note "Run in Rust; TypeScript not yet"
`comline build` verifies validators: an unknown validator name, a validator
put on a value of a type it doesn't apply to, a keyword argument that isn't
a property or whose literal type is wrong, or an undeclared `params.*`
reference all fail the build. Generated **Rust** code then
[runs them](#in-generated-code). Generated TypeScript doesn't yet.

A **validator** is a named, parameterised check you attach to a
[struct](structure.md) or [error](error.md) field.
[struct](structure.md) or [error](error.md) field, or to a
[`type` alias](#on-a-type-alias).

```ids linenums="1"
/// Checks a string's length is within bounds.
Expand Down Expand Up @@ -62,9 +62,15 @@ A `validate` block holds one or more `assert(condition, message)` calls.
`params.*` (this validator's properties), compared with `== != >= <= > <` and
joined by `and` / `or`. `assert(c, m)` passes when `c` is true. Numbers
compare by value, so a `u8` value against a `u32` property is fine.
- **`value` accessors** — `value` itself, `value.length` (only when every
target is `string` or an array), and `value.name` (the field's name, for
messages). Anything else is a build error at the validator.
- **`value` accessors**:
- `value` itself;
- `value.length`, only when every target is `string` or an array. A string's
length counts characters (Unicode scalar values), not bytes, in every
language; an array's counts elements.
- `value.name`, where the value sits, for messages: `handle`, `friends[1]`,
`author.handle`.

Anything else is a build error at the validator.
- **`message`** — an [interpolated string](error.md#message); `{value.…}` and
`{params.…}` placeholders are substituted.

Expand All @@ -87,3 +93,68 @@ struct Message {
Unbound properties fall back to their declared default. Errors' fields take
`@validators` too. Validators imported with `use` (such as
`std::validators::StringBounds`) are checked the same way as local ones.

`@validators` goes on a field or a `type` alias. On a struct, error, protocol or
function there's no single value for it to check, so it's a build error.

## On a type alias

Put `@validators` on a `type` alias to check every value of that alias,
wherever it's used:

```ids linenums="1"
use std::validators::StringBounds

@validators = [StringBounds(min_chars = 2, max_chars = 8)]
type Handle = string

struct Account {
handle: Handle
friends: Handle[]
}

protocol Accounts {
function rename(to: Handle) -> Account;
}
```

- The alias stays the type it stands for: `Handle` is still a `String` in
generated Rust. Adding a validator never changes a type's API or its wire
format.
- The validator's `for` target is checked once, against what the alias
resolves to.
- A field of the alias runs the alias's checks first, then its own.
- An alias of an alias collects the checks along the chain.
- An array of the alias (`Handle[]`) checks each element.
- An alias is the only way to check a **function argument**: arguments take no
annotations of their own, but `to: Handle` runs `Handle`'s checks.

`@deprecated` works on an alias too, and warns wherever the alias is used.

## In generated code

In Rust, every struct, error and function-params struct with something to
check implements `comline_runtime::validation::Validate`. That includes checks
on its own fields, through an alias, or further down in a nested struct or an
array element.

```rust
use comline_runtime::validation::Validate;

account.validate()?; // Err(ValidationErrors): every failed check
account.validate_first()?; // Err(ValidationError): the first one
```

Each `ValidationError` carries `field`, the path to the failing value
(`friends[1]`), and `message`, the validator's message with its placeholders
filled in (`friends[1] must be at least 2 characters`). An absent `optional`
field skips its checks.

Nothing calls `validate()` for you unless you ask:
[`validate_on_decode`](../../reference/congregation/comline-toml.md#validate_on_decode)
in `comline.toml` makes decoding run the checks, and makes a provider refuse a
call whose arguments fail them.

Properties are filled in when the code is generated: the argument you wrote,
else the property's default, with constants resolved. Nothing about the
validator exists at run time beyond the checks themselves.
7 changes: 5 additions & 2 deletions docs/docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ parse → resolve imports → validate → (freeze into .comline/) → (generate
| `comline add <name> <dir>` / `comline add <name> --git <uri> --commit <sha> --version <v>` | Add a dependency to `config.idp`, imported as `use <name>::…`: resolved first (a git pin is fetched into `.comline/deps-cache/`), then written with its `hash` pinned. `--no-hash` skips the pin. |
| `comline check` | Parse, resolve and validate. No `.comline/` write (beyond fetching git dependencies), no version bump — safe for editors, hooks, CI. |
| `comline build [--release] [--watch]` | Compile, validate, freeze a new immutable version into `.comline/`; print the changelog and the bump. `--release` is currently a no-op. |
| `comline generate [--target <lang>] [--out <dir>] [--layout <tpl>] [--mode <m>] [--external-std] [--no-external-std] [--dry-run] [--list] [--watch]` | Validate (no freeze), then write generated code. Location/layout from [`comline.toml`](congregation/comline-toml.md); targets from there or `config.idp`. |
| `comline generate [--target <lang>] [--out <dir>] [--layout <tpl>] [--mode <m>] [--external-std] [--no-external-std] [--validate-on-decode <when>] [--dry-run] [--list] [--watch]` | Validate (no freeze), then write generated code. Location/layout from [`comline.toml`](congregation/comline-toml.md); targets from there or `config.idp`. |
| `comline diff <old> [<new>]` | Show schema changes between two built versions. Each argument is a version (`0.2.0`), a commit hash (4+ chars), or `HEAD` (`<new>` default). |
| `comline clean [--dry-run]` | Remove generated code (the `generate` output root or files). Never touches `.comline/`. |
| `comline reset [--force] [--dry-run]` | **Destructive.** Delete `.comline/` — the whole version history, no undo — *and* generated code. Next `build` starts over at `0.0.1`. On a terminal it shows how many versions will be lost and makes you type `reset`; without a terminal it refuses (exit `2`) unless `--force`. |
Expand All @@ -49,7 +49,10 @@ reference std's own package — `comline_std` for rust, `@comline/std` for
typescript — instead of inlining it, and `--no-external-std` inlines it;
either overrides [`comline.toml`](congregation/comline-toml.md)'s
`[generate] external_std`, which itself defaults to on in `lib` mode and off
in `code` mode.
in `code` mode. `--validate-on-decode off|collect|first` overrides
[`[generate] validate_on_decode`](congregation/comline-toml.md#validate_on_decode),
whether decoding runs the schema's `@validators` (also
`COMLINE_GENERATE_VALIDATE_ON_DECODE`).
Each target's files are drawn as a tree. `--list` prints them one per line
instead, with the schema each came from — and, when several package versions
are generated at once, the version each line belongs to. `--dry-run` writes
Expand Down
24 changes: 23 additions & 1 deletion docs/docs/reference/congregation/comline-toml.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Defaults for every target.
| `package_versions` | `"latest"` \| `"all"` \| list | `"latest"` | Which package versions to emit — see [below](#package_versions). |
| `default_framing` | string | — | Wire framing for protocols that don't pick one with [`@framing`](../../guide/idl/protocol.md#annotations) — see [below](#default_framing). |
| `external_std` | bool | on in `lib`, off in `code` | Reference `std` as its own package instead of inlining it — see [below](#external_std). |
| `validate_on_decode` | `"off"` \| `"collect"` \| `"first"` | `"off"` | Whether decoding a value runs its [`@validators`](../../guide/idl/validator.md) — see [below](#validate_on_decode). |
| `only` | list of strings | — | Generate just these languages, out of the ones `config.idp` declares — see [below](#only). |

## `[[generate.target]]`
Expand All @@ -38,7 +39,7 @@ declare is an error. To generate *fewer* than the package declares, use
|---|---|---|
| `language` | string | **Required.** Must be one the manifest declares under `code_generation.languages`. |
| `lang_version` | string | Optional; selects a version-specific generator. Defaults to what `config.idp` declared. |
| `out`, `layout`, `mode`, `package_versions`, `default_framing`, `external_std` | — | Per-target override of the `[generate]` value. |
| `out`, `layout`, `mode`, `package_versions`, `default_framing`, `external_std`, `validate_on_decode` | — | Per-target override of the `[generate]` value. |
| `flatten` | bool | `lib`/`dylib` only, target-only (no `[generate]` default): write the crate straight into `out` instead of `out/<language>/`. See [below](#flatten). |

```toml
Expand Down Expand Up @@ -98,6 +99,27 @@ Neither package is published yet, so a `lib` target that references one needs
it linked locally (`npm link`, a path override) until that changes — which is
already true of `@comline/runtime` for any package with a protocol.

### `validate_on_decode`

Generated code always carries a schema's [`@validators`](../../guide/idl/validator.md)
checks as `validate()` / `validate_first()`, for you to call. This decides
whether **decoding** calls them too:

| Value | Decoding a value that fails its checks |
|---|---|
| `"off"` | succeeds; nothing runs until you call `validate()` |
| `"collect"` | fails, with every failed check |
| `"first"` | fails, with the first failed check |

It's yours to choose rather than the schema's: it changes what decoding costs,
and a form that wants every problem at once and a hot path that wants to stop
early want different things from the same type. With it on, a provider also
refuses a call whose arguments fail their checks, the same way it refuses
arguments that don't decode.

Today only the `rust` generator runs validators; `typescript` ignores this.
`--validate-on-decode` and `COMLINE_GENERATE_VALIDATE_ON_DECODE` override it.

### `only`

Generate a subset of the languages `config.idp` declares, for a checkout that
Expand Down
Loading