From 6e8bb0e13e076afca27e79741d01267efd7ec527 Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 5 Nov 2026 16:37:34 -0600 Subject: [PATCH 1/9] Prototype portable KV-backed Geo indexing and lookup --- .../2026-11-05-kv-geo-prototype-results.md | 100 ++ .../specs/2026-11-05-kv-geo-design.md | 225 +++ examples/geo-kv-prototype/Cargo.lock | 1246 +++++++++++++++++ examples/geo-kv-prototype/Cargo.toml | 20 + examples/geo-kv-prototype/README.md | 79 ++ examples/geo-kv-prototype/src/import.rs | 337 +++++ examples/geo-kv-prototype/src/index.rs | 767 ++++++++++ examples/geo-kv-prototype/src/main.rs | 237 ++++ examples/geo-kv-prototype/src/store.rs | 129 ++ 9 files changed, 3140 insertions(+) create mode 100644 docs/superpowers/plans/2026-11-05-kv-geo-prototype-results.md create mode 100644 docs/superpowers/specs/2026-11-05-kv-geo-design.md create mode 100644 examples/geo-kv-prototype/Cargo.lock create mode 100644 examples/geo-kv-prototype/Cargo.toml create mode 100644 examples/geo-kv-prototype/README.md create mode 100644 examples/geo-kv-prototype/src/import.rs create mode 100644 examples/geo-kv-prototype/src/index.rs create mode 100644 examples/geo-kv-prototype/src/main.rs create mode 100644 examples/geo-kv-prototype/src/store.rs diff --git a/docs/superpowers/plans/2026-11-05-kv-geo-prototype-results.md b/docs/superpowers/plans/2026-11-05-kv-geo-prototype-results.md new file mode 100644 index 00000000..24b2b59a --- /dev/null +++ b/docs/superpowers/plans/2026-11-05-kv-geo-prototype-results.md @@ -0,0 +1,100 @@ +# KV Geo prototype results + +- Date: 2026-11-05. +- Status: format/lookup experiment completed; production integration and remote KV acceptance remain open. +- Spec: [portable KV Geo](../specs/2026-11-05-kv-geo-design.md). +- Code and reproduction commands: [prototype README](../../../examples/geo-kv-prototype/README.md). +- Base revision: `72ceed7e2a802cfa536851dd20c95ca55ebea4fc`, plus the standalone prototype identified by the source fingerprint below. +- Environment: Linux x86_64, Rust 1.95.0, optimized Cargo release build, local file-backed artifact KV. +- Tested source fingerprint: `70227e734eee4709c7072a45a50dcc00ea15bd1c947c82596025a309e9c2ddbf`. Compute with `sha256sum examples/geo-kv-prototype/src/*.rs examples/geo-kv-prototype/Cargo.toml examples/geo-kv-prototype/Cargo.lock | sha256sum`. +- Tested release binary SHA-256: `dd9af4b519c1213f2094b6805a41307fc637137eded40751491384dbfb5e6878`. + +## Dataset + +Used the user's `GeoLite2-City-CSV_20261103.zip`, kept outside Git. Archive SHA-256 is `94745697bcd25751e38c9386260eeb29527f851ec9e3732cfe58aa62a07b9ab8`. ZIP CRC checks passed; a publisher-provided checksum was not available for comparison. + +| Input | Count | +| --- | --- | +| English location rows | 75,449 | +| IPv4 source networks | 3,632,756 | +| IPv6 source networks | 2,091,942 | +| Merged IPv4 geographic intervals | 1,371,640 | +| Merged IPv6 geographic intervals | 605,625 | +| IPv4 networks without country | 768 | +| IPv6 networks without country | 724 | + +The import joins geolocation IDs only. It does not substitute registered-country IDs or include names, coordinates or postal codes in the index. Source networks without country produce gaps, not guessed locations. + +## Index shape + +All objects were written and read back through the actual EdgeZero `KvHandle`. The transport was the prototype's local artifact-file `KvStore`, not a cloud service. + +| Object budget | Encoded total | Objects, including manifest | Uncached path objects, IPv4/IPv6 | Import, build and read-back | +| --- | --- | --- | --- | --- | +| 16 KiB | 40.423 MiB | 2595 | 3/3 | 1.482 s | +| 64 KiB | 40.302 MiB | 649 | 2/2 | 1.487 s | +| 256 KiB | 40.272 MiB | 165 | 2/2 | 1.490 s | +| 384 KiB | 40.269 MiB | 111 | 2/2 | 1.460 s | + +A resolver acquires the manifest with one additional read when opened. The path counts above exclude it and are maxima; root/directory gaps can return without a leaf read. + +The 64 KiB build contained 42,259,649 encoded bytes and 649 objects. Its largest object was 65,532 bytes. Both address families fit a root directory followed by a leaf. Increasing shards beyond 64 KiB reduced key count but did not remove another read from this dataset's path. + +Importer whole-process peak RSS was about 231.4 MiB. That process retains raw CIDRs for verification and builds merged ranges, so this is not serving memory. + +## Lookup checks and local timings + +For each size, checked 2,048 points per family in three passes: no object cache, an initially empty 4 MiB object cache, and a repeat using that same cache. All 49,152 comparisons across four sizes and three modes matched the independent source-CIDR oracle. These are repeated comparisons over the same 4,096-point workload, not 49,152 distinct random addresses or exhaustive dataset coverage. + +The sample mix includes family endpoints, stratified source-network endpoints and neighbors, and deterministic uniform random addresses. Each pass found 2,917 geographic locations and 1,179 no-matches. It is a correctness workload with correlated neighbors and many uncovered addresses, not a user-traffic distribution. + +| Object budget | Cache-disabled p95, microseconds | Repeated-cache p95, microseconds | Repeated-cache reads/query | Repeated-cache bytes/query | +| --- | --- | --- | --- | --- | +| 16 KiB | 63.5 | 46.5 | 0.240 | 3.83 KiB | +| 64 KiB | 158.5 | 142.9 | 0.228 | 14.51 KiB | +| 256 KiB | 524.9 | 514.6 | 0.184 | 46.51 KiB | +| 384 KiB | 1266.1 | 749.0 | 0.184 | 69.96 KiB | + +Timings include local file reads, hashing, validation and lookup. OS page cache was uncontrolled. They do not measure remote latency, cold storage, concurrency, provider billing or cache behavior across short-lived isolates. The mean read/byte rates are specific to this sample ordering and cache budget. + +A fresh 64 KiB `lookup` process resolved four sample inputs without loading source CSV or an inventory. It read one manifest, one directory and one leaf, retained 77,304 cache payload bytes, and peaked at 3,006,464 bytes of whole-process RSS. This proves a small lookup workload avoids the importer's full-data memory footprint. It does not establish maximum serving RSS under load. Cache limits count retained encoded payload, with allocator metadata and validation scratch memory in addition. + +## Country-code policy mismatch + +The real release contains `XK`, while EdgeZero's existing `CountryCode` rejects it. There are 393 IPv4 and 68 IPv6 source networks with that code. + +The initial strict import failed on this mismatch. To finish the format experiment without changing production country policy, the standalone prototype preserves syntactically valid source country codes in a separate DTO and reports whether core accepts them. A fresh lookup over an actual `XK` interval returned the preserved code with `core_country_supported: false`. No shared allowlist or application outcome changed. + +This is a decision needed before production implementation. Rejecting a complete release because it contains one unsupported code prevents using this dataset as supplied. Alternatives must explicitly define unsupported-code behavior rather than silently losing those intervals or calling them `NoMatch`. + +## Verification record + +| Check | Result | +| --- | --- | +| `cargo test --offline --locked --manifest-path examples/geo-kv-prototype/Cargo.toml` | 10 prototype tests passed | +| `cargo clippy --offline --locked --manifest-path examples/geo-kv-prototype/Cargo.toml --all-targets -- -D warnings` | Passed | +| Prototype and workspace `cargo fmt -- --check` commands | Passed | +| `cargo test --offline --locked -p edgezero-core` | 753 unit tests and enabled doctests passed; 13 existing doctests ignored | +| `cargo build --offline --locked --release --manifest-path examples/geo-kv-prototype/Cargo.toml` | Passed | +| Real-data build at 16/64/256/384 KiB | Every exported object read back and digest-checked | +| Repeat real-data builds | Same receipts and immutable object bytes accepted for every size | +| CSV oracle with cache off/on/repeated | 49,152 comparisons, zero mismatches | +| Fresh-process lookup, including real `XK` interval | No CSV/inventory preload; unsupported-domain flag preserved | +| Pre-PR `cargo test --offline --locked --workspace --all-targets` | 2,115 passed, six existing tests ignored | +| Pre-PR `cargo clippy --offline --locked --workspace --all-targets --all-features -- -D warnings` | Passed | +| Pre-PR `cargo check --offline --locked --workspace --all-targets --features 'fastly cloudflare spin'` | Passed | +| Pre-PR `cargo check --offline --locked -p edgezero-adapter-spin --target wasm32-wasip2 --features spin` | Passed | + +Synthetic tests cover multilevel traversal, deterministic output, cache eviction, IPv4/IPv6 zero and maximum addresses, empty families, gaps, country-only results, absent geolocation without registered-country fallback, quoted CSV fields, bad references, duplicate identities, overlaps, bad family/CIDR input, missing objects, wrong digests and malformed records with matching digests. Missing expected data is an error, not a no-match. + +Logs from this session are under `/tmp/geo-kv-prototype-*.log`. Dataset, artifacts and durable machine-readable measurements are under `~/.cache/edgezero/geo-kv-prototype/`; each `artifacts-k/` directory contains `receipt.json`, `inventory.json`, `build-report.json` and `verification-report.json`. These are local artifacts, not committed production data. + +## Next decisions + +64 KiB is a reasonable next remote-KV experiment: it preserved two-read lookup paths while transferring and validating less data than 256/384 KiB shards. The locally fastest 16 KiB objects needed a third uncached read, which may lose once network latency is included. No serving default or wire format is frozen by these results. + +Next work should decide country-code policy, exercise actual provider upload/read paths, and measure representative traffic and remote KV costs before changing request APIs or dispatch. City import and synthetic Country-shaped CSV were exercised; a full real Country release was not. + +Not verified: production async provider composition, request cancellation/deadlines, reader-IP admission on edge adapters, WASM runtime behavior, live KV latency/capacity, globally visible publication, crash-safe uploads, licensing review or deletion across deployed caches. No live credentials or cloud resources were used. The pre-PR workspace gates above passed without changing the prototype's measured source fingerprint; logs are under `/tmp/geo-kv-pr-*.log`. No new Fastly/Cloudflare WASM runtime execution or live-service acceptance was performed. + +The existing GitHub workflow base-branch filters do not include `feature/pluggable-geo-providers`, and root workspace checks do not include this standalone crate. The explicit prototype commands in this record are its local verification, not a claim of automated GitHub coverage. diff --git a/docs/superpowers/specs/2026-11-05-kv-geo-design.md b/docs/superpowers/specs/2026-11-05-kv-geo-design.md new file mode 100644 index 00000000..468bc06e --- /dev/null +++ b/docs/superpowers/specs/2026-11-05-kv-geo-design.md @@ -0,0 +1,225 @@ +# Portable KV-backed Geo + +- Status: draft for review. The format prototype is measured; production Geo integration and remote KV behavior are not implemented or verified. +- Date: 2026-11-05. +- Base: `72ceed7e2a802cfa536851dd20c95ca55ebea4fc`, the shared base of this branch and `feature/axum-local-geoip` before its MMDB implementation. +- Related work: [pluggable Geo providers](2026-10-22-pluggable-geo-providers-design.md), issues [#432](https://github.com/IABTechLab/edgezero/issues/432) and [#438](https://github.com/IABTechLab/edgezero/issues/438). + +## 1. Outcome and decisions + +Consumers can resolve an admitted reader IP to an estimated country and optional first-level subdivision without platform-provided geography, a runtime filesystem, or loading the entire dataset into memory. They supply a populated KV store through EdgeZero's existing store abstractions. + +Use GeoLite CSV as the initial import format. Transform it outside the serving runtime into an immutable, versioned IP index. Resolve requests asynchronously through `KvHandle`, with optional bounded in-memory caching. Geography remains an estimate; applications own jurisdiction recognition and policy. + +KV is the portable serving dependency, not a claim of better lookup performance than MMDB. Remote KV adds read cost, latency and failure modes. The first implementation must measure these costs before making production performance claims. + +Required invariants: + +- A deployment explicitly selects zero or one Geo provider. There is no automatic source selection or fallback. +- One immutable Geo outcome is resolved after ingress admission and before application middleware and handlers. +- The adapter owns reader-IP trust. The resolver receives a typed, admitted IP rather than parsing request headers. +- Request lookup uses exact-key reads, never key listing, full-dataset loading or per-prefix probing. +- Every request uses one pinned dataset generation, including cache hits and misses. +- Missing or corrupt required index data is a provider failure, not an unmatched address. +- Runtime Geo resolution requires no filesystem, local sidecar, MaxMind credentials or downloader. +- Portable code gains no native SDK, Tokio dependency, mandatory `Send` future or platform-specific timer. + +CloudFront assertions remain an optional selected provider. This work does not depend on the sibling MMDB implementation and does not remove it from that branch. + +## 2. Ownership and flow + +| Concern | Owner | +| --- | --- | +| Dataset acquisition, download integrity and licensing | Operator's update process | +| CSV validation, geographic projection and index construction | Offline importer | +| KV resource creation, credentials and backend bindings | Existing platform tooling and operator | +| Index lookup and decoding | Portable KV Geo implementation | +| Socket/platform client identity and forwarding trust | Adapter ingress | +| Per-request outcome and geographic value types | Portable Geo contract | +| Jurisdiction, consent, permissions and response caching | Application | + +```mermaid +flowchart TD + CSV[GeoLite City or Country CSV release] --> Import[Offline validation and index construction] + Import --> Publish[Publish immutable KV objects] + Publish --> Verify[Read back and verify generation] + Verify --> Select[Deploy with pinned manifest identity] + Request[Incoming request] --> Ingress[Adapter admission and checked reader IP] + Ingress --> Resolver[Portable async Geo resolution] + Select --> Resolver + Resolver --> Cache{Index object cached?} + Cache -->|Yes| Search[Search validated object] + Cache -->|No| Read[Read exact KV key and validate object] + Publish --> Read + Read --> Search + Search --> Next{Directory or leaf?} + Next -->|Directory| Cache + Next -->|Leaf or address gap| Outcome[Immutable request Geo outcome] + Outcome --> App[Application middleware and handlers] +``` + +## 3. Import contract + +Initially support GeoLite2 City and Country CSV releases. City is needed for subdivisions. For City, consume these files from the same archive: + +```text +GeoLite2-City-Blocks-IPv4.csv +GeoLite2-City-Blocks-IPv6.csv +GeoLite2-City-Locations-en.csv +``` + +Country uses the corresponding Country filenames and produces country-only locations. Read columns by header name so unrelated added columns do not change the projection. + +Join block `geoname_id` to location `geoname_id`. Keep `country_iso_code` and, for City, `subdivision_1_iso_code`. Do not substitute `registered_country_geoname_id` or `represented_country_geoname_id`; those identify different geographic facts. Names, coordinates, postal codes and network registration metadata are not needed in the serving index. + +Validate address-family membership, CIDRs, duplicate location identities, referenced locations and consumed geographic codes. A populated reference with no matching location is an import error. An absent geolocation reference or location without country yields no geographic coverage, not a guessed country. A subdivision without country is invalid. Country-only locations remain partial; subdivision validation checks syntax, not worldwide assignment or application recognition. + +Convert networks into sorted numeric intervals, separately for IPv4 and IPv6. Reject conflicting or overlapping source intervals rather than inventing precedence. Adjacent intervals with identical projected locations may be merged. Preserve gaps. Use overflow-safe address arithmetic, including the last address in each family. + +The importer emits deterministic objects, their keys and digests, and a publication inventory. Metadata records the dataset edition, source release identifier, source file digests, index schema version and importer version. Repeated import with the same source and format settings produces the same object bytes. + +The importer may use a filesystem and native tools. Those requirements belong to the publishing environment, not deployed applications. MaxMind authentication stays outside the serving runtime. + +## 4. KV index + +### Object layout + +Use a versioned tree of directory objects and leaf objects, separately addressable through exact KV keys. A small manifest identifies both family roots and the schema. Empty families are explicit. + +Conceptual keys: + +```text +geo/v1//manifest +geo/v1//v4/directory/ +geo/v1//v4/leaf/ +geo/v1//v6/directory/ +geo/v1//v6/leaf/ +``` + +A leaf contains sorted, disjoint inclusive intervals and their country and optional subdivision. A directory contains sorted, disjoint child envelopes, exact child keys and expected content digests. Envelopes include any internal gaps; the selected leaf confirms actual coverage. A directory gap proves no coverage without another read. + +Build leaves by encoded byte size, then build directories over contiguous groups until each family has one root. Additional directory levels allow the directory itself to exceed one KV value without requiring a whole-directory preload. Lookup chooses at most one child at each level by binary search. It never probes multiple prefixes or scans neighboring leaves. + +Use a versioned binary encoding with explicit record counts, address-family widths and network-byte-order addresses. Locations are stored with the intervals, avoiding a second location-record read. The exact byte schema, directory fanout and default object size must be recorded after the prototype; this draft does not claim they are settled. + +### Limits and validation + +All accesses go through `KvHandle`. Build to an explicit target backend value budget, including serialization overhead. The portable 25 MiB ceiling is not a promise every backend accepts that size. For example, the current native DynamoDB adapter caps values at 384 KiB and permits operators to lower that cap. + +The reader validates schema, digest, encoded lengths, counts, ordering, interval bounds, child bounds and geographic values before using an object. Child keys must remain within the selected generation and address family, and children must fit the parent's declared envelope. Expected digests bind decoded descendants to the selected manifest. Hashes identify bytes, not publisher authenticity. + +The implementation must specify per-object bytes, traversal work and memory budgets. Derive defaults from the supported backend limits and measured index shape, rather than inventing an arbitrary fixed tree depth. Reject an incompatible dataset explicitly; never skip validation or truncate data to fit. + +## 5. Async resolution and request lifecycle + +The base's `GeoProvider` is synchronous and assertion-only. KV resolution requires a new async execution boundary and a typed reader-IP input; it cannot be implemented by blocking on KV inside that existing trait. + +Retain the public synchronous supplied-provider contract for existing consumers. Add an object-safe asynchronous provider contract using the repository's non-`Send` future convention. Existing synchronous providers are wrapped at composition and return immediately. Native hosting retains the object ownership bounds it needs. The new provider owns its configured `KvHandle`; request input does not expose the request or a service registry. + +Give the new async contract discriminated assertion and reader-IP inputs, with source admission owned by adapters. Preserve the existing synchronous `GeoInput` assertion struct; the composition wrapper converts only the assertion variant into it. Add `NoMatch` and `MissingReaderIp` outcomes and reader-IP provenance. Existing country/subdivision value types and `RequestContext::geo()` remain the consumer interface. New public variants need release notes and exhaustive-match migration examples. + +Managed request sequence: + +1. Perform existing ingress validation, forwarding normalization and admission. Rejected or aborted ingress makes no Geo lookup. +2. Establish the selected input contract from adapter-private facts. Missing usable reader input produces `MissingReaderIp` without provider calls. +3. Await the selected provider once, with the request's remaining execution/cancellation budget. Traverse index objects and return one resolution. +4. Attach an owned immutable outcome before request preparers, application middleware and handlers. Response finalization never resolves again. + +A logical provider invocation can perform multiple KV reads. Without cache hits, read count is manifest acquisition plus one read per directory level and one leaf. The manifest may be retained for the configured instance lifetime. Concurrent cold requests may perform duplicate reads without changing correctness; request coalescing is not a first-version requirement. + +Geo code adds no retry loop. Backend retry and timeout behavior must be documented separately. Adapter integration supplies runtime-specific deadline/cancellation mechanics; portable lookup does not acquire a timer dependency. Cancellation releases request ownership without detached lookup work. A dropped future does not promise the underlying platform has aborted its physical I/O. + +| Condition | Outcome | +| --- | --- | +| Geo not selected | `NoSelection` | +| Selected, no admitted usable reader IP | `MissingReaderIp` | +| Validated directory or leaf proves address gap | `NoMatch` | +| Matched country, absent subdivision | Located, country only | +| Matched country and subdivision | Located, estimated subdivision | +| Expected object absent, invalid, unsupported or digest mismatch | Provider failure | +| KV error or Geo lookup deadline exhausted | Provider failure | + +Provider failure does not choose CloudFront, another dataset, or an older cache generation. Outcomes do not automatically choose HTTP status or permission policy. + +## 6. Configuration and adapter portability + +Configuration selects a logical KV store ID, dataset generation, expected manifest digest and reader-IP expectation. Resolve the logical ID through existing bindings; the Geo selection contains no physical table name, namespace credentials or filesystem path. Fail configuration if the ID is undeclared, the selection is unsupported, or the configured backend cannot serve the dataset's object budget. + +Use an explicit portable Geo selection that every managed adapter can consume. The exact Rust and manifest spelling remains a review decision. Selecting Geo must not silently create a new default store or reuse an application's default store by accident. + +Native hosting prepares the manifest after KV handles are ready and before readiness. Request-scoped edge runtimes may acquire it during request initialization and optionally retain it in runtime-supported memory. They do not need a process-wide startup phase or full-data validation on every invocation. Unsupported schemas or missing manifests fail initialization; missing descendants remain request-time provider failures. + +Every adapter must establish an admitted client IP independently of geographic metadata: + +| Adapter | Required integration | +| --- | --- | +| Axum | Reuse checked peer and trusted-forwarding normalization; support direct-peer or forwarded-required expectation. | +| Fastly | Use platform client-address facts with documented provenance. | +| Cloudflare | Use the platform's documented client-address mechanism and prove its replacement/trust contract, rather than accepting an arbitrary copied header. | +| Spin | Use a verified runtime client-address mechanism or an explicit trusted-ingress integration; return missing input if neither is available. | + +These are acceptance requirements, not assertions that the base already implements them. Preserve existing mapped-IPv4 normalization. Never geolocate a reverse proxy merely because its socket peer is available under a forwarded-required selection. + +KV makes lookup portable, but does not create trustworthy reader identity. A runtime with KV but no trustworthy client address cannot produce reader-IP geography. Report that limitation explicitly rather than weakening admission. + +## 7. Cache behavior + +Caching is optional and bounded by bytes. Cache validated immutable objects using store identity, dataset generation and object digest. Eviction only causes another KV read. Fresh workers and caches disabled entirely must produce the same outcomes as warm workers. + +Use in-memory caching where the runtime permits it, without requiring cross-request or cross-instance persistence. Do not cache reader IPs or add an IP-to-location cache. A missing expected object is a publication/storage failure, not a negative geolocation cache entry. + +Retiring a dataset includes stopping or reconfiguring instances that retain its objects in memory. A version pin prevents accidental mixing; it does not establish freshness or permission to keep using old data. + +## 8. Publication and lifecycle + +A complete offline build precedes publication. One publishing operation owns a generation; concurrent publishers use separate generation targets. Upload objects under their immutable keys, read them back and verify their expected bytes or digests, then publish the manifest last. A retry of the same build writes the same bytes. Never overwrite a published generation with different data. + +The portable KV API has no compare-and-swap or multi-key transaction. Immutable publication is therefore an operator/tooling ownership rule, not a storage-enforced lock. Deployment configuration pins the manifest digest. There is no mutable `active` pointer in the first version. + +A manifest written last is not a global readiness barrier on eventually consistent KV. Validate propagation through the selected provider's supported mechanisms before rollout. A successful uploader-region read-back does not prove all serving locations can read every descendant. If an object is temporarily unavailable, readers return provider failure without switching generations. Document the backend-specific rollout check and residual uncertainty. + +A stopped upload leaves an unselected partial generation. Re-running the same inventory converges; cleanup of abandoned generations uses the retained inventory outside request handling. Retire selected generations only after readers stop referencing them, subject to the dataset's mandatory update and deletion terms. Do not rely on KV TTL as proof of immediate deletion. + +The publishing workflow needs two outputs: an artifact containing the exact key/value inventory, and a way to upload and verify it for the selected backend. Runtime `KvHandle` portability does not imply a workstation can instantiate every edge SDK's store binding. Reuse existing provider tooling where available; otherwise add an explicit uploader or document an operator-supported upload workflow. CLI syntax and initial backend uploader coverage must be agreed before implementation is considered end-to-end complete. + +Keep dataset acquisition and MaxMind credentials outside these runtime APIs. Attribution and redistribution/deletion obligations apply to the derived KV index, retained export artifacts, caches and backups as well as the original CSV archive. + +## 9. Diagnostics + +Expose bounded categories for configuration, initialization and lookup failures. Record schema and loaded dataset identity in startup/initialization diagnostics, not per-IP labels. Useful metrics include lookup latency, KV reads, cache hits, bytes decoded and outcome category. + +Do not retain reader IPs, raw forwarding headers, credentials or raw KV errors in Geo outcomes or default diagnostics. KV keys and values need the existing storage logging discipline; index keys encode dataset structure, not individual request addresses. Dependency logging is a separate operator concern. + +## 10. Acceptance and delivery + +Deliver in checkable units: + +1. **Format prototype.** Build synthetic City/Country fixtures and a real permitted GeoLite release. Record encoded size, object count, depth, cold-read count, import cost and maximum decoded-object memory. Freeze schema and defaults only after these measurements. +2. **Importer and lookup.** Prove deterministic bytes and compare index results with an independent direct CSV-range lookup, including generated/random addresses, family boundaries, gaps, country-only records and subdivisions. Reject malformed references, overlaps and corrupt objects. Exercise actual `KvHandle` reads with caching off and on. +3. **Async contract and native integration.** Preserve existing CloudFront/supplied-provider behavior. Test checked-reader input, source rejection, request admission, deadlines, cancellation and immutable outcomes through managed Axum dispatch. No detached work, blocking KV bridge or response-finalization lookup. +4. **Edge adapter integration.** Run managed request tests for Fastly, Cloudflare and Spin, including forged address inputs and runtimes without usable reader facts. Compile all WASM targets without native dependency leakage. Generic data access alone is not adapter acceptance. +5. **Publication and deployed checks.** Exercise interrupted uploads, identical re-publication, read-back mismatch, incomplete visibility, pinned versions, missing descendants and retirement. Verify real provider upload/binding/read paths separately from mocks, with operator authorization for live resources. + +Performance acceptance must include cold and warm lookup latency, reads and bytes per request, memory, and KV cost at the intended traffic level. Test representative native remote KV and edge KV. The current DynamoDB binding uses one partition key per namespace, so workload capacity and hot-partition behavior need explicit evaluation; index sharding alone does not change that physical partitioning. + +The spec is ready for implementation planning when the byte format/object budgets, configuration spelling, reader-IP support matrix, publishing coverage and acceptable lookup budgets have been reviewed. Until then it is a design draft, not a production-readiness claim. + +## 11. Prototype findings + +The [standalone prototype](../../../examples/geo-kv-prototype/README.md) and [measurement record](../plans/2026-11-05-kv-geo-prototype-results.md) exercise the downloaded City CSV through `KvHandle` using a local artifact-store harness. A 64 KiB object budget produced about 40.3 MiB across 649 objects, with at most two object reads per lookup after manifest acquisition. This is not deployed-provider performance evidence or a frozen wire format. + +One production decision is now concrete: the CSV contains `XK`, which core rejects. The format experiment preserves it in a source DTO and reports the mismatch without changing core policy. Resolve how a complete dataset containing unsupported source codes maps into Geo outcomes before implementing the import validation policy in section 3. It must not silently become an address gap or an accepted core country. + +## 12. Scope boundaries and references + +Included: City/Country CSV import, versioned KV index, portable async lookup, optional object caching, explicit selection, checked reader inputs, adapter integration, publication/retirement contract and focused proofs. + +Deferred: remote geolocation APIs, automatic source fallback, automatic download/update scheduling, live version switching, richer city/coordinate output, jurisdiction catalogues, permission policy and mandatory MMDB support. Application response-cache variation remains an application responsibility. + +- [GeoLite formats and account requirements](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/). +- [CSV fields and join semantics](https://dev.maxmind.com/geoip/docs/databases/city-and-country/). +- [Account downloads](https://www.maxmind.com/en/accounts/current/geoip/downloads), choose City CSV for subdivisions. +- [Dummy City CSV sample](https://dev.maxmind.com/examples/GeoIP2-City-CSV_Example.zip). +- [KV interface and provider consistency](../../guide/kv.md). +- [Native DynamoDB limits and semantics](../../guide/dynamodb-kv.md). +- [Native startup lifecycle](2026-10-05-axum-production-startup-lifecycle-design.md). diff --git a/examples/geo-kv-prototype/Cargo.lock b/examples/geo-kv-prototype/Cargo.lock new file mode 100644 index 00000000..ce02cb52 --- /dev/null +++ b/examples/geo-kv-prototype/Cargo.lock @@ -0,0 +1,1246 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "adler2" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa" + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "alloc-no-stdlib" +version = "2.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc7bb162ec39d46ab1ca8c77bf72e890535becd1751bb45f64c597edb4c8c6b3" + +[[package]] +name = "alloc-stdlib" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0e76a019e91224d279006ff972f1e984179a6e9feb050adba6ce8274aef23195" +dependencies = [ + "alloc-no-stdlib", +] + +[[package]] +name = "anyhow" +version = "1.0.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3" + +[[package]] +name = "async-compression" +version = "0.4.43" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3976abdc8fe7d1133d43d304afd42abdf5bc3e1319d263d223bde07b5efc4be8" +dependencies = [ + "compression-codecs", + "compression-core", + "futures-io", + "pin-project-lite", +] + +[[package]] +name = "async-stream" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b5a71a6f37880a80d1d7f19efd781e4b5de42c88f0722cc13bcb6cc2cfe8476" +dependencies = [ + "async-stream-impl", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-stream-impl" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "async-trait" +version = "0.1.89" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "brotli" +version = "8.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5cc91aac060a7a1e25823bdccbfb6af1875b88f17c6daac97894eed8207166b3" +dependencies = [ + "alloc-no-stdlib", + "alloc-stdlib", + "brotli-decompressor", +] + +[[package]] +name = "brotli-decompressor" +version = "5.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5962523e1b92ce1b5e793d9169b9943eece10d39f62550bc04bb605d75b94924" +dependencies = [ + "alloc-no-stdlib", + "alloc-stdlib", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "bytes" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "compression-codecs" +version = "0.4.38" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce2548391e9c1929c21bf6aa2680af86fe4c1b33e6cea9ac1cfeec0bd11218cf" +dependencies = [ + "brotli", + "compression-core", + "flate2", + "memchr", +] + +[[package]] +name = "compression-core" +version = "0.4.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc14f565cf027a105f7a44ccf9e5b424348421a1d8952a8fc9d499d313107789" + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "crc32fast" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "crypto-common" +version = "0.1.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bfb12502f3fc46cca1bb51ac28df9d618d813cdc3d2f25b9fe775a34af26bb3" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "csv" +version = "1.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52cd9d68cf7efc6ddfaaee42e7288d3a99d613d4b50f76ce9827ae0c6e14f938" +dependencies = [ + "csv-core", + "itoa", + "ryu", + "serde_core", +] + +[[package]] +name = "csv-core" +version = "0.1.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "704a3c26996a80471189265814dbc2c257598b96b8a7feae2d31ace646bb9782" +dependencies = [ + "memchr", +] + +[[package]] +name = "darling" +version = "0.20.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee" +dependencies = [ + "darling_core", + "darling_macro", +] + +[[package]] +name = "darling_core" +version = "0.20.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d00b9596d185e565c2207a0b01f8bd1a135483d02d9b7b0a54b11da8d53412e" +dependencies = [ + "fnv", + "ident_case", + "proc-macro2", + "quote", + "strsim", + "syn 2.0.119", +] + +[[package]] +name = "darling_macro" +version = "0.20.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead" +dependencies = [ + "darling_core", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "displaydoc" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ac70aa55017e108007fbaf5aa0f54b021c98f92ff8af59d42eda9da96e3dd4f" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "edgezero-core" +version = "0.1.0" +dependencies = [ + "adler2", + "alloc-no-stdlib", + "alloc-stdlib", + "anyhow", + "async-compression", + "async-stream", + "async-trait", + "brotli", + "brotli-decompressor", + "bytes", + "compression-codecs", + "compression-core", + "crc32fast", + "edgezero-macros", + "flate2", + "futures", + "futures-util", + "http", + "http-body", + "log", + "matchit", + "miniz_oxide", + "ryu", + "serde", + "serde_json", + "serde_path_to_error", + "serde_urlencoded", + "sha2", + "simd-adler32", + "thiserror", + "toml", + "tower-service", + "tracing", + "url", + "validator", + "web-time", +] + +[[package]] +name = "edgezero-macros" +version = "0.1.0" +dependencies = [ + "log", + "proc-macro2", + "quote", + "serde", + "serde_json", + "syn 3.0.3", + "toml", + "validator", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "flate2" +version = "1.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c" +dependencies = [ + "crc32fast", + "miniz_oxide", +] + +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + +[[package]] +name = "form_urlencoded" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf" +dependencies = [ + "percent-encoding", +] + +[[package]] +name = "futures" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d" +dependencies = [ + "futures-channel", + "futures-core", + "futures-executor", + "futures-io", + "futures-sink", + "futures-task", + "futures-util", +] + +[[package]] +name = "futures-channel" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d" +dependencies = [ + "futures-core", + "futures-sink", +] + +[[package]] +name = "futures-core" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d" + +[[package]] +name = "futures-executor" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d" +dependencies = [ + "futures-core", + "futures-task", + "futures-util", +] + +[[package]] +name = "futures-io" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718" + +[[package]] +name = "futures-macro" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "futures-sink" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893" + +[[package]] +name = "futures-task" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393" + +[[package]] +name = "futures-util" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6" +dependencies = [ + "futures-channel", + "futures-core", + "futures-io", + "futures-macro", + "futures-sink", + "futures-task", + "memchr", + "pin-project-lite", + "slab", +] + +[[package]] +name = "generic-array" +version = "0.14.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4bb6743198531e02858aeaea5398fcc883e71851fcbcb5a2f773e2fb6cb1edf2" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "geo-kv-prototype" +version = "0.0.0" +dependencies = [ + "anyhow", + "async-trait", + "bytes", + "csv", + "edgezero-core", + "futures", + "ipnet", + "serde", + "serde_json", + "sha2", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "http" +version = "1.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6970f50e31d6fc17d3fa27329444bfa74e196cf62e95052a3f6fee181dba6425" +dependencies = [ + "bytes", + "itoa", +] + +[[package]] +name = "http-body" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184" +dependencies = [ + "bytes", + "http", +] + +[[package]] +name = "icu_collections" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c" +dependencies = [ + "displaydoc", + "potential_utf", + "utf8_iter", + "yoke", + "zerofrom", + "zerovec", +] + +[[package]] +name = "icu_locale_core" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29" +dependencies = [ + "displaydoc", + "litemap", + "tinystr", + "writeable", + "zerovec", +] + +[[package]] +name = "icu_normalizer" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4" +dependencies = [ + "icu_collections", + "icu_normalizer_data", + "icu_properties", + "icu_provider", + "smallvec", + "zerovec", +] + +[[package]] +name = "icu_normalizer_data" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38" + +[[package]] +name = "icu_properties" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de" +dependencies = [ + "icu_collections", + "icu_locale_core", + "icu_properties_data", + "icu_provider", + "zerotrie", + "zerovec", +] + +[[package]] +name = "icu_properties_data" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14" + +[[package]] +name = "icu_provider" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421" +dependencies = [ + "displaydoc", + "icu_locale_core", + "writeable", + "yoke", + "zerofrom", + "zerotrie", + "zerovec", +] + +[[package]] +name = "ident_case" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39" + +[[package]] +name = "idna" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de" +dependencies = [ + "idna_adapter", + "smallvec", + "utf8_iter", +] + +[[package]] +name = "idna_adapter" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714" +dependencies = [ + "icu_normalizer", + "icu_properties", +] + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown", +] + +[[package]] +name = "ipnet" +version = "2.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2" +dependencies = [ + "serde", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.105" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce57d20d1ea864ce2ac172ab472d409214f4fd359f0b2a2775abdf522e2af99e" +dependencies = [ + "cfg-if", + "futures-util", + "wasm-bindgen", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "litemap" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0" + +[[package]] +name = "log" +version = "0.4.30" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "616ec5685824bcc94416c6d4a7a446eea774a31efd7062c8480ba6fd06d7a6e5" + +[[package]] +name = "matchit" +version = "0.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8863b587001c1b9a8a4e36008cebc6b3612cb1226fe2de94858e06092687b608" + +[[package]] +name = "memchr" +version = "2.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8" + +[[package]] +name = "miniz_oxide" +version = "0.8.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316" +dependencies = [ + "adler2", + "simd-adler32", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "potential_utf" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564" +dependencies = [ + "zerovec", +] + +[[package]] +name = "proc-macro-error-attr2" +version = "2.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5" +dependencies = [ + "proc-macro2", + "quote", +] + +[[package]] +name = "proc-macro-error2" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802" +dependencies = [ + "proc-macro-error-attr2", + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dfbc457d0c7a0759a614551b11a6409e5951f6c7537be1f1b7682b9ae9230368" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "regex" +version = "1.12.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a" + +[[package]] +name = "rustversion" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d" + +[[package]] +name = "ryu" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "serde_json" +version = "1.0.150" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_path_to_error" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10a9ff822e371bb5403e391ecd83e182e0e77ba7f6fe0160b795797109d1b457" +dependencies = [ + "itoa", + "serde", + "serde_core", +] + +[[package]] +name = "serde_spanned" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26" +dependencies = [ + "serde_core", +] + +[[package]] +name = "serde_urlencoded" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd" +dependencies = [ + "form_urlencoded", + "itoa", + "ryu", + "serde", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "simd-adler32" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214" + +[[package]] +name = "slab" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" + +[[package]] +name = "smallvec" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "synstructure" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "thiserror" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.3", +] + +[[package]] +name = "tinystr" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d" +dependencies = [ + "displaydoc", + "zerovec", +] + +[[package]] +name = "toml" +version = "1.1.2+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee" +dependencies = [ + "indexmap", + "serde_core", + "serde_spanned", + "toml_datetime", + "toml_parser", + "toml_writer", + "winnow", +] + +[[package]] +name = "toml_datetime" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_parser" +version = "1.1.2+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526" +dependencies = [ + "winnow", +] + +[[package]] +name = "toml_writer" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db" + +[[package]] +name = "tower-service" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3" + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "url" +version = "2.5.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed" +dependencies = [ + "form_urlencoded", + "idna", + "percent-encoding", + "serde", +] + +[[package]] +name = "utf8_iter" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" + +[[package]] +name = "validator" +version = "0.20.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43fb22e1a008ece370ce08a3e9e4447a910e92621bb49b85d6e48a45397e7cfa" +dependencies = [ + "idna", + "once_cell", + "regex", + "serde", + "serde_derive", + "serde_json", + "url", + "validator_derive", +] + +[[package]] +name = "validator_derive" +version = "0.20.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7df16e474ef958526d1205f6dda359fdfab79d9aa6d54bafcb92dcd07673dca" +dependencies = [ + "darling", + "once_cell", + "proc-macro-error2", + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "wasm-bindgen" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 3.0.3", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "web-time" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "winnow" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0592e1c9d151f854e6fd382574c3a0855250e1d9b2f99d9281c6e6391af352f1" + +[[package]] +name = "writeable" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" + +[[package]] +name = "yoke" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "abe8c5fda708d9ca3df187cae8bfb9ceda00dd96231bed36e445a1a48e66f9ca" +dependencies = [ + "stable_deref_trait", + "yoke-derive", + "zerofrom", +] + +[[package]] +name = "yoke-derive" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "zerofrom" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272" +dependencies = [ + "zerofrom-derive", +] + +[[package]] +name = "zerofrom-derive" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "synstructure", +] + +[[package]] +name = "zerotrie" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf" +dependencies = [ + "displaydoc", + "yoke", + "zerofrom", +] + +[[package]] +name = "zerovec" +version = "0.11.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239" +dependencies = [ + "yoke", + "zerofrom", + "zerovec-derive", +] + +[[package]] +name = "zerovec-derive" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "zmij" +version = "1.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" diff --git a/examples/geo-kv-prototype/Cargo.toml b/examples/geo-kv-prototype/Cargo.toml new file mode 100644 index 00000000..13ee9b21 --- /dev/null +++ b/examples/geo-kv-prototype/Cargo.toml @@ -0,0 +1,20 @@ +[package] +name = "geo-kv-prototype" +version = "0.0.0" +edition = "2024" +publish = false + +# A format experiment, not a new production workspace member. +[workspace] + +[dependencies] +anyhow = "1" +async-trait = "0.1" +bytes = "=1.12.1" +csv = "=1.4.0" +edgezero-core = { path = "../../crates/edgezero-core", default-features = false } +futures = "0.3" +ipnet = { version = "=2.12.0", features = ["serde"] } +serde = { version = "1", features = ["derive"] } +serde_json = "=1.0.150" +sha2 = "0.10" diff --git a/examples/geo-kv-prototype/README.md b/examples/geo-kv-prototype/README.md new file mode 100644 index 00000000..776a3c5d --- /dev/null +++ b/examples/geo-kv-prototype/README.md @@ -0,0 +1,79 @@ +# KV Geo format prototype + +A standalone Rust experiment for [portable KV Geo](../../docs/superpowers/specs/2026-11-05-kv-geo-design.md). It imports GeoLite City/Country CSV, exports a compact IP index, and checks asynchronous point lookups through the real EdgeZero `KvHandle` against the original unmerged CIDRs. + +This is the format/lookup prototype, not production Geo integration. It does not change `GeoProvider`, request dispatch, adapters, store bindings or CLI commands. The crate has its own workspace and lockfile so CSV/import dependencies do not enter production builds. + +## Run + +From the repository root: + +```sh +cargo test --locked --manifest-path examples/geo-kv-prototype/Cargo.toml +cargo build --locked --release --manifest-path examples/geo-kv-prototype/Cargo.toml +``` + +The binary is `examples/geo-kv-prototype/target/release/geo-kv-prototype` unless `CARGO_TARGET_DIR` selects another directory. The commands below assume `BIN` points to that binary. + +Extract the City CSV archive outside the repository. The source directory must contain the English locations file and both Blocks files. For Country CSV, use the corresponding Country filenames. + +```sh +SOURCE="$HOME/.cache/edgezero/geo-kv-prototype/source/GeoLite2-City-CSV_20261103" +OUT="$HOME/.cache/edgezero/geo-kv-prototype/artifacts-64k" + +"$BIN" build "$SOURCE" "$OUT" 65536 +"$BIN" verify "$SOURCE" "$OUT" 2048 4194304 +"$BIN" lookup "$OUT" 1.0.1.1 ::1 +``` + +- `build` imports, merges adjacent equal locations, builds byte-bounded objects, exports them through `KvHandle`, reads every object back, and emits a pinned receipt, inventory and build report. Use a separate output directory for each object-size experiment. An identical rerun must reproduce the same receipt and object bytes. +- `verify` compares 2,048 points per family against an independent `ipnet::contains` lookup over source CIDRs. Points include family endpoints, stratified CIDR endpoints and neighbors, and deterministic random addresses. It measures cache-disabled, cache-enabled first-pass and cache-enabled repeated-pass lookups. The last two share a bounded cache, not a full dataset preload. The manifest read is measured separately from per-query object reads. +- `lookup` runs in a fresh process without reading CSV or an inventory. Results follow input order, without echoing addresses. A JSON `null` means no geographic coverage; a missing required object is an error. + +Generated CSV, index files and machine-readable reports stay outside Git. Publication artifacts contain derived MaxMind data and retain its licensing, attribution, update and deletion obligations. + +## Data and policy are separate + +The downloaded release includes `XK`. EdgeZero's existing `CountryCode` rejects that code. The prototype's six-byte location is a source-format DTO, not a `GeoLocation` or production `GeoResolution`. + +The prototype validates country syntax, preserves source codes, reports unsupported-code network counts and exposes `core_country_supported` in explicit lookup output. It does not add `XK` to EdgeZero's country allowlist, silently discard those intervals, or map them to another country. Production policy needs a decision before integrating this format with Geo outcomes. + +## Experimental encoding + +```mermaid +flowchart TD + Receipt[Pinned manifest key and SHA-256] --> Manifest[Family roots and object byte budget] + Manifest --> Directory[Sorted child envelopes and hashes] + Directory --> Leaf[Sorted address intervals and source locations] + Leaf --> Location[Country and optional subdivision] +``` + +Each binary object has a 12-byte header: + +| Bytes | Meaning | +| ----- | ----------------------------------- | +| 0..4 | Magic `EZP1` | +| 4 | Address family, 4 or 6 | +| 5 | Kind, 0 for leaf or 1 for directory | +| 6..8 | Big-endian level, 0 for leaf | +| 8..12 | Big-endian record count | + +Leaf records contain inclusive start and end addresses, then a six-byte location. IPv4 addresses occupy four bytes each; IPv6 addresses occupy sixteen. Location bytes are uppercase country code, subdivision length, and three zero-padded subdivision bytes. Leaf records are therefore 14 or 38 bytes. + +Directory records contain inclusive child-envelope bounds and a 32-byte child SHA-256. They occupy 40 or 64 bytes. Child level decreases by one. Child keys derive from generation, family and digest; no arbitrary child key is decoded from dataset bytes. Directories are grouped until each family has one root. No key listing or CIDR-prefix probing occurs. + +The JSON manifest binds source-file hashes, edition, object budget and both roots. `receipt.json` pins its exact key and digest. The experimental generation hashes schema number, object budget and source metadata. Changes to this encoding/build algorithm require a new format version; p1 is not frozen for consumers. + +Objects are validated before use or caching. The reader checks content digest, family, level, lengths, ordering, envelopes and source-code syntax. The cache retains validated object bytes and searches records without materializing all intervals. Its configured budget counts retained payload bytes, not allocator/hash-map overhead. A fetched object and validation scratch memory exist in addition to retained cache payload. + +## Measurement boundaries + +`store.rs` is a local file-backed artifact transport implementing `KvStore`. Its synchronous filesystem operations are intentionally a prototype harness, not a production async backend. The lookup algorithm itself only uses `KvHandle` and requires no filesystem API. Replacing this harness with actual remote KV and integrating cancellation belongs to the next stage. + +Local timings include file reads, SHA-256, validation and search. OS page cache is uncontrolled. Cache-disabled means the resolver object cache is disabled, not that disk or OS caches are cold. No simulated network delay is added. The sample mix is a correctness workload, not representative user traffic or a KV cost forecast. + +Linux `VmHWM` supplies whole-process peak RSS. Import/verify processes retain source CIDRs for the oracle, so their RSS is not serving memory. Use a fresh `lookup` process to measure a small serving workload without that retained dataset. + +The artifact writer demonstrates deterministic export/read-back, not crash-safe publication, distributed propagation, concurrent writer exclusion or a backend's upload API. Production retry, publication and retirement behavior remains specified separately. + +See [measured results](../../docs/superpowers/plans/2026-11-05-kv-geo-prototype-results.md). diff --git a/examples/geo-kv-prototype/src/import.rs b/examples/geo-kv-prototype/src/import.rs new file mode 100644 index 00000000..156b93b4 --- /dev/null +++ b/examples/geo-kv-prototype/src/import.rs @@ -0,0 +1,337 @@ +//! CSV import and independent, unmerged CIDR oracle for the format experiment. +use std::{collections::HashMap, fs::File, io::Read, net::IpAddr, path::Path}; + +use anyhow::{Context, Result, ensure}; +use ipnet::IpNet; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; + +use crate::index::{Location, Range, Source, address, hex}; + +#[derive(Debug, Deserialize)] +struct LocationRow { + geoname_id: u32, + country_iso_code: String, + #[serde(default)] + subdivision_1_iso_code: String, +} + +#[derive(Debug, Deserialize)] +struct BlockRow { + network: IpNet, + geoname_id: Option, +} + +#[derive(Clone, Debug)] +pub struct SourceRange { + pub network: IpNet, + pub location: Option, +} + +impl SourceRange { + pub fn start(&self) -> u128 { + address(self.network.network()).1 + } + pub fn end(&self) -> u128 { + address(self.network.broadcast()).1 + } +} + +#[derive(Debug, Serialize)] +pub struct FamilyStats { + pub source_networks: usize, + pub without_country: usize, + pub country_only: usize, + pub merged_intervals: usize, + pub unsupported_core_country_networks: std::collections::BTreeMap, +} + +pub struct Imported { + pub source: Source, + pub locations: usize, + pub raw4: Vec, + pub raw6: Vec, +} + +fn locations(reader: impl Read) -> Result>> { + let mut out = HashMap::new(); + for row in csv::Reader::from_reader(reader).deserialize::() { + let row = row?; + let location = if row.country_iso_code.is_empty() { + ensure!( + row.subdivision_1_iso_code.is_empty(), + "subdivision without country for geoname {}", + row.geoname_id + ); + None + } else { + Some( + Location::new(&row.country_iso_code, &row.subdivision_1_iso_code).with_context( + || format!("invalid location codes for geoname {}", row.geoname_id), + )?, + ) + }; + ensure!( + out.insert(row.geoname_id, location).is_none(), + "duplicate geoname identity {}", + row.geoname_id + ); + } + Ok(out) +} + +fn blocks( + reader: impl Read, + locations: &HashMap>, + family: u8, +) -> Result> { + let mut out = Vec::new(); + for row in csv::Reader::from_reader(reader).deserialize::() { + let row = row?; + ensure!( + address(row.network.addr()).0 == family, + "wrong CSV address family" + ); + ensure!( + row.network.addr() == row.network.network(), + "CIDR has host bits" + ); + let location = row + .geoname_id + .map(|id| { + locations + .get(&id) + .copied() + .with_context(|| format!("missing referenced geoname {id}")) + }) + .transpose()? + .flatten(); + out.push(SourceRange { + network: row.network, + location, + }); + } + out.sort_unstable_by_key(SourceRange::start); + for pair in out.windows(2) { + ensure!( + pair[0].end() < pair[1].start(), + "overlapping source networks" + ); + } + Ok(out) +} + +struct HashedReader { + file: File, + hash: Sha256, +} + +impl HashedReader { + fn open(path: &Path) -> Result { + Ok(Self { + file: File::open(path)?, + hash: Sha256::new(), + }) + } + fn finish(self) -> String { + hex(&self.hash.finalize()) + } +} + +impl Read for HashedReader { + fn read(&mut self, buffer: &mut [u8]) -> std::io::Result { + let n = self.file.read(buffer)?; + self.hash.update(&buffer[..n]); + Ok(n) + } +} + +pub fn load(directory: &Path) -> Result { + let edition = if directory.join("GeoLite2-City-Locations-en.csv").is_file() { + "GeoLite2-City" + } else { + "GeoLite2-Country" + }; + let files: Vec = ["Locations-en.csv", "Blocks-IPv4.csv", "Blocks-IPv6.csv"] + .iter() + .map(|suffix| format!("{edition}-{suffix}")) + .collect(); + // Hash the same byte stream CSV consumes, rather than re-reading mutable files. + let mut reader = HashedReader::open(&directory.join(&files[0]))?; + let locations = locations(&mut reader)?; + let mut hashes = vec![reader.finish()]; + let mut reader = HashedReader::open(&directory.join(&files[1]))?; + let raw4 = blocks(&mut reader, &locations, 4)?; + hashes.push(reader.finish()); + let mut reader = HashedReader::open(&directory.join(&files[2]))?; + let raw6 = blocks(&mut reader, &locations, 6)?; + hashes.push(reader.finish()); + let files = files.into_iter().zip(hashes).collect(); + Ok(Imported { + source: Source { + edition: edition.into(), + files, + }, + locations: locations.len(), + raw4, + raw6, + }) +} + +pub fn project(raw: &[SourceRange]) -> (Vec, FamilyStats) { + let mut out: Vec = Vec::new(); + let mut stats = FamilyStats { + source_networks: raw.len(), + without_country: 0, + country_only: 0, + merged_intervals: 0, + unsupported_core_country_networks: std::collections::BTreeMap::new(), + }; + for row in raw { + let Some(location) = row.location else { + stats.without_country += 1; + continue; + }; + if !location.core_country_supported() { + let code = std::str::from_utf8(&location.0[..2]).expect("validated source country"); + *stats + .unsupported_core_country_networks + .entry(code.into()) + .or_default() += 1; + } + if location.0[2] == 0 { + stats.country_only += 1; + } + let start = row.start(); + let end = row.end(); + if let Some(previous) = out.last_mut() + && previous.location == location + && previous.end.checked_add(1) == Some(start) + { + previous.end = end; + continue; + } + out.push(Range { + start, + end, + location, + }); + } + stats.merged_intervals = out.len(); + (out, stats) +} + +// Uses the original CIDRs and ipnet.contains, not the encoded/merged interval search. +pub fn oracle(raw: &[SourceRange], ip: IpAddr) -> Option { + let value = address(ip).1; + let i = raw.partition_point(|r| r.start() <= value); + if i == 0 { + return None; + } + raw[i - 1] + .network + .contains(&ip) + .then_some(raw[i - 1].location) + .flatten() +} + +pub fn samples(raw: &[SourceRange], family: u8, count: usize) -> Vec { + let max = if family == 4 { + u32::MAX.into() + } else { + u128::MAX + }; + let mut values = vec![0, max]; + let boundary_rows = (count / 8).min(raw.len()); + for i in 0..boundary_rows { + let row = &raw[i * raw.len() / boundary_rows]; + values.extend([row.start(), row.end()]); + if let Some(before) = row.start().checked_sub(1) { + values.push(before); + } + if let Some(after) = row.end().checked_add(1).filter(|n| *n <= max) { + values.push(after); + } + } + let mut state = 0x6a09_e667_f3bc_c909_u64; + let mut random = || { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + state + }; + while values.len() < count { + let n = (u128::from(random()) << 64) | u128::from(random()); + values.push(n & max); + } + values + .into_iter() + .map(|n| { + if family == 4 { + IpAddr::from((n as u32).to_be_bytes()) + } else { + IpAddr::from(n.to_be_bytes()) + } + }) + .collect() +} + +#[cfg(test)] +mod tests { + use super::*; + const LOCATIONS: &str = "geoname_id,country_iso_code,subdivision_1_iso_code,country_name\n1,US,CA,\"United, States\"\n2,GB,,United Kingdom\n3,,,\n"; + + #[test] + fn csv_join_country_only_and_no_registered_fallback() { + let loc = locations(LOCATIONS.as_bytes()).unwrap(); + let raw = blocks("network,geoname_id,registered_country_geoname_id\n1.0.0.0/24,,1\n1.0.1.0/24,1,2\n1.0.2.0/24,2,1\n1.0.3.0/24,3,1\n".as_bytes(), &loc, 4).unwrap(); + assert_eq!(raw[0].location, None); + assert_eq!(raw[1].location, Some(Location::new("US", "CA").unwrap())); + assert_eq!(raw[2].location, Some(Location::new("GB", "").unwrap())); + assert_eq!(raw[3].location, None); + assert_eq!(oracle(&raw, "1.0.1.255".parse().unwrap()), raw[1].location); + } + + #[test] + fn source_codes_are_preserved_without_changing_core_policy() { + let loc = locations("geoname_id,country_iso_code\n1,XK\n".as_bytes()).unwrap(); + let raw = blocks("network,geoname_id\n0.0.0.0/0,1\n".as_bytes(), &loc, 4).unwrap(); + let (ranges, stats) = project(&raw); + assert!(!ranges[0].location.core_country_supported()); + assert_eq!(stats.unsupported_core_country_networks.get("XK"), Some(&1)); + } + + #[test] + fn invalid_csv_data_is_rejected() { + assert!( + locations("geoname_id,country_iso_code,subdivision_1_iso_code\n1,,CA\n".as_bytes()) + .is_err() + ); + assert!(locations("geoname_id,country_iso_code\n1,US\n1,GB\n".as_bytes()).is_err()); + let loc = locations(LOCATIONS.as_bytes()).unwrap(); + for (csv, family) in [ + ("network,geoname_id\n1.0.0.0/24,999\n", 4), + ("network,geoname_id\n::/0,1\n", 4), + ("network,geoname_id\n1.0.0.1/24,1\n", 4), + ("network,geoname_id\n1.0.0.0/24,1\n1.0.0.0/25,2\n", 4), + ] { + assert!(blocks(csv.as_bytes(), &loc, family).is_err()); + } + } + + #[test] + fn projection_merges_only_adjacent_equal_locations_and_preserves_gaps() { + let loc = locations(LOCATIONS.as_bytes()).unwrap(); + let raw = blocks("network,geoname_id\n0.0.0.0/32,1\n0.0.0.1/32,1\n0.0.0.2/32,\n0.0.0.3/32,1\n255.255.255.255/32,2\n".as_bytes(), &loc, 4).unwrap(); + let (ranges, stats) = project(&raw); + assert_eq!(stats.without_country, 1); + assert_eq!(ranges.len(), 3); + assert_eq!((ranges[0].start, ranges[0].end), (0, 1)); + assert_eq!( + (ranges[2].start, ranges[2].end), + (u32::MAX.into(), u32::MAX.into()) + ); + let raw = blocks("network,geoname_id\n::/0,2\n".as_bytes(), &loc, 6).unwrap(); + assert_eq!(project(&raw).0[0].end, u128::MAX); + } +} diff --git a/examples/geo-kv-prototype/src/index.rs b/examples/geo-kv-prototype/src/index.rs new file mode 100644 index 00000000..169698f7 --- /dev/null +++ b/examples/geo-kv-prototype/src/index.rs @@ -0,0 +1,767 @@ +//! Experimental p1 encoding. No filesystem or runtime-specific API in lookup. +use std::{ + collections::{HashMap, HashSet, VecDeque}, + net::IpAddr, +}; + +use anyhow::{Context, Result, bail, ensure}; +use bytes::Bytes; +use edgezero_core::{ + geo::{CountryCode, SubdivisionCode}, + key_value_store::KvHandle, +}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; + +const HEADER: usize = 12; +const MAGIC: &[u8; 4] = b"EZP1"; + +pub fn digest(bytes: &[u8]) -> [u8; 32] { + Sha256::digest(bytes).into() +} +pub fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub struct Location(pub [u8; 6]); + +impl Location { + pub fn new(country: &str, subdivision: &str) -> Result { + // Source-format DTO, not GeoLocation. Preserve XK and report the domain mismatch. + ensure!( + country.len() == 2 && country.bytes().all(|b| b.is_ascii_alphabetic()), + "invalid source country syntax" + ); + let country = country.to_ascii_uppercase(); + let subdivision = if subdivision.is_empty() { + None + } else { + Some(SubdivisionCode::new(subdivision)?) + }; + let mut bytes = [0; 6]; + bytes[..2].copy_from_slice(country.as_bytes()); + if let Some(subdivision) = subdivision { + let s = subdivision.as_str().as_bytes(); + bytes[2] = s.len() as u8; + bytes[3..3 + s.len()].copy_from_slice(s); + } + Ok(Self(bytes)) + } + + fn decode(bytes: &[u8]) -> Result { + let raw: [u8; 6] = bytes.try_into()?; + let len = raw[2] as usize; + ensure!( + len <= 3 && raw[3 + len..].iter().all(|b| *b == 0), + "invalid location encoding" + ); + let canonical = Self::new( + std::str::from_utf8(&raw[..2])?, + std::str::from_utf8(&raw[3..3 + len])?, + )?; + ensure!(canonical.0 == raw, "noncanonical location"); + Ok(canonical) + } + + pub fn core_country_supported(self) -> bool { + CountryCode::new(std::str::from_utf8(&self.0[..2]).expect("validated source code")).is_ok() + } + + pub fn json(self) -> serde_json::Value { + let country = std::str::from_utf8(&self.0[..2]).expect("validated country"); + let subdivision = if self.0[2] == 0 { + None + } else { + Some( + std::str::from_utf8(&self.0[3..3 + self.0[2] as usize]) + .expect("validated subdivision"), + ) + }; + serde_json::json!({"country": country, "subdivision": subdivision, "core_country_supported": self.core_country_supported()}) + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Range { + pub start: u128, + pub end: u128, + pub location: Location, +} + +pub fn width(family: u8) -> Result { + match family { + 4 => Ok(4), + 6 => Ok(16), + _ => bail!("invalid address family"), + } +} + +pub fn address(ip: IpAddr) -> (u8, u128) { + match ip { + IpAddr::V4(ip) => (4, u32::from(ip).into()), + IpAddr::V6(ip) => (6, u128::from(ip)), + } +} + +fn max_address(family: u8) -> u128 { + if family == 4 { + u32::MAX.into() + } else { + u128::MAX + } +} +fn read_address(bytes: &[u8]) -> u128 { + let mut raw = [0; 16]; + raw[16 - bytes.len()..].copy_from_slice(bytes); + u128::from_be_bytes(raw) +} +fn write_address(out: &mut Vec, value: u128, width: usize) { + out.extend_from_slice(&value.to_be_bytes()[16 - width..]); +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +pub struct NodeRef { + pub start: u128, + pub end: u128, + pub hash: [u8; 32], + pub level: u16, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +pub struct Source { + pub edition: String, + pub files: Vec<(String, String)>, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +pub struct Manifest { + pub schema: u32, + pub generation: String, + pub object_bytes: usize, + pub source: Source, + pub ipv4: Option, + pub ipv6: Option, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +pub struct Receipt { + pub manifest_key: String, + pub manifest_hash: [u8; 32], +} + +#[derive(Debug, Serialize)] +pub struct InventoryEntry { + pub key: String, + pub sha256: String, + pub bytes: usize, +} + +pub fn node_key(generation: &str, family: u8, hash: &[u8; 32]) -> String { + format!("geo/p1/{generation}/v{family}/{}", hex(hash)) +} + +fn header(family: u8, level: u16, count: usize) -> Result> { + let mut bytes = MAGIC.to_vec(); + bytes.extend_from_slice(&[family, u8::from(level > 0)]); + bytes.extend_from_slice(&level.to_be_bytes()); + bytes.extend_from_slice(&u32::try_from(count)?.to_be_bytes()); + Ok(bytes) +} + +async fn publish_node( + kv: &KvHandle, + generation: &str, + family: u8, + reference: &NodeRef, + bytes: Vec, + inventory: &mut Vec, +) -> Result<()> { + let key = node_key(generation, family, &reference.hash); + inventory.push(InventoryEntry { + key: key.clone(), + sha256: hex(&reference.hash), + bytes: bytes.len(), + }); + kv.put_bytes(&key, bytes.into()).await?; + Ok(()) +} + +async fn build_family( + kv: &KvHandle, + generation: &str, + family: u8, + ranges: &[Range], + object_bytes: usize, + inventory: &mut Vec, +) -> Result> { + let w = width(family)?; + let leaf_capacity = (object_bytes - HEADER) / (2 * w + 6); + let directory_capacity = (object_bytes - HEADER) / (2 * w + 32); + ensure!( + leaf_capacity > 0 && directory_capacity >= 2, + "object budget cannot fit a branching directory" + ); + let mut previous_end = None; + for range in ranges { + ensure!( + range.start <= range.end && range.end <= max_address(family), + "invalid range" + ); + ensure!( + previous_end.is_none_or(|end| range.start > end), + "overlapping or unordered ranges" + ); + Location::decode(&range.location.0)?; + previous_end = Some(range.end); + } + let mut children = Vec::new(); + for chunk in ranges.chunks(leaf_capacity) { + let mut bytes = header(family, 0, chunk.len())?; + for range in chunk { + write_address(&mut bytes, range.start, w); + write_address(&mut bytes, range.end, w); + bytes.extend_from_slice(&range.location.0); + } + let reference = NodeRef { + start: chunk[0].start, + end: chunk[chunk.len() - 1].end, + hash: digest(&bytes), + level: 0, + }; + publish_node(kv, generation, family, &reference, bytes, inventory).await?; + children.push(reference); + } + while children.len() > 1 { + let mut parents = Vec::new(); + for chunk in children.chunks(directory_capacity) { + let level = chunk[0] + .level + .checked_add(1) + .context("tree depth overflow")?; + let mut bytes = header(family, level, chunk.len())?; + for child in chunk { + write_address(&mut bytes, child.start, w); + write_address(&mut bytes, child.end, w); + bytes.extend_from_slice(&child.hash); + } + let reference = NodeRef { + start: chunk[0].start, + end: chunk[chunk.len() - 1].end, + hash: digest(&bytes), + level, + }; + publish_node(kv, generation, family, &reference, bytes, inventory).await?; + parents.push(reference); + } + children = parents; + } + Ok(children.pop()) +} + +pub async fn build( + kv: &KvHandle, + source: Source, + v4: &[Range], + v6: &[Range], + object_bytes: usize, +) -> Result<(Receipt, Manifest, Vec)> { + // The minimum follows the encoding: header plus two IPv6 directory records. + ensure!( + (HEADER + 2 * (32 + 32)..=KvHandle::MAX_VALUE_SIZE).contains(&object_bytes), + "invalid object byte budget" + ); + let generation = hex(&digest(&serde_json::to_vec(&( + 1_u32, + object_bytes, + &source, + ))?)); + let mut inventory = Vec::new(); + let ipv4 = build_family(kv, &generation, 4, v4, object_bytes, &mut inventory).await?; + let ipv6 = build_family(kv, &generation, 6, v6, object_bytes, &mut inventory).await?; + let manifest = Manifest { + schema: 1, + generation: generation.clone(), + object_bytes, + source, + ipv4, + ipv6, + }; + let bytes = serde_json::to_vec(&manifest)?; + ensure!( + bytes.len() <= object_bytes, + "manifest exceeds target value budget" + ); + let receipt = Receipt { + manifest_key: format!("geo/p1/{generation}/manifest"), + manifest_hash: digest(&bytes), + }; + inventory.push(InventoryEntry { + key: receipt.manifest_key.clone(), + sha256: hex(&receipt.manifest_hash), + bytes: bytes.len(), + }); + kv.put_bytes(&receipt.manifest_key, bytes.into()).await?; + Ok((receipt, manifest, inventory)) +} + +#[derive(Clone)] +struct Object { + bytes: Bytes, + width: usize, + record_bytes: usize, + count: usize, +} + +impl Object { + fn record(&self, i: usize) -> &[u8] { + &self.bytes[HEADER + i * self.record_bytes..HEADER + (i + 1) * self.record_bytes] + } + + fn bounds(&self, i: usize) -> (u128, u128) { + let r = self.record(i); + ( + read_address(&r[..self.width]), + read_address(&r[self.width..2 * self.width]), + ) + } + + fn decode(bytes: Bytes, family: u8, reference: &NodeRef, max_bytes: usize) -> Result { + ensure!( + bytes.len() <= max_bytes && bytes.len() >= HEADER, + "invalid object length" + ); + ensure!(digest(&bytes) == reference.hash, "object digest mismatch"); + ensure!( + &bytes[..4] == MAGIC && bytes[4] == family, + "invalid object header" + ); + let level = u16::from_be_bytes(bytes[6..8].try_into()?); + ensure!( + level == reference.level && bytes[5] == u8::from(level > 0), + "invalid object level" + ); + let w = width(family)?; + let count = u32::from_be_bytes(bytes[8..12].try_into()?) as usize; + let record_bytes = 2 * w + if level == 0 { 6 } else { 32 }; + ensure!( + count > 0 + && count + .checked_mul(record_bytes) + .and_then(|n| n.checked_add(HEADER)) + == Some(bytes.len()), + "invalid object records" + ); + let object = Self { + bytes, + width: w, + record_bytes, + count, + }; + let mut previous_end = None; + let mut locations = HashSet::new(); + for i in 0..count { + let (start, end) = object.bounds(i); + ensure!( + start <= end && previous_end.is_none_or(|e| start > e), + "invalid record ordering" + ); + if level == 0 { + let raw: [u8; 6] = object.record(i)[2 * w..].try_into()?; + if locations.insert(raw) { + Location::decode(&raw)?; + } + } + previous_end = Some(end); + } + ensure!( + object.bounds(0).0 == reference.start && object.bounds(count - 1).1 == reference.end, + "child envelope mismatch" + ); + Ok(object) + } + + fn find(&self, ip: u128) -> Option { + let mut lo = 0; + let mut hi = self.count; + while lo < hi { + let mid = lo + (hi - lo) / 2; + if self.bounds(mid).0 <= ip { + lo = mid + 1; + } else { + hi = mid; + } + } + (lo > 0 && ip <= self.bounds(lo - 1).1).then(|| lo - 1) + } +} + +#[derive(Default, Debug, Serialize)] +pub struct CacheStats { + pub hits: u64, + pub peak_payload_bytes: usize, + pub max_object_bytes: usize, +} + +struct Cache { + budget: usize, + bytes: usize, + values: HashMap<[u8; 32], Object>, + order: VecDeque<[u8; 32]>, + stats: CacheStats, +} + +impl Cache { + fn new(budget: usize) -> Self { + Self { + budget, + bytes: 0, + values: HashMap::new(), + order: VecDeque::new(), + stats: CacheStats::default(), + } + } + + fn get(&mut self, hash: &[u8; 32]) -> Option { + let object = self.values.get(hash)?.clone(); + self.order.retain(|h| h != hash); + self.order.push_back(*hash); + self.stats.hits += 1; + Some(object) + } + + fn insert(&mut self, hash: [u8; 32], object: Object) { + let size = object.bytes.len(); + self.stats.max_object_bytes = self.stats.max_object_bytes.max(size); + if size > self.budget { + return; + } + while self.bytes + size > self.budget { + let hash = self.order.pop_front().expect("cache accounting"); + self.bytes -= self.values.remove(&hash).expect("cache entry").bytes.len(); + } + self.bytes += size; + self.values.insert(hash, object); + self.order.push_back(hash); + self.stats.peak_payload_bytes = self.stats.peak_payload_bytes.max(self.bytes); + } +} + +pub struct Resolver { + kv: KvHandle, + pub manifest: Manifest, + cache: Cache, +} + +impl Resolver { + pub async fn open(kv: KvHandle, receipt: &Receipt, cache_bytes: usize) -> Result { + let bytes = kv + .get_bytes(&receipt.manifest_key) + .await? + .context("missing manifest")?; + ensure!( + digest(&bytes) == receipt.manifest_hash, + "manifest digest mismatch" + ); + let manifest: Manifest = serde_json::from_slice(&bytes)?; + ensure!( + manifest.schema == 1 + && manifest.generation.len() == 64 + && manifest.generation.bytes().all(|b| b.is_ascii_hexdigit()), + "invalid manifest identity" + ); + ensure!( + receipt.manifest_key == format!("geo/p1/{}/manifest", manifest.generation), + "manifest key mismatch" + ); + ensure!( + (HEADER + 128..=KvHandle::MAX_VALUE_SIZE).contains(&manifest.object_bytes), + "invalid manifest object budget" + ); + for (family, root) in [(4, &manifest.ipv4), (6, &manifest.ipv6)] { + if let Some(root) = root { + ensure!( + root.start <= root.end && root.end <= max_address(family), + "invalid family root" + ); + } + } + Ok(Self { + kv, + manifest, + cache: Cache::new(cache_bytes), + }) + } + + pub fn cache_stats(&self) -> &CacheStats { + &self.cache.stats + } + + pub async fn lookup(&mut self, ip: IpAddr) -> Result> { + let (family, ip) = address(ip); + let Some(mut reference) = (if family == 4 { + &self.manifest.ipv4 + } else { + &self.manifest.ipv6 + }) + .clone() else { + return Ok(None); + }; + loop { + if ip < reference.start || ip > reference.end { + return Ok(None); + } + let object = if let Some(object) = self.cache.get(&reference.hash) { + object + } else { + let key = node_key(&self.manifest.generation, family, &reference.hash); + let bytes = self + .kv + .get_bytes(&key) + .await? + .context("missing required index object")?; + let object = Object::decode(bytes, family, &reference, self.manifest.object_bytes)?; + self.cache.insert(reference.hash, object.clone()); + object + }; + // Cached content still has to agree with this parent's envelope and level. + ensure!( + object.bounds(0).0 == reference.start + && object.bounds(object.count - 1).1 == reference.end + && u16::from_be_bytes(object.bytes[6..8].try_into()?) == reference.level, + "cached child descriptor mismatch" + ); + let Some(i) = object.find(ip) else { + return Ok(None); + }; + if reference.level == 0 { + return Ok(Some(Location( + object.record(i)[2 * object.width..].try_into()?, + ))); + } + let (start, end) = object.bounds(i); + reference = NodeRef { + start, + end, + hash: object.record(i)[2 * object.width..].try_into()?, + level: reference.level - 1, + }; + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use async_trait::async_trait; + use edgezero_core::key_value_store::{KvError, KvPage, KvStore}; + use futures::executor::block_on; + use std::{ + sync::{Arc, Mutex}, + time::Duration, + }; + + #[derive(Default)] + struct MemoryStore(Mutex>); + #[async_trait(?Send)] + impl KvStore for MemoryStore { + async fn get_bytes(&self, key: &str) -> Result, KvError> { + Ok(self.0.lock().unwrap().get(key).cloned()) + } + async fn put_bytes(&self, key: &str, bytes: Bytes) -> Result<(), KvError> { + self.0.lock().unwrap().insert(key.into(), bytes); + Ok(()) + } + async fn delete(&self, _: &str) -> Result<(), KvError> { + panic!("lookup must not delete") + } + async fn list_keys_page( + &self, + _: &str, + _: Option<&str>, + _: usize, + ) -> Result { + panic!("lookup must not list") + } + async fn put_bytes_with_ttl(&self, _: &str, _: Bytes, _: Duration) -> Result<(), KvError> { + panic!("no TTL") + } + } + fn source() -> Source { + Source { + edition: "synthetic".into(), + files: vec![], + } + } + fn ranges() -> Vec { + (0..10_000) + .map(|i| Range { + start: i * 4, + end: i * 4 + 1, + location: Location::new("US", if i % 2 == 0 { "CA" } else { "" }).unwrap(), + }) + .collect() + } + + #[test] + fn round_trip_multilevel_cache_and_determinism() { + block_on(async { + let raw = Arc::new(MemoryStore::default()); + let kv = KvHandle::new(raw.clone()); + let ranges = ranges(); + let (receipt, manifest, _) = + build(&kv, source(), &ranges, &ranges, 1024).await.unwrap(); + assert!(manifest.ipv6.as_ref().unwrap().level >= 3); + let before = raw.0.lock().unwrap().clone(); + let (again, _, _) = build(&kv, source(), &ranges, &ranges, 1024).await.unwrap(); + assert_eq!(receipt, again); + assert_eq!(before, *raw.0.lock().unwrap()); + for budget in [0, 1024, 1024 * 1024] { + let mut resolver = Resolver::open(kv.clone(), &receipt, budget).await.unwrap(); + for family in [4, 6] { + for n in 0..410_u128 { + let ip = if family == 4 { + IpAddr::from((n as u32).to_be_bytes()) + } else { + IpAddr::from(n.to_be_bytes()) + }; + let expected = ranges + .iter() + .find(|r| r.start <= n && n <= r.end) + .map(|r| r.location); + assert_eq!(resolver.lookup(ip).await.unwrap(), expected); + } + } + assert!(resolver.cache_stats().peak_payload_bytes <= budget); + } + }); + } + + #[test] + fn full_family_and_empty_family() { + block_on(async { + for family in [4, 6] { + let kv = KvHandle::new(Arc::new(MemoryStore::default())); + let ranges = [Range { + start: 0, + end: max_address(family), + location: Location::new("GB", "ENG").unwrap(), + }]; + let (receipt, _, _) = build( + &kv, + source(), + if family == 4 { &ranges } else { &[] }, + if family == 6 { &ranges } else { &[] }, + 1024, + ) + .await + .unwrap(); + let mut resolver = Resolver::open(kv, &receipt, 0).await.unwrap(); + for n in [0, max_address(family)] { + let ip = if family == 4 { + IpAddr::from((n as u32).to_be_bytes()) + } else { + IpAddr::from(n.to_be_bytes()) + }; + assert_eq!(resolver.lookup(ip).await.unwrap(), Some(ranges[0].location)); + } + let empty = if family == 4 { "::1" } else { "1.0.0.1" }; + assert_eq!(resolver.lookup(empty.parse().unwrap()).await.unwrap(), None); + } + }); + } + + #[test] + fn source_country_roundtrip_does_not_claim_domain_acceptance() { + block_on(async { + let kv = KvHandle::new(Arc::new(MemoryStore::default())); + let ranges = [Range { + start: 0, + end: 1, + location: Location::new("XK", "").unwrap(), + }]; + let (receipt, _, _) = build(&kv, source(), &ranges, &[], 1024).await.unwrap(); + let mut resolver = Resolver::open(kv, &receipt, 0).await.unwrap(); + let location = resolver + .lookup("0.0.0.1".parse().unwrap()) + .await + .unwrap() + .unwrap(); + assert_eq!(location, ranges[0].location); + assert!(!location.core_country_supported()); + }); + } + + #[test] + fn missing_and_corrupt_objects_are_errors_not_misses() { + block_on(async { + let raw = Arc::new(MemoryStore::default()); + let kv = KvHandle::new(raw.clone()); + let (receipt, manifest, _) = build(&kv, source(), &ranges(), &[], 1024).await.unwrap(); + let key = node_key( + &manifest.generation, + 4, + &manifest.ipv4.as_ref().unwrap().hash, + ); + let old = raw.0.lock().unwrap().remove(&key).unwrap(); + let mut resolver = Resolver::open(kv.clone(), &receipt, 0).await.unwrap(); + assert!( + resolver + .lookup("0.0.0.0".parse().unwrap()) + .await + .unwrap_err() + .to_string() + .contains("missing required") + ); + let mut corrupt = old.to_vec(); + corrupt[0] ^= 1; + raw.0.lock().unwrap().insert(key, corrupt.into()); + assert!( + resolver + .lookup("0.0.0.0".parse().unwrap()) + .await + .unwrap_err() + .to_string() + .contains("digest mismatch") + ); + let mut bad_receipt = receipt.clone(); + bad_receipt.manifest_hash[0] ^= 1; + assert!(Resolver::open(kv, &bad_receipt, 0).await.is_err()); + }); + } + + #[test] + fn malformed_records_with_matching_digest_are_rejected() { + let reference = |b: &[u8]| NodeRef { + start: 0, + end: 4, + hash: digest(b), + level: 0, + }; + let mut bytes = header(4, 0, 2).unwrap(); + for (start, end) in [(0, 2), (2, 4)] { + write_address(&mut bytes, start, 4); + write_address(&mut bytes, end, 4); + bytes.extend_from_slice(&Location::new("US", "CA").unwrap().0); + } + assert!(Object::decode(bytes.clone().into(), 4, &reference(&bytes), 140).is_err()); + let mut bytes = header(4, 0, 1).unwrap(); + write_address(&mut bytes, 0, 4); + write_address(&mut bytes, 4, 4); + bytes.extend_from_slice(&[b'U', b'S', 4, b'C', b'A', 0]); + assert!(Object::decode(bytes.clone().into(), 4, &reference(&bytes), 140).is_err()); + bytes.pop(); + assert!(Object::decode(bytes.clone().into(), 4, &reference(&bytes), 140).is_err()); + } + + #[test] + fn rejects_overlaps_and_too_small_objects() { + block_on(async { + let kv = KvHandle::new(Arc::new(MemoryStore::default())); + let mut ranges = ranges(); + ranges[1].start = 1; + assert!(build(&kv, source(), &ranges, &[], 140).await.is_err()); + assert!(build(&kv, source(), &[], &[], 139).await.is_err()); + }); + } +} diff --git a/examples/geo-kv-prototype/src/main.rs b/examples/geo-kv-prototype/src/main.rs new file mode 100644 index 00000000..13688c3e --- /dev/null +++ b/examples/geo-kv-prototype/src/main.rs @@ -0,0 +1,237 @@ +mod import; +mod index; +mod store; + +use std::{fs, net::IpAddr, path::Path, sync::Arc, time::Instant}; + +use anyhow::{Context, Result, bail, ensure}; +use edgezero_core::key_value_store::KvHandle; +use futures::executor::block_on; +use serde_json::json; + +use index::{Receipt, Resolver, digest, hex}; +use store::ArtifactStore; + +fn peak_rss_bytes() -> Option { + // Linux process high-water mark, including importer/oracle/backend/cache. + fs::read_to_string("/proc/self/status") + .ok()? + .lines() + .find_map(|line| { + line.strip_prefix("VmHWM:")? + .split_whitespace() + .next()? + .parse::() + .ok() + .map(|n| n * 1024) + }) +} + +fn write_json(path: &Path, value: &impl serde::Serialize) -> Result<()> { + fs::write(path, serde_json::to_vec_pretty(value)?)?; + Ok(()) +} + +fn read_receipt(out: &Path) -> Result { + Ok(serde_json::from_slice(&fs::read( + out.join("receipt.json"), + )?)?) +} + +async fn build(source: &Path, out: &Path, object_bytes: usize) -> Result<()> { + let started = Instant::now(); + let imported = import::load(source)?; + let import_seconds = started.elapsed().as_secs_f64(); + let (v4, stats4) = import::project(&imported.raw4); + let (v6, stats6) = import::project(&imported.raw6); + let projection_seconds = started.elapsed().as_secs_f64() - import_seconds; + fs::create_dir_all(out)?; + let store = Arc::new(ArtifactStore::new(out)); + let kv = KvHandle::new(store.clone()); + let build_started = Instant::now(); + let (receipt, manifest, inventory) = + index::build(&kv, imported.source, &v4, &v6, object_bytes).await?; + let build_seconds = build_started.elapsed().as_secs_f64(); + if out.join("receipt.json").exists() { + ensure!( + read_receipt(out)? == receipt, + "output already selects another generation; use a different output directory" + ); + } + let verify_started = Instant::now(); + for item in &inventory { + let bytes = kv + .get_bytes(&item.key) + .await? + .context("published object missing")?; + ensure!( + bytes.len() == item.bytes && hex(&digest(&bytes)) == item.sha256, + "published object read-back mismatch" + ); + } + let report = json!({ + "prototype": "p1, not a production format", + "transport": "local artifact files through KvHandle; no remote KV measurements", + "source": manifest.source, + "manifest_key": receipt.manifest_key, + "manifest_sha256": hex(&receipt.manifest_hash), + "locations": imported.locations, + "ipv4": stats4, + "ipv6": stats6, + "object_budget_bytes": object_bytes, + "objects_including_manifest": inventory.len(), + "encoded_bytes_including_manifest": inventory.iter().map(|i| i.bytes).sum::(), + "largest_object_bytes": inventory.iter().map(|i| i.bytes).max(), + "ipv4_lookup_objects": manifest.ipv4.as_ref().map(|r| usize::from(r.level) + 1), + "ipv6_lookup_objects": manifest.ipv6.as_ref().map(|r| usize::from(r.level) + 1), + "import_seconds_including_source_hashing": import_seconds, + "projection_seconds": projection_seconds, + "build_and_write_seconds": build_seconds, + "readback_seconds": verify_started.elapsed().as_secs_f64(), + "total_seconds": started.elapsed().as_secs_f64(), + "process_peak_rss_bytes": peak_rss_bytes(), + "store": store.stats(), + }); + write_json(&out.join("receipt.json"), &receipt)?; + write_json(&out.join("inventory.json"), &inventory)?; + write_json(&out.join("build-report.json"), &report)?; + println!("{}", serde_json::to_string_pretty(&report)?); + Ok(()) +} + +async fn measure( + name: &str, + resolver: &mut Resolver, + store: &ArtifactStore, + queries: &[(IpAddr, Option)], +) -> Result { + store.reset_stats(); + let mut times = Vec::with_capacity(queries.len()); + let mut read_histogram = std::collections::BTreeMap::new(); + let mut located = 0; + let hits_before = resolver.cache_stats().hits; + for (ip, expected) in queries { + let before = store.stats().reads; + let started = Instant::now(); + let actual = resolver.lookup(*ip).await?; + times.push(started.elapsed().as_secs_f64() * 1_000_000.0); + ensure!(actual == *expected, "CSV oracle mismatch"); + located += usize::from(actual.is_some()); + *read_histogram + .entry(store.stats().reads - before) + .or_insert(0_usize) += 1; + } + times.sort_by(f64::total_cmp); + let stats = store.stats(); + Ok(json!({ + "pass": name, "queries": queries.len(), "located": located, + "no_match": queries.len() - located, "oracle_mismatches": 0, + "mean_microseconds": times.iter().sum::() / times.len() as f64, + "p50_microseconds": times[times.len() / 2], + "p95_microseconds": times[(times.len() - 1) * 95 / 100], + "max_microseconds": times[times.len() - 1], + "reads_per_query_histogram": read_histogram, + "reads_per_query_mean": stats.reads as f64 / queries.len() as f64, + "bytes_per_query_mean": stats.read_bytes as f64 / queries.len() as f64, + "cache_hits": resolver.cache_stats().hits - hits_before, + "peak_cached_payload_bytes": resolver.cache_stats().peak_payload_bytes, + "max_decoded_object_bytes": resolver.cache_stats().max_object_bytes, + "store": stats, + })) +} + +async fn verify(source: &Path, out: &Path, count: usize, cache_bytes: usize) -> Result<()> { + ensure!( + count >= 2, + "at least two samples per family are needed for endpoints" + ); + let imported = import::load(source)?; + let mut queries = Vec::new(); + for (family, raw) in [(4, &imported.raw4), (6, &imported.raw6)] { + for ip in import::samples(raw, family, count) { + queries.push((ip, import::oracle(raw, ip))); + } + } + let receipt = read_receipt(out)?; + let store = Arc::new(ArtifactStore::new(out)); + let kv = KvHandle::new(store.clone()); + let mut disabled = Resolver::open(kv.clone(), &receipt, 0).await?; + ensure!( + disabled.manifest.source == imported.source, + "oracle source differs from indexed source" + ); + let mut reports = vec![measure("cache_disabled", &mut disabled, &store, &queries).await?]; + let mut enabled = Resolver::open(kv, &receipt, cache_bytes).await?; + reports.push(measure("cache_enabled_first_pass", &mut enabled, &store, &queries).await?); + reports.push(measure("cache_enabled_repeat", &mut enabled, &store, &queries).await?); + let report = json!({ + "transport": "local file artifact KV; OS page cache uncontrolled; not network latency", + "manifest_sha256": hex(&receipt.manifest_hash), + "sample_method": "per family: zero/max, stratified source CIDR endpoints and neighbors, deterministic uniform random remainder", + "samples_per_family": count, + "manifest_reads_per_resolver_initialization": 1, + "cache_payload_budget_bytes": cache_bytes, + "passes": reports, + "process_peak_rss_bytes_including_csv_oracle": peak_rss_bytes(), + }); + write_json(&out.join("verification-report.json"), &report)?; + println!("{}", serde_json::to_string_pretty(&report)?); + Ok(()) +} + +async fn lookup(out: &Path, ips: &[String]) -> Result<()> { + ensure!(!ips.is_empty(), "provide at least one IP address"); + let receipt = read_receipt(out)?; + let store = Arc::new(ArtifactStore::new(out)); + let mut resolver = + Resolver::open(KvHandle::new(store.clone()), &receipt, 4 * 1024 * 1024).await?; + let mut results = Vec::new(); + for text in ips { + let ip: IpAddr = text.parse()?; + results.push(resolver.lookup(ip).await?.map(index::Location::json)); + } + println!( + "{}", + serde_json::to_string_pretty(&json!({ + "results_in_input_order": results, + "store": store.stats(), "cache": resolver.cache_stats(), + "process_peak_rss_bytes_without_csv_import": peak_rss_bytes(), + }))? + ); + Ok(()) +} + +fn main() -> Result<()> { + let args: Vec = std::env::args().skip(1).collect(); + block_on(async { + match args.first().map(String::as_str) { + Some("build") if (3..=4).contains(&args.len()) => { + build( + Path::new(&args[1]), + Path::new(&args[2]), + args.get(3) + .map(|s| s.parse()) + .transpose()? + .unwrap_or(256 * 1024), + ) + .await + } + Some("verify") if (3..=5).contains(&args.len()) => { + verify( + Path::new(&args[1]), + Path::new(&args[2]), + args.get(3).map(|s| s.parse()).transpose()?.unwrap_or(2048), + args.get(4) + .map(|s| s.parse()) + .transpose()? + .unwrap_or(4 * 1024 * 1024), + ) + .await + } + Some("lookup") if args.len() >= 3 => lookup(Path::new(&args[1]), &args[2..]).await, + _ => bail!( + "usage: geo-kv-prototype build [object-bytes]\n geo-kv-prototype verify [samples-per-family] [cache-bytes]\n geo-kv-prototype lookup [ip ...]" + ), + } + }) +} diff --git a/examples/geo-kv-prototype/src/store.rs b/examples/geo-kv-prototype/src/store.rs new file mode 100644 index 00000000..67ef9d1e --- /dev/null +++ b/examples/geo-kv-prototype/src/store.rs @@ -0,0 +1,129 @@ +//! Local artifact transport only. The resolver itself knows only KvHandle. +//! Synchronous filesystem calls here are NOT a production async KV backend. +use std::{ + fs, + path::{Component, Path, PathBuf}, + sync::atomic::{AtomicU64, Ordering}, + time::Duration, +}; + +use async_trait::async_trait; +use bytes::Bytes; +use edgezero_core::key_value_store::{KvError, KvPage, KvStore}; +use serde::Serialize; + +#[derive(Default)] +pub struct ArtifactStore { + root: PathBuf, + reads: AtomicU64, + read_bytes: AtomicU64, + writes: AtomicU64, + write_bytes: AtomicU64, +} + +#[derive(Debug, Default, Clone, Serialize)] +pub struct StoreStats { + pub reads: u64, + pub read_bytes: u64, + pub writes: u64, + pub write_bytes: u64, +} + +impl ArtifactStore { + pub fn new(root: impl Into) -> Self { + Self { + root: root.into(), + ..Self::default() + } + } + + fn path(&self, key: &str) -> Result { + if !key.starts_with("geo/p1/") + || Path::new(key) + .components() + .any(|p| !matches!(p, Component::Normal(_))) + { + return Err(KvError::Validation("invalid artifact key".into())); + } + Ok(self.root.join(key)) + } + + pub fn stats(&self) -> StoreStats { + StoreStats { + reads: self.reads.load(Ordering::Relaxed), + read_bytes: self.read_bytes.load(Ordering::Relaxed), + writes: self.writes.load(Ordering::Relaxed), + write_bytes: self.write_bytes.load(Ordering::Relaxed), + } + } + + pub fn reset_stats(&self) { + self.reads.store(0, Ordering::Relaxed); + self.read_bytes.store(0, Ordering::Relaxed); + self.writes.store(0, Ordering::Relaxed); + self.write_bytes.store(0, Ordering::Relaxed); + } +} + +fn unsupported(operation: &str) -> KvError { + KvError::Unsupported { + operation: operation.into(), + } +} + +#[async_trait(?Send)] +impl KvStore for ArtifactStore { + async fn get_bytes(&self, key: &str) -> Result, KvError> { + self.reads.fetch_add(1, Ordering::Relaxed); + let value = match fs::read(self.path(key)?) { + Ok(value) => value, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(e) => return Err(KvError::Internal(e.into())), + }; + self.read_bytes + .fetch_add(value.len() as u64, Ordering::Relaxed); + Ok(Some(value.into())) + } + + async fn put_bytes(&self, key: &str, value: Bytes) -> Result<(), KvError> { + let path = self.path(key)?; + fs::create_dir_all( + path.parent() + .ok_or_else(|| KvError::Validation("missing parent".into()))?, + ) + .map_err(|e| KvError::Internal(e.into()))?; + // A rerun may repeat the same immutable publication, never replace different bytes. + match fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&path) + { + Ok(mut file) => { + use std::io::Write; + file.write_all(&value) + .map_err(|e| KvError::Internal(e.into()))?; + } + Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => { + let old = fs::read(path).map_err(|e| KvError::Internal(e.into()))?; + if old != value { + return Err(KvError::Validation("immutable artifact differs".into())); + } + } + Err(e) => return Err(KvError::Internal(e.into())), + } + self.writes.fetch_add(1, Ordering::Relaxed); + self.write_bytes + .fetch_add(value.len() as u64, Ordering::Relaxed); + Ok(()) + } + + async fn delete(&self, _: &str) -> Result<(), KvError> { + Err(unsupported("delete")) + } + async fn list_keys_page(&self, _: &str, _: Option<&str>, _: usize) -> Result { + Err(unsupported("list_keys_page")) + } + async fn put_bytes_with_ttl(&self, _: &str, _: Bytes, _: Duration) -> Result<(), KvError> { + Err(unsupported("put_bytes_with_ttl")) + } +} From c155e5de3438b75bc81668ec8de2b6feabb2100a Mon Sep 17 00:00:00 2001 From: Christian Date: Tue, 10 Nov 2026 14:05:12 -0600 Subject: [PATCH 2/9] fix(geo): refresh prototype lockfile for current core --- examples/geo-kv-prototype/Cargo.lock | 88 +++++++++++++++++++++++++--- 1 file changed, 79 insertions(+), 9 deletions(-) diff --git a/examples/geo-kv-prototype/Cargo.lock b/examples/geo-kv-prototype/Cargo.lock index ce02cb52..44579008 100644 --- a/examples/geo-kv-prototype/Cargo.lock +++ b/examples/geo-kv-prototype/Cargo.lock @@ -83,6 +83,12 @@ dependencies = [ "syn 2.0.119", ] +[[package]] +name = "base16ct" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fd307490d624467aa6f74b0eabb77633d1f758a7b25f12bceb0b22e08d9726f6" + [[package]] name = "block-buffer" version = "0.10.4" @@ -92,6 +98,15 @@ dependencies = [ "generic-array", ] +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + [[package]] name = "brotli" version = "8.0.4" @@ -149,6 +164,12 @@ version = "0.4.32" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cc14f565cf027a105f7a44ccf9e5b424348421a1d8952a8fc9d499d313107789" +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + [[package]] name = "cpufeatures" version = "0.2.17" @@ -158,6 +179,15 @@ dependencies = [ "libc", ] +[[package]] +name = "cpufeatures" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566" +dependencies = [ + "libc", +] + [[package]] name = "crc32fast" version = "1.5.0" @@ -177,6 +207,15 @@ dependencies = [ "typenum", ] +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + [[package]] name = "csv" version = "1.4.0" @@ -239,8 +278,19 @@ version = "0.10.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ - "block-buffer", - "crypto-common", + "block-buffer 0.10.4", + "crypto-common 0.1.6", +] + +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer 0.12.1", + "const-oid", + "crypto-common 0.2.2", ] [[package]] @@ -265,6 +315,7 @@ dependencies = [ "async-compression", "async-stream", "async-trait", + "base16ct", "brotli", "brotli-decompressor", "bytes", @@ -285,7 +336,7 @@ dependencies = [ "serde_json", "serde_path_to_error", "serde_urlencoded", - "sha2", + "sha2 0.11.1", "simd-adler32", "thiserror", "toml", @@ -452,7 +503,7 @@ dependencies = [ "ipnet", "serde", "serde_json", - "sha2", + "sha2 0.10.9", ] [[package]] @@ -481,6 +532,15 @@ dependencies = [ "http", ] +[[package]] +name = "hybrid-array" +version = "0.4.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "27f864f10dfb56725ce5ce5472bc52252c8f93a4ab86327122cebf62c5f59a17" +dependencies = [ + "typenum", +] + [[package]] name = "icu_collections" version = "2.2.0" @@ -856,8 +916,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ "cfg-if", - "cpufeatures", - "digest", + "cpufeatures 0.2.17", + "digest 0.10.7", +] + +[[package]] +name = "sha2" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.1", + "digest 0.11.3", ] [[package]] @@ -1061,12 +1132,11 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" [[package]] name = "validator" -version = "0.20.0" +version = "0.21.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "43fb22e1a008ece370ce08a3e9e4447a910e92621bb49b85d6e48a45397e7cfa" +checksum = "c3d68c6633c483df6780cc5277a417c7c2d1bceee2649d06c8ab6b0fd2dd3c81" dependencies = [ "idna", - "once_cell", "regex", "serde", "serde_derive", From 96c21d731501370044f11ca5832e85701e79a421 Mon Sep 17 00:00:00 2001 From: Christian Date: Tue, 10 Nov 2026 14:08:47 -0600 Subject: [PATCH 3/9] feat(core): finalize rendered admitted errors with owned geo outcome --- crates/edgezero-core/src/app.rs | 288 +++++++++++++++++++++++++++++++- docs/guide/native-geo.md | 7 + 2 files changed, 294 insertions(+), 1 deletion(-) diff --git a/crates/edgezero-core/src/app.rs b/crates/edgezero-core/src/app.rs index a2ea3020..39a2aa67 100644 --- a/crates/edgezero-core/src/app.rs +++ b/crates/edgezero-core/src/app.rs @@ -38,6 +38,9 @@ const DEFAULT_APP_NAME: &str = "EdgeZero App"; pub const FASTLY_ADAPTER: &str = "fastly"; /// Canonical adapter name for the Spin adapter. pub const SPIN_ADAPTER: &str = "spin"; +type AdmittedErrorResponseFinalizer = + Arc; + enum ErrorResponseRenderer { Custom(Arc Response + Send + Sync + 'static>), Default, @@ -53,6 +56,7 @@ pub enum FrameworkErrorDetail { /// Lightweight container around a `RouterService` that can be extended via hook implementations. pub struct App { + admitted_error_response_finalizer: Option, config_extraction_limits: ConfigExtractionLimits, detached_response_egress_decision_factory: DetachedResponseEgressDecisionFactory, error_response_renderer: ErrorResponseRenderer, @@ -329,6 +333,7 @@ impl App { let request_start = prepared.request_start(); let route = prepared.route_metadata().cloned(); let (resolved, admitted, completion) = prepared.into_parts(); + let error_geo = geo.clone(); let (result, route_error, mut failure) = self .router .dispatch_resolved_outcome( @@ -356,7 +361,20 @@ impl App { } else { None }; - self.render_error_response_with_message(error, fixed) + let required_headers = self + .admitted_error_response_finalizer + .as_ref() + .and_then(|_| error.required_response_headers().ok()); + let mut response = self.render_error_response_with_message(error, fixed); + if let Some(finalizer) = &self.admitted_error_response_finalizer { + let status = response.status(); + finalizer(&error_geo, &mut response); + *response.status_mut() = status; + if let Some(headers) = required_headers { + response.headers_mut().extend(headers); + } + } + response } }; let envelope = self @@ -566,6 +584,24 @@ impl App { &self.router } + /// Installs a synchronous finalizer for rendered admitted dispatch errors only. + /// + /// The callback runs exactly once after the configured error renderer, with the same + /// immutable geographic outcome supplied to the admitted request context. It does not + /// resolve geography again. Use middleware to finalize successful dispatch responses, + /// including application-returned error statuses. Detached errors, pre-admission errors, + /// and admitted body-conversion errors without a resolved outcome bypass this hook. + /// + /// `EdgeZero` preserves failure classification and reapplies the rendered status and + /// mandatory protocol headers after the callback. With no finalizer, behavior is unchanged. + #[inline] + pub fn set_admitted_error_response_finalizer(&mut self, finalizer: Finalizer) + where + Finalizer: Fn(&GeoOutcome, &mut Response) + Send + Sync + 'static, + { + self.admitted_error_response_finalizer = Some(Arc::new(finalizer)); + } + /// Installs validated typed-config extraction limits. /// /// # Errors @@ -708,6 +744,7 @@ impl App { S: Into, { Self { + admitted_error_response_finalizer: None, config_extraction_limits: ConfigExtractionLimits::default(), detached_response_egress_decision_factory: Arc::new(|head| { DetachedResponseEgressDecision::Send { @@ -885,6 +922,7 @@ mod tests { use crate::config_store::ConfigExtractionLimits; use crate::context::RequestContext; use crate::error::{EdgeError, StoreExtractionReason}; + use crate::geo::{GeoInput, GeoProvider, GeoResolution}; use crate::http::{ HeaderMap, HeaderValue, Method, StatusCode, Version, request_builder, response_builder, }; @@ -932,6 +970,15 @@ mod tests { struct GrantDropProbe(Arc); + struct CountingGeoProvider(GeoResolution, Arc); + + impl GeoProvider for CountingGeoProvider { + fn resolve(&self, _input: &GeoInput) -> GeoResolution { + self.1.fetch_add(1, Ordering::SeqCst); + self.0.clone() + } + } + #[derive(Clone)] struct AppEgressObserver(Arc>>); @@ -2665,6 +2712,245 @@ mod tests { ); } + #[test] + fn admitted_error_finalizer_retains_geo_without_resolving_again() { + use crate::geo::{ + CountryCode, GeoAssertion, GeoLocation, GeoProviderId, GeoSource, GeoStatus, + GeoTrustBasis, GeoUnavailableReason, + }; + + for resolution in [ + GeoResolution::Located(GeoLocation::new(CountryCode::new("US").unwrap(), None)), + GeoResolution::MissingMetadata, + ] { + let provider_calls = Arc::new(AtomicUsize::new(0)); + let provider = CountingGeoProvider(resolution, Arc::clone(&provider_calls)); + let status = match provider.resolve(&GeoInput { + country: GeoAssertion::absent(), + subdivision: GeoAssertion::absent(), + }) { + GeoResolution::Located(location) => GeoStatus::Located { + location, + source: GeoSource::CloudFront, + trust_basis: GeoTrustBasis::DeclaredProtectedAlb, + }, + GeoResolution::MissingMetadata => { + GeoStatus::Unavailable(GeoUnavailableReason::MissingMetadata) + } + GeoResolution::Error(_) => panic!("unexpected provider error"), + }; + let geo = GeoOutcome::Selected { + provider: GeoProviderId::new("counted").unwrap(), + status, + }; + let handler_geo = geo.clone(); + let router = RouterService::builder() + .get("/fails", move |mut ctx: RequestContext| { + let expected = handler_geo.clone(); + async move { + assert_eq!(ctx.geo(), &expected); + ctx.headers_mut() + .insert("cloudfront-viewer-country", HeaderValue::from_static("DE")); + ctx.extensions_mut().insert(GeoOutcome::NoSelection); + assert_eq!(ctx.geo(), &expected); + Err::<&str, _>(EdgeError::config_out_of_date("private-sentinel", "field")) + } + }) + .build(); + let mut app = App::new(router); + let renderer_calls = Arc::new(AtomicUsize::new(0)); + let rendered = Arc::clone(&renderer_calls); + app.set_error_response_renderer(move |error| { + rendered.fetch_add(1, Ordering::SeqCst); + assert_eq!(error.message(), "private-sentinel"); + Response::new(Body::from("application-owned")) + }); + let finalizer_calls = Arc::new(AtomicUsize::new(0)); + let finalized = Arc::clone(&finalizer_calls); + let finalizer_geo = geo.clone(); + app.set_admitted_error_response_finalizer(move |outcome, response| { + finalized.fetch_add(1, Ordering::SeqCst); + assert_eq!(outcome, &finalizer_geo); + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); + assert_eq!(response.headers()["retry-after"], "60"); + assert_eq!(response.body().as_bytes().unwrap(), b"application-owned"); + response + .headers_mut() + .insert("x-geo-finalized", HeaderValue::from_static("yes")); + response.headers_mut().remove("retry-after"); + *response.status_mut() = StatusCode::OK; + }); + let request = request_builder().uri("/fails").body(Body::empty()).unwrap(); + let head = IngressHeadParts::from_request( + &request, + IngressHeadAccounting::HostManaged, + IngressFraming::HostManaged, + ); + let IngressBeginOutcome::Admitted(prepared) = + app.begin_ingress(head, app.monotonic_now()).unwrap() + else { + panic!("admission"); + }; + let envelope = block_on(app.dispatch_admitted_with_geo( + prepared, + request, + FrameworkErrorDetail::CategoryOnly, + geo, + )); + assert_eq!( + envelope.failure_classification(), + Some(ResponseEgressFailureClassification::error( + "config_out_of_date" + )) + ); + let response = complete_envelope(envelope); + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); + assert_eq!(response.headers()["retry-after"], "60"); + assert_eq!(response.headers()["x-geo-finalized"], "yes"); + assert_eq!(provider_calls.load(Ordering::SeqCst), 1); + assert_eq!(renderer_calls.load(Ordering::SeqCst), 1); + assert_eq!(finalizer_calls.load(Ordering::SeqCst), 1); + } + } + + #[test] + fn admitted_error_finalizer_preserves_category_only_routing_errors() { + let mut app = App::new( + RouterService::builder() + .get("/known", |_ctx: RequestContext| async { + Ok::<_, EdgeError>("ok") + }) + .build(), + ); + let calls = Arc::new(AtomicUsize::new(0)); + let observed = Arc::clone(&calls); + app.set_admitted_error_response_finalizer(move |geo, response| { + observed.fetch_add(1, Ordering::SeqCst); + assert_eq!(geo, &GeoOutcome::NoSelection); + assert!( + !str::from_utf8(response.body().as_bytes().unwrap()) + .unwrap() + .contains("sentinel") + ); + response + .headers_mut() + .insert("x-robots-tag", HeaderValue::from_static("noindex")); + response.headers_mut().remove("allow"); + *response.status_mut() = StatusCode::OK; + }); + for (method, path, status, message) in [ + ( + Method::GET, + "/private-sentinel", + StatusCode::NOT_FOUND, + "route not found", + ), + ( + Method::from_bytes(b"PRIVATE-SENTINEL").unwrap(), + "/known", + StatusCode::METHOD_NOT_ALLOWED, + "method not allowed", + ), + ] { + let envelope = managed_dispatch(&app, method, path, FrameworkErrorDetail::CategoryOnly); + assert_eq!( + envelope.failure_classification(), + Some(ResponseEgressFailureClassification::routing( + if status == StatusCode::NOT_FOUND { + "not_found" + } else { + "method_not_allowed" + } + )) + ); + let response = complete_envelope(envelope); + assert_eq!(response.status(), status); + if status == StatusCode::METHOD_NOT_ALLOWED { + assert_eq!(response.headers()["allow"], "GET"); + } + assert_eq!(response.headers()["x-robots-tag"], "noindex"); + let payload: serde_json::Value = + serde_json::from_slice(response.body().as_bytes().unwrap()).unwrap(); + assert_eq!(payload["error"]["message"], message); + } + assert_eq!(calls.load(Ordering::SeqCst), 2); + } + + #[test] + fn admitted_error_finalizer_bypasses_ok_and_errors_without_geo_outcomes() { + let mut app = App::new( + RouterService::builder() + .get("/{status}", |ctx: RequestContext| async move { + let status = match ctx.uri().path() { + "/401" => StatusCode::UNAUTHORIZED, + "/404" => StatusCode::NOT_FOUND, + _ => StatusCode::OK, + }; + Ok::<_, EdgeError>( + response_builder() + .status(status) + .body(Body::empty()) + .unwrap(), + ) + }) + .build(), + ); + app.set_admitted_error_response_finalizer(|_, _| panic!("must not finalize")); + for (path, status) in [ + ("/200", StatusCode::OK), + ("/401", StatusCode::UNAUTHORIZED), + ("/404", StatusCode::NOT_FOUND), + ] { + let envelope = + managed_dispatch(&app, Method::GET, path, FrameworkErrorDetail::CategoryOnly); + assert!(envelope.failure_classification().is_none()); + assert_eq!(complete_envelope(envelope).status(), status); + } + let detached = app.detached_ingress_error_egress( + EdgeError::uri_too_long("private"), + Method::GET, + app.monotonic_now(), + ); + assert_eq!( + complete_dispatch(detached).status(), + StatusCode::URI_TOO_LONG + ); + let head = IngressHeadParts::new( + Method::GET, + "/200".parse().unwrap(), + Version::HTTP_11, + HeaderMap::new(), + ); + let IngressBeginOutcome::Admitted(prepared) = + app.begin_ingress(head, app.monotonic_now()).unwrap() + else { + panic!("admission"); + }; + assert_eq!( + complete_envelope( + app.admitted_error_egress(prepared, EdgeError::bad_request("conversion")) + ) + .status(), + StatusCode::BAD_REQUEST + ); + app.set_ingress_admission_policy(|_| AdmissionDecision::Refuse { + completion: ResponseEgressCompletion::empty(), + response: response_builder() + .status(StatusCode::FORBIDDEN) + .body(Body::empty()) + .unwrap(), + }); + let request = request_builder().uri("/200").body(Body::empty()).unwrap(); + let refused = block_on(app.dispatch_ingress( + request, + app.monotonic_now(), + IngressHeadAccounting::HostManaged, + IngressFraming::HostManaged, + )) + .unwrap(); + assert_eq!(complete_dispatch(refused).status(), StatusCode::FORBIDDEN); + } + #[test] fn geo_legacy_dispatch_ignores_forged_extensions() { use crate::geo::{GeoError, GeoOutcome, GeoProviderId, GeoStatus}; diff --git a/docs/guide/native-geo.md b/docs/guide/native-geo.md index 7a1b1f05..bb05cabd 100644 --- a/docs/guide/native-geo.md +++ b/docs/guide/native-geo.md @@ -200,6 +200,13 @@ The unmanaged request converter, legacy dispatch, Fastly, Cloudflare, and Spin e `NoSelection`. Returning a context to an ordinary request and constructing a new unmanaged context does not preserve its private Geo result. +For application-owned response headers, use middleware on `Ok(Response)` and install +`App::set_admitted_error_response_finalizer` for admitted dispatch errors. This optional +synchronous callback receives `&GeoOutcome` and `&mut Response` after the existing error +renderer, without resolving the provider again. EdgeZero preserves failure classification, +rendered status, and required protocol headers. Detached and pre-admission errors have no +resolved outcome and bypass the callback. + ## CloudFront assertion rules CloudFront must generate both `CloudFront-Viewer-Country` and From 72d04aa4ccc5c531b42b57a36f95f9d770ffc01c Mon Sep 17 00:00:00 2001 From: Christian Date: Fri, 13 Nov 2026 17:44:09 -0600 Subject: [PATCH 4/9] spec and plan --- .../plans/2026-11-13-kv-record-contract.md | 288 +++++++++++++++++ .../2026-11-13-kv-record-contract-design.md | 289 ++++++++++++++++++ 2 files changed, 577 insertions(+) create mode 100644 docs/superpowers/plans/2026-11-13-kv-record-contract.md create mode 100644 docs/superpowers/specs/2026-11-13-kv-record-contract-design.md diff --git a/docs/superpowers/plans/2026-11-13-kv-record-contract.md b/docs/superpowers/plans/2026-11-13-kv-record-contract.md new file mode 100644 index 00000000..e6014311 --- /dev/null +++ b/docs/superpowers/plans/2026-11-13-kv-record-contract.md @@ -0,0 +1,288 @@ +# Portable KV records implementation plan + +Status: implementation authorized and started on 2026-11-13. The user reviewed the [spec](../specs/2026-11-13-kv-record-contract-design.md) and requested this execution checklist. Focused core tests and the shared suite on real redb and Fastly/Viceroy have passed. DynamoDB SDK-wire and companion celld service tests have passed. Combined construction, DynamoDB Local, the EdgeZero celld provider and lint gates remain incomplete. + +Worktree: `/home/pav/worktrees/feat-kv-record-contract`, branch `feat/kv-record-contract`, initially based on #439 at `645703bf39f5965dad5b54d7907e28c9197a7034`. + +This file owns EdgeZero task-level progress and checks. The Obsidian plan at `Projects/Trusted Server/plans/Implement the Record Contract and Reconcile Provider Adoption.md` owns celld source/artifact coordination, TS #1536/#1537 reconciliation, publication and stack disposition. The spec owns behavior. Keep those responsibilities separate rather than maintain competing copies of this checklist. + +## Delivery contract + +- One EdgeZero PR closes #472 and #473, after #439 and before #454. Use forward merges; #495 remains outside the official stack. +- Implement the same public record contract on Fastly KV, PersistentKvStore, DynamoDB on Axum and celld on Axum. +- Install application-owned grouping before provider preparation. Ordinary and record operations share the same value and invalidate generations consistently. +- Preserve ordinary global listing without ordering guarantees. DynamoDB uses Scan for that operation and strong Query for explicit partition listing. +- Include complete pagination, text metadata with empty/absent normalization, native TTL behavior and explicit uncertain mutation outcomes without hidden replay. +- Reuse DynamoDB Local and local celld infrastructure. No Floci, hosted AWS/IAM qualification, migration implementation, deployment or main merge. + +## Execution order + +```mermaid +flowchart TD + A[0. Freeze baseline] --> B[1. Core types and conformance tests] + B --> C[2. Construction-time layout wiring] + C --> D[3. Persistent transactions] + C --> E[4. Fastly native operations] + C --> F[5. DynamoDB record writes and Query] + F --> G[6. Global Scan enumeration] + C --> H[7. Native feature and binding selection] + I[Companion celld protocol and rebuilt artifact] --> J[8. celld record binding] + H --> J + D --> K[9. Shared real-path acceptance] + E --> K + G --> K + J --> K + K --> L[10. Stable checks and coordination handoff] +``` + +Complete one behavior and its focused proof before building on it. After code changes run the affected `cargo test` selection. At a stable combined checkpoint run the broader gates below. Independent investigations may overlap; give each edited file, build directory and mutable service fixture one command owner. + +## 0. Freeze the baseline + +- [x] Create the isolated worktree from the recorded #439 head without changing unrelated main-checkout work. +- [x] Record the spec decisions, including unordered global Scan and strong partition Query. +- [x] Rechecked #439 ownership and head before edits: ChristianPavilonis, unchanged `645703bf`. Its ancestry includes recorded #422 `4ea31995`. No parent merge was needed. +- [x] Recorded clean code baseline plus the two untracked planning documents. Explicit Rust 1.98.1 is installed; shell default 1.93.1 is not used. `cargo +1.98.1 test --locked --offline -p edgezero-core` passed before source edits. +- [x] Confirm the reviewed spec's concrete choices: construction-time layouts, retained per-key u64 generation allocation, ordinary replacement clearing metadata and explicit celld no-expiry format. + +Exit: exact starting source and chosen design recorded. No repair to an unrelated prerequisite is hidden in this feature. + +## 1. Define the portable record contract and shared tests + +Files: + +- New `crates/edgezero-core/src/record_store.rs`, or an equivalently focused module. +- `crates/edgezero-core/src/key_value_store.rs`. +- `crates/edgezero-core/src/lib.rs` and focused public-API tests under `crates/edgezero-core/tests/`. + +Work: + +- [x] Add the record value, equality-only generation marker, write modes, acknowledged outcomes, page and validated partition types from the spec. +- [x] Add object-safe record methods/capability to `KvStore`, with Unsupported defaults. Keep ordinary-only implementations and test doubles compiling. +- [x] Add `KvHandle::require_records()` as a non-mutating view over that exact provider. Do not add a second provider registry or independently routed record wrapper. +- [x] Add payload-free uncertain-mutation errors. Keep definitive precondition rejection separate from transport/protocol failures and unconditional overwrite errors. +- [x] Implement text metadata normalization/validation, current key/value/page limits and query-bound continuation. Do not add binary metadata or an expiry envelope. +- [x] Update ordinary listing documentation so it does not promise alphabetical ordering on the new partitioned binding. Preserve existing ordinary-only behavior. +- [x] Add one reusable public-handle conformance suite. Keep provider setup/runtime differences in fixture factories, not in duplicated assertions. + +Tests to add: unsupported acquisition without I/O, exact body/text fidelity, empty/absent metadata, Add/CAS outcomes, same-row ordinary/record operations, replacement generation invalidation, no-expiry, continuation completeness and uncertainty classification. Provider-specific success is established in later tasks, not by an in-memory double alone. + +Check: `cargo test -p edgezero-core`. Add a public consumer compilation test for the final signatures. New test/type names are planned work, not existing APIs. + +## 2. Install layouts before stores are prepared + +Files: + +- `crates/edgezero-core/src/app.rs` and `store_registry.rs`. +- `crates/edgezero-adapter-axum/src/dev_server.rs` and `native_stores.rs`. +- Other adapters' existing registry construction paths as needed for portable declarations and unsupported capability handling. + +Work: + +- [x] Register immutable per-ID record requirements through App configure. Resolve full keys to stable enumerable prefixes without embedding TS key meanings in core. +- [ ] Validate declarations after App construction and before `build_production_stores`. Initializer acquisition must not change routing after handles exist. +- [ ] Thread the same declaration through production, development and provider constructors. Ready/deferred handles must already match the required layout/mode. +- [ ] Handle IDs that resolve to a shared backing name consistently; reject conflicting declarations before publishing stores. +- [ ] Keep ordinary arbitrary-prefix/global listing distinct from complete-partition validation. An empty global prefix must not fail merely because it is not a partition. +- [ ] Preserve existing source-selection validation, preparation ordering, immutable snapshots and no-fallback behavior. Do not rewrite the hosting lifecycle. + +Checks: core layout tests and focused Axum startup tests prove configure-before-prepare, acquisition without mutation, mismatched supplied handles, shared-name declarations, ordinary-only compatibility and unsupported providers. Run `cargo test -p edgezero-adapter-axum` with the selected relevant features after wiring. + +## 3. Implement transactional persistent records + +File: `crates/edgezero-adapter-axum/src/key_value_store.rs`, with colocated tests and existing adapter contract coverage. + +- [x] Add an explicit record-capable redb format; leave old ordinary-only files readable by their existing path. +- [x] Store body, metadata, generation and optional expiry together. Evaluate Add/CAS and write the new row inside one write transaction. +- [x] Keep durable generation history outside value expiry/deletion. Reopen/restart must not reset it or allow an old token to match a recreated key. +- [x] Route ordinary get/put/TTL-put/delete through the same record rows. Ordinary replacement clears metadata, changes generation and applies the requested expiry or no expiry. +- [x] Preserve the expiry recheck before lazy cleanup so cleanup cannot erase a concurrent fresh overwrite. +- [x] Implement strong partition pages with a read view and bounded scan batches. Preserve continuation when scanned expired rows or scan limits produce short/empty pages. + +Check: `cargo test -p edgezero-adapter-axum --lib key_value_store` plus the common record suite against a real temporary database. Include reopen, delete/recreate, cleanup races and ordinary/record coherence. + +## 4. Map native Fastly record operations + +Files: `crates/edgezero-adapter-fastly/src/key_value_store.rs`, `tests/contract.rs` and the existing Fastly integration fixture paths. + +- [x] Use `current_generation()`, native Add and `if_generation_match`. Never use the deprecated generation accessor or get-then-put CAS. +- [ ] Normalize native metadata into optional UTF-8 text and preserve native TTL behavior. Do not return a fabricated post-write generation. +- [ ] Implement authoritative exact existence with Strong prefix listing and exact key equality. Follow continuation; descendants and truncated pages cannot prove presence/absence. +- [ ] Preserve native Strong list cursors for partition pages. Keep groups as prefixes in the existing physical store. +- [ ] Map only definitive Add/CAS rejection to normal contention. Surface uncertain mutation completion without library replay. + +Check: add and execute the common suite under Viceroy using `scripts/test_integration.sh --adapter fastly --suite ci` with a nonzero selector for the new cases. SDK compilation/native Rust tests do not replace runtime proof. Keep current runtime/SDK pins unless an independently reviewed prerequisite requires change. + +## 5. Implement partitioned DynamoDB record writes and Query + +Files: + +- `crates/edgezero-store-aws/src/dynamodb_kv.rs`, `dynamodb_tests.rs`, settings and preparation code. +- Add focused record implementation modules rather than overloading unrelated snapshot code. + +Work: + +- [ ] Add explicit record-format preparation using existing native SDK clients, deadlines, table-key validation and redacted error handling. +- [ ] Encode format/namespace/group into distinct PK values. Separate value and per-key allocator SK kinds without changing the full logical key returned to callers. +- [ ] Allocate u64 markers atomically per key, then use native conditional PutItem for Add/CAS or unconditional replacement. No namespace-wide item is mutated by every insert. +- [ ] Retain allocator state across value delete/TTL; exclude it from user reads/listing. Prevent wrap/reset. Gaps and out-of-allocation-order unconditional commits are valid. +- [ ] Apply one absolute operation budget across allocation and value write. Preserve single-attempt SDK mutations. An allocator-only interruption must not be confused with an acknowledged value write. +- [ ] Implement authoritative strong GetItem and strong base-table Query for a declared partition. Carry LastEvaluatedKey through internal/filtered rows, even for an empty returned page. +- [ ] Keep ordinary point operations coherent with record rows. Use native TTL on the explicit new format and preserve the legacy ordinary binding's existing format/expiry behavior. + +Tests: different groups produce different actual PKs; no global allocator traffic; independent Add/CAS races; stale-token rejection after overwrite/delete/expiry/restart; lost acknowledgement; allocation-only failure; exhaustion; byte/text limits; Query continuation and namespace isolation. + +Check: extend existing SDK loopback fixtures and DynamoDB Local proof. The current `cargo test -p edgezero-store-aws --no-default-features --features dynamodb-kv` is an existing entry point; add/select the record feature before claiming record coverage. + +## 6. Retain unordered global enumeration with Scan + +Files: DynamoDB record provider, cursor encoding/validation, provider wire tests and local generated-consumer proof. + +- [ ] Implement ordinary `list_keys_page` across all groups and arbitrary/empty prefixes with native paginated Scan. Do not reject cross-group requests. +- [ ] Filter by the binding's versioned namespace, value-row kind and requested logical prefix. Return logical keys only, never allocator rows or foreign namespace values. +- [ ] Preserve native LastEvaluatedKey. The Scan limit counts evaluated rows, so short/empty output with continuation is valid. +- [ ] Allow a native continuation position that belongs to a filtered physical row. Validate cursor structure and bind the outer token to operation, binding, layout/format and requested prefix; do not treat the native position as a returned value. +- [ ] Reject Scan/Query cursor interchange. Keep strong partition listing and authoritative exact existence off this global Scan path. +- [ ] Do not sort all groups, rescan the complete namespace per output page or create a global index/catalog. Document broad read cost and the required Scan operation without adding hosted IAM acceptance. + +Check: local DynamoDB traversal over multiple groups/namespaces with allocator-only pages, empty prefixes, nonmatching prefixes, exact page multiples and live changes. In a quiescent dataset compare complete key sets and duplicate counts, not order. Verify page-sized work uses native continuation rather than restarting at the beginning. + +## 7. Add explicit native binding and feature selection + +Files: + +- `crates/edgezero-adapter-axum/src/native_bindings.rs`, `native_stores.rs`, `native_build.rs` and `Cargo.toml`. +- `crates/edgezero-store-aws/Cargo.toml`, `crates/edgezero-store-celld/Cargo.toml`. +- `crates/edgezero-cli/src/generator.rs` and generated consumers. +- `scripts/check_native_store_dependency_contract.sh`. + +Work: + +- [ ] Add strict, explicit record-mode selection and constructor arguments. Do not silently convert a raw KV binding or fall back to its old format. +- [ ] Add default-off record feature forwarding. Proposed names are `dynamodb-records` in the AWS crate, `aws-dynamodb-records` and `celld-records` in Axum; verify the final names and update commands/tests together. +- [ ] Preserve settings-only/default-off builds and current AWS/celld config/secret independence. Do not add native dependencies to core or WASM graphs. +- [ ] Extend generated consumer templates, strict validation and negative dependency tests. Record capability does not mean offline CLI can certify runtime permissions/connectivity. + +Checks: selected feature tests, generated Cargo checks, startup suites and dependency graph script, independently disabled and enabled. Broad all-features success alone is insufficient. + +## 8. Integrate the matching celld record service + +EdgeZero files: `crates/edgezero-store-celld/src/records.rs`, `client.rs`, settings/preparation, `tests/records_interop.rs`, native binding wiring and generated proof. + +The service change is mandatory companion work, not another EdgeZero PR or a deferred follow-up. Its implementation/publication is coordinated in Obsidian Phase 4. The available old service has neither record continuation nor ordinary no-expiry record writes. + +- [ ] Agree and implement the matching versioned protocol for record pagination and no-expiry. Coordinate server `scoped_record.rs`, service `kv.js` and required client/CLI paths in the celld repository; preserve full extension source, not just a pagination patch against an unpublished base. +- [ ] Prepare the EdgeZero binding with a read-only capability/version check for the required protocol. Old services must fail preparation, not silently downgrade. +- [ ] Route all ordinary methods on this binding through the record dataset, never `/value` raw KV. Keep raw-only bindings unchanged. +- [ ] Support actual non-expiring records, not a fabricated long TTL. Apply the reviewed fresh-format/no-migration approach consistently to read, write, Add/CAS, list, sweep and alarm behavior. +- [ ] Implement record keyset continuation, scope/prefix/mode-bound cursors, end detection and text metadata normalization. +- [ ] Rebuild the matching service from exact source/lockfile and record source/tree/toolchain/build command/binary hash through the coordination plan. Do not use the cached old binary as new-protocol evidence. + +Checks: wire fixtures for old/malformed protocols, uncertainty and cursor validation; then common public-handle tests against the actual rebuilt local service. Prove same-row ordinary access, metadata clearing, no-expiry across restart/sweep and complete pagination. No claim of fleet/rolling-upgrade or production isolation qualification. + +## 9. Exercise generated consumers and all four providers + +Existing files to extend: + +- `crates/edgezero-cli/tests/dynamodb_generated_proof.rs` and `native_aws_emulator_proof.rs`. +- `crates/edgezero-cli/tests/celld_generated_proof.rs`. +- `crates/edgezero-adapter-axum/tests/native_aws_startup.rs` and `native_celld_startup.rs`. +- `scripts/run_native_aws_integration.mjs`, `run_native_celld_integration.mjs` and their existing Node tests. + +- [ ] Run the same semantic assertions on Fastly, persistent redb, DynamoDB Local and real local celld. Independent writers must share the backing store; a client mutex is not an atomicity proof. +- [ ] Require nonzero execution of selected generated/service tests. Missing services or ignored tests cannot count as passes. +- [ ] Exercise same-row get/write/delete, ordinary TTL/no-TTL replacement, metadata, stale tokens and reopen/recreate behavior on actual providers. +- [ ] Fault-inject a lost response after value commit and prove no hidden replay. For the two-stage DynamoDB design, distinguish one allocation plus one value mutation from retrying either request. +- [ ] Exercise page continuation and live changes without asserting a frozen snapshot or alphabetical order. Include separate strong Query and ordinary global Scan cases. +- [ ] Retain the exact rebuilt celld artifact and source identity for TS adoption. Record local DynamoDB image/configuration and dummy endpoint/credential use. + +Existing runner entry points, to be extended before they count as new record acceptance: + +```sh +node --test scripts/run_native_aws_integration.test.mjs scripts/run_native_celld_integration.test.mjs +node scripts/run_native_aws_integration.mjs +CELLD_BIN=/absolute/path/to/rebuilt/celld node scripts/run_native_celld_integration.mjs +scripts/test_integration.sh --adapter fastly --suite ci +``` + +## 10. Stable checks and handoff + +- [ ] Run focused tests after each code change and the full required gates on a stable combined state. +- [ ] Run default-off/enabled feature graphs and target-matched SDK/runtime checks. Add the final record feature selections to the command record. +- [ ] Review the entire feature diff against the refreshed #439 parent. No #454 prototype, unrelated prerequisite repair or TS business policy belongs in this PR. +- [ ] Document record binding selection, native TTL/metadata limits, generation allocation cost, Scan versus Query, protocol/artifact requirements and migration exclusions. +- [ ] Give the Obsidian coordination plan an exact verified EdgeZero source/dependency/service checkpoint for TS adoption. Do not claim runtime readiness from the spec or historical receipts. +- [ ] Use the coordination plan for owned TS PR reconciliation, preserving old public heads, forward merge into #454 and official stack recreation. Do not execute publication, deployment or main merges from this checklist alone. + +Required stable EdgeZero commands: + +```sh +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings +cargo test --workspace --all-targets +cargo check --workspace --all-targets --features 'fastly cloudflare spin' +cargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin +``` + +Also run affected docs, demo/generated-consumer and other required WASM checks from current repository instructions. Internal superpowers documents are excluded by the existing docs Prettier configuration; do not present an ignored file as a formatting pass. + +## Requirement coverage + +| Spec requirements | Execution tasks | +| --- | --- | +| R1, R4, R7, R9 | 1, then provider implementations and 9 | +| R2, R11 | 2 and 7, plus startup acceptance in 9 | +| R3, R10 | 3, 4, 5 and 8 | +| R5 | 9 and 10 | +| R6 | 5, with stored-key/traffic assertions in 9 | +| R8 | 3, 4, 5 and 8, then 9 | +| R12 | 7 and 10 | +| R13 | 6 and global Scan acceptance in 9 | + +## Progress and evidence + +Implementation is in progress. Checkmarks record implemented behaviors with focused evidence, not a completed provider/startup qualification. Broad lint, generated-consumer and real-service gates remain separate. + +For each completed behavior, record the exact source/dependency/service inputs, command, result, elapsed time and log. Reuse results only while relevant inputs remain unchanged. Report baseline failures separately and do not relax bounds to turn them green. + +| Checkpoint | State | Evidence | +| --- | --- | --- | +| Core contract and layout | Functional checks passed; lint pending | `record_api`: 12 passed; `record_configuration`: 4 passed. Full core tests reached the subsequent Clippy command with task-local TMPDIR. Clippy failed on new-code conventions and ordinary-only default-method lint sites. | +| Persistent records | Focused checks and lib lint passed | 36 selected KV tests passed, including 9 record cases, the public shared suite, independent Add/CAS writers, delete/reopen, missing allocator state, delayed cleanup and expiry/recreation. A reproduced strong-existence expiry mismatch was fixed. Axum lib Clippy passed after source organization/style fixes. | +| Fastly records | Basic real-path suite passed; driver/edge cases pending | Release WASM fixture built; explicit repaired Viceroy 0.21.0 returned HTTP 200 and `executed_suites=1` after the public shared assertions. No native race/restart/ack-loss claim. | +| DynamoDB writes/Query/global Scan | SDK-wire and Local checks passed; lint/generated proof pending | Updated crate: 66 passed after endpoint/ARN/table identity correction and 3 scope regressions. Both explicitly selected Local tests passed against the existing pinned DynamoDB Local 3.3.1 image, including real group/allocator rows, shared suite, independent clients, complete Query/Scan and simulated physical expiry removal/reopen. | +| Feature/binding selection | Default-native checks passed; broader selections pending | Default Axum `record_`: 13 passed, `native_`: 63 passed. Explicit mode/layout/shared-name/ready/deferred checks executed. Platform/CLI/template and selected native-feature checks are still being collected. | +| Matching celld protocol and provider | Service tested; EdgeZero client in progress | 12 HTTP/protocol cases and 4 new real-process v2 cases passed. Startup neighbors: 10 default cases plus both explicitly selected TLS/60-second cases passed. EdgeZero records-v2 wire unit: 8 passed; real client/service interop remains pending. | +| Four-provider real-path conformance | Basic shared suite passed on all four; combined acceptance pending | Real redb, Fastly/Viceroy, DynamoDB Local and the EdgeZero client against rebuilt celld have executed the shared assertions. Celld additionally passed independent native races/global listing and committed PUT/DELETE dropped-reply proof. New automatic Fastly race/continuation/CI, generated consumers, broad gates and review remain pending. | +| Stable integration and coordination handoff | Pending | No TS adoption or public mutation yet. | + +Checkpoint logs and inputs live under `/home/pav/.cache/edgezero-kv-record-contract/`: + +- `core-redb-expanded.log`: API/configuration and persistent focused commands; builds 1.89s and 12.57s, persistent tests 0.24s. `first-record-checkpoints.sha256` records relevant inputs before the later redb cleanup/expiry test additions. +- `core-neighbors-lint.log`: first broad core attempt, 11 existing temporary-file writes failed with EDQUOT on `/tmp`. No application limit or test was weakened. Scoped TMPDIR rerun proceeded to Clippy; `core-neighbors-lint-disk-tmp.log` retains its lint diagnostics. Subsequent checks use the task-local disk-backed `tmp/` directory. +- `dynamodb-records-unit.log`: 63 SDK-wire/settings cases passed, build 27.75s and tests 3.07s. A neighboring ordinary-only feature run, crate lint and real DynamoDB Local proof are still required. +- `fastly-fixture-build.log` and `fastly-runtime-receipt.json`: build 25.35s; one real shared suite passed. Guest SHA-256 `e5fc4beb55042df62504d60e76db8475bf9429de9d1bd931e2fae90635128ddd`; selected runtime SHA-256 `c846e8db180beeb1477fb504ec1d89c09251203633cd98c7fd9d97abbd7b9755` at `/home/pav/.cache/rc-refresh-20261110/target-viceroy-repair/debug/viceroy`. The global 0.17.0 binary was not used. The earlier native Fastly lib-test attempt failed on unresolved SDK host imports and is not a test pass or established baseline regression. +- `celld-protocol-unit.log`, `celld-v2-runtime.log` and `celld-startup-neighbors.log`: 12 HTTP cases, 4 v2 real-process cases and 10 nonignored startup neighbors passed. Builds took 77s and 54.23s; v2 runtime tests took 0.77s. Both explicit ignored neighbors passed in `celld-startup-ignored.log`, 2 cases in 61.20s. +- `artifacts/celld-v2-f345ffd84455`: retained rebuilt service, SHA-256 `f345ffd844551c6dd3780cd2b8a5b9d2c0dc5ecd33522e17edea7ca5d2c53e39`, from companion base `9350a043860c4f41dccb692ec9de98ebd9dac3ae` plus `artifacts/celld-v2-source.patch`, SHA-256 `0427280d95893fb45aa28d085a42e75ab06e1de39b40cc676411234a11494980`. Source fingerprints are in `artifacts/celld-v2-source.sha256`; Cargo.lock SHA-256 is `d6a17422788aa910ba76bed9cf59ac15767f93fe107ebdb336954d0e9efe052b`. Rust 1.98.1 built the binary through the focused service integration command. Full source publication must include the base extension, not only this v2 delta. + +Additional current receipts: + +- `redb-native-check.log`: 36 persistent, 13 record-filtered and 63 native-filtered cases passed. `redb-expiry-red2.log` is the intentional failing expiry-consistency regression before the fix. `redb-style-tests.log` passed 36 cases again after moving the private implementation to `src/key_value_store_records.rs`; `axum-clippy-recheck.log` passed lib Clippy. +- `core-tests-recheck.log`: all core targets passed, including 1147 unit, 12 API, 4 configuration and 15 doc tests. `core-clippy-recheck.log` reduced new-code lint failures from 148 to 10; those last fixes and the generic pagination test-loop correction remain pending. +- `fastly-wasi-unit.log`: 9 provider tests passed on wasm32-wasip1 with the explicit repaired runtime, including 6 new cases. `fastly-harness-unit.log`: 4 nonzero/readiness/evidence validator tests passed. Expanded native races/1001-descendant proof still needs its new runtime receipt. +- `dynamodb-scope-test.log`: 66 SDK-wire/settings cases passed. `dynamodb-local-records.log`: 2 explicitly ignored Local cases and their isolated exact-test children passed, build 3.40s and execution 1.26s. The expiry case labels manual physical value-row deletion as simulation, not a Local/native AWS TTL sweep. `dynamodb-local-container.json` records the task-owned loopback container and pinned image digest. +- `celld-client-unit.log`: 8 wire cases passed. `celld-client-interop.log`: 8 current wire cases and 1 explicitly selected real-service case passed, the latter in 0.26s. The service used the retained f345 artifact and a fresh writable `identity` alias; both parent and worker temporary listeners were subsequently stopped. +- `record-runner-node-tests.log`: 18 tests passed, including mandatory old/new generated sentinels, both Local provider cases, zero-test rejection, interruption cleanup and credential/trust-input isolation. Existing runners are being extended, not replaced with another emulator framework. +- Public `docs/guide/kv-records.md` and links now document the actual API, explicit formats, native TTL/metadata, generation cost and Scan/Query distinction. Prettier 3.9.10 check passed for the five touched published-doc files. + +### Open dependency baseline question + +The added absolute no-Tokio check found that pinned Cloudflare `worker =0.8.5` already brings `tokio` into its WASM graph. Root Cargo/lock, core Cargo and Cloudflare Cargo inputs are unchanged from #439. The inverse graph is `tokio -> worker -> edgezero-adapter-cloudflare`; `cloudflare-sdk-tokio-baseline.log` records selected features. The user was asked whether this pre-existing SDK dependency is outside the record work or a separate prerequisite. No SDK pin, global lint or dependency policy has been relaxed. The stricter graph gate remains incomplete while that scope question is open. + +## References + +- [Reviewed spec](../specs/2026-11-13-kv-record-contract-design.md). +- [EdgeZero #472](https://github.com/IABTechLab/edgezero/issues/472) and [#473](https://github.com/IABTechLab/edgezero/issues/473). +- [Native AWS foundation plan](2026-10-07-native-store-providers.md) and [celld foundation plan](2026-10-15-native-celld-store-providers.md), historical receipts rather than new record proof. +- Obsidian coordination plan: `Projects/Trusted Server/plans/Implement the Record Contract and Reconcile Provider Adoption.md`. +- Obsidian decisions: `Projects/Trusted Server/notes/Record Contract and Provider Reconciliation Decisions.md`. diff --git a/docs/superpowers/specs/2026-11-13-kv-record-contract-design.md b/docs/superpowers/specs/2026-11-13-kv-record-contract-design.md new file mode 100644 index 00000000..494f5fb6 --- /dev/null +++ b/docs/superpowers/specs/2026-11-13-kv-record-contract-design.md @@ -0,0 +1,289 @@ +# Portable KV records and native providers + +Status: draft for review. Requirements were agreed on 2026-11-13; implementation and runtime acceptance have not started. Concrete type names and the technical choices called out below remain subject to spec review. + +Delivery: one EdgeZero PR closes [#472](https://github.com/IABTechLab/edgezero/issues/472) and [#473](https://github.com/IABTechLab/edgezero/issues/473), stacked after [#439](https://github.com/IABTechLab/edgezero/pull/439) and before [#454](https://github.com/IABTechLab/edgezero/pull/454). Planning baseline is `645703bf39f5965dad5b54d7907e28c9197a7034`, including #414 and #422. Recheck heads before implementation or publication. + +The [EdgeZero implementation plan](../plans/2026-11-13-kv-record-contract.md) owns repository-level execution tasks and checks. The Obsidian plan at `Projects/Trusted Server/plans/Implement the Record Contract and Reconcile Provider Adoption.md` owns cross-repository celld/TS coordination, dependencies, publication and stack disposition. The user requested the repository plan after reviewing this spec, superseding the earlier single-plan restriction. This spec remains the behavior contract. The interview record is `Projects/Trusted Server/notes/Record Contract and Provider Reconciliation Decisions.md`. + +## Goal + +Let an application require one record-capable KV binding and use the same code on native Fastly KV, Axum's local persistent store, DynamoDB on Axum, or celld on Axum. EdgeZero owns provider mechanics. The application owns key grouping, content validation, consent rules and operation-specific conflict handling. + +A record binding guarantees generation-bearing reads, atomic conditional writes, authoritative exact-key existence and strongly consistent partition-local pagination. It must not simulate atomicity with a read followed by an unconditional put or a process-local lock. + +Trusted Server is the first consumer. Its shared identity graph already depends on these operations. This spec does not move identity policy into EdgeZero or implement the whole server-side identity roadmap. + +## Scope and requirements + +| ID | Requirement | +| --- | --- | +| R1 | Add record operations to the existing `KvStore` contract and a validated `RecordHandle` over the same provider and logical store. Preserve ordinary-only bindings. | +| R2 | Declare an immutable application-owned key layout once before provider preparation. Preserve full logical keys. Resolve point operations through that layout and validate complete partition prefixes for record listing. | +| R3 | Within a record binding, ordinary and record operations reach the same value. Every ordinary overwrite invalidates previous generations. Ordinary no-TTL put remains non-expiring. | +| R4 | Supply generation-bearing lookup, Add, Overwrite, generation-match write, definitive precondition-failed outcome, strong exact existence, delete and resumable partition listing. | +| R5 | Implement and run one contract suite on Fastly under Viceroy, `PersistentKvStore`, DynamoDB Local and a rebuilt matching celld service. | +| R6 | DynamoDB uses distinct namespace-qualified partition-key values for distinct declared partitions. No namespace-wide item is updated on every record insert. | +| R7 | Metadata is optional UTF-8 text. Missing and empty are equivalent; preserve nonempty text exactly. Bodies remain arbitrary bytes. | +| R8 | Every record partition-list page uses the promised strong partition query. Pagination is live, not a frozen cross-page snapshot. Only absence of continuation establishes traversal completion. | +| R9 | Report ambiguous mutation completion explicitly and do not automatically replay the mutation. The application owns reconciliation and conflict policy. | +| R10 | Use documented provider TTL behavior. Do not add a common exact-deadline read/list guarantee or an expiry envelope. Expiry and physical cleanup are separate. | +| R11 | Reject unsupported record bindings during initialization without a mutating probe. Missing capability, invalid layout, unavailable provider and runtime write failure are not interchangeable. | +| R12 | Keep native SDK/runtime dependencies optional and out of core/WASM graphs. Extend existing DynamoDB Local and celld infrastructure rather than add Floci or another emulator framework. | +| R13 | Retain ordinary arbitrary-prefix/global listing on the partitioned DynamoDB binding through paginated Scan, without an ordering guarantee. It is separate from strong partition listing. | + +Out of scope: hosted AWS/IAM qualification, data migration implementation, production load certification, deployment, main merges, after-response work, new identity-provider policy, Cloudflare/Spin record implementations, and DynamoDB/celld execution on WASM. Existing unrelated acceptance obligations are not completed by this work. + +## Existing gaps + +- `edgezero-core/src/key_value_store.rs` has ordinary KV operations but no portable record contract. The default `exists` is a read, not an authoritative existence promise. +- Fastly already supplies generations, Add, conditional insert and strong list; the EdgeZero wrapper drops those distinctions. +- `PersistentKvStore` stores body and expiry only. It needs transactional metadata/generation state without breaking ordinary-only files. +- #414's DynamoDB provider uses `pk = namespace`, `sk = key`. It supplies strong reads/query but not conditional records or distributed logical partitions. +- #422's celld record client is separate from raw KV. The service stores raw values in `__kv` and conditional records in `__kv_records`. Record listing has no continuation, and every record write currently requires a positive TTL. +- TS has provider mechanics in its Fastly adapter and in the older Axum DynamoDB/celld line. The target is one shared TS `EcKvStore` over this contract, not another adapter implementation. + +## Boundaries and construction + +```mermaid +flowchart TD + A[Application configure declares per-ID layouts] --> B[Validate source selection and layout] + B --> C[Prepare one provider per logical binding] + C --> D[Publish existing KvRegistry and handles] + D --> E[Initializer requires record capability] + E --> F[Shared application record operations] + F --> G[Fastly native KV] + F --> H[Persistent redb] + F --> I[DynamoDB namespace and partition] + F --> J[celld record dataset] +``` + +Keep `BoundKvStore = KvHandle` and the existing per-ID registry. Do not introduce another provider registry or independently routed record dataset behind the same binding. + +Register layout requirements through the portable application's existing configure path. Axum currently builds/configures the App before `build_production_stores`; pass the declarations into that preparation. The initializer runs after stores are prepared, so it may validate/acquire a handle but must not install or change routing. Thread the same declarations through local development and platform registry construction. + +Supplied ready/deferred handles must already satisfy the requested layout and record mode. Reject mismatches before publishing stores. Existing local preparation deduplicates by backing-store name; two IDs that resolve to the same store cannot silently use conflicting layouts. + +Capability checking establishes supported operations and applicable local/protocol compatibility, not hosted write authorization or permanent availability. Fastly's request initialization may perform platform binding checks; do not invent a process-wide startup phase it does not have. + +## Public API direction + +Names below are a proposed spelling of the agreed API, not existing Rust definitions. Place portable record types in a focused core module and add object-safe record operations to the existing provider trait. Unsupported defaults must return `KvError::Unsupported`, never fabricate success. Preserve the existing local-future model and `Send + Sync` provider boundary. + +```rust +// Portable configure phase, before provider preparation. +app.require_record_store("identity", TsIdentityLayout)?; + +// Initializer: no layout argument and no routing mutation. +let identity = stores.kv("identity")?.require_records()?; + +let record = identity.lookup(&ec_id).await?; +let outcome = identity.insert(&ec_id, write).await?; +let held = identity.exists_strong(&ec_id).await?; +let partition = identity.partition(&partition_prefix)?; +let page = identity.list_partition(&partition, cursor, limit).await?; +identity.delete(&ec_id).await?; +``` + +The earlier interview sketch combined declaration and acquisition as `require_records(id, layout)`. The implementation separates these phases because `PreparedStores` is already immutable at initializer time. Grouping is still declared once; point operations still take only full keys. + +Conceptual values: + +```rust +struct Record { + body: Bytes, + metadata: Option, + generation: RecordGeneration, +} + +struct RecordWrite<'a> { + body: &'a [u8], + metadata: Option<&'a str>, + ttl: Option, + mode: RecordWriteMode, +} + +enum RecordWriteMode { + Add, + Overwrite, + IfGenerationMatch(RecordGeneration), +} + +enum RecordWriteOutcome { + Written, + PreconditionFailed, +} + +struct RecordPage { + keys: Vec, + cursor: Option, +} +``` + +A successful insert acknowledges the write, not its new generation. Fastly cannot atomically return that generation on insert, and a subsequent read might observe a competing writer. Consumers must not assume every lookup is strongly consistent; conditional writes reject stale generations and authoritative operations handle decisions about absence. + +### Layout contract + +The application supplies a stable, versioned declaration that resolves a full key to its enumerable partition prefix and validates a proposed complete partition prefix. The prefix must actually prefix the key. Partitions must be disjoint and stable for the lifetime of the stored format. No Unicode normalization or delimiter rewriting is permitted. + +Use validated prefix values at the provider boundary. Check resolved membership on point operations and returned keys on listing. Layout identity/version participates in binding compatibility and cursor identity. Application-specific parsing stays in the application; EdgeZero must not parse EC IDs or consent markers. Do not build an unrelated universal templating or partition-directory framework. + +TS's declaration will recognize existing root keys `.` and completion markers `__ts_ec_withdrawal_complete__::`. Roots group by IP hash; markers group by their marker prefix and EC ID. TS declares key-shape rules, not every individual IP hash. The rules and their non-overlap need focused TS tests. + +Changing a layout changes data placement. Treat it as an explicit format/cutover choice, not a mutable request option or automatic migration. + +Complete-partition validation applies to `list_partition`, not ordinary arbitrary-prefix/global enumeration. Do not reject an ordinary prefix merely because it spans groups or does not resolve to one complete partition. Ordinary listing must still enforce the logical binding's namespace, format and requested prefix. + +### Value identity and replacement + +Within a record-capable binding, ordinary get/put/TTL-put/delete/exists and record operations route to the same value row. Ordinary reads return the body without a hidden record JSON envelope. Ordinary puts are unconditional replacements with no metadata and a fresh generation; test that stale CAS fails afterward. A no-TTL put removes the prior expiry rather than substituting a long lifetime. + +This is an explicit technical design choice for review. Ordinary-only stores keep their current format and behavior. Opting into records must not silently redirect existing raw celld data or the legacy DynamoDB namespace into a different dataset. + +### Metadata and limits + +Normalize absent and empty metadata to `None`. Reject invalid UTF-8 returned as portable metadata rather than perform lossy decoding. Preserve nonempty text and body bytes exactly. + +Retain existing key/value/page validation and documented provider limits. Fastly 0.13.1 accepts 2000 metadata bytes; celld currently accepts 1024. Do not promise 2000 bytes on unchanged celld or encode binary metadata into the body. State the effective metadata limit in capability documentation; shared conformance values must fit every selected provider. No unrelated dependency or arbitrary new size restriction is required. + +## Atomicity, generations and outcomes + +Add may not overwrite an existing record under the backend's documented expiry rules. A generation-match write applies only when that same key still has the expected generation. Overwrite has no generation precondition. Comparison and mutation must be one provider-atomic operation or transaction. + +Generations are equality tokens scoped to a key/binding, not timestamps, sequence numbers for business decisions, or authorization. A stale token must not become valid after ordinary overwrite, deletion/recreation, expiry/recreation or restart. Deleting a value must not delete the state needed to prevent token reuse. + +Draft generation design: retain opaque `u64` markers for compatibility with current Fastly/celld and TS primitives. A `RecordGeneration` newtype exposes equality, not ordering/arithmetic promises. DynamoDB uses retained per-key allocation described below. Allocation gaps are valid. Acknowledged writes need not expose numerically increasing markers when competing unconditional writers commit out of allocation order. + +The cost of this choice is two DynamoDB write operations per attempted record insert and retained per-key history. An opaque random 128-bit generation could avoid that history and one request, but would change the TS generation representation and provide probabilistic uniqueness. This draft chooses deterministic retained allocation; review its cost before implementation. + +A definitive provider rejection maps to `PreconditionFailed` only for Add or generation-match. Failure of an unconditional overwrite is not normal contention. Validation, unsupported capability, unavailability and malformed responses remain errors. + +After a request may have mutated storage, a lost response/timeout must produce a payload-free error indicating the mutation may have committed. Do not turn it into `PreconditionFailed`, success, or a hidden retry. Keep SDK mutation attempts at one. Reads and application reconciliation do not authorize replay of the original mutation. Apply this to delete as well as insert; a repeated delete could erase a newly recreated value. + +## Pagination and existence + +`list_partition` requires a validated complete partition prefix. It returns logical full keys with an opaque continuation; alphabetical order is not a portable requirement. Continue until `cursor = None`; a short or empty page can still have a cursor. Exact page-size multiples must not lose the final continuation decision. + +Bind continuation to the logical store/backing scope, format/layout version, requested prefix or partition, and operation/query mode. A global Scan cursor is not interchangeable with a partition Query cursor. Preserve and validate native continuation rather than replace it with a guessed last key where the provider does not support that. Reject foreign, malformed or nonadvancing cursors without leaking credentials or record content. + +For a quiescent partition, traversal must return every matching visible key exactly once. Pages are strongly read, but the whole traversal is not a snapshot. Do not promise inclusion of keys inserted behind a consumed cursor or an atomic cleanup protocol against concurrent writers. + +Strong existence checks exact key equality against authoritative provider state. An ordinary lookup miss, a neighbouring prefix match, or an inconclusive page budget cannot prove absence. Return an error if authority cannot be established. Expired-but-retained rows may remain visible under native TTL rules; neither existence nor listing is a physical-erasure certificate. + +### Global ordinary listing + +The user approved retaining `list_keys_page(prefix, cursor, limit)` across all groups, including the empty prefix, without an alphabetical ordering guarantee. This supersedes the earlier proposal to reject cross-partition requests. Existing ordinary-only bindings retain their current behavior; update the common documentation so it does not promise universal lexical ordering on the new record binding. + +DynamoDB implements this operation with native paginated Scan. Apply the binding's versioned namespace and value-row-kind filters and the requested logical key prefix; never expose allocator rows or another namespace's values. Do not collect/sort the full namespace per output page or add a global index/catalog. Scan reads broadly and its filters do not make those reads cheap; document the read cost and required Scan operation without claiming hosted IAM qualification. + +The Scan limit bounds evaluated items, not matching returned keys. Return at most the requested number of logical keys, preserve LastEvaluatedKey, and allow empty/short pages with continuation. The native continuation may refer to a filtered internal row or another namespace's physical row: validate its structure and bind the outer cursor to the requested operation/scope, but do not require that native scan position itself be a returned logical value. Namespace/prefix filters remain mandatory on every page. + +Global enumeration carries ordinary-KV consistency and no cross-page snapshot or ordering promise. It must not replace the strongly consistent base-table Query used by `list_partition`, and must not be used as the authority for exact-key absence. In a quiescent store, following Scan continuation to completion returns all matching logical values exactly once, ignoring order. + +```mermaid +flowchart TD + A[DynamoDB listing request] --> B{Operation} + B --> C[Record partition listing] + C --> D[Strong Query on one declared PK] + B --> E[Ordinary arbitrary-prefix or global listing] + E --> F[Paginated Scan with binding filters] + F --> G[Unordered keys and native continuation] +``` + +## Provider implementations + +### Fastly native KV + +Use the pinned SDK's `current_generation()`, not deprecated `generation()`. Map Add and generation-match to native insert options and map definitive precondition rejection separately from transport/platform errors. Use native metadata and TTL; normalize empty metadata without a body envelope. + +Use explicit Strong list mode and native continuation. Partitions are logical prefixes within the existing store, not separate provisioned Fastly stores. Strong existence may enumerate the full key prefix and test exact equality, following continuation until absence is established or the provider/runtime returns an error. Do not infer presence from descendants or absence from a truncated page. + +Run record contracts in Viceroy. SDK compilation alone does not prove atomic races, expiry behavior or generation non-reuse. + +### Local persistent store + +Use an explicit versioned record table in redb; do not silently reinterpret the old ordinary-only row tuple. Store body, text metadata, generation and optional expiry together. Conditional evaluation and replacement occur in one write transaction. Ordinary record-mode methods use that same transaction path. + +Keep durable allocator state separate from value expiry/deletion. Reopening the database must not reset it. Preserve the existing expired-row recheck before lazy cleanup so cleanup cannot remove a concurrent fresh overwrite. + +Use a read view per record page with bounded scan batches and exclusive continuation. Scan caps or skipped expired keys must preserve a cursor, even for an empty returned page. Native redb expiry may hide values before physical cleanup; no shared exact TTL deadline is added. + +### DynamoDB on Axum + +Keep the provider native/default-off and use the existing AWS SDK/preparation infrastructure. Introduce an explicit record-format selection rather than silently reusing the legacy ordinary schema. + +A collision-free encoding of format version, namespace and declared partition prefix forms PK. Value SK is a reserved value-kind prefix plus the unchanged full logical key; a different reserved kind identifies that key's allocator. These encodings must remain disjoint for arbitrary valid logical keys. Returned keys never contain provider kind prefixes. + +Retain one allocator item per logical key in that same partition. Allocate a unique `u64` candidate through native atomic UpdateItem with a non-wrapping ceiling condition, then conditionally PutItem on the value row. Add and CAS use native value-row conditions, not a client read/check. Ordinary replacement uses the same allocation path. Allocator rows have no TTL, are not removed by value delete and never appear in user listings. No item is updated by inserts into every group. + +An allocation can succeed without a subsequent value write. Gaps and losing-CAS allocations are acceptable. A value-write acknowledgement can be lost. Use one absolute operation budget across both requests, retain single-attempt mutations, and distinguish error stages without implying success. + +Use strongly consistent base-table GetItem for authoritative existence and strongly consistent Query on the exact derived PK for record pages. Preserve LastEvaluatedKey, including on empty pages. Exclude internal allocator rows correctly without dropping continuation. Do not use a GSI for strong record listing. + +Keep ordinary arbitrary-prefix/global enumeration available through the separate filtered Scan path described above. Preserve native scan position across all evaluated rows, bind cursors to Scan versus Query and the requested scope, and make no global ordering or strong snapshot claim. + +Use native DynamoDB TTL for the new record format; do not introduce a portable logical-expiry envelope or claim exact deletion timing. Keep the legacy ordinary binding's existing expiry behavior unchanged. The new format is explicit and does not migrate or fall back to old rows. + +Document required SDK operations and new format implications. No hosted AWS/IAM acceptance or provisioned-capacity claim is required here. DynamoDB Local proves the local integration path, not production service behavior. + +### celld on Axum + +Add an explicit record-capable binding mode. All its ordinary operations use the conditional-record dataset, including no-TTL puts. Existing ordinary-only `/value` bindings stay raw KV; do not mix the two datasets under one binding. + +The companion service contribution must make the full current scoped-store/record implementation reviewable, not publish only a cursor patch against an unavailable base. Existing local source is `denoland/celld` branch `feat/scoped-store-cas` at `9350a043860c4f41dccb692ec9de98ebd9dac3ae`; no published PR/fork was verified during planning. + +The new record protocol must supply: + +- A read-only capability/version handshake proving pagination and no-expiry support before publishing the binding. +- Keyset continuation across record prefix, optional cursor, limit and relevant list mode, returning keys plus continuation. +- Scope/prefix-bound cursor validation and authoritative end detection. Raw KV cursors are not record cursors. +- An explicit non-expiring record representation for ordinary no-TTL writes. Do not invent a maximum TTL or put the value in raw KV instead. +- Atomic preconditions/generation allocation for every record-mode overwrite, including ordinary puts. + +Draft format choice: a fresh versioned record format with nullable expiry, leaving the existing positive-expiry-only record table/protocol and raw KV data untouched. The new server must treat no expiry correctly in reads, Add/CAS checks, listing, sweeps and alarm scheduling. No automatic table/data migration is included. Older services must fail preparation for the new binding instead of silently downgrading. + +celld's existing namespace-local transactional allocator can remain if it preserves generation lifetime. The no-namespace-wide DynamoDB mutation-item requirement does not impose an unrelated celld allocator redesign. + +Rebuild the server from the exact new source/lockfile and record source commit, tree, toolchain, build command and binary hash. Pin that rebuild into both EdgeZero and TS acceptance. The cached 0.6.2 binary and earlier receipts do not implement or prove the new protocol. + +## Acceptance and evidence + +| Proof | Required observation | +| --- | --- | +| Shared suite on all four providers | Exercise the public record handle, not only a provider-specific helper or in-memory double. | +| Same-row coherence | Ordinary write appears in record lookup and vice versa; ordinary replacement/TTL-put invalidates prior CAS; delete is visible through both views. | +| Conditional races | Independent writers racing Add or the same generation produce exactly one acknowledged winner. Do not substitute a single-process client mutex. | +| Generation lifetime | Old generations fail after ordinary overwrite, deletion/recreation, expiry/recreation and restart. Allocation gaps, failure between allocation/value write and allocator exhaustion do not reuse markers. | +| Strong absence | Stale lookup misses cannot override authoritative existence; prefix neighbours and multi-page exact-key checks do not yield false conclusions. | +| Distributed DynamoDB keys | Different declared groups produce different PK values. No namespace-wide mutation allocator traffic. Query targets only the declared group. | +| Pagination | Multiple pages, exact multiples, more than one provider page, empty/short continuation pages, expired rows, prefix neighbours, live changes and foreign/nonadvancing cursors. Stop only on no continuation; do not require alphabetical order. | +| Global ordinary listing | Scan traverses values across multiple groups, supports arbitrary/empty prefixes, excludes other namespaces and allocator rows, and preserves continuation through filtered-only pages. In a quiescent store compare key sets for completeness/uniqueness. Reject Scan/Query cursor interchange. | +| Metadata/body | Empty/missing normalization, exact nonempty multibyte text, actual provider byte limits, malformed UTF-8 rejection and binary body fidelity. | +| No-expiry | Ordinary no-TTL replacement remains non-expiring through the record-capable provider, including celld restart/sweep. | +| Ambiguous mutations | Lose a response after commit; report possible completion and observe no hidden replay. Definitive contention remains distinguishable. | +| Initialization | Unsupported/old-protocol/layout-mismatched stores fail before accepting identity work, without a mutation probe or fallback. Ordinary-only bindings remain usable. | +| Isolation | Required default-off and enabled native feature graphs, generated consumers and portable WASM checks pass without native dependencies in core. | + +Extend `scripts/run_native_aws_integration.mjs`, the matching celld runner, existing provider wire fixtures and Fastly/Viceroy contracts. Do not add another emulator stack. New test selectors must be nonzero and must not pass by skipping unavailable services. + +At stable checkpoints run repository-required formatting, strict Clippy, workspace tests, feature/target compilation and affected docs/consumer checks. Match evidence to exact source, dependencies, service artifact and environment. Source inspection, historical receipts and all-features compilation are not runtime conformance. + +## Trusted Server adoption boundary + +The coordinating plan repurposes user-owned TS #1536 for the shared core `EcKvStore` adapter plus DynamoDB adoption and #1537 for celld through that same adapter. TS owns its layout and caller-specific merge/retry/defer behavior. Port persistence-before-cookie, concurrent enrichment, withdrawal and restart tests before deleting duplicate Fastly/Axum provider mechanics. + +Awaitable core #1526 is a prerequisite, not something this EdgeZero spec delivers. Existing TS PR ancestry includes unrelated HTTP, Geo and consent work; preserving public commit history does not mean retaining all of that work in the new review diffs. Review an explicit disposition manifest before repurposing. Do not edit Aram's #1506/#1559 or Dhruv's #1487. + +## Review checkpoint + +Before code changes, review this draft's concrete technical choices: pre-preparation layout registration, retained per-key DynamoDB generation history, ordinary replacement metadata behavior and fresh celld format/no-expiry support. These are exposed for review rather than silently added to the interview decisions. The listing follow-up is settled: preserve global ordinary enumeration through unordered paginated Scan, while record partition listing uses strong Query. + +Implementation completion stops at tested, review-ready PRs and a reconciled official stack. No deploy, main merge, data migration, release tag or image publication is implied. + +## References + +- [EdgeZero #472](https://github.com/IABTechLab/edgezero/issues/472) and [#473](https://github.com/IABTechLab/edgezero/issues/473). +- Native AWS [#414](https://github.com/IABTechLab/edgezero/pull/414), combined celld [#422](https://github.com/IABTechLab/edgezero/pull/422), superseded [#463](https://github.com/IABTechLab/edgezero/pull/463). +- Native provider design: [AWS](2026-10-07-native-store-providers-design.md) and [celld](2026-10-15-native-celld-store-providers-design.md). +- Read-only TS [identity design #1559](https://github.com/IABTechLab/trusted-server/pull/1559), especially EZI-2a/EZI-2b, AX-4 and DEC-3/DEC-6. This spec records the later user decision to preserve owned PR identities while replacing their provider architecture. +- Shared TS consumer [#1587](https://github.com/IABTechLab/trusted-server/issues/1587), awaitable prerequisite [#1526](https://github.com/IABTechLab/trusted-server/pull/1526), adoption units [#1536](https://github.com/IABTechLab/trusted-server/pull/1536) and [#1537](https://github.com/IABTechLab/trusted-server/pull/1537). From 455c256147195a5ad8e29c2622a58c298cdeff9b Mon Sep 17 00:00:00 2001 From: Christian Date: Fri, 13 Nov 2026 18:38:44 -0600 Subject: [PATCH 5/9] Add prepared KV records across native providers --- Cargo.lock | 1 + crates/edgezero-adapter-axum/Cargo.toml | 2 + .../edgezero-adapter-axum/src/dev_server.rs | 407 +++++- .../src/key_value_store.rs | 125 +- .../src/key_value_store_records.rs | 932 +++++++++++++ .../src/native_bindings.rs | 81 +- .../edgezero-adapter-axum/src/native_build.rs | 24 + .../src/native_stores.rs | 214 ++- .../src/templates/Cargo.toml.hbs | 2 + crates/edgezero-adapter-cloudflare/src/lib.rs | 93 +- .../src/request.rs | 16 + .../src/key_value_store.rs | 560 +++++++- crates/edgezero-adapter-fastly/src/lib.rs | 80 +- crates/edgezero-adapter-fastly/src/request.rs | 96 +- .../src/key_value_store.rs | 4 + crates/edgezero-adapter-spin/src/lib.rs | 93 +- crates/edgezero-adapter-spin/src/request.rs | 15 + crates/edgezero-cli/src/config.rs | 53 + crates/edgezero-cli/src/generator.rs | 2 + .../tests/celld_generated_proof.rs | 139 ++ .../tests/native_aws_emulator_proof.rs | 221 +++- crates/edgezero-core/src/app.rs | 47 + crates/edgezero-core/src/key_value_store.rs | 145 ++- crates/edgezero-core/src/lib.rs | 6 + crates/edgezero-core/src/record_store.rs | 710 ++++++++++ crates/edgezero-core/src/store_registry.rs | 29 +- crates/edgezero-core/tests/record_api.rs | 588 +++++++++ .../tests/record_configuration.rs | 182 +++ crates/edgezero-store-aws/Cargo.toml | 2 + crates/edgezero-store-aws/src/dynamodb_kv.rs | 82 +- .../src/dynamodb_records.rs | 861 ++++++++++++ .../src/dynamodb_records_tests.rs | 1159 +++++++++++++++++ .../edgezero-store-aws/src/dynamodb_tests.rs | 14 +- crates/edgezero-store-aws/src/lib.rs | 12 + .../tests/dynamodb_records_local.rs | 586 +++++++++ crates/edgezero-store-celld/Cargo.toml | 3 + crates/edgezero-store-celld/README.md | 37 + crates/edgezero-store-celld/src/client.rs | 94 +- crates/edgezero-store-celld/src/kv.rs | 8 +- crates/edgezero-store-celld/src/lib.rs | 7 +- .../edgezero-store-celld/src/preparation.rs | 54 + crates/edgezero-store-celld/src/records_v2.rs | 613 +++++++++ .../src/records_v2_tests.rs | 388 ++++++ .../tests/records_v2_interop.rs | 440 +++++++ .../crates/fixture-fastly/Cargo.toml | 6 + .../crates/fixture-fastly/fastly.toml | 3 + .../fixture-fastly/src/bin/record_contract.rs | 99 ++ .../crates/fixture-fastly/src/record_cases.rs | 136 ++ crates/edgezero-test-harness/src/main.rs | 17 +- crates/edgezero-test-harness/src/runners.rs | 4 + .../edgezero-test-harness/src/runners/ci.rs | 20 + .../src/runners/record_contract.rs | 387 ++++++ docs/.vitepress/config.mts | 1 + docs/guide/dynamodb-kv.md | 4 + docs/guide/kv-records.md | 257 ++++ docs/guide/kv.md | 22 +- docs/guide/native-celld-stores.md | 4 +- .../plans/2026-11-13-kv-record-contract.md | 130 +- .../check_native_store_dependency_contract.sh | 62 +- scripts/run_native_aws_integration.mjs | 50 +- scripts/run_native_aws_integration.test.mjs | 33 +- scripts/run_native_celld_integration.mjs | 14 +- scripts/run_native_celld_integration.test.mjs | 12 +- 63 files changed, 10267 insertions(+), 221 deletions(-) create mode 100644 crates/edgezero-adapter-axum/src/key_value_store_records.rs create mode 100644 crates/edgezero-core/src/record_store.rs create mode 100644 crates/edgezero-core/tests/record_api.rs create mode 100644 crates/edgezero-core/tests/record_configuration.rs create mode 100644 crates/edgezero-store-aws/src/dynamodb_records.rs create mode 100644 crates/edgezero-store-aws/src/dynamodb_records_tests.rs create mode 100644 crates/edgezero-store-aws/tests/dynamodb_records_local.rs create mode 100644 crates/edgezero-store-celld/README.md create mode 100644 crates/edgezero-store-celld/src/records_v2.rs create mode 100644 crates/edgezero-store-celld/src/records_v2_tests.rs create mode 100644 crates/edgezero-store-celld/tests/records_v2_interop.rs create mode 100644 crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/bin/record_contract.rs create mode 100644 crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/record_cases.rs create mode 100644 crates/edgezero-test-harness/src/runners/record_contract.rs create mode 100644 docs/guide/kv-records.md diff --git a/Cargo.lock b/Cargo.lock index 5d4df6a5..7fb6c7bf 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1444,6 +1444,7 @@ dependencies = [ "serde", "serde_json", "sha2 0.11.1", + "tempfile", "thiserror 2.0.21", "tokio", "tokio-rustls", diff --git a/crates/edgezero-adapter-axum/Cargo.toml b/crates/edgezero-adapter-axum/Cargo.toml index d9eeffa1..2ab92f1d 100644 --- a/crates/edgezero-adapter-axum/Cargo.toml +++ b/crates/edgezero-adapter-axum/Cargo.toml @@ -35,8 +35,10 @@ aws-appconfig-agent = ["axum", "edgezero-store-aws/appconfig-agent"] aws-secrets-manager = ["axum", "edgezero-store-aws/secrets-manager"] aws-stores = ["aws-appconfig-agent", "aws-secrets-manager"] aws-dynamodb = ["axum", "edgezero-store-aws/dynamodb-kv"] +aws-dynamodb-records = ["aws-dynamodb", "edgezero-store-aws/dynamodb-records"] celld-config = ["axum", "edgezero-store-celld/config"] celld-kv = ["axum", "edgezero-store-celld/kv"] +celld-records = ["celld-kv", "edgezero-store-celld/records-v2"] geo-cloudfront = ["axum"] cli = [ "native-bindings", diff --git a/crates/edgezero-adapter-axum/src/dev_server.rs b/crates/edgezero-adapter-axum/src/dev_server.rs index 8d5ba1e6..4590fb57 100644 --- a/crates/edgezero-adapter-axum/src/dev_server.rs +++ b/crates/edgezero-adapter-axum/src/dev_server.rs @@ -56,6 +56,7 @@ use edgezero_core::secret_store::SecretHandle; use edgezero_core::store_registry::{ BoundSecretStore, ConfigRegistry, ConfigStoreBinding, KvRegistry, SecretRegistry, StoreRegistry, }; +use edgezero_core::{RecordLayout, RecordStoreRequirements}; use log::{Level, LevelFilter}; use simple_logger::SimpleLogger; use tokio::task::yield_now; @@ -403,11 +404,24 @@ fn stable_store_name_hash(store_name: &str) -> u64 { hash } +#[cfg(test)] fn kv_handle_from_path(kv_path: &Path) -> anyhow::Result { + kv_handle_from_path_with_records(kv_path, None) +} + +fn kv_handle_from_path_with_records( + kv_path: &Path, + layout: Option<&RecordLayout>, +) -> anyhow::Result { if let Some(parent) = kv_path.parent() { fs::create_dir_all(parent).context("failed to create KV store directory")?; } - let kv_store = Arc::new(PersistentKvStore::new(kv_path).context("failed to create KV store")?); + let store = match layout { + Some(declared_layout) => PersistentKvStore::new_records(kv_path, declared_layout.clone()), + None => PersistentKvStore::new(kv_path), + } + .context("failed to create KV store")?; + let kv_store = Arc::new(store); log::info!("KV store: {}", kv_path.display()); Ok(KvHandle::new(kv_store)) } @@ -1066,7 +1080,12 @@ where log::warn!(target: BOOT_LOG_TARGET, "{warning}"); } let stores = PreparedStores { - kv: build_kv_registry(metadata.kv, &env, kv_init_requirement(metadata))?, + kv: build_kv_registry( + metadata.kv, + &env, + kv_init_requirement(metadata), + app.record_store_requirements(), + )?, config: build_config_registry(metadata.config, &env, config_limits)?, secrets: build_secret_registry(metadata.secrets, &env), }; @@ -1314,7 +1333,13 @@ where init_logging::(options.logging_level())?; let mut app = App::build::(crate::AXUM_PLATFORM) .map_err(|_error| anyhow::anyhow!("application configuration failed"))?; - let stores = build_production_stores(metadata, &options, sources).await?; + let stores = build_production_stores_with_records( + metadata, + &options, + sources, + app.record_store_requirements(), + ) + .await?; let transport = AxumOutboundClient::try_transport() .map_err(|_error| failure("transport", "HTTP client", "initialization failed"))?; initialize(&mut app, &stores).await.map_err(|error| { @@ -1357,15 +1382,32 @@ fn check_data_write_access(root: &Path) -> anyhow::Result<()> { Err(failure("stores", "DATA_DIR", "write-probe unavailable")) } +#[cfg(test)] +async fn build_production_stores( + metadata: StoresMetadata, + options: &AxumRunOptions, + sources: NativeSourcePlan, +) -> anyhow::Result { + build_production_stores_with_records( + metadata, + options, + sources, + &RecordStoreRequirements::new(), + ) + .await +} + #[expect( clippy::too_many_lines, reason = "keep all sequential staging in one ownership scope before atomic publication" )] -async fn build_production_stores( +async fn build_production_stores_with_records( metadata: StoresMetadata, options: &AxumRunOptions, sources: NativeSourcePlan, + requirements: &RecordStoreRequirements, ) -> anyhow::Result { + sources.validate_record_requirements(requirements, options)?; let mut config = BTreeMap::new(); let mut kv = BTreeMap::new(); let mut secrets = BTreeMap::new(); @@ -1434,7 +1476,11 @@ async fn build_production_stores( } for (id, source) in sources.kv { let handle = match source { - NativeStoreSource::BuiltIn(builtin) => sessions.prepare_kv(builtin, id).await?, + NativeStoreSource::BuiltIn(builtin) => { + sessions + .prepare_kv(builtin, id, requirements.get(id)) + .await? + } local @ (NativeStoreSource::Local | NativeStoreSource::Ready(_) | NativeStoreSource::Prepare(_)) => { @@ -1446,8 +1492,12 @@ async fn build_production_stores( if let Some(handle) = handles.get(&name) { return Ok(handle.clone()); } - let store = PersistentKvStore::new(root.join(kv_store_file_name(&name))) - .map_err(|_error| failure("kv", id, "database unavailable or locked"))?; + let path = root.join(kv_store_file_name(&name)); + let store = match requirements.get(id) { + Some(layout) => PersistentKvStore::new_records(path, layout.clone()), + None => PersistentKvStore::new(path), + } + .map_err(|_error| failure("kv", id, "database unavailable or locked"))?; let handle = KvHandle::new(Arc::new(store)); handles.insert(name, handle.clone()); Ok(handle) @@ -1457,6 +1507,7 @@ async fn build_production_stores( }; kv.insert(id.to_owned(), handle); } + validate_prepared_record_bindings(&kv, requirements)?; let environment_secrets = SecretHandle::new(Arc::new(EnvSecretStore::new())); for (id, source) in sources.secrets { let binding = match source { @@ -1482,6 +1533,40 @@ async fn build_production_stores( }) } +fn validate_prepared_record_bindings( + handles: &BTreeMap, + requirements: &RecordStoreRequirements, +) -> anyhow::Result<()> { + for (id, layout) in requirements { + let records = handles + .get(id) + .ok_or_else(|| failure("kv", id, "required record binding missing"))? + .require_records() + .map_err(|error| failure("kv", id, native_stores::record_capability_failure(&error)))?; + if records.binding().layout != *layout { + return Err(failure("kv", id, "supplied record layout mismatch")); + } + } + let mut scopes = BTreeMap::new(); + for (id, handle) in handles { + if let Ok(records) = handle.require_records() { + let binding = records.binding(); + let scope = (binding.scope.clone(), binding.format.clone()); + if scopes + .insert(scope, binding.layout.clone()) + .is_some_and(|previous| previous != binding.layout) + { + return Err(failure( + "kv", + id, + "shared backing store has conflicting record layouts", + )); + } + } + } + Ok(()) +} + fn finish_registry( metadata: Option, handles: BTreeMap, @@ -1507,7 +1592,9 @@ fn build_kv_registry( kv_meta: Option, env: &EnvConfig, init: KvInitRequirement, + requirements: &RecordStoreRequirements, ) -> anyhow::Result> { + validate_local_record_declarations(kv_meta, env, requirements)?; let Some(meta) = kv_meta else { return Ok(None); }; @@ -1521,14 +1608,15 @@ fn build_kv_registry( continue; } let kv_path = kv_store_path(&store_name); - let handle = match kv_handle_from_path(&kv_path) { + let layout = requirements.get(*id); + let handle = match kv_handle_from_path_with_records(&kv_path, layout) { Ok(handle) => handle, Err(_error) => match init { - KvInitRequirement::Optional => { + KvInitRequirement::Optional if layout.is_none() => { log::warn!(target: BOOT_LOG_TARGET, "KV store id `{id}` unavailable; omitting optional binding"); continue; } - KvInitRequirement::Required => { + KvInitRequirement::Required | KvInitRequirement::Optional => { return Err(failure("KV", id, "declared database unavailable or locked")); } }, @@ -1548,6 +1636,41 @@ fn build_kv_registry( Ok(StoreRegistry::from_parts(by_id, default_id)) } +fn validate_local_record_declarations( + metadata: Option, + env: &EnvConfig, + requirements: &RecordStoreRequirements, +) -> anyhow::Result<()> { + for id in requirements.keys() { + if !metadata.is_some_and(|meta| meta.ids.contains(&id.as_str())) { + return Err(failure( + "kv", + id, + "record declaration uses an undeclared ID", + )); + } + } + // None requests the ordinary table, not an inherited record format. Mixing + // it with a declared layout would silently redirect one alias's value rows. + let mut layouts = BTreeMap::new(); + if let Some(meta) = metadata { + for id in meta.ids { + let layout = requirements.get(*id); + if layouts + .insert(env.store_name("kv", id), layout) + .is_some_and(|previous| previous != layout) + { + return Err(failure( + "kv", + id, + "shared backing store has conflicting record layouts", + )); + } + } + } + Ok(()) +} + /// Build the per-request config registry from the per-id local-file stores. /// /// Each declared id reads `.edgezero/local-config-.json`. A missing @@ -4150,6 +4273,270 @@ mod integration_tests { } } +#[cfg(test)] +mod record_preparation_tests { + use std::cell::Cell; + use std::fs; + use std::rc::Rc; + use std::sync::Arc; + + use edgezero_core::RecordKeyLayout; + use edgezero_core::key_value_store::{KvError, NoopKvStore}; + use edgezero_core::record_store::contract::assert_record_contract; + use edgezero_store_aws::settings::DynamoDbKvSettings; + use futures::executor::block_on; + + use super::*; + use crate::native_stores::{NativeKvSource, NativeStoreOverrides}; + + struct Groups(u32); + + impl RecordKeyLayout for Groups { + fn id(&self) -> &'static str { + "native.record-groups" + } + + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + key.split_once('/') + .and_then(|(_, suffix)| key.strip_suffix(suffix)) + .ok_or_else(|| KvError::Validation("unknown group".to_owned())) + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.matches('/').count() == 1 { + Ok(()) + } else { + Err(KvError::Validation("incomplete group".to_owned())) + } + } + + fn version(&self) -> u32 { + self.0 + } + } + + fn metadata() -> StoresMetadata { + StoresMetadata { + kv: Some(StoreMetadata { + ids: &["first", "second"], + default: "first", + }), + ..StoresMetadata::default() + } + } + + fn requirements() -> RecordStoreRequirements { + RecordStoreRequirements::from([("first".to_owned(), RecordLayout::new(Groups(1)).unwrap())]) + } + + fn options() -> AxumRunOptions { + AxumRunOptions::new("127.0.0.1:19001".parse().unwrap()).unwrap() + } + + #[test] + fn local_record_preparation_publishes_the_declared_layout() { + let root = tempfile::tempdir().unwrap(); + let options = options().with_data_dir(root.path()).unwrap(); + let plan = + NativeSourcePlan::validate(metadata(), &options, NativeStoreOverrides::default()) + .unwrap(); + let stores = block_on(build_production_stores_with_records( + metadata(), + &options, + plan, + &requirements(), + )) + .unwrap(); + let records = stores.kv().unwrap().named("first").unwrap(); + assert_eq!( + records + .require_records() + .unwrap() + .binding() + .layout + .version(), + 1 + ); + stores + .kv() + .unwrap() + .named("second") + .unwrap() + .require_records() + .unwrap_err(); + block_on(assert_record_contract(&records, "group/")); + } + + #[test] + fn unknown_record_id_fails_before_opening_local_files() { + let root = tempfile::tempdir().unwrap(); + let options = options().with_data_dir(root.path()).unwrap(); + let plan = + NativeSourcePlan::validate(metadata(), &options, NativeStoreOverrides::default()) + .unwrap(); + let unknown = RecordStoreRequirements::from([( + "unknown".to_owned(), + RecordLayout::new(Groups(1)).unwrap(), + )]); + assert!( + block_on(build_production_stores_with_records( + metadata(), + &options, + plan, + &unknown + )) + .is_err() + ); + assert_eq!(fs::read_dir(root.path()).unwrap().count(), 0); + } + + #[test] + fn shared_local_backing_names_cannot_mix_layouts_or_formats() { + let root = tempfile::tempdir().unwrap(); + let options = AxumRunOptions::from_vars([ + ("EDGEZERO__STORES__KV__FIRST__NAME", "shared"), + ("EDGEZERO__STORES__KV__SECOND__NAME", "shared"), + ]) + .unwrap() + .with_data_dir(root.path()) + .unwrap(); + let mut required = requirements(); + for second in [None, Some(RecordLayout::new(Groups(2)).unwrap())] { + if let Some(layout) = second { + required.insert("second".to_owned(), layout); + } + let plan = + NativeSourcePlan::validate(metadata(), &options, NativeStoreOverrides::default()) + .unwrap(); + assert!( + block_on(build_production_stores_with_records( + metadata(), + &options, + plan, + &required + )) + .is_err() + ); + assert_eq!(fs::read_dir(root.path()).unwrap().count(), 0); + } + required.insert("second".to_owned(), RecordLayout::new(Groups(1)).unwrap()); + let plan = + NativeSourcePlan::validate(metadata(), &options, NativeStoreOverrides::default()) + .unwrap(); + let stores = block_on(build_production_stores_with_records( + metadata(), + &options, + plan, + &required, + )) + .unwrap(); + stores + .kv() + .unwrap() + .named("second") + .unwrap() + .require_records() + .unwrap(); + } + + #[test] + fn ready_mismatch_is_rejected_before_deferred_work_is_polled() { + let polled = Rc::new(Cell::new(false)); + let observed = Rc::clone(&polled); + let mut overrides = NativeStoreOverrides::default(); + overrides + .insert_kv("first", KvHandle::new(Arc::new(NoopKvStore))) + .unwrap(); + overrides + .prepare_kv("second", async move { + observed.set(true); + Ok(KvHandle::new(Arc::new(NoopKvStore))) + }) + .unwrap(); + let options = options(); + let plan = NativeSourcePlan::validate(metadata(), &options, overrides).unwrap(); + assert!( + block_on(build_production_stores_with_records( + metadata(), + &options, + plan, + &requirements() + )) + .is_err() + ); + assert!(!polled.get()); + } + + #[test] + fn deferred_handles_must_already_be_record_capable_before_publication() { + let mut overrides = NativeStoreOverrides::default(); + overrides + .prepare_kv("first", async { Ok(KvHandle::new(Arc::new(NoopKvStore))) }) + .unwrap(); + overrides + .insert_kv("second", KvHandle::new(Arc::new(NoopKvStore))) + .unwrap(); + let options = options(); + let plan = NativeSourcePlan::validate(metadata(), &options, overrides).unwrap(); + assert!( + block_on(build_production_stores_with_records( + metadata(), + &options, + plan, + &requirements() + )) + .is_err() + ); + } + + #[test] + fn raw_builtin_is_not_implicitly_upgraded_and_record_mode_requires_layout() { + let root = tempfile::tempdir().unwrap(); + let options = options().with_data_dir(root.path()).unwrap(); + let settings: DynamoDbKvSettings = toml::from_str( + "region='us-east-1'\ntable='record-fixture'\nnamespace='record-fixture'", + ) + .unwrap(); + let mut raw_plan = + NativeSourcePlan::validate(metadata(), &options, NativeStoreOverrides::default()) + .unwrap(); + raw_plan.kv.insert( + "first", + NativeStoreSource::BuiltIn(NativeKvSource::DynamoDb(settings.clone())), + ); + assert!( + block_on(build_production_stores_with_records( + metadata(), + &options, + raw_plan, + &requirements() + )) + .is_err() + ); + let mut record_plan = + NativeSourcePlan::validate(metadata(), &options, NativeStoreOverrides::default()) + .unwrap(); + record_plan.kv.insert( + "first", + NativeStoreSource::BuiltIn(NativeKvSource::DynamoDbRecords(settings)), + ); + assert!( + block_on(build_production_stores_with_records( + metadata(), + &options, + record_plan, + &RecordStoreRequirements::new() + )) + .is_err() + ); + assert_eq!(fs::read_dir(root.path()).unwrap().count(), 0); + } +} + #[cfg(test)] mod native_store_tests { use super::*; diff --git a/crates/edgezero-adapter-axum/src/key_value_store.rs b/crates/edgezero-adapter-axum/src/key_value_store.rs index 3383f7b1..c4d45d03 100644 --- a/crates/edgezero-adapter-axum/src/key_value_store.rs +++ b/crates/edgezero-adapter-axum/src/key_value_store.rs @@ -43,13 +43,20 @@ //! - The database file path acts as the namespace identifier, similar to //! how Cloudflare uses bindings and Fastly uses store names. +#[path = "key_value_store_records.rs"] +mod records; + use std::ops::Bound; use std::path::Path; use std::time::Duration; use async_trait::async_trait; use bytes::Bytes; -use edgezero_core::key_value_store::{KvError, KvPage, KvStore}; +use edgezero_core::key_value_store::{KvError, KvHandle, KvPage, KvStore}; +use edgezero_core::record_store::{ + Record, RecordBinding, RecordLayout, RecordPage, RecordPartition, RecordWrite, + RecordWriteOutcome, +}; use redb::{Database, ReadableDatabase as _, ReadableTable as _, TableDefinition}; use std::time::SystemTime; @@ -66,6 +73,7 @@ type KvTable<'txn> = redb::Table<'txn, &'static str, (&'static [u8], Option, } impl PersistentKvStore { @@ -168,7 +176,10 @@ impl PersistentKvStore { })?; // Initialize the table - let store = Self { db }; + let store = Self { + db, + record_binding: None, + }; let write_txn = store.begin_write()?; { let _table = Self::open_table(&write_txn)?; @@ -178,6 +189,42 @@ impl PersistentKvStore { Ok(store) } + /// Opens an explicit record format without converting ordinary-only rows. + /// + /// All ordinary methods on this binding access these record rows. Generation + /// history remains outside value deletion and expiry. + /// + /// # Errors + /// Returns an error for unavailable storage, incompatible layout or incomplete + /// persistent format state. Existing state is never reset to repair an error. + #[inline] + #[expect( + clippy::unnecessary_debug_formatting, + reason = "Path Debug preserves and escapes non-UTF-8 path bytes for cursor scope identity" + )] + pub fn new_records

(path: P, layout: RecordLayout) -> Result + where + P: AsRef, + { + let db = Database::create(path.as_ref()).map_err(|_error| KvError::Unavailable)?; + let absolute = path + .as_ref() + .canonicalize() + .map_err(|_error| KvError::Unavailable)?; + let record_binding = RecordBinding { + format: records::FORMAT.to_owned(), + layout, + max_metadata_bytes: KvHandle::MAX_VALUE_SIZE, + scope: format!("redb:{absolute:?}"), + }; + let store = Self { + db, + record_binding: Some(record_binding), + }; + records::initialize(&store)?; + Ok(store) + } + fn open_table(txn: &redb::WriteTransaction) -> Result, KvError> { txn.open_table(KV_TABLE) .map_err(|err| KvError::Internal(anyhow::anyhow!("failed to open table: {err}"))) @@ -196,6 +243,9 @@ impl PersistentKvStore { impl KvStore for PersistentKvStore { #[inline] async fn delete(&self, key: &str) -> Result<(), KvError> { + if self.record_binding.is_some() { + return records::ordinary_delete(self, key); + } let write_txn = self.begin_write()?; let mut table = Self::open_table(&write_txn)?; table @@ -212,6 +262,9 @@ impl KvStore for PersistentKvStore { #[inline] async fn get_bytes(&self, key: &str) -> Result, KvError> { + if self.record_binding.is_some() { + return records::ordinary_lookup(self, key); + } let read_txn = self .db .begin_read() @@ -277,6 +330,9 @@ impl KvStore for PersistentKvStore { cursor: Option<&str>, limit: usize, ) -> Result { + if self.record_binding.is_some() { + return records::ordinary_page(self, prefix, cursor, limit); + } let mut live_keys = Vec::with_capacity(limit.saturating_add(1)); let mut scan_cursor = cursor.map(str::to_owned); let mut reached_end = false; @@ -388,6 +444,9 @@ impl KvStore for PersistentKvStore { #[inline] async fn put_bytes(&self, key: &str, value: Bytes) -> Result<(), KvError> { + if self.record_binding.is_some() { + return records::ordinary_put(self, key, value.as_ref(), None); + } let write_txn = self.begin_write()?; let mut table = Self::open_table(&write_txn)?; table @@ -404,6 +463,9 @@ impl KvStore for PersistentKvStore { value: Bytes, ttl: Duration, ) -> Result<(), KvError> { + if self.record_binding.is_some() { + return records::ordinary_put(self, key, value.as_ref(), Some(ttl)); + } let expires_at = SystemTime::now() .checked_add(ttl) .ok_or_else(|| KvError::Internal(anyhow::anyhow!("ttl overflows system time")))?; @@ -417,6 +479,65 @@ impl KvStore for PersistentKvStore { drop(table); Self::commit(write_txn) } + + #[inline] + fn record_binding(&self) -> Option { + self.record_binding.clone() + } + + #[inline] + async fn record_delete(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + records::delete(self, key, partition) + } + + #[inline] + async fn record_exists_strong( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result { + records::exists(self, key, partition) + } + + #[inline] + async fn record_insert( + &self, + key: &str, + partition: &RecordPartition, + write: RecordWrite<'_>, + ) -> Result { + records::insert(self, key, partition, write) + } + + #[inline] + async fn record_list_partition( + &self, + partition: &RecordPartition, + cursor: Option<&str>, + limit: usize, + ) -> Result { + let binding = self + .record_binding + .as_ref() + .ok_or_else(|| KvError::Unsupported { + operation: "records".to_owned(), + })?; + if binding.layout.partition(partition.prefix())? != *partition { + return Err(KvError::Validation( + "record partition does not match".to_owned(), + )); + } + records::page(self, partition.prefix(), cursor, limit, "partition") + } + + #[inline] + async fn record_lookup( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result, KvError> { + records::lookup(self, key, partition) + } } #[cfg(test)] diff --git a/crates/edgezero-adapter-axum/src/key_value_store_records.rs b/crates/edgezero-adapter-axum/src/key_value_store_records.rs new file mode 100644 index 00000000..e55adca5 --- /dev/null +++ b/crates/edgezero-adapter-axum/src/key_value_store_records.rs @@ -0,0 +1,932 @@ +//! Explicit persistent record format. Ordinary-only tables are not reinterpreted. + +use std::ops::Bound; +use std::time::{Duration, SystemTime}; + +use bytes::Bytes; +use edgezero_core::key_value_store::{KvError, KvHandle, KvPage}; +use edgezero_core::record_store::{ + Record, RecordBinding, RecordGeneration, RecordPage, RecordPartition, RecordWrite, + RecordWriteMode, RecordWriteOutcome, +}; +use redb::{ReadableDatabase as _, ReadableTable as _, TableDefinition, TableError}; +use serde::{Deserialize, Serialize}; + +use super::PersistentKvStore; + +const ROWS: TableDefinition<&str, RecordRow<'static>> = TableDefinition::new("edgezero-records-v1"); +const STATE: TableDefinition<&str, (u64, &str, u32)> = + TableDefinition::new("edgezero-records-state-v1"); +const STATE_KEY: &str = "state"; +pub(super) const FORMAT: &str = "redb-records-v1"; + +type RecordRow<'row> = (&'row [u8], Option<&'row str>, u64, Option); + +#[derive(Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +struct Cursor { + after: String, + layout: String, + mode: String, + prefix: String, + scope: String, + version: u32, +} + +fn binding(store: &PersistentKvStore) -> Result<&RecordBinding, KvError> { + store + .record_binding + .as_ref() + .ok_or_else(|| KvError::Unsupported { + operation: "records".to_owned(), + }) +} + +pub(super) fn initialize(store: &PersistentKvStore) -> Result<(), KvError> { + let binding = binding(store)?; + let read = store + .db + .begin_read() + .map_err(|_error| KvError::Unavailable)?; + let rows_exist = match read.open_table(ROWS) { + Ok(_) => true, + Err(TableError::TableDoesNotExist(_)) => false, + Err(_) => return Err(KvError::Unavailable), + }; + let state_exists = match read.open_table(STATE) { + Ok(_) => true, + Err(TableError::TableDoesNotExist(_)) => false, + Err(_) => return Err(KvError::Unavailable), + }; + if rows_exist != state_exists { + return Err(KvError::Unavailable); + } + if state_exists { + let state = read + .open_table(STATE) + .map_err(|_error| KvError::Unavailable)?; + let saved = state + .get(STATE_KEY) + .map_err(|_error| KvError::Unavailable)? + .ok_or(KvError::Unavailable)?; + let (_, id, version) = saved.value(); + if id != binding.layout.id() || version != binding.layout.version() { + return Err(invalid("persistent record layout does not match")); + } + return Ok(()); + } + drop(read); + let write = store + .db + .begin_write() + .map_err(|_error| KvError::Unavailable)?; + { + let _rows = write + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + let mut state = write + .open_table(STATE) + .map_err(|_error| KvError::Unavailable)?; + let _previous_state = state + .insert( + STATE_KEY, + (0, binding.layout.id(), binding.layout.version()), + ) + .map_err(|_error| KvError::Unavailable)?; + } + write.commit().map_err(|_error| KvError::MutationUncertain) +} + +fn invalid(message: &str) -> KvError { + KvError::Validation(message.to_owned()) +} + +fn validate_partition( + store: &PersistentKvStore, + key: &str, + partition: &RecordPartition, +) -> Result<(), KvError> { + if binding(store)?.layout.resolve(key)? != *partition { + return Err(invalid("record partition does not match the key")); + } + Ok(()) +} + +pub(super) fn lookup( + store: &PersistentKvStore, + key: &str, + partition: &RecordPartition, +) -> Result, KvError> { + validate_partition(store, key, partition)?; + let read = store + .db + .begin_read() + .map_err(|_error| KvError::Unavailable)?; + let table = read + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + let result = match table.get(key).map_err(|_error| KvError::Unavailable)? { + None => return Ok(None), + Some(row) => { + let (body, metadata, generation, expiry) = row.value(); + if generation == 0 { + return Err(KvError::Unavailable); + } + if PersistentKvStore::is_expired(expiry) { + None + } else { + Some(Record { + body: Bytes::copy_from_slice(body), + generation: RecordGeneration::from_u64(generation), + metadata: metadata.filter(|text| !text.is_empty()).map(str::to_owned), + }) + } + } + }; + drop(table); + drop(read); + if result.is_none() { + cleanup_expired(store, &[key.to_owned()])?; + } + Ok(result) +} + +pub(super) fn exists( + store: &PersistentKvStore, + key: &str, + partition: &RecordPartition, +) -> Result { + validate_partition(store, key, partition)?; + let read = store + .db + .begin_read() + .map_err(|_error| KvError::Unavailable)?; + let table = read + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + // Authoritative existence uses the same native redb TTL visibility as + // lookup, conditional writes and partition listing, not mere disk presence. + Ok(table + .get(key) + .map_err(|_error| KvError::Unavailable)? + .is_some_and(|row| !PersistentKvStore::is_expired(row.value().3))) +} + +pub(super) fn insert( + store: &PersistentKvStore, + key: &str, + partition: &RecordPartition, + value: RecordWrite<'_>, +) -> Result { + validate_partition(store, key, partition)?; + if value.body.len() > KvHandle::MAX_VALUE_SIZE + || value.metadata.is_some_and(|text| { + text.len() > binding(store).map_or(0, |capability| capability.max_metadata_bytes) + }) + { + return Err(invalid("record exceeds the provider byte limit")); + } + let replacement_expiry = value + .ttl + .map(|ttl| { + if ttl.is_zero() { + return Err(invalid("record TTL must be positive")); + } + SystemTime::now() + .checked_add(ttl) + .map(PersistentKvStore::system_time_to_millis) + .ok_or_else(|| invalid("record TTL cannot be represented")) + }) + .transpose()?; + let write = store + .db + .begin_write() + .map_err(|_error| KvError::Unavailable)?; + { + let mut rows = write + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + let current = rows + .get(key) + .map_err(|_error| KvError::Unavailable)? + .map(|row| { + let (_, _, generation, expiry) = row.value(); + (generation, expiry) + }); + let live = current.filter(|(_, expiry)| !PersistentKvStore::is_expired(*expiry)); + match value.mode { + RecordWriteMode::Add if live.is_some() => { + return Ok(RecordWriteOutcome::PreconditionFailed); + } + RecordWriteMode::IfGenerationMatch(expected) + if live.map(|(generation, _)| generation) != Some(expected.as_u64()) => + { + return Ok(RecordWriteOutcome::PreconditionFailed); + } + RecordWriteMode::Add + | RecordWriteMode::Overwrite + | RecordWriteMode::IfGenerationMatch(_) => {} + } + let mut state = write + .open_table(STATE) + .map_err(|_error| KvError::Unavailable)?; + let counter = { + let saved = state + .get(STATE_KEY) + .map_err(|_error| KvError::Unavailable)? + .ok_or(KvError::Unavailable)?; + let (counter, id, version) = saved.value(); + let binding = binding(store)?; + if id != binding.layout.id() + || version != binding.layout.version() + || current.is_some_and(|(generation, _)| generation == 0 || generation > counter) + { + return Err(KvError::Unavailable); + } + counter + }; + let next = counter.checked_add(1).ok_or(KvError::Unavailable)?; + let binding = binding(store)?; + state + .insert( + STATE_KEY, + (next, binding.layout.id(), binding.layout.version()), + ) + .map_err(|_error| KvError::Unavailable)?; + let _previous_row = rows + .insert( + key, + ( + value.body, + value.metadata.filter(|text| !text.is_empty()), + next, + replacement_expiry, + ), + ) + .map_err(|_error| KvError::Unavailable)?; + } + write + .commit() + .map_err(|_error| KvError::MutationUncertain)?; + Ok(RecordWriteOutcome::Written) +} + +pub(super) fn delete( + store: &PersistentKvStore, + key: &str, + partition: &RecordPartition, +) -> Result<(), KvError> { + validate_partition(store, key, partition)?; + let write = store + .db + .begin_write() + .map_err(|_error| KvError::Unavailable)?; + { + let mut rows = write + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + if rows + .remove(key) + .map_err(|_error| KvError::Unavailable)? + .is_none() + { + return Ok(()); + } + } + write.commit().map_err(|_error| KvError::MutationUncertain) +} + +pub(super) fn ordinary_lookup( + store: &PersistentKvStore, + key: &str, +) -> Result, KvError> { + let partition = binding(store)?.layout.resolve(key)?; + Ok(lookup(store, key, &partition)?.map(|record| record.body)) +} + +pub(super) fn ordinary_put( + store: &PersistentKvStore, + key: &str, + body: &[u8], + ttl: Option, +) -> Result<(), KvError> { + let partition = binding(store)?.layout.resolve(key)?; + match insert( + store, + key, + &partition, + RecordWrite { + body, + metadata: None, + ttl, + mode: RecordWriteMode::Overwrite, + }, + )? { + RecordWriteOutcome::Written => Ok(()), + RecordWriteOutcome::PreconditionFailed => Err(KvError::Unavailable), + } +} + +pub(super) fn ordinary_delete(store: &PersistentKvStore, key: &str) -> Result<(), KvError> { + let partition = binding(store)?.layout.resolve(key)?; + delete(store, key, &partition) +} + +fn cleanup_expired(store: &PersistentKvStore, keys: &[String]) -> Result<(), KvError> { + if keys.is_empty() { + return Ok(()); + } + let write = store + .db + .begin_write() + .map_err(|_error| KvError::Unavailable)?; + { + let mut table = write + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + for key in keys { + let expired = table + .get(key.as_str()) + .map_err(|_error| KvError::Unavailable)? + .is_some_and(|row| PersistentKvStore::is_expired(row.value().3)); + if expired { + table + .remove(key.as_str()) + .map_err(|_error| KvError::Unavailable)?; + } + } + } + write.commit().map_err(|_error| KvError::Unavailable) +} + +pub(super) fn page( + store: &PersistentKvStore, + prefix: &str, + cursor: Option<&str>, + limit: usize, + mode: &str, +) -> Result { + if prefix.len() > KvHandle::MAX_KEY_SIZE + || prefix.chars().any(char::is_control) + || limit == 0 + || limit > KvHandle::MAX_LIST_PAGE_SIZE + { + return Err(invalid("invalid record listing arguments")); + } + let binding = binding(store)?; + let after = cursor + .map(|text| { + let token: Cursor = serde_json::from_str(text) + .map_err(|_error| invalid("invalid persistent record cursor"))?; + if token.scope != binding.scope + || token.layout != binding.layout.id() + || token.version != binding.layout.version() + || token.prefix != prefix + || token.mode != mode + || !token.after.starts_with(prefix) + { + return Err(invalid("persistent record cursor does not match the query")); + } + binding.layout.resolve(&token.after)?; + Ok(token.after) + }) + .transpose()?; + let read = store + .db + .begin_read() + .map_err(|_error| KvError::Unavailable)?; + let table = read + .open_table(ROWS) + .map_err(|_error| KvError::Unavailable)?; + let mut range = match after.as_deref() { + Some(position) => table.range::<&str>((Bound::Excluded(position), Bound::Unbounded)), + None => table.range(prefix..), + } + .map_err(|_error| KvError::Unavailable)?; + let mut keys = Vec::with_capacity(limit.saturating_add(1)); + let mut expired = Vec::new(); + let mut scanned = 0; + let mut last = None; + let mut complete = false; + let scan_limit = PersistentKvStore::LIST_SCAN_BATCH_SIZE * PersistentKvStore::MAX_SCAN_BATCHES; + while scanned < scan_limit && keys.len() <= limit { + let Some(entry) = range.next() else { + complete = true; + break; + }; + let (stored_key, row) = entry.map_err(|_error| KvError::Unavailable)?; + let key = stored_key.value(); + if !key.starts_with(prefix) { + complete = true; + break; + } + binding.layout.resolve(key)?; + last = Some(key.to_owned()); + scanned = scanned.saturating_add(1); + if PersistentKvStore::is_expired(row.value().3) { + expired.push(key.to_owned()); + } else { + keys.push(key.to_owned()); + } + } + drop(range); + drop(table); + drop(read); + let next = if keys.len() > limit { + keys.truncate(limit); + keys.last().cloned() + } else if complete { + None + } else { + last + }; + let next_cursor = next + .map(|last_key| { + serde_json::to_string(&Cursor { + after: last_key, + layout: binding.layout.id().to_owned(), + mode: mode.to_owned(), + prefix: prefix.to_owned(), + scope: binding.scope.clone(), + version: binding.layout.version(), + }) + .map_err(|_error| KvError::Unavailable) + }) + .transpose()?; + cleanup_expired(store, &expired)?; + Ok(RecordPage { + keys, + cursor: next_cursor, + }) +} + +pub(super) fn ordinary_page( + store: &PersistentKvStore, + prefix: &str, + cursor: Option<&str>, + limit: usize, +) -> Result { + let page = page(store, prefix, cursor, limit, "ordinary")?; + Ok(KvPage { + keys: page.keys, + cursor: page.cursor, + }) +} + +#[cfg(test)] +mod tests { + use std::{ + path::Path, + sync::{Arc, Barrier}, + thread::{sleep, spawn}, + }; + + use edgezero_core::key_value_store::KvStore as _; + use edgezero_core::record_store::{ + RecordKeyLayout, RecordLayout, contract::assert_record_contract, + }; + use futures::executor::block_on; + + use super::*; + + struct Layout; + + impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "persistent-test" + } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + key.find('/') + .and_then(|split| key.get(..=split)) + .ok_or_else(|| invalid("missing group")) + } + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + match prefix.strip_suffix('/') { + Some(group) if !group.is_empty() && !group.contains('/') => Ok(()), + _ => Err(invalid("invalid group")), + } + } + fn version(&self) -> u32 { + 1 + } + } + + fn layout() -> RecordLayout { + RecordLayout::new(Layout).expect("should validate layout") + } + + fn open(path: &Path) -> KvHandle { + KvHandle::new(Arc::new( + PersistentKvStore::new_records(path, layout()).expect("should open records"), + )) + } + + #[test] + fn ordinary_overwrite_shares_row_clears_metadata_and_invalidates_generation() { + let dir = tempfile::tempdir().expect("should create directory"); + let kv = open(&dir.path().join("records.redb")); + let records = kv.require_records().expect("should require records"); + block_on(async { + assert_eq!( + records + .insert( + "a/key", + RecordWrite { + body: b"one", + metadata: Some("json"), + ttl: None, + mode: RecordWriteMode::Add + } + ) + .await + .expect("should insert"), + RecordWriteOutcome::Written + ); + let old = records + .lookup("a/key") + .await + .expect("should read") + .expect("should find row"); + kv.put_bytes("a/key", Bytes::from_static(b"two")) + .await + .expect("should overwrite"); + let new = records + .lookup("a/key") + .await + .expect("should read") + .expect("should find row"); + assert_eq!(new.body, Bytes::from_static(b"two")); + assert_eq!(new.metadata, None); + assert_ne!(old.generation, new.generation); + assert_eq!( + records + .insert( + "a/key", + RecordWrite { + body: b"stale", + metadata: None, + ttl: None, + mode: RecordWriteMode::IfGenerationMatch(old.generation) + } + ) + .await + .expect("should reject stale token"), + RecordWriteOutcome::PreconditionFailed + ); + assert_eq!( + kv.get_bytes("a/key").await.expect("should read ordinary"), + Some(Bytes::from_static(b"two")) + ); + }); + } + + #[test] + fn generation_history_survives_delete_and_reopen() { + let dir = tempfile::tempdir().expect("should create directory"); + let path = dir.path().join("records.redb"); + let old = { + let kv = open(&path); + let records = kv.require_records().expect("should require records"); + block_on(async { + kv.put_bytes("a/key", Bytes::from_static(b"one")) + .await + .expect("should write"); + let old = records + .lookup("a/key") + .await + .expect("should read") + .expect("should find row") + .generation; + records.delete("a/key").await.expect("should delete"); + old + }) + }; + let kv = open(&path); + let records = kv.require_records().expect("should require records"); + block_on(async { + kv.put_bytes("a/key", Bytes::from_static(b"two")) + .await + .expect("should recreate"); + assert_ne!( + records + .lookup("a/key") + .await + .expect("should read") + .expect("should find row") + .generation, + old + ); + assert_eq!( + records + .insert( + "a/key", + RecordWrite { + body: b"stale", + metadata: None, + ttl: None, + mode: RecordWriteMode::IfGenerationMatch(old) + } + ) + .await + .expect("should reject stale token"), + RecordWriteOutcome::PreconditionFailed + ); + }); + } + + #[test] + fn record_and_global_pages_have_complete_separate_continuation() { + let dir = tempfile::tempdir().expect("should create directory"); + let kv = open(&dir.path().join("records.redb")); + let records = kv.require_records().expect("should require records"); + block_on(async { + for key in ["a/one", "a/two", "b/three", "b/four"] { + kv.put_bytes(key, Bytes::from_static(b"value")) + .await + .expect("should insert"); + } + let partition = records.partition("a/").expect("should resolve group"); + let mut keys = Vec::new(); + let mut cursor = None; + loop { + let page = records + .list_partition(&partition, cursor.as_deref(), 1) + .await + .expect("should list group"); + keys.extend(page.keys); + cursor = page.cursor; + if cursor.is_none() { + break; + } + } + assert_eq!(keys, vec!["a/one", "a/two"]); + let first = kv + .list_keys_page("", None, 1) + .await + .expect("should list globally"); + records + .list_partition(&partition, first.cursor.as_deref(), 1) + .await + .expect_err("global cursor must not cross into partition traversal"); + let mut global_keys = first.keys; + let mut global_cursor = first.cursor; + while global_cursor.is_some() { + let page = kv + .list_keys_page("", global_cursor.as_deref(), 1) + .await + .expect("should continue globally"); + global_keys.extend(page.keys); + global_cursor = page.cursor; + } + global_keys.sort(); + assert_eq!(global_keys, vec!["a/one", "a/two", "b/four", "b/three"]); + }); + } + + #[test] + fn delayed_expiry_cleanup_preserves_a_fresh_overwrite() { + let dir = tempfile::tempdir().expect("should create directory"); + let store = PersistentKvStore::new_records(dir.path().join("records.redb"), layout()) + .expect("should open store"); + ordinary_put(&store, "a/key", b"old", None).expect("should write original"); + let partition = layout().resolve("a/key").expect("should resolve key"); + let generation = lookup(&store, "a/key", &partition) + .expect("should read original") + .expect("should find original") + .generation; + let write = store.db.begin_write().expect("should begin expiry fixture"); + let mut table = write.open_table(ROWS).expect("should open rows"); + table + .insert( + "a/key", + (b"old".as_slice(), None, generation.as_u64(), Some(0)), + ) + .expect("should make original row expired"); + drop(table); + write.commit().expect("should commit expiry fixture"); + // A reader has observed the expired key but has not acquired its cleanup + // transaction yet. A different writer replaces it in that interval. + let observed_expired = vec!["a/key".to_owned()]; + ordinary_put(&store, "a/key", b"fresh", None).expect("should overwrite old expiry"); + cleanup_expired(&store, &observed_expired).expect("should recheck current expiry"); + assert_eq!( + ordinary_lookup(&store, "a/key").expect("should read fresh value"), + Some(Bytes::from_static(b"fresh")) + ); + } + + #[test] + fn public_handle_shared_record_contract() { + let dir = tempfile::tempdir().expect("should create directory"); + let kv = open(&dir.path().join("records.redb")); + block_on(assert_record_contract(&kv, "a/")); + } + + fn race(kv: &KvHandle, mode: RecordWriteMode) -> Vec { + let barrier = Arc::new(Barrier::new(3)); + let writers: Vec<_> = [b"left".as_slice(), b"right".as_slice()] + .into_iter() + .map(|body| { + let writer_kv = kv.clone(); + let writer_barrier = Arc::clone(&barrier); + spawn(move || { + let records = writer_kv.require_records().expect("should require records"); + writer_barrier.wait(); + block_on(records.insert( + "a/race", + RecordWrite { + body, + metadata: None, + ttl: None, + mode, + }, + )) + .expect("should complete conditional write") + }) + }) + .collect(); + barrier.wait(); + writers + .into_iter() + .map(|writer| writer.join().expect("should join writer")) + .collect() + } + + #[test] + fn independent_writers_have_one_add_and_one_cas_winner() { + let dir = tempfile::tempdir().expect("should create directory"); + let kv = open(&dir.path().join("records.redb")); + let records = kv.require_records().expect("should require records"); + let outcomes = race(&kv, RecordWriteMode::Add); + assert_eq!( + outcomes + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::Written) + .count(), + 1 + ); + assert_eq!( + outcomes + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::PreconditionFailed) + .count(), + 1 + ); + let generation = block_on(records.lookup("a/race")) + .expect("should read") + .expect("should find row") + .generation; + let cas_outcomes = race(&kv, RecordWriteMode::IfGenerationMatch(generation)); + assert_eq!( + cas_outcomes + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::Written) + .count(), + 1 + ); + assert_eq!( + cas_outcomes + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::PreconditionFailed) + .count(), + 1 + ); + } + + #[test] + fn missing_allocator_state_is_not_recreated_after_all_values_were_deleted() { + let dir = tempfile::tempdir().expect("should create directory"); + let path = dir.path().join("records.redb"); + let store = PersistentKvStore::new_records(&path, layout()).expect("should open"); + ordinary_put(&store, "a/key", b"one", None).expect("should write"); + ordinary_delete(&store, "a/key").expect("should delete value only"); + let write = store + .db + .begin_write() + .expect("should begin corruption fixture"); + assert!( + write + .delete_table(STATE) + .expect("should remove allocator fixture") + ); + write.commit().expect("should commit corruption fixture"); + drop(store); + assert!(PersistentKvStore::new_records(&path, layout()).is_err()); + } + + #[test] + fn explicit_record_format_does_not_convert_ordinary_rows() { + let dir = tempfile::tempdir().expect("should create directory"); + let path = dir.path().join("records.redb"); + let ordinary = PersistentKvStore::new(&path).expect("should open ordinary store"); + block_on(ordinary.put_bytes("a/key", Bytes::from_static(b"ordinary"))) + .expect("should write ordinary row"); + drop(ordinary); + let records = PersistentKvStore::new_records(&path, layout()) + .expect("should open explicit record format"); + assert_eq!( + ordinary_lookup(&records, "a/key").expect("should read fresh record format"), + None + ); + ordinary_put(&records, "a/key", b"record", None).expect("should write record row"); + drop(records); + let reopened = PersistentKvStore::new(&path).expect("should reopen ordinary format"); + assert_eq!( + block_on(reopened.get_bytes("a/key")).expect("should preserve ordinary row"), + Some(Bytes::from_static(b"ordinary")) + ); + } + + #[test] + fn empty_expired_page_keeps_continuation_and_cleanup_cannot_reset_generation() { + let dir = tempfile::tempdir().expect("should create directory"); + let store = PersistentKvStore::new_records(dir.path().join("records.redb"), layout()) + .expect("should open"); + let partition = layout().partition("a/").expect("should group"); + for index in 0_usize..40_usize { + insert( + &store, + &format!("a/{index:03}"), + &partition, + RecordWrite { + body: b"old", + metadata: None, + ttl: Some(Duration::from_millis(1)), + mode: RecordWriteMode::Add, + }, + ) + .expect("should insert expired fixture"); + } + let old_generation = { + let read = store + .db + .begin_read() + .expect("should inspect expiry fixture"); + let table = read.open_table(ROWS).expect("should open expired rows"); + let row = table + .get("a/000") + .expect("should read old row") + .expect("should retain expired row until cleanup"); + RecordGeneration::from_u64(row.value().2) + }; + sleep(Duration::from_millis(20)); + assert!( + !exists(&store, "a/000", &partition).expect("should check native expiry visibility"), + "strong exact existence must agree with this provider's expiry-aware partition listing" + ); + ordinary_put(&store, "a/zzz", b"live", None).expect("should insert live tail"); + let first = page(&store, "a/", None, 1, "partition").expect("should scan first page"); + assert!(first.keys.is_empty()); + assert!(first.cursor.is_some()); + let second = + page(&store, "a/", first.cursor.as_deref(), 1, "partition").expect("should continue"); + assert_eq!(second.keys, vec!["a/zzz"]); + assert!( + second.cursor.is_none(), + "the live tail completes this traversal" + ); + let kv = KvHandle::new(Arc::new(store)); + let records = kv.require_records().expect("should require records"); + block_on(async { + assert_eq!( + records + .insert( + "a/000", + RecordWrite { + body: b"recreated", + metadata: None, + ttl: None, + mode: RecordWriteMode::Add, + } + ) + .await + .expect("should recreate expired value"), + RecordWriteOutcome::Written + ); + let recreated = records + .lookup("a/000") + .await + .expect("should read recreated value") + .expect("should find recreated value"); + assert_ne!(recreated.generation, old_generation); + assert_eq!( + records + .insert( + "a/000", + RecordWrite { + body: b"stale", + metadata: None, + ttl: None, + mode: RecordWriteMode::IfGenerationMatch(old_generation), + } + ) + .await + .expect("should reject stale generation after expiry"), + RecordWriteOutcome::PreconditionFailed + ); + }); + } +} diff --git a/crates/edgezero-adapter-axum/src/native_bindings.rs b/crates/edgezero-adapter-axum/src/native_bindings.rs index 8838a25a..4ebbb555 100644 --- a/crates/edgezero-adapter-axum/src/native_bindings.rs +++ b/crates/edgezero-adapter-axum/src/native_bindings.rs @@ -95,8 +95,10 @@ pub struct NativeProviderCapabilities { pub appconfig_agent: bool, pub secrets_manager: bool, pub dynamodb: bool, + pub dynamodb_records: bool, pub celld_config: bool, pub celld_kv: bool, + pub celld_records: bool, } /// Strict configuration dispatch. Each provider validates its own settings. @@ -115,6 +117,10 @@ pub enum NativeConfigBinding { pub enum NativeKvBinding { #[serde(rename = "aws-dynamodb")] DynamoDb(DynamoDbKvSettings), + #[serde(rename = "aws-dynamodb-records")] + DynamoDbRecords(DynamoDbKvSettings), + #[serde(rename = "celld-records")] + CelldRecords(CelldKvSettings), #[serde(rename = "celld-kv")] Celld(CelldKvSettings), } @@ -371,13 +377,14 @@ impl NativeStoreBindings { let mut celld_mappings = 0_u64; for binding in self.kv.values() { let counter = match binding { - NativeKvBinding::DynamoDb(settings) => { + NativeKvBinding::DynamoDb(settings) + | NativeKvBinding::DynamoDbRecords(settings) => { settings .validate() .map_err(|_category| NativeBindingsError::Invalid)?; &mut aws_mappings } - NativeKvBinding::Celld(settings) => { + NativeKvBinding::Celld(settings) | NativeKvBinding::CelldRecords(settings) => { settings .validate() .map_err(|_category| NativeBindingsError::Invalid)?; @@ -680,7 +687,9 @@ impl<'declarations> NativeSourceSelection<'declarations> { }) || (!bindings.secrets.is_empty() && !capabilities.secrets_manager) || bindings.kv.values().any(|binding| match binding { NativeKvBinding::DynamoDb(_) => !capabilities.dynamodb, + NativeKvBinding::DynamoDbRecords(_) => !capabilities.dynamodb_records, NativeKvBinding::Celld(_) => !capabilities.celld_kv, + NativeKvBinding::CelldRecords(_) => !capabilities.celld_records, }) { return Err(NativeBindingsError::UnsupportedProvider); @@ -804,6 +813,73 @@ mod tests { const CELLD: &str = "version=1\n[config.app]\nprovider='celld-config'\nendpoint='http://127.0.0.1:8788'\ndefault_key='settings'\n[config.app.access]\nmode='trusted-network'\n[config.app.documents.settings]\nstore='config-fixture'\nkey='settings'\n[kv.state]\nprovider='celld-kv'\nendpoint='http://127.0.0.1:8788'\nstore='state-fixture'\n[kv.state.access]\nmode='trusted-network'\n"; + #[test] + fn record_binding_tags_require_their_own_compiled_capability() -> Result<(), NativeBindingsError> + { + let limits = NativeBindingsLimits::default(); + let metadata = StoresMetadata { + kv: Some(StoreMetadata { + ids: &["state"], + default: "state", + }), + ..StoresMetadata::default() + }; + for (provider, settings, records) in [ + ( + "aws-dynamodb-records", + "region='us-east-1'\ntable='edgezero-kv'\nnamespace='records'", + NativeProviderCapabilities { + dynamodb_records: true, + ..Default::default() + }, + ), + ( + "celld-records", + "endpoint='http://127.0.0.1:8788'\nstore='state-fixture'\n[kv.state.access]\nmode='trusted-network'", + NativeProviderCapabilities { + celld_records: true, + ..Default::default() + }, + ), + ] { + let input = format!("version=1\n[kv.state]\nprovider='{provider}'\n{settings}"); + let document = NativeStoreBindings::parse(&input, limits)?; + let raw_only = NativeProviderCapabilities { + dynamodb: true, + celld_kv: true, + ..Default::default() + }; + assert_eq!( + NativeSourceSelection::validate( + metadata, + &document, + &NativeSuppliedIds::default(), + &EnvConfig::default(), + raw_only, + limits, + None + ) + .err() + .expect("record provider requires its own compiled capability"), + NativeBindingsError::UnsupportedProvider, + ); + NativeSourceSelection::validate( + metadata, + &document, + &NativeSuppliedIds::default(), + &EnvConfig::default(), + records, + limits, + None, + )?; + assert!( + NativeStoreBindings::parse(&format!("{input}\nunknown_record_option=true"), limits) + .is_err() + ); + } + Ok(()) + } + #[test] fn celld_schema_is_structural_strict_and_settings_only() -> Result<(), NativeBindingsError> { let limits = NativeBindingsLimits::default(); @@ -876,6 +952,7 @@ mod tests { celld_config: mask & 4 != 0, celld_kv: mask & 8 != 0, secrets_manager: false, + ..Default::default() }; let accepted = (!aws_config || caps.appconfig_agent) && (!aws_state || caps.dynamodb) diff --git a/crates/edgezero-adapter-axum/src/native_build.rs b/crates/edgezero-adapter-axum/src/native_build.rs index d76d6239..1be31325 100644 --- a/crates/edgezero-adapter-axum/src/native_build.rs +++ b/crates/edgezero-adapter-axum/src/native_build.rs @@ -190,6 +190,30 @@ mod tests { values.iter().map(|value| (*value).to_owned()).collect() } + #[test] + fn record_provider_features_are_explicit_and_preserve_native_cargo_controls() { + let defaults = NativeCargoSelection::normalize_args(&[], &[]).unwrap(); + assert!(defaults.requested_features.is_empty()); + let selected = NativeCargoSelection::normalize_args( + &strings(&["aws-dynamodb-records"]), + &strings(&["--no-default-features", "--features", "celld-records"]), + ) + .unwrap(); + assert_eq!( + selected.requested_features, + strings(&["aws-dynamodb-records", "celld-records"]) + ); + assert!(selected.no_default_features); + assert_eq!( + selected.cargo_args(), + strings(&[ + "--no-default-features", + "--features", + "aws-dynamodb-records,celld-records" + ]) + ); + } + #[test] fn controls_after_separator_are_runtime_args() { let selection = NativeCargoSelection::normalize_args( diff --git a/crates/edgezero-adapter-axum/src/native_stores.rs b/crates/edgezero-adapter-axum/src/native_stores.rs index b283dbf8..7bf76c5f 100644 --- a/crates/edgezero-adapter-axum/src/native_stores.rs +++ b/crates/edgezero-adapter-axum/src/native_stores.rs @@ -12,7 +12,7 @@ use std::future::Future; use std::pin::Pin; use edgezero_core::app::StoresMetadata; -use edgezero_core::key_value_store::KvHandle; +use edgezero_core::key_value_store::{KvError, KvHandle}; use edgezero_core::store_registry::{BoundSecretStore, ConfigStoreBinding}; #[cfg(any( feature = "aws-appconfig-agent", @@ -22,6 +22,7 @@ use edgezero_core::store_registry::{BoundSecretStore, ConfigStoreBinding}; feature = "celld-kv" ))] use edgezero_core::time::MonotonicClock; +use edgezero_core::{RecordLayout, RecordStoreRequirements}; #[cfg(any( feature = "aws-appconfig-agent", feature = "aws-secrets-manager", @@ -91,6 +92,11 @@ pub(crate) enum NativeConfigSource { pub(crate) enum NativeKvSource { DynamoDb(DynamoDbKvSettings), + DynamoDbRecords(DynamoDbKvSettings), + CelldRecords { + settings: CelldKvSettings, + token: Option, + }, Celld { settings: CelldKvSettings, token: Option, @@ -305,8 +311,10 @@ impl NativeSourcePlan { appconfig_agent: cfg!(feature = "aws-appconfig-agent"), secrets_manager: cfg!(feature = "aws-secrets-manager"), dynamodb: cfg!(feature = "aws-dynamodb"), + dynamodb_records: cfg!(feature = "aws-dynamodb-records"), celld_config: cfg!(feature = "celld-config"), celld_kv: cfg!(feature = "celld-kv"), + celld_records: cfg!(feature = "celld-records"), }, options.bindings_limits(), options.preparation_policies(), @@ -347,23 +355,11 @@ impl NativeSourcePlan { }, )?, kv: resolve(selection.kv, overrides.kv, "kv", |id| { - match bindings + let binding = bindings .kv() .get(id) - .ok_or_else(|| failure("kv", id, "invalid binding"))? - { - NativeKvBinding::DynamoDb(settings) => { - Ok(NativeKvSource::DynamoDb(settings.clone())) - } - NativeKvBinding::Celld(settings) => Ok(NativeKvSource::Celld { - token: settings - .access - .token_env() - .map(|name| options.celld_token(name)) - .transpose()?, - settings: settings.clone(), - }), - } + .ok_or_else(|| failure("kv", id, "invalid binding"))?; + native_kv_source(binding, options) })?, secrets: resolve(selection.secrets, overrides.secrets, "secrets", |id| { bindings @@ -384,6 +380,99 @@ impl NativeSourcePlan { Ok(plan) } + /// Validate all record declarations before sessions, futures or local stores open. + /// A missing declaration retains ordinary constructor/format semantics. Aliases + /// cannot inherit a record layout without silently redirecting their value table. + pub(crate) fn validate_record_requirements( + &self, + requirements: &RecordStoreRequirements, + options: &AxumRunOptions, + ) -> anyhow::Result<()> { + for id in requirements.keys() { + if !self.kv.contains_key(id.as_str()) { + return Err(failure( + "kv", + id, + "record declaration uses an undeclared ID", + )); + } + } + let mut scopes = BTreeMap::new(); + for (id, source) in &self.kv { + let layout = requirements.get(*id); + let scope = match source { + NativeStoreSource::BuiltIn( + NativeKvSource::DynamoDb(_) | NativeKvSource::Celld { .. }, + ) if layout.is_some() => { + return Err(failure( + "kv", + id, + "raw binding does not support declared records", + )); + } + NativeStoreSource::BuiltIn(NativeKvSource::DynamoDbRecords(settings)) => { + if layout.is_none() { + return Err(failure( + "kv", + id, + "record binding requires an application layout", + )); + } + Some(vec![ + "dynamodb-records".to_owned(), + settings.region.clone(), + settings.table.clone(), + settings.namespace.clone(), + ]) + } + NativeStoreSource::BuiltIn(NativeKvSource::CelldRecords { settings, .. }) => { + if layout.is_none() { + return Err(failure( + "kv", + id, + "record binding requires an application layout", + )); + } + Some(vec![ + "celld-records".to_owned(), + settings.endpoint.clone(), + settings.store.clone(), + ]) + } + NativeStoreSource::Local => Some(vec![ + "local".to_owned(), + options.env_config().store_name("kv", id), + ]), + NativeStoreSource::Ready(handle) if layout.is_some() => { + let records = handle + .require_records() + .map_err(|error| failure("kv", id, record_capability_failure(&error)))?; + if layout != Some(&records.binding().layout) { + return Err(failure("kv", id, "supplied record layout mismatch")); + } + None + } + NativeStoreSource::Ready(_) + | NativeStoreSource::Prepare(_) + | NativeStoreSource::BuiltIn( + NativeKvSource::DynamoDb(_) | NativeKvSource::Celld { .. }, + ) => None, + }; + if let Some(backing_scope) = scope + && scopes + .insert(backing_scope, layout) + .is_some_and(|previous| previous != layout) + { + return Err(failure( + "kv", + id, + "shared backing store has conflicting record layouts", + )); + } + } + Ok(()) + } + #[cfg_attr( not(any( feature = "aws-appconfig-agent", @@ -412,7 +501,9 @@ impl NativeSourcePlan { }) || self.kv.values().any(|source| { matches!( source, - NativeStoreSource::BuiltIn(NativeKvSource::DynamoDb(_)) + NativeStoreSource::BuiltIn( + NativeKvSource::DynamoDb(_) | NativeKvSource::DynamoDbRecords(_) + ) ) }) || self .secrets @@ -444,7 +535,9 @@ impl NativeSourcePlan { }) || self.kv.values().any(|source| { matches!( source, - NativeStoreSource::BuiltIn(NativeKvSource::Celld { .. }) + NativeStoreSource::BuiltIn( + NativeKvSource::Celld { .. } | NativeKvSource::CelldRecords { .. } + ) ) }) { Some( @@ -463,6 +556,53 @@ impl NativeSourcePlan { } } +fn native_kv_source( + binding: &NativeKvBinding, + options: &AxumRunOptions, +) -> anyhow::Result { + match binding { + NativeKvBinding::DynamoDb(settings) => Ok(NativeKvSource::DynamoDb(settings.clone())), + NativeKvBinding::DynamoDbRecords(settings) => { + Ok(NativeKvSource::DynamoDbRecords(settings.clone())) + } + NativeKvBinding::Celld(settings) | NativeKvBinding::CelldRecords(settings) => { + let token = settings + .access + .token_env() + .map(|name| options.celld_token(name)) + .transpose()?; + let cloned_settings = settings.clone(); + if matches!(binding, NativeKvBinding::CelldRecords(_)) { + Ok(NativeKvSource::CelldRecords { + settings: cloned_settings, + token, + }) + } else { + Ok(NativeKvSource::Celld { + settings: cloned_settings, + token, + }) + } + } + } +} + +#[inline] +pub(crate) fn record_capability_failure(error: &KvError) -> &'static str { + match error { + KvError::Unsupported { .. } => "record operations unsupported", + KvError::Validation(_) => "invalid record binding", + KvError::Internal(_) + | KvError::LimitExceeded { .. } + | KvError::MutationUncertain + | KvError::NotFound { .. } + | KvError::OpenFailed { .. } + | KvError::Serialization(_) + | KvError::Unavailable + | _ => "record capability acquisition failed", + } +} + fn resolve( selection: BTreeMap<&'static str, NativeBindingSource>, mut supplied: BTreeMap>, @@ -592,8 +732,46 @@ impl NativePreparationSessions { &mut self, builtin: NativeKvSource, id: &str, + declared_layout: Option<&RecordLayout>, ) -> anyhow::Result { match builtin { + NativeKvSource::DynamoDbRecords(settings) => { + let layout = + declared_layout.ok_or_else(|| failure("kv", id, "record layout required"))?; + #[cfg(feature = "aws-dynamodb-records")] + { + self.aws + .as_mut() + .ok_or_else(|| failure("kv", id, "invalid preparation session"))? + .prepare_dynamodb_records(&settings, layout) + .await + .map_err(|category| failure("kv", id, &category.to_string())) + } + #[cfg(not(feature = "aws-dynamodb-records"))] + { + let _: (DynamoDbKvSettings, &RecordLayout) = (settings, layout); + Err(failure("kv", id, "record provider not compiled")) + } + } + NativeKvSource::CelldRecords { settings, token } => { + let layout = + declared_layout.ok_or_else(|| failure("kv", id, "record layout required"))?; + #[cfg(feature = "celld-records")] + { + self.celld + .as_mut() + .ok_or_else(|| failure("kv", id, "invalid preparation session"))? + .prepare_record_store(&settings, layout, token.as_ref()) + .await + .map_err(|category| failure("kv", id, &category.to_string())) + } + #[cfg(not(feature = "celld-records"))] + { + let _: (CelldKvSettings, &RecordLayout, Option) = + (settings, layout, token); + Err(failure("kv", id, "record provider not compiled")) + } + } NativeKvSource::DynamoDb(settings) => { #[cfg(feature = "aws-dynamodb")] { diff --git a/crates/edgezero-adapter-axum/src/templates/Cargo.toml.hbs b/crates/edgezero-adapter-axum/src/templates/Cargo.toml.hbs index 59728e4f..bf2a14ea 100644 --- a/crates/edgezero-adapter-axum/src/templates/Cargo.toml.hbs +++ b/crates/edgezero-adapter-axum/src/templates/Cargo.toml.hbs @@ -31,5 +31,7 @@ aws-appconfig-agent = ["edgezero-adapter-axum/aws-appconfig-agent"] aws-secrets-manager = ["edgezero-adapter-axum/aws-secrets-manager"] aws-stores = ["aws-appconfig-agent", "aws-secrets-manager"] aws-dynamodb = ["edgezero-adapter-axum/aws-dynamodb"] +aws-dynamodb-records = ["edgezero-adapter-axum/aws-dynamodb-records"] celld-config = ["edgezero-adapter-axum/celld-config"] celld-kv = ["edgezero-adapter-axum/celld-kv"] +celld-records = ["edgezero-adapter-axum/celld-records"] diff --git a/crates/edgezero-adapter-cloudflare/src/lib.rs b/crates/edgezero-adapter-cloudflare/src/lib.rs index 82075e69..4e329d2e 100644 --- a/crates/edgezero-adapter-cloudflare/src/lib.rs +++ b/crates/edgezero-adapter-cloudflare/src/lib.rs @@ -39,6 +39,8 @@ use edgezero_core::context::RuntimeVariables; use edgezero_core::env_config::EnvConfig; #[cfg(any(test, all(feature = "cloudflare", target_arch = "wasm32")))] use edgezero_core::error::EdgeError; +#[cfg(any(test, all(feature = "cloudflare", target_arch = "wasm32")))] +use edgezero_core::key_value_store::KvError; #[cfg(all(feature = "cloudflare", target_arch = "wasm32"))] use edgezero_core::manifest::BakedManifest; #[cfg(any(test, all(feature = "cloudflare", target_arch = "wasm32")))] @@ -74,7 +76,24 @@ fn build_target_app() -> Result where A: Hooks, { - App::build::(CLOUDFLARE_PLATFORM) + let app = App::build::(CLOUDFLARE_PLATFORM)?; + let metadata = A::stores().kv; + if app + .record_store_requirements() + .keys() + .any(|id| !metadata.is_some_and(|meta| meta.ids.contains(&id.as_str()))) + { + return Err( + KvError::Validation("record declaration uses an undeclared ID".to_owned()).into(), + ); + } + if !app.record_store_requirements().is_empty() { + return Err(KvError::Unsupported { + operation: "records".to_owned(), + } + .into()); + } + Ok(app) } /// # Errors @@ -244,6 +263,78 @@ where .await } +#[cfg(test)] +mod record_requirement_tests { + use edgezero_core::app::{App, Hooks, StoreMetadata, StoresMetadata}; + use edgezero_core::http::StatusCode; + use edgezero_core::key_value_store::KvError; + use edgezero_core::router::RouterService; + use edgezero_core::{EdgeError, RecordKeyLayout}; + + struct Layout; + impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "platform.record-test" + } + + fn partition_prefix<'key>(&self, _key: &'key str) -> Result<&'key str, KvError> { + Ok("group/") + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix == "group/" { + Ok(()) + } else { + Err(KvError::Validation("incomplete partition".to_owned())) + } + } + + fn version(&self) -> u32 { + 1 + } + } + + struct Required; + #[expect( + clippy::missing_trait_methods, + reason = "record startup fixture retains unrelated Hooks defaults" + )] + impl Hooks for Required { + fn configure(app: &mut App) -> Result<(), EdgeError> { + app.require_record_store("identity", Layout)?; + Ok(()) + } + fn routes() -> Result { + Ok(RouterService::builder().build()) + } + fn stores() -> StoresMetadata { + StoresMetadata { + kv: DECLARED.then_some(StoreMetadata { + ids: &["identity"], + default: "identity", + }), + ..StoresMetadata::default() + } + } + } + + #[test] + fn record_requirement_refuses_unsupported_platform_before_store_preparation() { + let error = super::build_target_app::>() + .err() + .expect("unsupported records"); + assert_eq!(error.status(), StatusCode::NOT_IMPLEMENTED); + let undeclared = super::build_target_app::>() + .err() + .expect("undeclared record ID"); + assert_eq!(undeclared.status(), StatusCode::BAD_REQUEST); + } +} + #[cfg(test)] mod platform_tests { use edgezero_core::app::{App, Hooks}; diff --git a/crates/edgezero-adapter-cloudflare/src/request.rs b/crates/edgezero-adapter-cloudflare/src/request.rs index 58835f92..12ab39fa 100644 --- a/crates/edgezero-adapter-cloudflare/src/request.rs +++ b/crates/edgezero-adapter-cloudflare/src/request.rs @@ -840,6 +840,22 @@ where core_request.extensions_mut().insert(variables); } let (config_registry, kv_registry, secret_registry) = synthesise_store_registries(stores); + if !app.record_store_requirements().is_empty() { + let validation = kv_registry + .as_ref() + .ok_or_else(|| { + edgezero_core::key_value_store::KvError::Validation( + "required record registry missing".to_owned(), + ) + }) + .and_then(|registry| { + registry.validate_record_requirements(app.record_store_requirements()) + }); + if let Err(error) = validation { + return deliver(app.admitted_error_egress(prepared, error.into())) + .map_err(|error| edge_error_to_worker(&error)); + } + } if let Some(registry) = config_registry { core_request.extensions_mut().insert(registry); } diff --git a/crates/edgezero-adapter-fastly/src/key_value_store.rs b/crates/edgezero-adapter-fastly/src/key_value_store.rs index b7551cb3..97270654 100644 --- a/crates/edgezero-adapter-fastly/src/key_value_store.rs +++ b/crates/edgezero-adapter-fastly/src/key_value_store.rs @@ -10,9 +10,15 @@ use async_trait::async_trait; use bytes::Bytes; use edgezero_core::key_value_store::{KvError, KvPage, KvStore, KvStoreOpenFailure}; use edgezero_core::logging::BOOT_LOG_TARGET; -use fastly::kv_store::{KVStore, KVStoreError}; +use edgezero_core::record_store::{ + Record, RecordBinding, RecordGeneration, RecordLayout, RecordPage, RecordPartition, + RecordWrite, RecordWriteMode, RecordWriteOutcome, +}; +use fastly::kv_store::{InsertMode, KVStore, KVStoreError, ListMode}; use fastly_shared::FastlyStatus; +use serde::{Deserialize, Serialize}; use std::collections::HashSet; +use std::str::from_utf8; use std::sync::{Mutex, OnceLock, PoisonError}; use std::time::Duration; @@ -20,6 +26,7 @@ use std::time::Duration; /// /// Opens compiled declarations lazily per request, or wraps an explicitly opened handle. pub struct FastlyKvStore { + records: Option, source: FastlyKvStoreSource, } @@ -31,6 +38,17 @@ enum FastlyKvStoreSource { Opened(KVStore), } +#[derive(Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +struct OrdinaryCursor { + format: String, + layout: String, + native: String, + prefix: String, + scope: String, + version: u32, +} + impl FastlyKvStore { pub(crate) fn declared(name: &'static str) -> Self { Self { @@ -38,6 +56,14 @@ impl FastlyKvStore { name, opened: OnceLock::new(), }, + records: None, + } + } + + pub(crate) fn declared_records(name: &'static str, layout: RecordLayout) -> Self { + Self { + records: Some(record_binding(name, layout)), + ..Self::declared(name) } } @@ -51,9 +77,37 @@ impl FastlyKvStore { open_result(KVStore::open(name)).map_err(|reason| KvError::OpenFailed { reason })?; Ok(Self { source: FastlyKvStoreSource::Opened(store), + records: None, + }) + } + + /// Open a record-capable Fastly store with its immutable application layout. + /// + /// # Errors + /// Returns [`KvError::OpenFailed`] when the native binding cannot be opened. + #[inline] + pub fn open_records(name: &str, layout: RecordLayout) -> Result { + Ok(Self { + records: Some(record_binding(name, layout)), + ..Self::open(name)? }) } + fn validate_partition(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + let binding = self.records.as_ref().ok_or_else(unsupported_records)?; + if binding.layout.resolve(key)? != *partition { + return Err(KvError::Validation("record partition mismatch".to_owned())); + } + Ok(()) + } + + fn validate_point(&self, key: &str) -> Result<(), KvError> { + if let Some(binding) = &self.records { + binding.layout.resolve(key)?; + } + Ok(()) + } + fn with_store( &self, operation: impl FnOnce(&KVStore) -> Result, @@ -76,7 +130,11 @@ impl FastlyKvStore { impl KvStore for FastlyKvStore { #[inline] async fn delete(&self, key: &str) -> Result<(), KvError> { + self.validate_point(key)?; self.with_store(|store| { + if self.records.is_some() { + return delete_record(store, key); + } store .delete(key) .map_err(|err| KvError::Internal(anyhow::anyhow!("delete failed: {err}"))) @@ -90,6 +148,7 @@ impl KvStore for FastlyKvStore { #[inline] async fn get_bytes(&self, key: &str) -> Result, KvError> { + self.validate_point(key)?; self.with_store(|store| match store.lookup(key) { Ok(mut response) => { let bytes = response.take_body_bytes(); @@ -107,6 +166,19 @@ impl KvStore for FastlyKvStore { cursor: Option<&str>, limit: usize, ) -> Result { + if let Some(binding) = &self.records { + let native_cursor = decode_ordinary_cursor(binding, prefix, cursor)?; + let page = self.with_store(|store| { + strong_list_page(store, prefix, native_cursor.as_deref(), limit) + })?; + for key in &page.keys { + binding.layout.resolve(key)?; + } + return Ok(KvPage { + keys: page.keys, + cursor: encode_ordinary_cursor(binding, prefix, page.cursor)?, + }); + } let limit_u32 = u32::try_from(limit) .map_err(|_e| KvError::Validation("list limit exceeds u32".to_owned()))?; @@ -134,7 +206,21 @@ impl KvStore for FastlyKvStore { #[inline] async fn put_bytes(&self, key: &str, value: Bytes) -> Result<(), KvError> { + self.validate_point(key)?; self.with_store(|store| { + if self.records.is_some() { + return insert_record( + store, + key, + RecordWrite { + body: value.as_ref(), + metadata: None, + ttl: None, + mode: RecordWriteMode::Overwrite, + }, + ) + .map(|_| ()); + } store .insert(key, value.as_ref()) .map_err(|err| KvError::Internal(anyhow::anyhow!("insert failed: {err}"))) @@ -148,7 +234,21 @@ impl KvStore for FastlyKvStore { value: Bytes, ttl: Duration, ) -> Result<(), KvError> { + self.validate_point(key)?; self.with_store(|store| { + if self.records.is_some() { + return insert_record( + store, + key, + RecordWrite { + body: value.as_ref(), + metadata: None, + ttl: Some(ttl), + mode: RecordWriteMode::Overwrite, + }, + ) + .map(|_| ()); + } store .build_insert() .time_to_live(ttl) @@ -156,6 +256,279 @@ impl KvStore for FastlyKvStore { .map_err(|err| KvError::Internal(anyhow::anyhow!("insert with ttl failed: {err}"))) }) } + + #[inline] + fn record_binding(&self) -> Option { + self.records.clone() + } + + #[inline] + async fn record_delete(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + self.validate_partition(key, partition)?; + self.with_store(|store| delete_record(store, key)) + } + + #[inline] + async fn record_exists_strong( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result { + self.validate_partition(key, partition)?; + self.with_store(|store| { + exact_exists(key, |cursor| strong_list_page(store, key, cursor, 1000)) + }) + } + + #[inline] + async fn record_insert( + &self, + key: &str, + partition: &RecordPartition, + write: RecordWrite<'_>, + ) -> Result { + self.validate_partition(key, partition)?; + self.with_store(|store| insert_record(store, key, write)) + } + + #[inline] + async fn record_list_partition( + &self, + partition: &RecordPartition, + cursor: Option<&str>, + limit: usize, + ) -> Result { + let binding = self.records.as_ref().ok_or_else(unsupported_records)?; + if binding.layout.partition(partition.prefix())? != *partition { + return Err(KvError::Validation("record partition mismatch".to_owned())); + } + let page = + self.with_store(|store| strong_list_page(store, partition.prefix(), cursor, limit))?; + for key in &page.keys { + self.validate_partition(key, partition)?; + } + Ok(RecordPage { + keys: page.keys, + cursor: page.cursor, + }) + } + + #[inline] + async fn record_lookup( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result, KvError> { + self.validate_partition(key, partition)?; + self.with_store(|store| match store.lookup(key) { + Ok(mut response) => Ok(Some(Record { + generation: RecordGeneration::from_u64(response.current_generation()), + metadata: decode_metadata(response.metadata().as_deref())?, + body: Bytes::from(response.take_body_bytes()), + })), + Err(KVStoreError::ItemNotFound) => Ok(None), + Err(_) => Err(KvError::Unavailable), + }) + } +} + +fn record_binding(name: &str, layout: RecordLayout) -> RecordBinding { + RecordBinding { + layout, + scope: name.to_owned(), + format: "fastly-native-v1".to_owned(), + max_metadata_bytes: 2000, + } +} + +fn unsupported_records() -> KvError { + KvError::Unsupported { + operation: "records".to_owned(), + } +} + +fn decode_metadata(metadata: Option<&[u8]>) -> Result, KvError> { + metadata + .filter(|bytes| !bytes.is_empty()) + .map(|bytes| { + from_utf8(bytes).map(str::to_owned).map_err(|_error| { + KvError::Internal(anyhow::anyhow!("invalid UTF-8 record metadata")) + }) + }) + .transpose() +} + +fn insert_record( + store: &KVStore, + key: &str, + write: RecordWrite<'_>, +) -> Result { + let conditional = !matches!(write.mode, RecordWriteMode::Overwrite); + let mut request = store.build_insert(); + request = match write.mode { + RecordWriteMode::Add => request.mode(InsertMode::Add), + RecordWriteMode::Overwrite => request.mode(InsertMode::Overwrite), + RecordWriteMode::IfGenerationMatch(generation) => { + request.if_generation_match(generation.as_u64()) + } + }; + if let Some(metadata) = write.metadata.filter(|text| !text.is_empty()) { + if metadata.len() > 2000 { + return Err(KvError::Validation( + "Fastly metadata exceeds 2000 bytes".to_owned(), + )); + } + request = request.metadata(metadata); + } + if let Some(ttl) = write.ttl { + request = request.time_to_live(ttl); + } + insert_outcome(request.execute(key, write.body), conditional) +} + +fn insert_outcome( + result: Result<(), KVStoreError>, + conditional: bool, +) -> Result { + match result { + Ok(()) => Ok(RecordWriteOutcome::Written), + Err(KVStoreError::ItemPreconditionFailed) if conditional => { + Ok(RecordWriteOutcome::PreconditionFailed) + } + Err(error) => Err(mutation_error(&error)), + } +} + +fn mutation_error(error: &KVStoreError) -> KvError { + match error { + KVStoreError::InvalidKey + | KVStoreError::InvalidStoreOptions + | KVStoreError::ItemBadRequest + | KVStoreError::ItemPayloadTooLarge => { + KvError::Validation("Fastly rejected record mutation".to_owned()) + } + KVStoreError::ItemPreconditionFailed | KVStoreError::ItemNotFound => { + KvError::Internal(anyhow::anyhow!("unexpected record mutation rejection")) + } + KVStoreError::TooManyRequests => KvError::Unavailable, + KVStoreError::InvalidStoreHandle + | KVStoreError::StoreNotFound(_) + | KVStoreError::Unexpected(_) + | _ => KvError::MutationUncertain, + } +} + +fn delete_record(store: &KVStore, key: &str) -> Result<(), KvError> { + match store.delete(key) { + Ok(()) | Err(KVStoreError::ItemNotFound) => Ok(()), + Err(error) => Err(mutation_error(&error)), + } +} + +fn strong_list_page( + store: &KVStore, + prefix: &str, + cursor: Option<&str>, + limit: usize, +) -> Result { + let limit_u32 = u32::try_from(limit) + .map_err(|_error| KvError::Validation("list limit exceeds u32".to_owned()))?; + let pending = store + .as_handle() + .list( + ListMode::Strong, + cursor.map(str::to_owned), + Some(limit_u32), + Some(prefix.to_owned()), + ) + .map_err(|_error| KvError::Unavailable)?; + let page = store + .pending_list_wait(pending) + .map_err(|_error| KvError::Unavailable)?; + if !matches!(page.mode(), ListMode::Strong) + || page.keys().iter().any(|key| !key.starts_with(prefix)) + { + return Err(KvError::Internal(anyhow::anyhow!( + "invalid strong list response" + ))); + } + let next = page.next_cursor().filter(|token| !token.is_empty()); + if next.as_deref().is_some_and(|token| Some(token) == cursor) { + return Err(KvError::Internal(anyhow::anyhow!( + "nonadvancing list cursor" + ))); + } + Ok(KvPage { + keys: page.into_keys(), + cursor: next, + }) +} + +fn exact_exists( + key: &str, + mut list: impl FnMut(Option<&str>) -> Result, +) -> Result { + let mut cursor = None; + let mut seen = HashSet::new(); + loop { + let page = list(cursor.as_deref())?; + if page.keys.iter().any(|candidate| candidate == key) { + return Ok(true); + } + let Some(next) = page.cursor else { + return Ok(false); + }; + if next.is_empty() || !seen.insert(next.clone()) { + return Err(KvError::Internal(anyhow::anyhow!( + "nonadvancing existence cursor" + ))); + } + cursor = Some(next); + } +} + +fn decode_ordinary_cursor( + binding: &RecordBinding, + prefix: &str, + cursor: Option<&str>, +) -> Result, KvError> { + let Some(token) = cursor else { + return Ok(None); + }; + let parsed: OrdinaryCursor = serde_json::from_str(token) + .map_err(|_error| KvError::Validation("invalid ordinary list cursor".to_owned()))?; + if parsed.scope != binding.scope + || parsed.format != binding.format + || parsed.layout != binding.layout.id() + || parsed.version != binding.layout.version() + || parsed.prefix != prefix + || parsed.native.is_empty() + { + return Err(KvError::Validation( + "ordinary list cursor scope mismatch".to_owned(), + )); + } + Ok(Some(parsed.native)) +} + +fn encode_ordinary_cursor( + binding: &RecordBinding, + prefix: &str, + cursor: Option, +) -> Result, KvError> { + cursor + .map(|native| { + serde_json::to_string(&OrdinaryCursor { + scope: binding.scope.clone(), + format: binding.format.clone(), + layout: binding.layout.id().to_owned(), + version: binding.layout.version(), + prefix: prefix.to_owned(), + native, + }) + .map_err(KvError::from) + }) + .transpose() } fn open_result( @@ -217,8 +590,193 @@ mod tests { use super::*; use edgezero_core::EdgeError; use edgezero_core::http::StatusCode; + use edgezero_core::record_store::RecordKeyLayout; use std::sync::atomic::{AtomicUsize, Ordering}; + struct DotLayout; + + impl RecordKeyLayout for DotLayout { + fn id(&self) -> &'static str { + "fastly-test" + } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + let end = key + .find('.') + .ok_or_else(|| KvError::Validation("missing partition".to_owned()))?; + key.get(..=end) + .ok_or_else(|| KvError::Validation("invalid partition".to_owned())) + } + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('.') && prefix.matches('.').count() == 1 { + Ok(()) + } else { + Err(KvError::Validation("invalid partition".to_owned())) + } + } + fn version(&self) -> u32 { + 1 + } + } + + #[test] + fn record_binding_is_immutable_and_acquisition_needs_no_native_io() { + let layout = RecordLayout::new(DotLayout).unwrap(); + let store = FastlyKvStore::declared_records("test-store", layout.clone()); + let binding = store.record_binding().unwrap(); + assert_eq!(binding.layout, layout); + assert_eq!(binding.max_metadata_bytes, 2000); + store.validate_point("group.key").unwrap(); + store.validate_point("invalid").unwrap_err(); + store + .validate_partition("group.key", &layout.partition("other.").unwrap()) + .unwrap_err(); + assert!( + FastlyKvStore::declared("ordinary") + .record_binding() + .is_none() + ); + FastlyKvStore::declared("ordinary") + .validate_point("invalid") + .unwrap(); + } + + #[test] + fn metadata_is_text_and_preserves_nonempty_bytes() { + assert_eq!(decode_metadata(None).unwrap(), None); + assert_eq!(decode_metadata(Some(b"")).unwrap(), None); + assert_eq!( + decode_metadata(Some("a\u{96ea}\n".as_bytes())) + .unwrap() + .as_deref(), + Some("a\u{96ea}\n") + ); + decode_metadata(Some(&[0xff, 0xfe])).unwrap_err(); + } + + #[test] + fn only_conditional_precondition_rejection_is_contention() { + for conditional in [true, false] { + assert!(matches!( + insert_outcome(Ok(()), conditional), + Ok(RecordWriteOutcome::Written) + )); + assert!(matches!( + insert_outcome( + Err(KVStoreError::Unexpected(FastlyStatus::ERROR)), + conditional + ), + Err(KvError::MutationUncertain) + )); + assert!(matches!( + insert_outcome(Err(KVStoreError::TooManyRequests), conditional), + Err(KvError::Unavailable) + )); + } + assert!(matches!( + insert_outcome(Err(KVStoreError::ItemPreconditionFailed), true), + Ok(RecordWriteOutcome::PreconditionFailed) + )); + assert!(matches!( + insert_outcome(Err(KVStoreError::ItemPreconditionFailed), false), + Err(KvError::Internal(_)) + )); + let error = mutation_error(&KVStoreError::StoreNotFound( + "secret-provider-payload".to_owned(), + )); + assert!(matches!(error, KvError::MutationUncertain)); + assert!(!format!("{error:?} {error}").contains("secret-provider-payload")); + } + + #[test] + fn exact_existence_follows_empty_and_neighbour_pages_until_exact_match() { + let mut calls = 0_usize; + assert!( + exact_exists("group.key", |cursor| { + calls += 1; + match calls { + 1 => { + assert_eq!(cursor, None); + Ok(KvPage { + keys: vec!["group.key.child".to_owned()], + cursor: Some("one".to_owned()), + }) + } + 2 => { + assert_eq!(cursor, Some("one")); + Ok(KvPage { + keys: vec![], + cursor: Some("two".to_owned()), + }) + } + 3 => { + assert_eq!(cursor, Some("two")); + Ok(KvPage { + keys: vec!["group.key".to_owned()], + cursor: None, + }) + } + _ => panic!("unexpected extra page"), + } + }) + .unwrap() + ); + assert_eq!(calls, 3); + } + + #[test] + fn exact_existence_does_not_infer_absence_before_end_or_error() { + let mut calls = 0_usize; + assert!( + !exact_exists("group.key", |_| { + calls += 1; + Ok(KvPage { + keys: vec!["group.key.child".to_owned()], + cursor: (calls == 1).then(|| "next".to_owned()), + }) + }) + .unwrap() + ); + assert_eq!(calls, 2); + assert!(matches!( + exact_exists("group.key", |_| Err(KvError::Unavailable)), + Err(KvError::Unavailable) + )); + exact_exists("group.key", |_| { + Ok(KvPage { + keys: vec![], + cursor: Some("stuck".to_owned()), + }) + }) + .unwrap_err(); + } + + #[test] + fn ordinary_cursor_preserves_native_position_and_rejects_other_scopes() { + let layout = RecordLayout::new(DotLayout).unwrap(); + let binding = record_binding("first", layout.clone()); + let cursor = encode_ordinary_cursor(&binding, "", Some("native-position".to_owned())) + .unwrap() + .unwrap(); + assert_eq!( + decode_ordinary_cursor(&binding, "", Some(&cursor)) + .unwrap() + .as_deref(), + Some("native-position") + ); + decode_ordinary_cursor(&binding, "group.", Some(&cursor)).unwrap_err(); + decode_ordinary_cursor(&record_binding("other", layout), "", Some(&cursor)).unwrap_err(); + decode_ordinary_cursor(&binding, "", Some("native-position")).unwrap_err(); + decode_ordinary_cursor( + &binding, + "", + Some(r#"{"scope":"first","native":"record-cursor"}"#), + ) + .unwrap_err(); + } + #[test] fn open_failure_classification_drops_provider_payloads() { assert_eq!(open_result(Ok(Some(7_u8))), Ok(7)); diff --git a/crates/edgezero-adapter-fastly/src/lib.rs b/crates/edgezero-adapter-fastly/src/lib.rs index 791e2f00..2ee14f2c 100644 --- a/crates/edgezero-adapter-fastly/src/lib.rs +++ b/crates/edgezero-adapter-fastly/src/lib.rs @@ -43,6 +43,8 @@ use edgezero_core::error::EdgeError; #[cfg(feature = "fastly")] use edgezero_core::http::{Extensions, Response}; #[cfg(any(feature = "fastly", test))] +use edgezero_core::key_value_store::KvError; +#[cfg(any(feature = "fastly", test))] use edgezero_core::manifest::ResolvedLoggingConfig; #[cfg(feature = "fastly")] use edgezero_core::time::MonotonicClock; @@ -104,7 +106,18 @@ fn build_target_app() -> Result where A: Hooks, { - App::build::(FASTLY_PLATFORM) + let app = App::build::(FASTLY_PLATFORM)?; + let metadata = A::stores().kv; + if app + .record_store_requirements() + .keys() + .any(|id| !metadata.is_some_and(|meta| meta.ids.contains(&id.as_str()))) + { + return Err( + KvError::Validation("record declaration uses an undeclared ID".to_owned()).into(), + ); + } + Ok(app) } #[cfg(feature = "fastly")] @@ -341,6 +354,71 @@ mod fastly_logging_tests { } } +#[cfg(test)] +mod record_requirement_tests { + use edgezero_core::app::{App, Hooks, StoreMetadata, StoresMetadata}; + use edgezero_core::key_value_store::KvError; + use edgezero_core::router::RouterService; + use edgezero_core::{EdgeError, RecordKeyLayout}; + + struct Layout; + impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "platform.record-test" + } + + fn partition_prefix<'key>(&self, _key: &'key str) -> Result<&'key str, KvError> { + Ok("group/") + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix == "group/" { + Ok(()) + } else { + Err(KvError::Validation("incomplete partition".to_owned())) + } + } + + fn version(&self) -> u32 { + 1 + } + } + + struct Required; + #[expect( + clippy::missing_trait_methods, + reason = "record startup fixture retains unrelated Hooks defaults" + )] + impl Hooks for Required { + fn configure(app: &mut App) -> Result<(), EdgeError> { + app.require_record_store("identity", Layout)?; + Ok(()) + } + fn routes() -> Result { + Ok(RouterService::builder().build()) + } + fn stores() -> StoresMetadata { + StoresMetadata { + kv: DECLARED.then_some(StoreMetadata { + ids: &["identity"], + default: "identity", + }), + ..StoresMetadata::default() + } + } + } + + #[test] + fn record_layouts_must_name_declared_ids_without_opening_a_provider() { + super::build_target_app::>().unwrap(); + assert!(super::build_target_app::>().is_err()); + } +} + #[cfg(test)] mod platform_tests { use edgezero_core::app::{App, Hooks}; diff --git a/crates/edgezero-adapter-fastly/src/request.rs b/crates/edgezero-adapter-fastly/src/request.rs index 214d85e9..9fb2eb44 100644 --- a/crates/edgezero-adapter-fastly/src/request.rs +++ b/crates/edgezero-adapter-fastly/src/request.rs @@ -179,7 +179,11 @@ impl<'app> FastlyService<'app> { ConfigSource::None => None, }; let kv = match self.kv { - Some(source) => resolve_kv_handle(&source.name, source.required)?, + Some(source) => resolve_kv_handle( + &source.name, + source.required, + self.app.record_store_requirements().get("default"), + )?, None => None, }; let secrets = match self.secrets { @@ -412,6 +416,17 @@ fn dispatch_core_request( // use `ctx.{config,kv,secret}_store_default()` or the // `Kv` / `Config` / `Secrets` extractors. let (config_registry, kv_registry, secret_registry) = synthesise_store_registries(stores); + if !app.record_store_requirements().is_empty() { + let validation = kv_registry + .as_ref() + .ok_or_else(|| KvError::Validation("required record registry missing".to_owned())) + .and_then(|registry| { + registry.validate_record_requirements(app.record_store_requirements()) + }); + if let Err(error) = validation { + return DispatchOutcome::Detached(app.admitted_error_egress(prepared, error.into())); + } + } if let Some(registry) = config_registry { core_request.extensions_mut().insert(registry); } @@ -605,7 +620,8 @@ where with_callback(|| { init_fastly_abi(); let request_start = app.monotonic_now(); - let kv_registry = build_kv_registry(stores.kv); + let kv_registry = + build_kv_registry_with_records(stores.kv, app.record_store_requirements()); let config_registry = build_config_registry(stores.config); let secret_registry = build_secret_registry(stores.secrets); let outcome = dispatch_with_handles( @@ -688,13 +704,24 @@ fn synthesise_store_registries( (config_registry, kv_registry, secret_registry) } +#[cfg(test)] fn build_kv_registry(kv_meta: Option) -> Option { + build_kv_registry_with_records(kv_meta, &edgezero_core::RecordStoreRequirements::new()) +} + +fn build_kv_registry_with_records( + kv_meta: Option, + requirements: &edgezero_core::RecordStoreRequirements, +) -> Option { let meta = kv_meta?; let mut by_id: BTreeMap = BTreeMap::new(); for id in meta.ids { by_id.insert( (*id).to_owned(), - KvHandle::new(Arc::new(FastlyKvStore::declared(id))), + KvHandle::new(Arc::new(match requirements.get(*id) { + Some(layout) => FastlyKvStore::declared_records(id, layout.clone()), + None => FastlyKvStore::declared(id), + })), ); } let default_id = meta.default.to_owned(); @@ -1036,11 +1063,18 @@ fn map_edge_error(err: &EdgeError) -> FastlyError { fn resolve_kv_handle( kv_store_name: &str, kv_required: bool, + layout: Option<&edgezero_core::RecordLayout>, ) -> Result, FastlyNoAttempt> { - match FastlyKvStore::open(kv_store_name) { + let opened = match layout { + Some(declared_layout) => { + FastlyKvStore::open_records(kv_store_name, declared_layout.clone()) + } + None => FastlyKvStore::open(kv_store_name), + }; + match opened { Ok(store) => Ok(Some(KvHandle::new(Arc::new(store)))), Err(err) => { - if kv_required { + if kv_required || layout.is_some() { let reason = if let KvError::OpenFailed { reason } = &err { *reason } else { @@ -1168,6 +1202,58 @@ mod synthesis_tests { SecretHandle::new(Arc::new(NoopSecretStore)) } + #[test] + fn declared_record_registry_keeps_the_configured_layout_without_provider_io() { + struct Layout; + impl edgezero_core::RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "fastly.record-registry" + } + + fn partition_prefix<'key>(&self, _key: &'key str) -> Result<&'key str, KvError> { + Ok("group/") + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix == "group/" { + Ok(()) + } else { + Err(KvError::Validation("incomplete partition".to_owned())) + } + } + + fn version(&self) -> u32 { + 3 + } + } + let requirements = edgezero_core::RecordStoreRequirements::from([( + "record_binding_not_opened".to_owned(), + edgezero_core::RecordLayout::new(Layout).unwrap(), + )]); + let registry = build_kv_registry_with_records( + Some(StoreMetadata { + default: "record_binding_not_opened", + ids: &["record_binding_not_opened", "ordinary_binding_not_opened"], + }), + &requirements, + ) + .unwrap(); + registry + .validate_record_requirements(&requirements) + .unwrap(); + let records = registry.default().unwrap().require_records().unwrap(); + assert_eq!(records.binding().layout.version(), 3); + registry + .named("ordinary_binding_not_opened") + .unwrap() + .require_records() + .unwrap_err(); + } + #[test] fn declared_kv_registry_construction_never_opens_a_provider() { let registry = build_kv_registry(Some(StoreMetadata { diff --git a/crates/edgezero-adapter-spin/src/key_value_store.rs b/crates/edgezero-adapter-spin/src/key_value_store.rs index e52953b3..1e81d6d9 100644 --- a/crates/edgezero-adapter-spin/src/key_value_store.rs +++ b/crates/edgezero-adapter-spin/src/key_value_store.rs @@ -125,6 +125,10 @@ impl SpinKvStore { } } +#[expect( + clippy::missing_trait_methods, + reason = "Spin implements ordinary KV and inherits unsupported record defaults" +)] #[async_trait(?Send)] impl KvStore for SpinKvStore { #[inline] diff --git a/crates/edgezero-adapter-spin/src/lib.rs b/crates/edgezero-adapter-spin/src/lib.rs index b90540f7..7adcf435 100644 --- a/crates/edgezero-adapter-spin/src/lib.rs +++ b/crates/edgezero-adapter-spin/src/lib.rs @@ -38,6 +38,8 @@ use edgezero_core::env_config::EnvConfig; #[cfg(any(test, all(feature = "spin", target_arch = "wasm32")))] use edgezero_core::error::EdgeError; #[cfg(any(test, all(feature = "spin", target_arch = "wasm32")))] +use edgezero_core::key_value_store::KvError; +#[cfg(any(test, all(feature = "spin", target_arch = "wasm32")))] use edgezero_core::outbound::OutboundScopeOwner; #[cfg(all(feature = "spin", target_arch = "wasm32"))] use edgezero_core::time::MonotonicClock; @@ -109,7 +111,24 @@ fn build_target_app(platform: edgezero_core::PlatformMetadata) -> Result(platform) + let app = App::build::(platform)?; + let metadata = A::stores().kv; + if app + .record_store_requirements() + .keys() + .any(|id| !metadata.is_some_and(|meta| meta.ids.contains(&id.as_str()))) + { + return Err( + KvError::Validation("record declaration uses an undeclared ID".to_owned()).into(), + ); + } + if !app.record_store_requirements().is_empty() { + return Err(KvError::Unsupported { + operation: "records".to_owned(), + } + .into()); + } + Ok(app) } #[cfg(any(test, all(feature = "spin", target_arch = "wasm32")))] @@ -324,6 +343,78 @@ mod callback_scope_tests { } } +#[cfg(test)] +mod record_requirement_tests { + use edgezero_core::app::{App, Hooks, StoreMetadata, StoresMetadata}; + use edgezero_core::http::StatusCode; + use edgezero_core::key_value_store::KvError; + use edgezero_core::router::RouterService; + use edgezero_core::{EdgeError, RecordKeyLayout}; + + struct Layout; + impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "platform.record-test" + } + + fn partition_prefix<'key>(&self, _key: &'key str) -> Result<&'key str, KvError> { + Ok("group/") + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix == "group/" { + Ok(()) + } else { + Err(KvError::Validation("incomplete partition".to_owned())) + } + } + + fn version(&self) -> u32 { + 1 + } + } + + struct Required; + #[expect( + clippy::missing_trait_methods, + reason = "record startup fixture retains unrelated Hooks defaults" + )] + impl Hooks for Required { + fn configure(app: &mut App) -> Result<(), EdgeError> { + app.require_record_store("identity", Layout)?; + Ok(()) + } + fn routes() -> Result { + Ok(RouterService::builder().build()) + } + fn stores() -> StoresMetadata { + StoresMetadata { + kv: DECLARED.then_some(StoreMetadata { + ids: &["identity"], + default: "identity", + }), + ..StoresMetadata::default() + } + } + } + + #[test] + fn record_requirement_refuses_unsupported_platform_before_store_preparation() { + let error = super::build_target_app::>(super::SPIN_PLATFORM) + .err() + .expect("unsupported records"); + assert_eq!(error.status(), StatusCode::NOT_IMPLEMENTED); + let undeclared = super::build_target_app::>(super::SPIN_PLATFORM) + .err() + .expect("undeclared record ID"); + assert_eq!(undeclared.status(), StatusCode::BAD_REQUEST); + } +} + #[cfg(test)] mod platform_tests { use edgezero_core::app::{App, Hooks}; diff --git a/crates/edgezero-adapter-spin/src/request.rs b/crates/edgezero-adapter-spin/src/request.rs index 96a4ca61..f733613d 100644 --- a/crates/edgezero-adapter-spin/src/request.rs +++ b/crates/edgezero-adapter-spin/src/request.rs @@ -455,6 +455,21 @@ where // legacy bare handles are synthesised into a one-id registry // at the dispatch boundary. let (config_registry, kv_registry, secret_registry) = synthesise_store_registries(stores); + if !app.record_store_requirements().is_empty() { + let validation = kv_registry + .as_ref() + .ok_or_else(|| { + edgezero_core::key_value_store::KvError::Validation( + "required record registry missing".to_owned(), + ) + }) + .and_then(|registry| { + registry.validate_record_requirements(app.record_store_requirements()) + }); + if let Err(error) = validation { + return Ok(deliver(app.admitted_error_egress(prepared, error.into()))?); + } + } if let Some(registry) = config_registry { core_request.extensions_mut().insert(registry); } diff --git a/crates/edgezero-cli/src/config.rs b/crates/edgezero-cli/src/config.rs index e64c0cfb..4544ef84 100644 --- a/crates/edgezero-cli/src/config.rs +++ b/crates/edgezero-cli/src/config.rs @@ -1575,8 +1575,10 @@ fn load_validated_native_bindings( appconfig_agent: true, secrets_manager: true, dynamodb: true, + dynamodb_records: true, celld_config: true, celld_kv: true, + celld_records: true, }, limits, None, @@ -2886,6 +2888,57 @@ environment = "production" profile = "envelope" "#; + #[cfg(feature = "edgezero-adapter-axum")] + #[test] + fn record_bindings_validation_is_structural_without_runtime_preparation() { + let manifest_text = r#" +[app] +name = "demo-app" +[adapters.axum.adapter] +crate = "crates/demo-axum" +[adapters.axum.commands] +build = "echo" +deploy = "echo" +serve = "echo" +[stores.kv] +ids = ["state"] +default = "state" +"#; + let (dir, manifest, app_config) = setup_project(manifest_text, "not used"); + fs::remove_file(app_config).expect("structural record validation does not load app TOML"); + for (provider, settings) in [ + ( + "aws-dynamodb-records", + "region='us-east-1'\ntable='record-fixture'\nnamespace='record-fixture'", + ), + ( + "celld-records", + "endpoint='https://unreachable.invalid'\nstore='record-fixture'\n[kv.state.access]\nmode='authenticated'\ntoken_env='ABSENT_RECORD_TOKEN'", + ), + ] { + let path = dir.path().join("record-bindings.toml"); + fs::write( + &path, + format!("version=1\n[kv.state]\nprovider='{provider}'\n{settings}"), + ) + .expect("write explicit record binding"); + let args = ConfigValidateArgs { + app_config: None, + adapter: Some("axum".to_owned()), + store_bindings: Some(path), + target: None, + target_features: Vec::new(), + target_all_features: false, + target_no_default_features: false, + manifest: manifest.clone(), + no_env: false, + strict: false, + }; + run_config_validate(&args) + .expect("record settings validate without tokens, SDKs or service I/O"); + } + } + #[cfg(feature = "edgezero-adapter-axum")] #[test] fn bindings_only_validation_does_not_require_app_toml() { diff --git a/crates/edgezero-cli/src/generator.rs b/crates/edgezero-cli/src/generator.rs index 2c2a9c0e..c7a2ffcf 100644 --- a/crates/edgezero-cli/src/generator.rs +++ b/crates/edgezero-cli/src/generator.rs @@ -2009,8 +2009,10 @@ git = "https://github.com/stackpop/spin-rust-sdk" "aws-appconfig-agent", "aws-secrets-manager", "aws-dynamodb", + "aws-dynamodb-records", "celld-config", "celld-kv", + "celld-records", ] { assert_eq!( features.get(feature), diff --git a/crates/edgezero-cli/tests/celld_generated_proof.rs b/crates/edgezero-cli/tests/celld_generated_proof.rs index 8d996fd2..c7cc3e70 100644 --- a/crates/edgezero-cli/tests/celld_generated_proof.rs +++ b/crates/edgezero-cli/tests/celld_generated_proof.rs @@ -444,6 +444,94 @@ fn main() -> anyhow::Result<()> { project } + fn generate_records(root: &Path) -> PathBuf { + let project = generate(root); + let manifest = project.join("edgezero.toml"); + let manifest_source = fs::read_to_string(&manifest).expect("generated record route"); + fs::write( + &manifest, + manifest_source.replacen( + "handler = \"celld_proof_core::handlers::celld_probe\"", + "handler = \"celld_proof_core::handlers::celld_record_probe\"", + 1, + ), + ) + .expect("record route declaration"); + let core_manifest = project.join("crates/celld-proof-core/Cargo.toml"); + let dependencies = fs::read_to_string(&core_manifest).expect("generated core dependencies"); + fs::write( + &core_manifest, + dependencies.replacen( + "edgezero-core = { workspace = true }", + "edgezero-core = { workspace = true, features = [\"test-utils\"] }", + 1, + ), + ) + .expect("portable shared-contract helper in fixture only"); + let core_lib = project.join("crates/celld-proof-core/src/lib.rs"); + let lib_source = fs::read_to_string(&core_lib).expect("generated configure hook"); + let mut configured = lib_source.replacen( + "fn configure_app(app: &mut EdgeZeroApp) -> Result<(), EdgeError> {", + "fn configure_app(app: &mut EdgeZeroApp) -> Result<(), EdgeError> {\n app.require_record_store(\"state\", RecordFixtureLayout)?;", 1, + ); + configured.push_str( + r#" +struct RecordFixtureLayout; +impl edgezero_core::RecordKeyLayout for RecordFixtureLayout { + fn id(&self) -> &'static str { "generated-celld-records-v2" } + fn version(&self) -> u32 { 1 } + fn validate(&self) -> Result<(), edgezero_core::key_value_store::KvError> { Ok(()) } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, edgezero_core::key_value_store::KvError> { + if key.starts_with("probe/") { Ok("probe/") } + else { Err(edgezero_core::key_value_store::KvError::Validation("outside fixture group".to_owned())) } + } + fn validate_partition(&self, prefix: &str) -> Result<(), edgezero_core::key_value_store::KvError> { + if prefix == "probe/" { Ok(()) } + else { Err(edgezero_core::key_value_store::KvError::Validation("incomplete fixture group".to_owned())) } + } +} +"#, + ); + fs::write(core_lib, configured).expect("declare layout before native preparation"); + let handlers = project.join("crates/celld-proof-core/src/handlers.rs"); + let mut handler_source = fs::read_to_string(&handlers).expect("generated record handler"); + handler_source.push_str(r#" +#[action] +pub async fn celld_record_probe( + edgezero_core::extractor::State(state): edgezero_core::extractor::State>, + kv: edgezero_core::extractor::Kv, +) -> Result, EdgeError> { + let store = kv.default().ok_or_else(|| EdgeError::service_unavailable("KV missing"))?; + let records = store.require_records()?; + let run_contract = !state.mutated.swap(true, std::sync::atomic::Ordering::SeqCst) + && std::env::var("CELLD_PROOF_MODE").unwrap_or_default() == "records-put"; + if run_contract { + edgezero_core::record_store::contract::assert_record_contract(&store, "probe/").await; + store.put_bytes("probe/persist", b"native-no-expiry".to_vec().into()).await?; + } + let row = records.lookup("probe/persist").await? + .ok_or_else(|| EdgeError::service_unavailable("record missing"))?; + let strong = records.exists_strong("probe/persist").await?; + Ok(Text::new(format!("greeting={};record-contract={run_contract};body={:?};metadata={:?};strong={strong}", + state.greeting, row.body.as_ref(), row.metadata))) +} +"#); + fs::write(handlers, handler_source) + .expect("shared public contract through generated route"); + project + } + + fn record_bindings(project: &Path, endpoint: &str) -> PathBuf { + let path = bindings(project, endpoint, true); + let text = fs::read_to_string(&path).expect("generated native record bindings"); + fs::write( + &path, + text.replacen("provider = \"celld-kv\"", "provider = \"celld-records\"", 1), + ) + .expect("explicit fresh-record provider, not raw auto-upgrade"); + path + } + fn bindings(project: &Path, endpoint: &str, authenticated: bool) -> PathBuf { let path = project.join("celld-bindings.toml"); let access = if authenticated { @@ -736,6 +824,57 @@ store = "state" } } + #[test] + #[ignore = "requires explicit v2 CELLD_BIN, curl and OpenSSL; shared record contract through actual generated TLS route"] + fn generated_native_consumer_uses_embedded_celld_over_tls_records_v2() { + let temp = TempDir::new().expect("generated v2 fixture"); + let service = Celld::start(temp.path()); + service.publish(temp.path(), "record-first-config"); + let project = generate_records(temp.path()); + let binding = record_bindings(&project, &service.endpoint); + let binary = build(&project, Some("celld-config,celld-records")); + let mut first = App::start( + &binary, + &project, + &binding, + "records-put", + Some((APP_TOKEN, &service.cert)), + ); + let response = first.request(); + assert!( + response.contains("200 OK") + && response.contains("record-contract=true") + && response.contains("greeting=record-first-config") + && response.contains("metadata=None") + && response.contains("strong=true"), + "generated public record contract: {response}" + ); + service.publish(temp.path(), "record-second-config"); + let again = first.request(); + assert!( + again.contains("greeting=record-first-config") + && again.contains("record-contract=false"), + "contract runs once and config remains a snapshot: {again}" + ); + drop(first); + drop(service); + let restarted_service = Celld::start(temp.path()); + let restarted_binding = record_bindings(&project, &restarted_service.endpoint); + let mut second = App::start( + &binary, + &project, + &restarted_binding, + "records-read", + Some((APP_TOKEN, &restarted_service.cert)), + ); + let restarted_response = second.request(); + assert!(restarted_response.contains("200 OK") && restarted_response.contains("record-contract=false") + && restarted_response.contains("greeting=record-second-config") && restarted_response.contains("metadata=None") + && restarted_response.contains("strong=true") && restarted_response.contains("body=[110, 97, 116, 105, 118, 101, 45, 110, 111, 45, 101, 120, 112, 105, 114, 121]"), + "generated v2 no-expiry row survives service and app restart: {restarted_response}"); + assert!(!project.join(".edgezero").exists()); + } + #[test] #[ignore = "requires explicit modified CELLD_BIN, curl and OpenSSL; runs actual generated consumer over verified TLS"] fn generated_native_consumer_uses_embedded_celld_over_tls() { diff --git a/crates/edgezero-cli/tests/native_aws_emulator_proof.rs b/crates/edgezero-cli/tests/native_aws_emulator_proof.rs index c494f761..701be951 100644 --- a/crates/edgezero-cli/tests/native_aws_emulator_proof.rs +++ b/crates/edgezero-cli/tests/native_aws_emulator_proof.rs @@ -18,7 +18,7 @@ mod tests { use std::io::{self, Read as _, Write as _}; use std::net::{Ipv4Addr, SocketAddr, TcpListener, TcpStream}; use std::path::{Path, PathBuf}; - use std::process::{Child, Command, ExitStatus, Stdio}; + use std::process::{self, Child, Command, ExitStatus, Stdio}; use std::str::from_utf8; use std::thread; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; @@ -961,6 +961,225 @@ region = "us-east-1" ); } + const RECORDS_CORE_SOURCE: &str = r#" +pub mod config; +mod handlers; +use edgezero_core::{EdgeError, RecordKeyLayout}; +use edgezero_core::key_value_store::KvError; +struct Groups; +impl RecordKeyLayout for Groups { + fn id(&self) -> &'static str { "generated-record-groups" } + fn version(&self) -> u32 { 1 } + fn validate(&self) -> Result<(), KvError> { Ok(()) } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + let end = key.find('/').ok_or_else(|| KvError::Validation("missing group".to_owned()))?; + key.get(..=end).ok_or_else(|| KvError::Validation("invalid group".to_owned())) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.bytes().filter(|byte| *byte == b'/').count() == 1 { + Ok(()) + } else { Err(KvError::Validation("incomplete group".to_owned())) } + } +} +fn configure_app(app: &mut edgezero_core::app::App) -> Result<(), EdgeError> { + app.require_record_store("primary", Groups)?; + Ok(()) +} +edgezero_core::app!("../../edgezero.toml", configure = crate::configure_app); +"#; + + const RECORDS_HANDLER_SOURCE: &str = r#" +use edgezero_core::{action, EdgeError}; +use edgezero_core::extractor::Kv; +use edgezero_core::body::Body; +use edgezero_core::http::{Response, StatusCode, response_builder}; +use serde_json::json; +#[action] +pub async fn proof(kv: Kv) -> Result { + let store = kv.named("primary").ok_or_else(|| EdgeError::service_unavailable("missing record store"))?; + edgezero_core::record_store::contract::assert_record_contract(&store, "group/").await; + response_builder().status(StatusCode::OK).header("content-type", "application/json") + .body(Body::json(&json!({"records": true})).map_err(EdgeError::internal)?) + .map_err(EdgeError::internal) +} +"#; + + const RECORDS_MAIN_SOURCE: &str = r#" +use std::time::{Duration, Instant}; +use edgezero_adapter_axum::dev_server::run_production_app_with_initializer; +use edgezero_core::EdgeError; +fn main() -> anyhow::Result<()> { + run_production_app_with_initializer::(|_app, stores| { + Box::pin(async move { + stores.kv().and_then(|registry| registry.named_ref("primary")) + .ok_or_else(|| EdgeError::service_unavailable("missing prepared KV"))? + .require_records()?; + let entered = std::env::var("PROOF_ENTERED").map_err(EdgeError::internal)?; + std::fs::write(entered, "records-ready").map_err(EdgeError::internal)?; + let release = std::path::PathBuf::from(std::env::var("PROOF_RELEASE").map_err(EdgeError::internal)?); + let deadline = Instant::now() + Duration::from_secs(45); + while !release.is_file() { + if Instant::now() >= deadline { + return Err(EdgeError::service_unavailable("initializer gate timed out")); + } + std::thread::sleep(Duration::from_millis(10)); + } + Ok(()) + }) + })?; + Ok(()) +} +"#; + + fn install_record_consumer(project: &Path) { + install_consumer(project); + let manifest_path = project.join("edgezero.toml"); + let mut manifest: toml::Value = + toml::from_str(&fs::read_to_string(&manifest_path).expect("manifest")) + .expect("manifest TOML"); + manifest["stores"] = + toml::Value::try_from(json!({"kv": {"ids": ["primary"], "default": "primary"}})) + .expect("record store declaration"); + manifest["adapters"]["axum"]["build"]["features"] = + toml::Value::try_from(vec!["aws-dynamodb-records"]) + .expect("explicit record provider feature"); + fs::write( + manifest_path, + toml::to_string(&manifest).expect("record manifest TOML"), + ) + .expect("write record manifest"); + let core = project.join("crates/native-aws-probe-core"); + let core_manifest = core.join("Cargo.toml"); + let mut dependencies: toml::Value = + toml::from_str(&fs::read_to_string(&core_manifest).expect("core Cargo manifest")) + .expect("core TOML"); + dependencies["dependencies"]["edgezero-core"] + .as_table_mut() + .expect("core dependency table") + .entry("features") + .or_insert_with(|| toml::Value::Array(Vec::new())) + .as_array_mut() + .expect("core feature array") + .push(toml::Value::String("test-utils".to_owned())); + let core_toml = toml::to_string(&dependencies).expect("core TOML"); + assert!( + !core_toml.contains("aws-sdk"), + "generated portable core must not depend on AWS SDK" + ); + fs::write(core_manifest, core_toml).expect("shared public contract helper feature"); + fs::write(core.join("src/lib.rs"), RECORDS_CORE_SOURCE) + .expect("configure-time record layout"); + fs::write(core.join("src/handlers.rs"), RECORDS_HANDLER_SOURCE) + .expect("generated public record contract handler"); + fs::write( + project.join("crates/native-aws-probe-adapter-axum/src/main.rs"), + RECORDS_MAIN_SOURCE, + ) + .expect("prepared record initializer"); + } + + // The same local dummy authorization shape as native_aws_test_admin.mjs. + // DynamoDB Local does not verify the signature or timestamp; no hosted + // service can be reached because Services validates a literal loopback URL. + fn record_table_admin(services: &Services, operation: &str, input: &Value) -> Value { + let address = services + .dynamodb + .strip_prefix("http://") + .expect("validated local endpoint") + .parse() + .expect("literal Local address"); + let headers = format!( + "Content-Type: application/x-amz-json-1.0\r\nX-Amz-Target: DynamoDB_20120810.{operation}\r\nX-Amz-Date: 20260101T000000Z\r\nAuthorization: AWS4-HMAC-SHA256 Credential=DUMMYLOCAL/20260101/us-east-1/dynamodb/aws4_request, SignedHeaders=host;x-amz-date, Signature={}\r\n", + "0".repeat(64) + ); + let (status, body) = http( + address, + "POST", + "/", + &headers, + &serde_json::to_vec(input).expect("local table JSON"), + ) + .expect("bounded local table request"); + assert_eq!( + status, + 200, + "local table {operation}: {}", + String::from_utf8_lossy(&body) + ); + serde_json::from_slice(&body).expect("local table JSON response") + } + + #[test] + #[ignore = "requires pinned local services; run node scripts/run_native_aws_integration.mjs"] + fn generated_native_aws_emulator_contracts_records() { + let services = Services::required(); + phase("AWS records proof: scaffold explicit record-only consumer"); + let temp = TempDir::new().expect("temporary generated record project"); + let mut new = Command::new(env!("CARGO_BIN_EXE_edgezero")); + new.args(["new", "native-aws-probe", "--dir"]) + .arg(temp.path()); + checked(new, WAIT, "real edgezero new for records"); + let project = temp.path().join("native-aws-probe"); + install_record_consumer(&project); + let target = env::var_os("EDGEZERO_GENERATED_TEST_TARGET_DIR").map_or_else( + || project.join("target"), + |value| { + let path = PathBuf::from(value); + if path.is_absolute() { + path + } else { + project.join(path) + } + }, + ); + let mut build = Command::new(env!("CARGO_BIN_EXE_edgezero")); + build + .args(["build", "--adapter", "axum", "--", "--offline"]) + .current_dir(&project) + .env("CARGO_TARGET_DIR", &target); + checked( + build, + BUILD_WAIT, + "generated explicit record-provider build", + ); + let binary = target.join("debug/native-aws-probe-adapter-axum"); + assert!(binary.is_file(), "generated record executable"); + let table = format!( + "edgezero-generated-records-{}-{}", + process::id(), + SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("clock") + .as_nanos() + ); + record_table_admin( + &services, + "CreateTable", + &json!({ + "TableName": table, "BillingMode": "PAY_PER_REQUEST", + "KeySchema": [{"AttributeName": "pk", "KeyType": "HASH"}, {"AttributeName": "sk", "KeyType": "RANGE"}], + "AttributeDefinitions": [{"AttributeName": "pk", "AttributeType": "S"}, {"AttributeName": "sk", "AttributeType": "S"}] + }), + ); + let bindings = project.join("record-bindings.toml"); + fs::write(&bindings, format!("version = 1\n[kv.primary]\nprovider = \"aws-dynamodb-records\"\nregion = \"us-east-1\"\ntable = {}\nnamespace = \"generated-records\"\n", toml_string(&table))).expect("explicit record bindings"); + phase("AWS records proof: public shared contract through generated HTTP handler"); + let mut first = App::spawn(&project, &binary, &bindings, &services, &services.token); + first.ready("records-ready"); + assert_eq!(first.call(&json!({})), json!({"records": true})); + drop(first); + let mut reopened = App::spawn(&project, &binary, &bindings, &services, &services.token); + reopened.ready("records-ready"); + assert_eq!(reopened.call(&json!({})), json!({"records": true})); + drop(reopened); + assert!( + !project.join("must-not-create-data").exists(), + "record binding must not fall back to local files" + ); + record_table_admin(&services, "DeleteTable", &json!({"TableName": table})); + phase("AWS records proof: generated record consumer passed"); + } + #[test] #[expect( clippy::too_many_lines, diff --git a/crates/edgezero-core/src/app.rs b/crates/edgezero-core/src/app.rs index da9b2fbc..0cf49741 100644 --- a/crates/edgezero-core/src/app.rs +++ b/crates/edgezero-core/src/app.rs @@ -15,9 +15,11 @@ use crate::ingress::{ IngressDispatchOutcome, IngressFraming, IngressHead, IngressHeadAccounting, IngressHeadLimits, IngressHeadParts, PreparedIngress, apply_admission_policy, default_admission_policy, }; +use crate::key_value_store::KvError; use crate::logging::BOOT_LOG_TARGET; use crate::manifest::{BakedManifest, ResolvedLoggingConfig}; use crate::platform::PlatformMetadata; +use crate::record_store::{RecordKeyLayout, RecordLayout, RecordStoreRequirements}; use crate::response_egress::{ DEFAULT_RESPONSE_WRITE_BUDGET, DetachedResponseEgressDecision, DetachedResponseEgressDecisionFactory, DetachedResponseEgressHead, ResponseEgressCompletion, @@ -61,6 +63,8 @@ pub struct App { monotonic_clock: MonotonicClock, name: String, platform: PlatformMetadata, + record_store_requirements: RecordStoreRequirements, + record_store_requirements_sealed: bool, response_egress_observer: ResponseEgressObserverHandle, response_egress_policy: ResponseEgressPolicyCallback, router: RouterService, @@ -429,6 +433,7 @@ impl App { "application configuration changed adapter-selected platform metadata" ))); } + app.record_store_requirements_sealed = true; Ok(app) } @@ -503,6 +508,13 @@ impl App { self.router.prepare_received_head(headers, extensions); } + /// Borrow immutable requirements for store preparation and ready-handle validation. + #[must_use] + #[inline] + pub fn record_store_requirements(&self) -> &RecordStoreRequirements { + &self.record_store_requirements + } + fn render_error_response(&self, error: EdgeError) -> Response { self.render_error_response_with_message(error, None) } @@ -536,6 +548,39 @@ impl App { response } + /// Declare a record layout once during configuration, before provider preparation. + /// Declarations cannot be replaced or added after hook-based construction finishes. + /// Low-level constructors leave configuration open until the caller prepares stores. + /// + /// # Errors + /// Rejects duplicate/empty logical IDs, invalid layouts or post-configuration changes. + #[inline] + pub fn require_record_store( + &mut self, + id: Id, + layout: KeyLayout, + ) -> Result<(), KvError> + where + Id: Into, + KeyLayout: RecordKeyLayout + 'static, + { + if self.record_store_requirements_sealed { + return Err(KvError::Validation( + "record store declarations are already sealed".to_owned(), + )); + } + let logical_id = id.into(); + if logical_id.is_empty() || self.record_store_requirements.contains_key(&logical_id) { + return Err(KvError::Validation( + "record store ID is empty or already declared".to_owned(), + )); + } + let declaration = RecordLayout::new(layout)?; + self.record_store_requirements + .insert(logical_id, declaration); + Ok(()) + } + fn response_egress_envelope( &self, response: Response, @@ -741,6 +786,8 @@ impl App { monotonic_clock: MonotonicClock::default(), name: name.into(), platform, + record_store_requirements: RecordStoreRequirements::new(), + record_store_requirements_sealed: false, response_egress_observer: ResponseEgressObserverHandle::default(), response_egress_policy: Arc::new(default_response_egress_policy), router, diff --git a/crates/edgezero-core/src/key_value_store.rs b/crates/edgezero-core/src/key_value_store.rs index 7cd99381..8381a36b 100644 --- a/crates/edgezero-core/src/key_value_store.rs +++ b/crates/edgezero-core/src/key_value_store.rs @@ -60,6 +60,10 @@ use serde::{Deserialize, Serialize}; use web_time::Instant; use crate::error::EdgeError; +use crate::record_store::{ + Record, RecordBinding, RecordHandle, RecordPage, RecordPartition, RecordWrite, + RecordWriteOutcome, +}; // --------------------------------------------------------------------------- // Contract test macro @@ -225,9 +229,10 @@ macro_rules! key_value_store_contract_tests { let mut cursor = None; let mut seen = std::collections::HashSet::new(); + let mut seen_cursors = std::collections::HashSet::new(); let mut collected = Vec::new(); - for _ in 0..expected.len() { + loop { let page = store .list_keys_page("", cursor.as_deref(), 1) .await @@ -242,9 +247,11 @@ macro_rules! key_value_store_contract_tests { } cursor = page.cursor; - if cursor.is_none() { - break; - } + let Some(next_cursor) = &cursor else { break }; + assert!( + seen_cursors.insert(next_cursor.clone()), + "repeated continuation in pagination" + ); } collected.sort(); @@ -331,6 +338,11 @@ pub enum KvError { #[error("kv backend limit exceeded: {message}")] LimitExceeded { message: String }, + /// A mutation may have committed, but its acknowledgement was lost. + /// The caller owns reconciliation. Replaying insert or delete is not safe by default. + #[error("kv mutation may have committed")] + MutationUncertain, + /// The requested key was not found (used by `delete` when strict). #[error("key not found: {key}")] NotFound { key: String }, @@ -757,7 +769,16 @@ impl KvHandle { Ok(updated) } - fn validate_key(key: &str) -> Result<(), KvError> { + /// Acquire a record view over this exact prepared provider, without I/O or mutation. + /// + /// # Errors + /// Returns Unsupported for ordinary-only bindings or validation errors for invalid bindings. + #[inline] + pub fn require_records(&self) -> Result { + RecordHandle::new(Arc::clone(&self.store)) + } + + pub(crate) fn validate_key(key: &str) -> Result<(), KvError> { if key.is_empty() { return Err(KvError::Validation("key cannot be empty".to_owned())); } @@ -781,7 +802,7 @@ impl KvHandle { Ok(()) } - fn validate_list_limit(limit: usize) -> Result<(), KvError> { + pub(crate) fn validate_list_limit(limit: usize) -> Result<(), KvError> { if limit == 0 { return Err(KvError::Validation( "list limit must be greater than zero".to_owned(), @@ -797,7 +818,7 @@ impl KvHandle { Ok(()) } - fn validate_prefix(prefix: &str) -> Result<(), KvError> { + pub(crate) fn validate_prefix(prefix: &str) -> Result<(), KvError> { if prefix.len() > Self::MAX_KEY_SIZE { return Err(KvError::Validation(format!( "prefix length {} exceeds limit of {} bytes", @@ -813,7 +834,7 @@ impl KvHandle { Ok(()) } - fn validate_ttl(ttl: Duration) -> Result<(), KvError> { + pub(crate) fn validate_ttl(ttl: Duration) -> Result<(), KvError> { if ttl < Self::MIN_TTL { return Err(KvError::Validation(format!( "TTL {ttl:?} is less than minimum of at least 60 seconds" @@ -827,7 +848,7 @@ impl KvHandle { Ok(()) } - fn validate_value(value: &[u8]) -> Result<(), KvError> { + pub(crate) fn validate_value(value: &[u8]) -> Result<(), KvError> { if value.len() > Self::MAX_VALUE_SIZE { return Err(KvError::Validation(format!( "value size {} exceeds limit of {} bytes", @@ -844,6 +865,9 @@ impl From for EdgeError { fn from(err: KvError) -> Self { match err { KvError::NotFound { key } => EdgeError::not_found(format!("kv key: {key}")), + KvError::MutationUncertain => { + EdgeError::service_unavailable("kv mutation may have committed") + } KvError::Unavailable | KvError::OpenFailed { reason: KvStoreOpenFailure::Unavailable, @@ -893,9 +917,9 @@ pub struct KvPage { /// /// # Pre-validation contract /// -/// This trait is always called through [`KvHandle`], which validates all +/// This trait is called through [`KvHandle`] or [`RecordHandle`], which validate /// inputs (key length/format, value size, TTL bounds, list limits) before -/// delegating here. Implementations may therefore assume that: +/// delegating here. Record handles also validate application layout membership. Implementations may therefore assume that: /// - Keys are non-empty and within [`KvHandle::MAX_KEY_SIZE`] /// - Values are within [`KvHandle::MAX_VALUE_SIZE`] /// - TTLs are within `[MIN_TTL, MAX_TTL]` @@ -925,7 +949,10 @@ pub trait KvStore: Send + Sync { /// Retrieve raw bytes for a key. Returns `Ok(None)` if the key does not exist. async fn get_bytes(&self, key: &str) -> Result, KvError>; - /// List keys in lexicographic order, returning at most `limit` keys. + /// List keys matching an arbitrary prefix, returning at most `limit` keys. + /// Ordering and consistency are provider-specific. Empty/short pages may + /// carry continuation; only its absence ends traversal. This is separate + /// from authoritative record partition listing. /// /// The `cursor` is opaque. Pass the cursor from a previous page back to /// continue listing. Implementations should keep memory usage bounded to a @@ -940,21 +967,95 @@ pub trait KvStore: Send + Sync { /// Store raw bytes for a key, overwriting any existing value. async fn put_bytes(&self, key: &str, value: Bytes) -> Result<(), KvError>; - /// Store raw bytes with a time-to-live. After `ttl` has elapsed the key - /// should be treated as expired. Eviction timing is backend-specific: + /// Store raw bytes with a provider-native time-to-live. + /// Visibility after expiry and physical removal are backend-specific; + /// there is no portable exact-deadline read or listing guarantee: /// - **Axum (`PersistentKvStore`)**: lazy eviction — expired keys are removed /// on the next `get_bytes` call for that key, or by any `list_keys` scan /// that reaches them. Keys neither read nor listed after expiration remain /// in the store's `.edgezero/kv--.redb` file, which grows /// without bound on long-running dev servers. - /// - **Fastly/Cloudflare**: eviction is managed by the platform and is not - /// guaranteed to be immediate. + /// - **Fastly/Cloudflare**: expiry and eviction are managed by the platform; + /// physical removal is not guaranteed to be immediate. + /// - **`DynamoDB` record bindings**: native TTL marks a row for asynchronous + /// deletion. Reads, authoritative existence and listings can still expose + /// the expired row until `DynamoDB` removes it. Legacy ordinary bindings + /// retain their existing logical-expiry behavior. async fn put_bytes_with_ttl( &self, key: &str, value: Bytes, ttl: Duration, ) -> Result<(), KvError>; + + /// Return immutable, already-prepared record capability without I/O. + /// Default ordinary-only providers advertise no record operations. + #[inline] + fn record_binding(&self) -> Option { + None + } + + /// Delete the same value ordinary KV uses, preserving generation allocation history. + /// Do not retry a delete after ambiguous completion. + #[inline] + async fn record_delete(&self, _key: &str, _partition: &RecordPartition) -> Result<(), KvError> { + Err(KvError::Unsupported { + operation: "record_delete".to_owned(), + }) + } + + /// Establish exact-key existence against authoritative state, not a lookup miss. + #[inline] + async fn record_exists_strong( + &self, + _key: &str, + _partition: &RecordPartition, + ) -> Result { + Err(KvError::Unsupported { + operation: "record_exists_strong".to_owned(), + }) + } + + /// Atomically evaluate Add/CAS and replace the same value ordinary KV uses. + /// Do not replay mutations. Uncertain completion must remain a distinct error. + #[inline] + async fn record_insert( + &self, + _key: &str, + _partition: &RecordPartition, + _write: RecordWrite<'_>, + ) -> Result { + Err(KvError::Unsupported { + operation: "record_insert".to_owned(), + }) + } + + /// Strongly read one complete partition using the native inner continuation. + /// Validate native continuation structure and progress, preserve empty pages + /// with continuation, and never substitute ordinary global enumeration. + #[inline] + async fn record_list_partition( + &self, + _partition: &RecordPartition, + _cursor: Option<&str>, + _limit: usize, + ) -> Result { + Err(KvError::Unsupported { + operation: "record_list_partition".to_owned(), + }) + } + + /// Read a record, retaining native lookup consistency. Decode metadata as UTF-8. + #[inline] + async fn record_lookup( + &self, + _key: &str, + _partition: &RecordPartition, + ) -> Result, KvError> { + Err(KvError::Unsupported { + operation: "record_lookup".to_owned(), + }) + } } // --------------------------------------------------------------------------- @@ -978,6 +1079,10 @@ pub struct NoopKvStore; #[cfg(any(test, feature = "test-utils"))] #[async_trait(?Send)] +#[expect( + clippy::missing_trait_methods, + reason = "ordinary-only provider intentionally retains Unsupported record defaults" +)] impl KvStore for NoopKvStore { #[inline] async fn delete(&self, _key: &str) -> Result<(), KvError> { @@ -1045,6 +1150,10 @@ mod tests { } #[async_trait(?Send)] + #[expect( + clippy::missing_trait_methods, + reason = "ordinary-only provider intentionally retains Unsupported record defaults" + )] impl KvStore for Failed { async fn delete(&self, _key: &str) -> Result<(), KvError> { Err(KvError::OpenFailed { reason: self.0 }) @@ -1077,6 +1186,10 @@ mod tests { } #[async_trait(?Send)] + #[expect( + clippy::missing_trait_methods, + reason = "ordinary-only provider intentionally retains Unsupported record defaults" + )] impl KvStore for MockStore { async fn delete(&self, key: &str) -> Result<(), KvError> { let mut data = self.data.lock().unwrap(); diff --git a/crates/edgezero-core/src/lib.rs b/crates/edgezero-core/src/lib.rs index 303a3ab1..d42841a4 100644 --- a/crates/edgezero-core/src/lib.rs +++ b/crates/edgezero-core/src/lib.rs @@ -40,6 +40,7 @@ pub mod params; pub mod platform; pub mod pre_dispatch; pub mod probe; +pub mod record_store; pub mod responder; pub mod response; pub mod response_egress; @@ -118,6 +119,11 @@ pub use platform::{ PlatformUnknownReason, }; pub use pre_dispatch::PreDispatchRequest; +pub use record_store::{ + Record, RecordBinding, RecordGeneration, RecordHandle, RecordKeyLayout, RecordLayout, + RecordPage, RecordPartition, RecordStoreRequirements, RecordWrite, RecordWriteMode, + RecordWriteOutcome, +}; pub use response_egress::{ DEFAULT_RESPONSE_WRITE_BUDGET, DetachedResponseEgressDecision, DetachedResponseEgressDecisionFactory, DetachedResponseEgressHead, diff --git a/crates/edgezero-core/src/record_store.rs b/crates/edgezero-core/src/record_store.rs new file mode 100644 index 00000000..f3ba6edf --- /dev/null +++ b/crates/edgezero-core/src/record_store.rs @@ -0,0 +1,710 @@ +//! Portable conditional records over the same value rows as ordinary KV. +//! +//! Record capability is established during provider preparation, without a write +//! probe. Lookup consistency and TTL visibility remain provider-specific. Exact +//! existence and every partition-list page are authoritative. A traversal is live, +//! not a snapshot, and ends only when its continuation is absent. + +/// Reusable assertions over the public handle for real provider fixtures. +/// +/// These do not establish native race, restart, expiry timing or fault-injection +/// behavior. Those require independent provider-specific runtime checks. +#[cfg(any(test, feature = "test-utils"))] +pub mod contract { + #![expect( + clippy::expect_used, + clippy::missing_assert_message, + clippy::unwrap_used, + reason = "shared test assertions intentionally panic at the failed provider operation or assertion" + )] + + use std::collections::BTreeSet; + + use bytes::Bytes; + + use super::{RecordGeneration, RecordHandle, RecordWrite, RecordWriteMode, RecordWriteOutcome}; + use crate::key_value_store::KvHandle; + + async fn assert_coherence(kv: &KvHandle, records: &RecordHandle, partition_prefix: &str) { + let key = format!("{partition_prefix}contract-main"); + assert_conditionals(kv, records, &key).await; + assert_replacements(kv, records, &key).await; + assert_deletion(kv, records, &key).await; + } + + async fn assert_conditionals(kv: &KvHandle, records: &RecordHandle, key: &str) { + let body = b"\x00\xffrecord\x00"; + assert!(!records.exists_strong(key).await.unwrap()); + let mut add = write(body, RecordWriteMode::Add); + add.metadata = Some("\u{00e9}\u{6587}\u{5b57}"); + assert_eq!( + records.insert(key, add).await.unwrap(), + RecordWriteOutcome::Written + ); + assert_eq!( + records.insert(key, add).await.unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + let first = records.lookup(key).await.unwrap().unwrap(); + assert_eq!(first.body.as_ref(), body); + assert_eq!(first.metadata.as_deref(), add.metadata); + assert_eq!(kv.get_bytes(key).await.unwrap().unwrap().as_ref(), body); + assert!(records.exists_strong(key).await.unwrap()); + assert!( + !records + .exists_strong(&format!("{key}-missing")) + .await + .unwrap() + ); + + let mut cas = write(b"cas", RecordWriteMode::IfGenerationMatch(first.generation)); + cas.metadata = add.metadata; + assert_eq!( + records.insert(key, cas).await.unwrap(), + RecordWriteOutcome::Written + ); + assert_eq!( + records.insert(key, cas).await.unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + } + + async fn assert_deletion(kv: &KvHandle, records: &RecordHandle, key: &str) { + let before_delete = records.lookup(key).await.unwrap().unwrap().generation; + records.delete(key).await.unwrap(); + assert_eq!(kv.get_bytes(key).await.unwrap(), None); + assert!(!records.exists_strong(key).await.unwrap()); + records + .insert(key, write(b"recreated", RecordWriteMode::Add)) + .await + .unwrap(); + assert_stale(records, key, before_delete).await; + let mut empty_metadata = write(b"\x00\xffrecord\x00", RecordWriteMode::Overwrite); + empty_metadata.metadata = Some(""); + records.insert(key, empty_metadata).await.unwrap(); + assert_eq!(records.lookup(key).await.unwrap().unwrap().metadata, None); + kv.delete(key).await.unwrap(); + assert!(!records.exists_strong(key).await.unwrap()); + } + + async fn assert_pagination(kv: &KvHandle, records: &RecordHandle, partition_prefix: &str) { + let partition = records + .partition(partition_prefix) + .expect("complete partition"); + let expected: BTreeSet = (0_usize..4_usize) + .map(|index| format!("{partition_prefix}contract-page-{index}")) + .collect(); + for key in &expected { + records + .insert(key, write(b"\x00\xffrecord\x00", RecordWriteMode::Add)) + .await + .unwrap(); + } + let mut cursor = None; + let mut seen_cursors = BTreeSet::new(); + let mut actual = BTreeSet::new(); + loop { + let page = records + .list_partition(&partition, cursor.as_deref(), 2) + .await + .unwrap(); + assert!(page.keys.len() <= 2); + for key in page.keys { + assert!(actual.insert(key), "duplicate partition key"); + } + cursor = page.cursor; + let Some(next) = &cursor else { break }; + assert!( + seen_cursors.insert(next.clone()), + "repeated partition continuation" + ); + } + assert_eq!(actual, expected); + + // Comparing sets does not require provider ordering. Empty-prefix ordinary + // listing is independent of complete-partition validation. + let mut ordinary_cursor = None; + let mut ordinary_seen_cursors = BTreeSet::new(); + let mut ordinary_keys = BTreeSet::new(); + loop { + let page = kv + .list_keys_page("", ordinary_cursor.as_deref(), 2) + .await + .unwrap(); + assert!(page.keys.len() <= 2); + for key in page.keys { + assert!(ordinary_keys.insert(key), "duplicate ordinary key"); + } + ordinary_cursor = page.cursor; + let Some(next) = &ordinary_cursor else { break }; + assert!( + ordinary_seen_cursors.insert(next.clone()), + "repeated ordinary continuation" + ); + } + assert_eq!(ordinary_keys, expected); + for key in expected { + records.delete(&key).await.unwrap(); + } + } + + /// Exercise coherence, atomic precondition outcomes, generation invalidation, + /// text/body fidelity and complete unordered partition/global pagination. + /// + /// The fixture must provide an isolated empty store and accept keys of the + /// form of the supplied prefix followed by `contract-*`. Run in a quiescent fixture + /// where acknowledged writes are observable by subsequent lookups. This is a + /// fixture condition, not a portable strong-lookup promise. No mutation is retried. + /// + /// # Panics + /// Panics on a failed contract assertion or provider error. + #[inline] + pub async fn assert_record_contract(kv: &KvHandle, partition_prefix: &str) { + let records = kv.require_records().expect("prepared record binding"); + assert_coherence(kv, &records, partition_prefix).await; + assert_pagination(kv, &records, partition_prefix).await; + } + + async fn assert_replacements(kv: &KvHandle, records: &RecordHandle, key: &str) { + let before_ordinary = records.lookup(key).await.unwrap().unwrap().generation; + kv.put_bytes(key, Bytes::from_static(b"ordinary")) + .await + .unwrap(); + let ordinary = records.lookup(key).await.unwrap().unwrap(); + assert_eq!(ordinary.body.as_ref(), b"ordinary"); + assert_eq!(ordinary.metadata, None); + assert_ne!(ordinary.generation, before_ordinary); + assert_stale(records, key, before_ordinary).await; + + let mut with_metadata = write(b"metadata", RecordWriteMode::Overwrite); + with_metadata.metadata = Some("\u{00e9}\u{6587}\u{5b57}"); + records.insert(key, with_metadata).await.unwrap(); + let before_ttl = records.lookup(key).await.unwrap().unwrap().generation; + kv.put_bytes_with_ttl(key, Bytes::from_static(b"ttl"), KvHandle::MIN_TTL) + .await + .unwrap(); + assert_stale(records, key, before_ttl).await; + let ttl_record = records.lookup(key).await.unwrap().unwrap(); + assert_eq!(ttl_record.body.as_ref(), b"ttl"); + assert_eq!(ttl_record.metadata, None); + kv.put_bytes(key, Bytes::from_static(b"no-expiry")) + .await + .unwrap(); + assert_stale(records, key, ttl_record.generation).await; + } + + async fn assert_stale(records: &RecordHandle, key: &str, generation: RecordGeneration) { + assert_eq!( + records + .insert( + key, + write( + b"must-not-commit", + RecordWriteMode::IfGenerationMatch(generation) + ) + ) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed, + ); + } + + fn write(body: &[u8], mode: RecordWriteMode) -> RecordWrite<'_> { + RecordWrite { + body, + metadata: None, + mode, + ttl: None, + } + } +} + +use std::collections::BTreeMap; +use std::fmt; +use std::sync::Arc; +use std::time::Duration; + +use bytes::Bytes; +use serde::{Deserialize, Serialize}; + +use crate::key_value_store::{KvError, KvHandle, KvStore}; + +/// An equality token scoped to one key and binding, not an ordered counter. +#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)] +pub struct RecordGeneration(u64); + +impl RecordGeneration { + /// Encode a token for a provider. The number has no ordering guarantee. + #[must_use] + #[inline] + pub const fn as_u64(self) -> u64 { + self.0 + } + + /// Construct a token from its provider representation. + #[must_use] + #[inline] + pub const fn from_u64(value: u64) -> Self { + Self(value) + } +} + +/// A record's exact body, optional UTF-8 metadata and generation. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct Record { + /// Exact value bytes, without an envelope. + pub body: Bytes, + /// Equality token for this key and binding. + pub generation: RecordGeneration, + /// Nonempty UTF-8 text. Empty metadata is normalized to absence. + pub metadata: Option, +} + +/// A replacement whose precondition is evaluated atomically by the provider. +#[derive(Clone, Copy, Debug)] +pub struct RecordWrite<'value> { + /// Replacement value bytes. + pub body: &'value [u8], + /// Optional UTF-8 text, normalized before provider I/O. + pub metadata: Option<&'value str>, + /// Provider-atomic precondition. + pub mode: RecordWriteMode, + /// Native provider TTL, or no expiry. + pub ttl: Option, +} + +/// Atomic replacement preconditions. Overwrite has no generation precondition. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum RecordWriteMode { + /// Write only when the key does not exist under native expiry rules. + Add, + /// Write only while this key still has the supplied generation. + IfGenerationMatch(RecordGeneration), + /// Replace without a generation precondition. + Overwrite, +} + +/// An acknowledged mutation or a definitive Add/CAS precondition rejection. +/// +/// Acknowledgement does not return a new generation. A later lookup may observe +/// another writer. Ambiguous completion is [`KvError::MutationUncertain`], not an +/// outcome, and must not cause automatic replay. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum RecordWriteOutcome { + /// Definitive Add/CAS rejection. The replacement did not occur. + PreconditionFailed, + /// The provider acknowledged the write, without returning its generation. + Written, +} + +/// Full logical keys from one strong partition page, without ordering promises. +/// Empty and short pages may have continuation. Only `cursor == None` ends a traversal. +#[derive(Clone, Debug, Default, Eq, PartialEq)] +pub struct RecordPage { + /// Opaque continuation. Only absence establishes completion. + pub cursor: Option, + /// Full logical keys, without an ordering guarantee. + pub keys: Vec, +} + +/// Application-owned rules for stable, disjoint, enumerable key prefixes. +/// +/// Implementations validate their rules in [`Self::validate`]. Equal id/version +/// pairs must describe identical rules for the lifetime of the stored format. +/// Partition prefixes must be disjoint and must preserve the original key bytes. +/// Changing placement requires an explicit format/cutover choice, not new rules +/// under the same identity. Core does not parse application key formats. +pub trait RecordKeyLayout: Send + Sync { + /// Stable application format identity. + fn id(&self) -> &str; + + /// Resolve a full key to a borrowed, complete partition prefix of that key. + /// + /// # Errors + /// Rejects keys outside the declared format or with ambiguous membership. + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError>; + + /// Validate the rule declaration, including stable, non-overlapping partitions. + /// + /// # Errors + /// Returns a validation error when the declaration cannot satisfy the contract. + fn validate(&self) -> Result<(), KvError>; + + /// Accept only complete enumerable partition prefixes, never partial groups. + /// + /// # Errors + /// Rejects a prefix that does not identify exactly one declared partition. + fn validate_partition(&self, prefix: &str) -> Result<(), KvError>; + + /// Version of the stored key layout, captured when the declaration is validated. + fn version(&self) -> u32; +} + +/// Validated application rules with a captured immutable identity. +#[derive(Clone)] +pub struct RecordLayout { + id: String, + rules: Arc, + version: u32, +} + +impl fmt::Debug for RecordLayout { + #[inline] + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("RecordLayout").finish_non_exhaustive() + } +} + +#[expect( + clippy::missing_trait_methods, + reason = "retain default ne as the negation of captured-identity equality" +)] +impl PartialEq for RecordLayout { + #[inline] + fn eq(&self, other: &Self) -> bool { + self.id == other.id && self.version == other.version + } +} + +#[expect( + clippy::missing_trait_methods, + reason = "Eq marker assertion methods are unstable compiler machinery" +)] +impl Eq for RecordLayout {} + +impl RecordLayout { + /// Captured stable layout identity. + #[must_use] + #[inline] + pub fn id(&self) -> &str { + &self.id + } + + /// Validate a declaration and capture its stable identity once. + /// + /// # Errors + /// Returns declaration errors or rejects an empty layout identity. + #[inline] + pub fn new(layout: KeyLayout) -> Result + where + KeyLayout: RecordKeyLayout + 'static, + { + layout.validate()?; + let id = layout.id().to_owned(); + if id.is_empty() { + return Err(KvError::Validation( + "record layout identity is empty".to_owned(), + )); + } + Ok(Self { + version: layout.version(), + id, + rules: Arc::new(layout), + }) + } + + /// Validate a complete enumerable prefix. Ordinary listing does not use this check. + /// + /// # Errors + /// Rejects invalid prefixes or incomplete/undeclared partitions. + #[inline] + pub fn partition(&self, prefix: &str) -> Result { + KvHandle::validate_prefix(prefix)?; + self.rules.validate_partition(prefix)?; + Ok(RecordPartition { + layout_id: self.id.clone(), + layout_version: self.version, + prefix: prefix.to_owned(), + }) + } + + /// Resolve a full logical key and validate its actual prefix membership. + /// + /// # Errors + /// Rejects invalid keys, undeclared keys or an inconsistent layout result. + #[inline] + pub fn resolve(&self, key: &str) -> Result { + KvHandle::validate_key(key)?; + let prefix = self.rules.partition_prefix(key)?; + if !key.starts_with(prefix) { + return Err(KvError::Validation( + "record layout resolved a prefix outside the key".to_owned(), + )); + } + self.partition(prefix) + } + + /// Captured layout version. + #[must_use] + #[inline] + pub const fn version(&self) -> u32 { + self.version + } +} + +/// A complete partition validated against an immutable application layout. +#[derive(Clone, Eq, PartialEq)] +pub struct RecordPartition { + layout_id: String, + layout_version: u32, + prefix: String, +} + +impl fmt::Debug for RecordPartition { + #[inline] + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("RecordPartition").finish_non_exhaustive() + } +} + +impl RecordPartition { + /// Full logical prefix identifying this complete partition. + #[must_use] + #[inline] + pub fn prefix(&self) -> &str { + &self.prefix + } +} + +/// Immutable per-logical-ID requirements installed during application configuration. +pub type RecordStoreRequirements = BTreeMap; + +/// Prepared record capability for one provider and backing scope. +/// +/// `scope` must be stable, non-secret and identify the bound backing namespace. +/// `format` identifies its explicit stored record format. These identities bind +/// cursors; they are not permissions or availability guarantees. Providers must +/// route ordinary operations to the same value rows, invalidate generations on +/// every replacement, and retain generation history across deletion and expiry. +#[derive(Clone)] +pub struct RecordBinding { + /// Explicit provider record-format identity. + pub format: String, + /// Immutable, validated application layout. + pub layout: RecordLayout, + /// Actual provider limit in UTF-8 bytes, excluding absent/empty metadata. + pub max_metadata_bytes: usize, + /// Stable, non-secret backing namespace identity. Omitted from Debug output. + pub scope: String, +} + +impl fmt::Debug for RecordBinding { + #[inline] + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("RecordBinding").finish_non_exhaustive() + } +} + +#[derive(Deserialize, Serialize)] +struct RecordCursor { + cursor: String, + format: String, + layout_id: String, + layout_version: u32, + operation: String, + partition: String, + scope: String, +} + +/// A validated record view over the exact provider used by its ordinary KV handle. +#[derive(Clone)] +pub struct RecordHandle { + binding: RecordBinding, + store: Arc, +} + +impl fmt::Debug for RecordHandle { + #[inline] + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("RecordHandle").finish_non_exhaustive() + } +} + +impl RecordHandle { + /// Immutable capability captured from the prepared provider. + #[must_use] + #[inline] + pub fn binding(&self) -> &RecordBinding { + &self.binding + } + + fn decode_cursor(&self, partition: &RecordPartition, encoded: &str) -> Result { + let cursor: RecordCursor = serde_json::from_str(encoded) + .map_err(|_error| KvError::Validation("record cursor is invalid".to_owned()))?; + if cursor.scope != self.binding.scope + || cursor.format != self.binding.format + || cursor.layout_id != self.binding.layout.id + || cursor.layout_version != self.binding.layout.version + || cursor.partition != partition.prefix + || cursor.operation != "partition" + || cursor.cursor.is_empty() + { + return Err(KvError::Validation( + "record cursor scope mismatch".to_owned(), + )); + } + Ok(cursor.cursor) + } + + /// Delete a record without replay. Deleting a missing value succeeds. + /// + /// # Errors + /// Returns validation/provider errors, including possible mutation completion. + #[inline] + pub async fn delete(&self, key: &str) -> Result<(), KvError> { + let partition = self.binding.layout.resolve(key)?; + self.store.record_delete(key, &partition).await + } + + /// Check exact-key existence against authoritative provider state. + /// + /// # Errors + /// Returns validation/provider errors when authority cannot be established. + #[inline] + pub async fn exists_strong(&self, key: &str) -> Result { + let partition = self.binding.layout.resolve(key)?; + self.store.record_exists_strong(key, &partition).await + } + + /// Atomically replace a record. This method never retries a mutation. + /// + /// # Errors + /// Returns input/provider failures, including possible completion as + /// [`KvError::MutationUncertain`]. An unconditional rejection is not contention. + #[inline] + pub async fn insert( + &self, + key: &str, + mut write: RecordWrite<'_>, + ) -> Result { + let partition = self.binding.layout.resolve(key)?; + KvHandle::validate_value(write.body)?; + if let Some(ttl) = write.ttl { + KvHandle::validate_ttl(ttl)?; + } + write.metadata = write.metadata.filter(|text| !text.is_empty()); + if write + .metadata + .is_some_and(|text| text.len() > self.binding.max_metadata_bytes) + { + return Err(KvError::Validation( + "record metadata exceeds the provider byte limit".to_owned(), + )); + } + let mode = write.mode; + let outcome = self.store.record_insert(key, &partition, write).await?; + if mode == RecordWriteMode::Overwrite && outcome == RecordWriteOutcome::PreconditionFailed { + return Err(KvError::Internal(anyhow::anyhow!( + "record provider rejected an unconditional write as contention" + ))); + } + Ok(outcome) + } + + /// Read one strongly consistent page from a complete partition. + /// + /// The opaque continuation is bound to backing scope, format, layout and + /// partition/query mode. Providers receive their native continuation only. + /// The traversal is live, unordered and complete only when continuation is absent. + /// + /// # Errors + /// Rejects foreign/malformed/nonadvancing cursors, invalid partitions/limits, + /// invalid provider keys or provider failures. + #[inline] + pub async fn list_partition( + &self, + partition: &RecordPartition, + cursor: Option<&str>, + limit: usize, + ) -> Result { + KvHandle::validate_list_limit(limit)?; + if self.binding.layout.partition(partition.prefix())? != *partition { + return Err(KvError::Validation( + "record partition layout mismatch".to_owned(), + )); + } + let decoded = cursor + .map(|encoded| self.decode_cursor(partition, encoded)) + .transpose()?; + let mut page = self + .store + .record_list_partition(partition, decoded.as_deref(), limit) + .await?; + if page.keys.len() > limit + || page.keys.iter().any(|key| { + !self + .binding + .layout + .resolve(key) + .is_ok_and(|resolved| resolved == *partition) + }) + { + return Err(KvError::Internal(anyhow::anyhow!( + "record provider returned an invalid partition page" + ))); + } + if page + .cursor + .as_deref() + .is_some_and(|next| next.is_empty() || Some(next) == decoded.as_deref()) + { + return Err(KvError::Internal(anyhow::anyhow!( + "record provider returned a nonadvancing continuation" + ))); + } + page.cursor = page + .cursor + .map(|native| { + serde_json::to_string(&RecordCursor { + scope: self.binding.scope.clone(), + format: self.binding.format.clone(), + layout_id: self.binding.layout.id.clone(), + layout_version: self.binding.layout.version, + partition: partition.prefix.clone(), + operation: "partition".to_owned(), + cursor: native, + }) + .map_err(KvError::from) + }) + .transpose()?; + Ok(page) + } + + /// Read a generation-bearing record. A miss need not prove authoritative absence. + /// + /// # Errors + /// Returns key/layout validation errors or provider read/decoding failures. + #[inline] + pub async fn lookup(&self, key: &str) -> Result, KvError> { + let partition = self.binding.layout.resolve(key)?; + let mut result = self.store.record_lookup(key, &partition).await?; + if let Some(record) = &mut result + && record.metadata.as_deref() == Some("") + { + record.metadata = None; + } + Ok(result) + } + + #[inline] + pub(crate) fn new(store: Arc) -> Result { + let binding = store.record_binding().ok_or_else(|| KvError::Unsupported { + operation: "records".to_owned(), + })?; + if binding.scope.is_empty() || binding.format.is_empty() { + return Err(KvError::Validation( + "record binding scope and format must be declared".to_owned(), + )); + } + Ok(Self { binding, store }) + } + + /// Validate a complete partition without I/O or routing mutation. + /// + /// # Errors + /// Rejects invalid or incomplete partition prefixes. + #[inline] + pub fn partition(&self, prefix: &str) -> Result { + self.binding.layout.partition(prefix) + } +} diff --git a/crates/edgezero-core/src/store_registry.rs b/crates/edgezero-core/src/store_registry.rs index 28180fb6..204cf658 100644 --- a/crates/edgezero-core/src/store_registry.rs +++ b/crates/edgezero-core/src/store_registry.rs @@ -26,7 +26,8 @@ use std::collections::BTreeMap; use bytes::Bytes; use crate::config_store::{BoundedStoreRead, ConfigStoreHandle}; -use crate::key_value_store::KvHandle; +use crate::key_value_store::{KvError, KvHandle}; +use crate::record_store::RecordStoreRequirements; use crate::secret_store::{SecretError, SecretHandle}; use crate::time::{Deadline, MonotonicClock}; @@ -250,6 +251,32 @@ impl StoreRegistry { } } +impl StoreRegistry { + /// Validate already-prepared handles against immutable application requirements. + /// This does not install layouts, alter routing or perform provider I/O. + /// + /// # Errors + /// Rejects missing IDs, unsupported record capability and layout mismatches. + #[inline] + pub fn validate_record_requirements( + &self, + requirements: &RecordStoreRequirements, + ) -> Result<(), KvError> { + for (id, layout) in requirements { + let handle = self.named_ref(id).ok_or_else(|| { + KvError::Validation("required record store is not registered".to_owned()) + })?; + let records = handle.require_records()?; + if records.binding().layout != *layout { + return Err(KvError::Validation( + "prepared record store layout does not match its declaration".to_owned(), + )); + } + } + Ok(()) + } +} + /// Registry of per-id KV handles. pub type KvRegistry = StoreRegistry; /// Registry of per-id config bindings (handle + default key). diff --git a/crates/edgezero-core/tests/record_api.rs b/crates/edgezero-core/tests/record_api.rs new file mode 100644 index 00000000..f98fc54a --- /dev/null +++ b/crates/edgezero-core/tests/record_api.rs @@ -0,0 +1,588 @@ +//! Public API and boundary tests. Scripted providers here are not native-provider +//! conformance evidence for atomicity, generation lifetime, storage or TTL. + +#[cfg(test)] +mod tests { + #![expect( + clippy::assertions_on_result_states, + reason = "boundary rejection matrices check error presence; classification is asserted separately" + )] + + use std::collections::VecDeque; + use std::sync::Arc; + use std::sync::Mutex; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::time::Duration; + + use async_trait::async_trait; + use bytes::Bytes; + use edgezero_core::key_value_store::{KvError, KvHandle, KvPage, KvStore}; + use edgezero_core::{ + Record, RecordBinding, RecordGeneration, RecordKeyLayout, RecordLayout, RecordPage, + RecordPartition, RecordWrite, RecordWriteMode, RecordWriteOutcome, + }; + use futures::executor::block_on; + + struct Groups(u32); + + impl RecordKeyLayout for Groups { + fn id(&self) -> &'static str { + "test.groups" + } + + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + let (group, suffix) = key + .split_once('/') + .ok_or_else(|| KvError::Validation("missing group".to_owned()))?; + if group.is_empty() || suffix.is_empty() { + return Err(KvError::Validation("missing group/key".to_owned())); + } + key.strip_suffix(suffix) + .ok_or_else(|| KvError::Validation("missing group prefix".to_owned())) + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.len() > 1 && prefix.ends_with('/') && prefix.matches('/').count() == 1 { + Ok(()) + } else { + Err(KvError::Validation("incomplete group".to_owned())) + } + } + + fn version(&self) -> u32 { + self.0 + } + } + + struct Ordinary; + + #[expect( + clippy::missing_trait_methods, + reason = "ordinary provider intentionally uses unsupported record defaults" + )] + #[async_trait(?Send)] + impl KvStore for Ordinary { + async fn delete(&self, _: &str) -> Result<(), KvError> { + panic!("unexpected I/O") + } + + async fn get_bytes(&self, _: &str) -> Result, KvError> { + panic!("unexpected I/O") + } + + async fn list_keys_page( + &self, + _: &str, + _: Option<&str>, + _: usize, + ) -> Result { + panic!("unexpected I/O") + } + + async fn put_bytes(&self, _: &str, _: Bytes) -> Result<(), KvError> { + panic!("unexpected I/O") + } + + async fn put_bytes_with_ttl(&self, _: &str, _: Bytes, _: Duration) -> Result<(), KvError> { + panic!("unexpected I/O") + } + } + + struct Scripted { + binding: RecordBinding, + calls: AtomicUsize, + cursors: Mutex>>, + outcome: RecordWriteOutcome, + pages: Mutex>, + uncertain: bool, + writes: Mutex>>, + } + + impl Scripted { + fn handle(self) -> (Arc, KvHandle) { + let store = Arc::new(self); + let handle = KvHandle::new(Arc::::clone(&store)); + (store, handle) + } + + fn new() -> Self { + Self { + binding: RecordBinding { + layout: RecordLayout::new(Groups(1)).unwrap(), + scope: "test-resource-sensitive".to_owned(), + format: "test-v1".to_owned(), + max_metadata_bytes: 4, + }, + calls: AtomicUsize::new(0), + cursors: Mutex::new(Vec::new()), + outcome: RecordWriteOutcome::Written, + pages: Mutex::new(VecDeque::new()), + uncertain: false, + writes: Mutex::new(Vec::new()), + } + } + } + + #[expect( + clippy::missing_trait_methods, + reason = "ordinary existence default is unrelated to scripted record operations" + )] + #[async_trait(?Send)] + impl KvStore for Scripted { + async fn delete(&self, _: &str) -> Result<(), KvError> { + panic!("unexpected ordinary I/O") + } + + async fn get_bytes(&self, _: &str) -> Result, KvError> { + panic!("unexpected ordinary I/O") + } + + async fn list_keys_page( + &self, + _: &str, + _: Option<&str>, + _: usize, + ) -> Result { + panic!("unexpected ordinary I/O") + } + + async fn put_bytes(&self, _: &str, _: Bytes) -> Result<(), KvError> { + panic!("unexpected ordinary I/O") + } + + async fn put_bytes_with_ttl(&self, _: &str, _: Bytes, _: Duration) -> Result<(), KvError> { + panic!("unexpected ordinary I/O") + } + + fn record_binding(&self) -> Option { + Some(self.binding.clone()) + } + + async fn record_delete(&self, _: &str, _: &RecordPartition) -> Result<(), KvError> { + self.calls.fetch_add(1, Ordering::SeqCst); + if self.uncertain { + Err(KvError::MutationUncertain) + } else { + Ok(()) + } + } + + async fn record_exists_strong( + &self, + _: &str, + _: &RecordPartition, + ) -> Result { + self.calls.fetch_add(1, Ordering::SeqCst); + Ok(true) + } + + async fn record_insert( + &self, + _: &str, + partition: &RecordPartition, + write: RecordWrite<'_>, + ) -> Result { + assert_eq!(partition.prefix(), "a/"); + self.calls.fetch_add(1, Ordering::SeqCst); + self.writes + .lock() + .unwrap() + .push(write.metadata.map(str::to_owned)); + if self.uncertain { + Err(KvError::MutationUncertain) + } else { + Ok(self.outcome) + } + } + + async fn record_list_partition( + &self, + _: &RecordPartition, + cursor: Option<&str>, + _: usize, + ) -> Result { + self.calls.fetch_add(1, Ordering::SeqCst); + self.cursors.lock().unwrap().push(cursor.map(str::to_owned)); + Ok(self + .pages + .lock() + .unwrap() + .pop_front() + .expect("scripted page")) + } + + async fn record_lookup( + &self, + _: &str, + partition: &RecordPartition, + ) -> Result, KvError> { + assert_eq!(partition.prefix(), "a/"); + self.calls.fetch_add(1, Ordering::SeqCst); + Ok(Some(Record { + body: Bytes::from_static(b"\x00\xff"), + metadata: Some(String::new()), + generation: RecordGeneration::from_u64(u64::MAX), + })) + } + } + + fn write(metadata: Option<&str>, mode: RecordWriteMode) -> RecordWrite<'_> { + RecordWrite { + body: b"\x00\xff", + metadata, + ttl: None, + mode, + } + } + + #[test] + fn ordinary_only_acquisition_and_defaults_are_unsupported_without_io() { + let store: Arc = Arc::new(Ordinary); + assert!(matches!( + KvHandle::new(Arc::clone(&store)).require_records(), + Err(KvError::Unsupported { .. }) + )); + let partition = RecordLayout::new(Groups(1)) + .unwrap() + .resolve("a/k") + .unwrap(); + block_on(async { + assert!(matches!( + store.record_lookup("a/k", &partition).await, + Err(KvError::Unsupported { .. }) + )); + assert!(matches!( + store + .record_insert("a/k", &partition, write(None, RecordWriteMode::Add)) + .await, + Err(KvError::Unsupported { .. }) + )); + assert!(matches!( + store.record_exists_strong("a/k", &partition).await, + Err(KvError::Unsupported { .. }) + )); + assert!(matches!( + store.record_delete("a/k", &partition).await, + Err(KvError::Unsupported { .. }) + )); + assert!(matches!( + store.record_list_partition(&partition, None, 1).await, + Err(KvError::Unsupported { .. }) + )); + }); + } + + #[test] + fn acquisition_is_pure_and_debug_does_not_expose_binding_scope() { + let (store, kv) = Scripted::new().handle(); + let records = kv.require_records().unwrap(); + let _again = kv.clone().require_records().unwrap(); + let partition = records.partition("a/").unwrap(); + assert_eq!(store.calls.load(Ordering::SeqCst), 0); + for debug in [ + format!("{records:?}"), + format!("{:?}", records.binding()), + format!("{partition:?}"), + ] { + assert!(!debug.contains("sensitive")); + assert!(!debug.contains("a/")); + } + } + + #[test] + fn generation_preserves_all_u64_bits_and_only_promises_equality() { + let token = RecordGeneration::from_u64(u64::MAX); + assert_eq!(token.as_u64(), u64::MAX); + assert_eq!(token, RecordGeneration::from_u64(u64::MAX)); + assert_ne!(token, RecordGeneration::from_u64(0)); + } + + #[test] + fn layout_requires_complete_partition_and_preserves_full_key() { + let layout = RecordLayout::new(Groups(1)).unwrap(); + assert_eq!( + layout.resolve("\u{3b1}/\u{00e9}.x").unwrap().prefix(), + "\u{3b1}/" + ); + assert!(layout.partition("").is_err()); + assert!(layout.partition("\u{3b1}").is_err()); + assert!(layout.partition("\u{3b1}/nested/").is_err()); + assert!(layout.resolve("ungrouped").is_err()); + assert!(layout.resolve("/empty").is_err()); + assert!(layout.resolve("a/\n").is_err()); + } + + #[test] + fn invalid_binding_is_not_unsupported_or_availability() { + let mut provider = Scripted::new(); + provider.binding.scope.clear(); + assert!(matches!( + provider.handle().1.require_records(), + Err(KvError::Validation(_)) + )); + } + + #[test] + fn input_validation_precedes_io_and_metadata_counts_utf8_bytes() { + let (store, kv) = Scripted::new().handle(); + let records = kv.require_records().unwrap(); + block_on(async { + assert!( + records + .insert("ungrouped", write(None, RecordWriteMode::Add)) + .await + .is_err() + ); + assert!( + records + .insert( + "a/k", + write(Some("\u{3b1}\u{3b1}\u{3b1}"), RecordWriteMode::Add) + ) + .await + .is_err() + ); + let mut invalid_ttl = write(None, RecordWriteMode::Add); + invalid_ttl.ttl = Some(Duration::from_secs(1)); + assert!(records.insert("a/k", invalid_ttl).await.is_err()); + assert!( + records + .list_partition(&records.partition("a/").unwrap(), None, 0) + .await + .is_err() + ); + assert_eq!(store.calls.load(Ordering::SeqCst), 0); + assert_eq!( + records + .insert("a/k", write(Some("\u{3b1}\u{3b1}"), RecordWriteMode::Add)) + .await + .unwrap(), + RecordWriteOutcome::Written + ); + }); + assert_eq!( + store.writes.lock().unwrap().as_slice(), + &[Some("\u{3b1}\u{3b1}".to_owned())] + ); + } + + #[test] + fn empty_metadata_normalizes_and_lookup_keeps_binary_body_and_generation() { + let (store, kv) = Scripted::new().handle(); + let records = kv.require_records().unwrap(); + block_on(async { + records + .insert("a/k", write(Some(""), RecordWriteMode::Overwrite)) + .await + .unwrap(); + records + .insert("a/k", write(None, RecordWriteMode::Overwrite)) + .await + .unwrap(); + let record = records.lookup("a/k").await.unwrap().unwrap(); + assert_eq!(record.body.as_ref(), b"\x00\xff"); + assert_eq!(record.metadata, None); + assert_eq!(record.generation.as_u64(), u64::MAX); + assert!(records.exists_strong("a/k").await.unwrap()); + }); + assert_eq!(store.writes.lock().unwrap().as_slice(), &[None, None]); + } + + #[test] + fn definitive_preconditions_are_not_unconditional_failures_or_uncertainty() { + let mut provider = Scripted::new(); + provider.outcome = RecordWriteOutcome::PreconditionFailed; + let (_, kv) = provider.handle(); + let records = kv.require_records().unwrap(); + block_on(async { + for mode in [ + RecordWriteMode::Add, + RecordWriteMode::IfGenerationMatch(RecordGeneration::from_u64(5)), + ] { + assert_eq!( + records.insert("a/k", write(None, mode)).await.unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + } + assert!(matches!( + records + .insert("a/k", write(None, RecordWriteMode::Overwrite)) + .await, + Err(KvError::Internal(_)) + )); + }); + } + + #[test] + fn ambiguous_insert_and_delete_are_never_replayed() { + let mut provider = Scripted::new(); + provider.uncertain = true; + let (store, kv) = provider.handle(); + let records = kv.require_records().unwrap(); + block_on(async { + assert!(matches!( + records + .insert("a/k", write(None, RecordWriteMode::Add)) + .await, + Err(KvError::MutationUncertain) + )); + assert_eq!(store.calls.load(Ordering::SeqCst), 1); + assert!(matches!( + records.delete("a/k").await, + Err(KvError::MutationUncertain) + )); + assert_eq!(store.calls.load(Ordering::SeqCst), 2); + }); + assert_eq!( + KvError::MutationUncertain.to_string(), + "kv mutation may have committed" + ); + } + + #[test] + fn empty_page_continues_and_provider_receives_only_native_cursor() { + let provider = Scripted::new(); + provider.pages.lock().unwrap().extend([ + RecordPage { + keys: vec![], + cursor: Some("native-next".to_owned()), + }, + RecordPage { + keys: vec!["a/one".to_owned()], + cursor: None, + }, + ]); + let (store, kv) = provider.handle(); + let records = kv.require_records().unwrap(); + let partition = records.partition("a/").unwrap(); + block_on(async { + let first = records.list_partition(&partition, None, 1).await.unwrap(); + assert!(first.keys.is_empty()); + assert!(first.cursor.is_some()); + let second = records + .list_partition(&partition, first.cursor.as_deref(), 1) + .await + .unwrap(); + assert_eq!(second.keys, ["a/one"]); + assert!(second.cursor.is_none()); + }); + assert_eq!( + store.cursors.lock().unwrap().as_slice(), + &[None, Some("native-next".to_owned())] + ); + } + + #[test] + fn foreign_cursor_scope_layout_partition_and_operation_are_rejected_before_io() { + let provider = Scripted::new(); + provider.pages.lock().unwrap().push_back(RecordPage { + keys: vec![], + cursor: Some("native".to_owned()), + }); + let (store, kv) = provider.handle(); + let records = kv.require_records().unwrap(); + let partition = records.partition("a/").unwrap(); + block_on(async { + let cursor = records + .list_partition(&partition, None, 1) + .await + .unwrap() + .cursor + .unwrap(); + for (field, value) in [ + ("scope", serde_json::json!("another-scope")), + ("format", serde_json::json!("another-format")), + ("layout_id", serde_json::json!("another-layout")), + ("layout_version", serde_json::json!(2_u32)), + ("partition", serde_json::json!("b/")), + ("operation", serde_json::json!("scan")), + ("cursor", serde_json::json!("")), + ] { + let mut foreign: serde_json::Value = serde_json::from_str(&cursor).unwrap(); + foreign[field] = value; + assert!(matches!( + records + .list_partition(&partition, Some(&foreign.to_string()), 1) + .await, + Err(KvError::Validation(_)) + )); + } + assert!( + records + .list_partition(&partition, Some("not a cursor"), 1) + .await + .is_err() + ); + assert!( + records + .list_partition(&partition, Some(r#"{"cursor":"x","prefix":"a/"}"#), 1) + .await + .is_err() + ); + let foreign_partition = RecordLayout::new(Groups(2)) + .unwrap() + .partition("a/") + .unwrap(); + assert!( + records + .list_partition(&foreign_partition, None, 1) + .await + .is_err() + ); + }); + assert_eq!(store.calls.load(Ordering::SeqCst), 1); + } + + #[test] + fn invalid_provider_pages_and_nonadvancing_continuation_are_errors() { + for keys in [ + vec!["b/foreign".to_owned()], + vec!["a/one".to_owned(), "a/two".to_owned()], + vec!["ungrouped".to_owned()], + ] { + let provider = Scripted::new(); + provider + .pages + .lock() + .unwrap() + .push_back(RecordPage { keys, cursor: None }); + let (_, kv) = provider.handle(); + let records = kv.require_records().unwrap(); + block_on(async { + assert!(matches!( + records + .list_partition(&records.partition("a/").unwrap(), None, 1) + .await, + Err(KvError::Internal(_)) + )); + }); + } + let provider = Scripted::new(); + provider.pages.lock().unwrap().extend([ + RecordPage { + keys: vec![], + cursor: Some("same".to_owned()), + }, + RecordPage { + keys: vec![], + cursor: Some("same".to_owned()), + }, + ]); + let (_, kv) = provider.handle(); + let records = kv.require_records().unwrap(); + let partition = records.partition("a/").unwrap(); + block_on(async { + let first = records.list_partition(&partition, None, 1).await.unwrap(); + assert!(matches!( + records + .list_partition(&partition, first.cursor.as_deref(), 1) + .await, + Err(KvError::Internal(_)) + )); + }); + } +} diff --git a/crates/edgezero-core/tests/record_configuration.rs b/crates/edgezero-core/tests/record_configuration.rs new file mode 100644 index 00000000..8e48458a --- /dev/null +++ b/crates/edgezero-core/tests/record_configuration.rs @@ -0,0 +1,182 @@ +//! Configuration and supplied-handle validation only. No provider I/O is needed. + +#[cfg(test)] +mod tests { + use std::sync::Arc; + use std::time::Duration; + + use async_trait::async_trait; + use bytes::Bytes; + use edgezero_core::app::{App, Hooks}; + use edgezero_core::key_value_store::{KvError, KvHandle, KvPage, KvStore}; + use edgezero_core::platform::PlatformMetadata; + use edgezero_core::router::RouterService; + use edgezero_core::store_registry::KvRegistry; + use edgezero_core::{ + EdgeError, RecordBinding, RecordKeyLayout, RecordLayout, RecordStoreRequirements, + }; + + struct Layout(u32); + + impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "configuration.test" + } + + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + key.strip_prefix("group/") + .and_then(|suffix| key.strip_suffix(suffix)) + .ok_or_else(|| KvError::Validation("unknown group".to_owned())) + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix == "group/" { + Ok(()) + } else { + Err(KvError::Validation("unknown partition".to_owned())) + } + } + + fn version(&self) -> u32 { + self.0 + } + } + + struct Configured; + + #[expect( + clippy::missing_trait_methods, + reason = "configuration fixture intentionally retains unrelated Hooks defaults" + )] + impl Hooks for Configured { + fn configure(app: &mut App) -> Result<(), EdgeError> { + app.require_record_store("identity", Layout(1))?; + Ok(()) + } + + fn routes() -> Result { + Ok(RouterService::builder().build()) + } + } + + struct Unconfigured; + #[expect( + clippy::missing_trait_methods, + reason = "configuration fixture intentionally retains Hooks defaults" + )] + impl Hooks for Unconfigured { + fn routes() -> Result { + Ok(RouterService::builder().build()) + } + } + + struct Prepared(Option); + + #[expect( + clippy::missing_trait_methods, + reason = "prepared binding fixture intentionally retains unsupported operation defaults" + )] + #[async_trait(?Send)] + impl KvStore for Prepared { + async fn delete(&self, _: &str) -> Result<(), KvError> { + panic!("unexpected I/O") + } + + async fn get_bytes(&self, _: &str) -> Result, KvError> { + panic!("unexpected I/O") + } + + async fn list_keys_page( + &self, + _: &str, + _: Option<&str>, + _: usize, + ) -> Result { + panic!("unexpected I/O") + } + + async fn put_bytes(&self, _: &str, _: Bytes) -> Result<(), KvError> { + panic!("unexpected I/O") + } + + async fn put_bytes_with_ttl(&self, _: &str, _: Bytes, _: Duration) -> Result<(), KvError> { + panic!("unexpected I/O") + } + + fn record_binding(&self) -> Option { + self.0.clone() + } + } + + fn handle(version: u32) -> KvHandle { + KvHandle::new(Arc::new(Prepared(Some(RecordBinding { + layout: RecordLayout::new(Layout(version)).unwrap(), + scope: "test".to_owned(), + format: "test-v1".to_owned(), + max_metadata_bytes: 1024, + })))) + } + + #[test] + fn configure_installs_immutable_per_id_requirements_before_preparation() { + let mut app = App::build::(PlatformMetadata::default()).unwrap(); + let requirements = app.record_store_requirements().clone(); + assert_eq!(requirements.len(), 1); + assert_eq!(requirements["identity"].id(), "configuration.test"); + assert_eq!(requirements["identity"].version(), 1); + assert!(app.require_record_store("another", Layout(1)).is_err()); + assert!(app.require_record_store("identity", Layout(2)).is_err()); + KvRegistry::single_id("identity".to_owned(), handle(1)) + .validate_record_requirements(&requirements) + .unwrap(); + } + + #[test] + fn low_level_declarations_cannot_replace_existing_layout() { + let mut app = App::new(RouterService::builder().build()); + assert!(app.require_record_store("", Layout(1)).is_err()); + app.require_record_store("identity", Layout(1)).unwrap(); + assert!(app.require_record_store("identity", Layout(2)).is_err()); + assert_eq!(app.record_store_requirements()["identity"].version(), 1); + } + + #[test] + fn defaults_preserve_ordinary_only_construction() { + let app = App::build::(PlatformMetadata::default()).unwrap(); + assert!(app.record_store_requirements().is_empty()); + let ordinary = KvHandle::new(Arc::new(Prepared(None))); + KvRegistry::single_id("ordinary".to_owned(), ordinary) + .validate_record_requirements(app.record_store_requirements()) + .unwrap(); + } + + #[test] + fn supplied_handles_must_already_have_matching_capability_and_layout() { + let requirements = RecordStoreRequirements::from([( + "identity".to_owned(), + RecordLayout::new(Layout(1)).unwrap(), + )]); + let wrong_layout = KvRegistry::single_id("identity".to_owned(), handle(2)); + assert!(matches!( + wrong_layout.validate_record_requirements(&requirements), + Err(KvError::Validation(_)) + )); + let missing_id = KvRegistry::single_id("another".to_owned(), handle(1)); + assert!(matches!( + missing_id.validate_record_requirements(&requirements), + Err(KvError::Validation(_)) + )); + let ordinary = KvRegistry::single_id( + "identity".to_owned(), + KvHandle::new(Arc::new(Prepared(None))), + ); + assert!(matches!( + ordinary.validate_record_requirements(&requirements), + Err(KvError::Unsupported { .. }) + )); + } +} diff --git a/crates/edgezero-store-aws/Cargo.toml b/crates/edgezero-store-aws/Cargo.toml index cc94bf64..98fa1313 100644 --- a/crates/edgezero-store-aws/Cargo.toml +++ b/crates/edgezero-store-aws/Cargo.toml @@ -16,6 +16,7 @@ snapshot-stores = ["dep:edgezero-core", "dep:bytes", "dep:async-trait", "dep:tok appconfig-agent = ["snapshot-stores", "dep:reqwest", "dep:futures-util"] secrets-manager = ["snapshot-stores", "dep:aws-config", "dep:aws-sdk-secretsmanager", "dep:aws-smithy-runtime-api", "dep:aws-types"] dynamodb-kv = ["snapshot-stores", "dep:aws-config", "dep:aws-sdk-dynamodb", "dep:aws-smithy-runtime-api", "dep:aws-types", "dep:serde_json"] +dynamodb-records = ["dynamodb-kv"] [dependencies] percent-encoding = { workspace = true } @@ -39,6 +40,7 @@ tokio = { workspace = true, optional = true } [dev-dependencies] base64 = { workspace = true } +edgezero-core = { workspace = true, features = ["test-utils"] } futures = { workspace = true } serde_json = { workspace = true } tempfile = { workspace = true } diff --git a/crates/edgezero-store-aws/src/dynamodb_kv.rs b/crates/edgezero-store-aws/src/dynamodb_kv.rs index 2e6f36fe..6076d967 100644 --- a/crates/edgezero-store-aws/src/dynamodb_kv.rs +++ b/crates/edgezero-store-aws/src/dynamodb_kv.rs @@ -1,8 +1,14 @@ //! Live native `DynamoDB` KV operations. SDK models never cross the portable boundary. +#[cfg(feature = "dynamodb-records")] +#[path = "dynamodb_records.rs"] +mod records; + use std::collections::HashMap; use std::future::Future; use std::sync::Arc; +#[cfg(feature = "dynamodb-records")] +use std::sync::Mutex; use std::time::{Duration, SystemTime, UNIX_EPOCH}; use async_trait::async_trait; @@ -46,6 +52,8 @@ struct DynamoDbStore { clock: MonotonicClock, limits: DynamoDbKvLimits, namespace: String, + #[cfg(feature = "dynamodb-records")] + record_identity: Option, region: String, table: String, } @@ -70,6 +78,14 @@ impl AwsPreparation { &mut self, settings: &DynamoDbKvSettings, ) -> Result { + let store = self.prepare_dynamodb_store(settings).await?; + Ok(KvHandle::new(Arc::new(store))) + } + + async fn prepare_dynamodb_store( + &mut self, + settings: &DynamoDbKvSettings, + ) -> Result { self.validate_dynamodb_mapping(settings)?; let key = (settings.region.clone(), settings.limits); let client = if let Some(client) = self.dynamodb_clients.get(&key) { @@ -113,22 +129,31 @@ impl AwsPreparation { &self, settings: &DynamoDbKvSettings, client: Client, - ) -> Result { + ) -> Result { let budget = self.dynamodb_budget(settings)?; - let result = timeout( - budget, - client - .describe_table() - .table_name(&settings.table) - .customize() - .config_override( - aws_sdk_dynamodb::Config::builder() - .retry_config(RetryConfig::standard().with_max_attempts(1)) - .timeout_config(timeout_policy(settings.limits, budget)), - ) - .send(), - ) - .await; + #[cfg(feature = "dynamodb-records")] + let endpoint_capture = Arc::new(Mutex::new(None)); + let preflight = client + .describe_table() + .table_name(&settings.table) + .customize() + .config_override( + aws_sdk_dynamodb::Config::builder() + .retry_config(RetryConfig::standard().with_max_attempts(1)) + .timeout_config(timeout_policy(settings.limits, budget)), + ); + #[cfg(feature = "dynamodb-records")] + let observed_request = { + let observer = Arc::clone(&endpoint_capture); + preflight.mutate_request(move |http_request| { + if let Ok(mut observed) = observer.lock() { + *observed = records::endpoint_identity(http_request.uri()); + } + }) + }; + #[cfg(not(feature = "dynamodb-records"))] + let observed_request = preflight; + let result = timeout(budget, observed_request.send()).await; self.check_deadline()?; let output = result .map_err(|_elapsed| PreparationError::Deadline)? @@ -140,13 +165,19 @@ impl AwsPreparation { .and_then(|table| validate_table(table, &settings.table)); self.check_deadline()?; validation?; - let handle = KvHandle::new(Arc::new(DynamoDbStore::new( - settings, - client, - self.clock.clone(), - ))); + let prepared = DynamoDbStore::new(settings, client, self.clock.clone()); + #[cfg(feature = "dynamodb-records")] + let store = { + let mut with_identity = prepared; + with_identity.record_identity = endpoint_capture.lock().ok().and_then(|captured| { + records::prepared_identity(captured.as_deref(), output.table.as_ref()?) + }); + with_identity + }; + #[cfg(not(feature = "dynamodb-records"))] + let store = prepared; self.check_deadline()?; - Ok(handle) + Ok(store) } #[cfg(test)] @@ -156,7 +187,8 @@ impl AwsPreparation { client: Client, ) -> Result { self.validate_dynamodb_mapping(settings)?; - self.finish_dynamodb_preparation(settings, client).await + let store = self.finish_dynamodb_preparation(settings, client).await?; + Ok(KvHandle::new(Arc::new(store))) } fn validate_dynamodb_mapping( @@ -234,6 +266,8 @@ impl DynamoDbStore { namespace: settings.namespace.clone(), region: settings.region.clone(), table: settings.table.clone(), + #[cfg(feature = "dynamodb-records")] + record_identity: None, } } @@ -298,6 +332,10 @@ impl DynamoDbStore { } } +#[expect( + clippy::missing_trait_methods, + reason = "legacy ordinary bindings deliberately retain the Unsupported record defaults" +)] #[async_trait(?Send)] impl KvStore for DynamoDbStore { #[inline] diff --git a/crates/edgezero-store-aws/src/dynamodb_records.rs b/crates/edgezero-store-aws/src/dynamodb_records.rs new file mode 100644 index 00000000..36d341c7 --- /dev/null +++ b/crates/edgezero-store-aws/src/dynamodb_records.rs @@ -0,0 +1,861 @@ +//! Explicit partitioned record format. Allocators survive value deletion and TTL. + +use std::collections::HashMap; +use std::future::Future; +use std::sync::Arc; +use std::time::{Duration, SystemTime, UNIX_EPOCH}; + +use async_trait::async_trait; +use aws_sdk_dynamodb::error::{ProvideErrorMetadata, SdkError}; +use aws_sdk_dynamodb::primitives::Blob; +use aws_sdk_dynamodb::types::{AttributeValue, ReturnValue, TableDescription}; +use bytes::Bytes; +use edgezero_core::key_value_store::{KvError, KvHandle, KvPage, KvStore}; +use edgezero_core::time::Deadline; +use serde::{Deserialize, Serialize}; +use tokio::time::timeout; + +use super::{ + DynamoDbStore, MAX_CURSOR_BYTES, MAX_ITEM_BYTES, MAX_KEY_BYTES, MAX_PAGE_KEYS, + expiry_from_duration, invalid_input, read_unsigned, +}; +use crate::PreparationError; +use crate::preparation::AwsPreparation; +use crate::settings::DynamoDbKvSettings; +use edgezero_core::record_store::{ + Record, RecordBinding, RecordGeneration, RecordLayout, RecordPage, RecordPartition, + RecordWrite, RecordWriteMode, RecordWriteOutcome, +}; + +const FORMAT: &str = "edgezero-dynamodb-records-v1"; +const MAX_PARTITION_BYTES: usize = 2_048; + +struct DynamoDbRecordStore { + binding: RecordBinding, + inner: DynamoDbStore, + partition_base: String, +} + +pub(super) struct PreparedRecordIdentity { + endpoint: String, + table_arn: Option, + table_id: Option, +} + +#[derive(Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +struct RecordCursor { + format: String, + layout_id: String, + layout_version: u32, + mode: String, + pk: String, + prefix: String, + scope: String, + sk: String, +} + +#[expect( + clippy::multiple_inherent_impl, + reason = "record preparation shares the ordinary provider's cached client and table preflight" +)] +impl AwsPreparation { + /// Prepare a fresh partitioned record binding, never the legacy ordinary format. + /// + /// Uses `GetItem`, `UpdateItem`, `PutItem`, `DeleteItem`, strong base-table Query and + /// unordered Scan. Scan evaluates rows across the table before filtering. + /// Allocators have no TTL and must not be deleted with value rows. Native TTL + /// uses `ttl_epoch_s`; expiry is asynchronous and not filtered on reads. + /// + /// # Errors + /// Returns a sanitized preparation failure without writing a probe record. + #[inline] + pub async fn prepare_dynamodb_records( + &mut self, + settings: &DynamoDbKvSettings, + layout: &RecordLayout, + ) -> Result { + let partition_base = record_partition_base(settings, layout)?; + let inner = self.prepare_dynamodb_store(settings).await?; + Ok(KvHandle::new(Arc::new(DynamoDbRecordStore::new( + inner, + layout.clone(), + partition_base, + )?))) + } +} + +#[expect( + clippy::arbitrary_source_item_ordering, + reason = "keep prepared identity, point operations and native pagination helpers grouped" +)] +impl DynamoDbRecordStore { + fn new( + inner: DynamoDbStore, + layout: RecordLayout, + partition_base: String, + ) -> Result { + let identity = inner + .record_identity + .as_ref() + .ok_or(PreparationError::Malformed)?; + let table_arn = identity + .table_arn + .as_deref() + .filter(|arn| !arn.is_empty()) + .ok_or(PreparationError::Malformed)?; + let scope = serde_json::to_string(&( + &identity.endpoint, + table_arn, + &identity.table_id, + &inner.region, + &inner.table, + &inner.namespace, + )) + .map_err(|_error| PreparationError::Malformed)?; + Ok(Self { + binding: RecordBinding { + layout, + scope, + format: FORMAT.to_owned(), + // Metadata may use the full configured payload budget when the + // body is empty. Inserts also enforce their combined byte size. + max_metadata_bytes: inner.limits.max_value_bytes, + }, + inner, + partition_base, + }) + } + + fn partition_key(&self, partition: &RecordPartition) -> Result { + if self.binding.layout.partition(partition.prefix())? != *partition { + return Err(invalid_input()); + } + let pk = format!("{}{}", self.partition_base, partition.prefix()); + if pk.len() > MAX_PARTITION_BYTES { + return Err(invalid_input()); + } + Ok(pk) + } + + fn key( + &self, + key: &str, + partition: &RecordPartition, + kind: &str, + ) -> Result, KvError> { + if self.binding.layout.resolve(key)? != *partition { + return Err(invalid_input()); + } + Ok(physical_key( + self.partition_key(partition)?, + format!("{kind}#{key}"), + )) + } + + async fn allocate( + &self, + key: &str, + partition: &RecordPartition, + deadline: Deadline, + ) -> Result { + // Failure here cannot have changed a value row. An unobserved allocation + // is merely a gap, and is never represented as a successful value write. + self.inner + .bounded(deadline, async { + let output = self + .inner + .client + .update_item() + .table_name(&self.inner.table) + .set_key(Some(self.key(key, partition, "g")?)) + .update_expression("SET #generation = if_not_exists(#generation, :zero) + :one") + .condition_expression( + "attribute_not_exists(#generation) OR #generation < :ceiling", + ) + .expression_attribute_names("#generation", "generation") + .expression_attribute_values(":zero", AttributeValue::N("0".to_owned())) + .expression_attribute_values(":one", AttributeValue::N("1".to_owned())) + .expression_attribute_values( + ":ceiling", + AttributeValue::N(u64::MAX.to_string()), + ) + .return_values(ReturnValue::UpdatedNew) + .customize() + .config_override(self.inner.operation_policy(deadline)?) + .send() + .await + .map_err(|_error| KvError::Unavailable)?; + let generation = output + .attributes + .as_ref() + .and_then(|attributes| attributes.get("generation")) + .and_then(|value| value.as_n().ok()) + .ok_or(KvError::Unavailable) + .and_then(|value| read_unsigned(value))?; + if generation == 0 { + return Err(KvError::Unavailable); + } + Ok(RecordGeneration::from_u64(generation)) + }) + .await + } + + async fn mutate( + &self, + deadline: Deadline, + operation: impl Future>, + ) -> Result { + let remaining = deadline + .remaining_at(self.inner.clock.now()) + .filter(|remaining| !remaining.is_zero()) + .ok_or(KvError::Unavailable)?; + let result = timeout(remaining, operation) + .await + .map_err(|_elapsed| KvError::MutationUncertain)?; + if deadline.is_expired_at(self.inner.clock.now()) { + return Err(KvError::MutationUncertain); + } + result + } + + async fn insert( + &self, + key: &str, + partition: &RecordPartition, + write: RecordWrite<'_>, + ) -> Result { + let deadline = self.inner.runtime_deadline()?; + let metadata = write.metadata.filter(|text| !text.is_empty()); + if write + .body + .len() + .checked_add(metadata.map_or(0, str::len)) + .is_none_or(|bytes| bytes > self.inner.limits.max_value_bytes) + { + return Err(invalid_input()); + } + let expiry = write + .ttl + .map(|ttl| { + let now = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map_err(|_error| KvError::Unavailable)?; + // Native TTL is in seconds. Rounding up never shortens the requested TTL. + Ok::(expiry_from_duration(now, ttl)?.div_ceil(1_000)) + }) + .transpose()?; + let mut item = self.key(key, partition, "r")?; + item.insert("v".to_owned(), AttributeValue::B(Blob::new(write.body))); + item.insert("format".to_owned(), AttributeValue::S(FORMAT.to_owned())); + item.insert("kind".to_owned(), AttributeValue::S("r".to_owned())); + if let Some(text) = metadata { + item.insert("metadata".to_owned(), AttributeValue::S(text.to_owned())); + } + if let Some(expires) = expiry { + item.insert( + "ttl_epoch_s".to_owned(), + AttributeValue::N(expires.to_string()), + ); + } + // Include the maximum generation before allocation so oversized values + // never consume an allocator request. + item.insert( + "generation".to_owned(), + AttributeValue::N(u64::MAX.to_string()), + ); + validate_record_size(&item)?; + let generation = self.allocate(key, partition, deadline).await?; + item.insert( + "generation".to_owned(), + AttributeValue::N(generation.as_u64().to_string()), + ); + // Resolve the remaining policy before constructing the mutation future. + // Deadline exhaustion after allocation alone remains Unavailable. + let policy = self.inner.operation_policy(deadline)?; + let mut request = self + .inner + .client + .put_item() + .table_name(&self.inner.table) + .set_item(Some(item)); + match write.mode { + RecordWriteMode::Overwrite => {} + RecordWriteMode::Add => { + request = request + .condition_expression("attribute_not_exists(#pk)") + .expression_attribute_names("#pk", "pk"); + } + RecordWriteMode::IfGenerationMatch(expected) => { + request = request + .condition_expression("#generation = :expected") + .expression_attribute_names("#generation", "generation") + .expression_attribute_values( + ":expected", + AttributeValue::N(expected.as_u64().to_string()), + ); + } + } + self.mutate(deadline, async { + match request.customize().config_override(policy).send().await { + Ok(_) => Ok(RecordWriteOutcome::Written), + Err(error) + if write.mode != RecordWriteMode::Overwrite + && error.as_service_error().is_some_and(|service| { + service.is_conditional_check_failed_exception() + }) => + { + Ok(RecordWriteOutcome::PreconditionFailed) + } + Err(error) => Err(mutation_error(&error)), + } + }) + .await + } + + async fn lookup( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result, KvError> { + let deadline = self.inner.runtime_deadline()?; + let expected = self.key(key, partition, "r")?; + self.inner + .bounded(deadline, async { + let output = self + .inner + .client + .get_item() + .table_name(&self.inner.table) + .set_key(Some(expected.clone())) + .consistent_read(true) + .customize() + .config_override(self.inner.operation_policy(deadline)?) + .send() + .await + .map_err(|_error| KvError::Unavailable)?; + output + .item + .map(|mut item| { + if expected + .iter() + .any(|(name, value)| item.get(name) != Some(value)) + { + return Err(KvError::Unavailable); + } + validate_value_kind(&item)?; + let generation = match item.remove("generation") { + Some(AttributeValue::N(value)) => { + RecordGeneration::from_u64(read_unsigned(&value)?) + } + _ => return Err(KvError::Unavailable), + }; + let metadata = match item.remove("metadata") { + None => None, + Some(AttributeValue::S(text)) + if text.len() <= self.inner.limits.max_value_bytes => + { + (!text.is_empty()).then_some(text) + } + _ => return Err(KvError::Unavailable), + }; + let body = match item.remove("v") { + Some(AttributeValue::B(body)) + if body.as_ref().len() <= self.inner.limits.max_value_bytes => + { + Bytes::from(body.into_inner()) + } + _ => return Err(KvError::Unavailable), + }; + if body + .len() + .checked_add(metadata.as_deref().map_or(0, str::len)) + .is_none_or(|bytes| bytes > self.inner.limits.max_value_bytes) + { + return Err(KvError::Unavailable); + } + // Check native TTL's shape but never impose a hard expiry filter. + if let Some(ttl) = item.get("ttl_epoch_s") { + let number = ttl.as_n().map_err(|_error| KvError::Unavailable)?; + read_unsigned(number)?; + } + Ok(Record { + body, + metadata, + generation, + }) + }) + .transpose() + }) + .await + } + + async fn delete_value(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + let deadline = self.inner.runtime_deadline()?; + let attributes = self.key(key, partition, "r")?; + let policy = self.inner.operation_policy(deadline)?; + self.mutate(deadline, async { + self.inner + .client + .delete_item() + .table_name(&self.inner.table) + .set_key(Some(attributes)) + .customize() + .config_override(policy) + .send() + .await + .map_err(|error| mutation_error(&error))?; + Ok(()) + }) + .await + } + + fn decode_cursor( + &self, + mode: &str, + prefix: &str, + encoded: &str, + partition: Option<&RecordPartition>, + ) -> Result, KvError> { + if encoded.len() > MAX_CURSOR_BYTES { + return Err(invalid_input()); + } + let cursor: RecordCursor = + serde_json::from_str(encoded).map_err(|_error| invalid_input())?; + if cursor.format != FORMAT + || cursor.layout_id != self.binding.layout.id() + || cursor.layout_version != self.binding.layout.version() + || cursor.scope != self.binding.scope + || cursor.mode != mode + || cursor.prefix != prefix + { + return Err(invalid_input()); + } + let key = physical_key(cursor.pk, cursor.sk); + validate_position(&key).map_err(|_error| invalid_input())?; + if let Some(expected_partition) = partition + && key.get("pk") != Some(&AttributeValue::S(self.partition_key(expected_partition)?)) + { + return Err(invalid_input()); + } + Ok(key) + } + + fn next_cursor( + &self, + mode: &str, + prefix: &str, + after: Option<&HashMap>, + next_position: Option>, + partition: Option<&RecordPartition>, + ) -> Result, KvError> { + let Some(position) = next_position.filter(|candidate| !candidate.is_empty()) else { + return Ok(None); + }; + let (pk, sk) = validate_position(&position)?; + if after == Some(&position) { + return Err(KvError::Unavailable); + } + if let Some(expected_partition) = partition + && (pk != self.partition_key(expected_partition)? + || after + .and_then(|key| key.get("sk")) + .and_then(|value| value.as_s().ok()) + .is_some_and(|previous| sk <= previous.as_str())) + { + return Err(KvError::Unavailable); + } + let encoded = serde_json::to_string(&RecordCursor { + format: FORMAT.to_owned(), + layout_id: self.binding.layout.id().to_owned(), + layout_version: self.binding.layout.version(), + mode: mode.to_owned(), + pk: pk.to_owned(), + prefix: prefix.to_owned(), + scope: self.binding.scope.clone(), + sk: sk.to_owned(), + }) + .map_err(|_error| KvError::Unavailable)?; + if encoded.len() > MAX_CURSOR_BYTES { + return Err(KvError::Unavailable); + } + Ok(Some(encoded)) + } + + fn listed_keys( + &self, + items: Vec>, + prefix: &str, + limit: usize, + partition: Option<&RecordPartition>, + ) -> Result, KvError> { + if items.len() > limit { + return Err(KvError::Unavailable); + } + items + .into_iter() + .map(|item| { + validate_value_kind(&item)?; + let (pk, sk) = validate_position_item(&item)?; + let key = sk + .strip_prefix("r#") + .filter(|key| key.starts_with(prefix)) + .ok_or(KvError::Unavailable)?; + let resolved = self + .binding + .layout + .resolve(key) + .map_err(|_error| KvError::Unavailable)?; + if pk != self.partition_key(&resolved)? + || partition.is_some_and(|expected| expected != &resolved) + { + return Err(KvError::Unavailable); + } + Ok(key.to_owned()) + }) + .collect() + } + + async fn page( + &self, + prefix: &str, + partition: Option<&RecordPartition>, + cursor: Option<&str>, + limit: usize, + ) -> Result { + let deadline = self.inner.runtime_deadline()?; + if prefix.len() > MAX_KEY_BYTES || prefix.chars().any(char::is_control) { + return Err(invalid_input()); + } + if limit == 0 || limit > MAX_PAGE_KEYS { + return Err(invalid_input()); + } + let mode = if partition.is_some() { "query" } else { "scan" }; + let after = cursor + .map(|encoded| self.decode_cursor(mode, prefix, encoded, partition)) + .transpose()?; + let native_limit = i32::try_from(limit).map_err(|_error| invalid_input())?; + let names = HashMap::from([ + ("#pk".to_owned(), "pk".to_owned()), + ("#sk".to_owned(), "sk".to_owned()), + ("#format".to_owned(), "format".to_owned()), + ("#kind".to_owned(), "kind".to_owned()), + ]); + let base_values = HashMap::from([ + (":format".to_owned(), AttributeValue::S(FORMAT.to_owned())), + (":kind".to_owned(), AttributeValue::S("r".to_owned())), + ( + ":prefix".to_owned(), + AttributeValue::S(format!("r#{prefix}")), + ), + ]); + self.inner.bounded(deadline, async { + let (items, position) = if let Some(query_partition) = partition { + let mut values = base_values; + values.insert(":pk".to_owned(), AttributeValue::S(self.partition_key(query_partition)?)); + let output = self.inner.client.query().table_name(&self.inner.table).consistent_read(true) + .scan_index_forward(true).limit(native_limit) + .key_condition_expression("#pk = :pk") + .filter_expression("#format = :format AND #kind = :kind") + .projection_expression("#pk, #sk, #format, #kind") + .set_expression_attribute_names(Some(names)) + // Query cannot filter a key attribute. Full membership is + // checked below, while the partition PK already bounds keys. + .set_expression_attribute_values(Some(values.into_iter().filter(|(name, _)| name != ":prefix").collect())) + .set_exclusive_start_key(after.clone()) + .customize().config_override(self.inner.operation_policy(deadline)?) + .send().await.map_err(|_error| KvError::Unavailable)?; + (output.items.unwrap_or_default(), output.last_evaluated_key) + } else { + let mut values = base_values; + values.insert(":binding".to_owned(), AttributeValue::S(self.partition_base.clone())); + let output = self.inner.client.scan().table_name(&self.inner.table).limit(native_limit) + .filter_expression("begins_with(#pk, :binding) AND #format = :format AND #kind = :kind AND begins_with(#sk, :prefix)") + .projection_expression("#pk, #sk, #format, #kind") + .set_expression_attribute_names(Some(names)).set_expression_attribute_values(Some(values)) + .set_exclusive_start_key(after.clone()) + .customize().config_override(self.inner.operation_policy(deadline)?) + .send().await.map_err(|_error| KvError::Unavailable)?; + (output.items.unwrap_or_default(), output.last_evaluated_key) + }; + if partition.is_some() { + validate_query_positions(&items, after.as_ref(), position.as_ref())?; + } + let keys = self.listed_keys(items, prefix, limit, partition)?; + let next = self.next_cursor(mode, prefix, after.as_ref(), position, partition)?; + Ok(KvPage { keys, cursor: next }) + }).await + } +} + +#[expect( + clippy::arbitrary_source_item_ordering, + reason = "keep prepared capability and ordinary methods ahead of the record boundary" +)] +#[expect( + clippy::missing_trait_methods, + reason = "ordinary exists deliberately delegates to the same strong get-bytes path" +)] +#[async_trait(?Send)] +impl KvStore for DynamoDbRecordStore { + fn record_binding(&self) -> Option { + Some(self.binding.clone()) + } + + async fn get_bytes(&self, key: &str) -> Result, KvError> { + let partition = self.binding.layout.resolve(key)?; + Ok(self + .lookup(key, &partition) + .await? + .map(|record| record.body)) + } + + async fn put_bytes(&self, key: &str, value: Bytes) -> Result<(), KvError> { + let partition = self.binding.layout.resolve(key)?; + self.insert( + key, + &partition, + RecordWrite { + body: &value, + metadata: None, + ttl: None, + mode: RecordWriteMode::Overwrite, + }, + ) + .await?; + Ok(()) + } + + async fn put_bytes_with_ttl( + &self, + key: &str, + value: Bytes, + ttl: Duration, + ) -> Result<(), KvError> { + let partition = self.binding.layout.resolve(key)?; + self.insert( + key, + &partition, + RecordWrite { + body: &value, + metadata: None, + ttl: Some(ttl), + mode: RecordWriteMode::Overwrite, + }, + ) + .await?; + Ok(()) + } + + async fn delete(&self, key: &str) -> Result<(), KvError> { + self.delete_value(key, &self.binding.layout.resolve(key)?) + .await + } + + async fn list_keys_page( + &self, + prefix: &str, + cursor: Option<&str>, + limit: usize, + ) -> Result { + self.page(prefix, None, cursor, limit).await + } + + async fn record_lookup( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result, KvError> { + self.lookup(key, partition).await + } + + async fn record_insert( + &self, + key: &str, + partition: &RecordPartition, + write: RecordWrite<'_>, + ) -> Result { + self.insert(key, partition, write).await + } + + async fn record_exists_strong( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result { + Ok(self.lookup(key, partition).await?.is_some()) + } + + async fn record_delete(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + self.delete_value(key, partition).await + } + + async fn record_list_partition( + &self, + partition: &RecordPartition, + cursor: Option<&str>, + limit: usize, + ) -> Result { + self.partition_key(partition)?; + let page = self + .page(partition.prefix(), Some(partition), cursor, limit) + .await?; + Ok(RecordPage { + keys: page.keys, + cursor: page.cursor, + }) + } +} + +pub(super) fn endpoint_identity(uri: &str) -> Option { + let mut endpoint = url::Url::parse(uri).ok()?; + endpoint.host_str()?; + endpoint.set_username("").ok()?; + endpoint.set_password(None).ok()?; + endpoint.set_query(None); + endpoint.set_fragment(None); + Some(endpoint.into()) +} + +pub(super) fn prepared_identity( + endpoint: Option<&str>, + table: &TableDescription, +) -> Option { + Some(PreparedRecordIdentity { + endpoint: endpoint?.to_owned(), + table_arn: table.table_arn.clone(), + table_id: table.table_id.clone(), + }) +} + +fn mutation_error(error: &SdkError) -> KvError +where + E: ProvideErrorMetadata, +{ + if matches!(error, SdkError::ConstructionFailure(_)) + || error.as_service_error().is_some_and(|_service| { + error + .raw_response() + .is_some_and(|response| (400..500).contains(&response.status().as_u16())) + }) + { + KvError::Unavailable + } else { + KvError::MutationUncertain + } +} + +fn physical_key(pk: String, sk: String) -> HashMap { + HashMap::from([ + ("pk".to_owned(), AttributeValue::S(pk)), + ("sk".to_owned(), AttributeValue::S(sk)), + ]) +} + +fn record_partition_base( + settings: &DynamoDbKvSettings, + layout: &RecordLayout, +) -> Result { + // Byte lengths delimit arbitrary namespace/layout identities without escaping + // or collisions. The final remainder is the unchanged declared partition. + settings.validate()?; + let base = format!( + "{FORMAT}#{}:{}{}:{}{}:", + settings.namespace.len(), + settings.namespace, + layout.id().len(), + layout.id(), + layout.version() + ); + if base + .len() + .checked_add(MAX_KEY_BYTES) + .is_none_or(|size| size > MAX_PARTITION_BYTES) + { + return Err(PreparationError::InvalidBinding); + } + Ok(base) +} + +fn validate_position(key: &HashMap) -> Result<(&str, &str), KvError> { + if key.len() != 2 { + return Err(KvError::Unavailable); + } + validate_position_item(key) +} + +fn validate_position_item(item: &HashMap) -> Result<(&str, &str), KvError> { + match (item.get("pk"), item.get("sk")) { + (Some(AttributeValue::S(pk)), Some(AttributeValue::S(sk))) + if !pk.is_empty() + && pk.len() <= MAX_PARTITION_BYTES + && !sk.is_empty() + && sk.len() <= 1_024 => + { + Ok((pk, sk)) + } + _ => Err(KvError::Unavailable), + } +} + +fn validate_query_positions( + items: &[HashMap], + after: Option<&HashMap>, + next_position: Option<&HashMap>, +) -> Result<(), KvError> { + let mut previous = after.map(validate_position).transpose()?.map(|(_, sk)| sk); + for item in items { + let (_, sk) = validate_position_item(item)?; + if previous.is_some_and(|last| sk <= last) { + return Err(KvError::Unavailable); + } + previous = Some(sk); + } + if let Some(position) = next_position.filter(|candidate| !candidate.is_empty()) { + let (_, sk) = validate_position(position)?; + if previous.is_some_and(|last| sk < last) { + return Err(KvError::Unavailable); + } + } + Ok(()) +} + +fn validate_value_kind(item: &HashMap) -> Result<(), KvError> { + if item.get("format") != Some(&AttributeValue::S(FORMAT.to_owned())) + || item.get("kind") != Some(&AttributeValue::S("r".to_owned())) + { + return Err(KvError::Unavailable); + } + Ok(()) +} + +#[expect( + clippy::iter_over_hash_type, + reason = "checked item-byte summation is independent of attribute order" +)] +#[expect( + clippy::wildcard_enum_match_arm, + reason = "all non-scalar attribute types are rejected uniformly in the record format" +)] +fn validate_record_size(item: &HashMap) -> Result<(), KvError> { + let mut size = 0_usize; + for (name, value) in item { + let bytes = match value { + AttributeValue::S(text) | AttributeValue::N(text) => text.len(), + AttributeValue::B(blob) => blob.as_ref().len(), + _ => return Err(invalid_input()), + }; + size = size + .checked_add(name.len()) + .and_then(|with_name| with_name.checked_add(bytes)) + .ok_or_else(invalid_input)?; + } + if size > MAX_ITEM_BYTES { + return Err(invalid_input()); + } + Ok(()) +} + +#[cfg(test)] +#[path = "dynamodb_records_tests.rs"] +mod tests; diff --git a/crates/edgezero-store-aws/src/dynamodb_records_tests.rs b/crates/edgezero-store-aws/src/dynamodb_records_tests.rs new file mode 100644 index 00000000..93e431f6 --- /dev/null +++ b/crates/edgezero-store-aws/src/dynamodb_records_tests.rs @@ -0,0 +1,1159 @@ +//! Real SDK wire tests. The local HTTP peer models conditional row mutations; +//! `DynamoDB` Local acceptance remains a separate integration check. + +#![expect( + clippy::arbitrary_source_item_ordering, + reason = "group fixture state, protocol helpers and behavior cases by responsibility" +)] + +use super::*; +use crate::dynamodb_kv::sdk_tests::{active_table, loopback_client, read_request}; +use crate::settings::{AwsPreparationLimits, DynamoDbKvLimits}; +use edgezero_core::record_store::{RecordKeyLayout, contract}; +use edgezero_core::time::MonotonicClock; +use serde_json::{Value, json}; +use std::collections::{BTreeMap, BTreeSet, VecDeque}; +use std::io::{ErrorKind, Write as _}; +use std::net::{SocketAddr, TcpListener}; +use std::sync::{ + Mutex, + atomic::{AtomicBool, Ordering}, +}; +use std::thread::{self, JoinHandle}; +use tokio::runtime::{Builder as RuntimeBuilder, Runtime}; + +#[derive(Clone, Copy)] +struct Groups; + +impl RecordKeyLayout for Groups { + fn id(&self) -> &'static str { + "wire-groups" + } + fn version(&self) -> u32 { + 1 + } + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + let end = key.find('/').ok_or_else(invalid_input)?; + key.get(..=end).ok_or_else(invalid_input) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.bytes().filter(|byte| *byte == b'/').count() == 1 { + Ok(()) + } else { + Err(invalid_input()) + } + } +} + +#[derive(Default)] +struct State { + rows: BTreeMap<(String, String), Value>, + requests: Vec<(String, Value)>, + query_pages: VecDeque, + scan_pages: VecDeque, + failure: Option<(String, bool, u16, Value)>, + allocation_delay: Option, + describe_response: Option, +} + +struct Fixture { + address: SocketAddr, + state: Arc>, + stopped: Arc, + worker: Option>, +} + +impl Fixture { + fn start() -> Self { + let listener = TcpListener::bind(("127.0.0.1", 0)).unwrap(); + listener.set_nonblocking(true).unwrap(); + let address = listener.local_addr().unwrap(); + let shared_state = Arc::new(Mutex::new(State::default())); + let stopped = Arc::new(AtomicBool::new(false)); + let worker_state = Arc::clone(&shared_state); + let worker_stopped = Arc::clone(&stopped); + let worker = thread::spawn(move || { + while !worker_stopped.load(Ordering::Acquire) { + match listener.accept() { + Ok((mut stream, _)) => { + stream + .set_read_timeout(Some(Duration::from_secs(3))) + .unwrap(); + stream + .set_write_timeout(Some(Duration::from_secs(3))) + .unwrap(); + let Some((target, wire_body)) = read_request(&mut stream) else { + continue; + }; + let body: Value = serde_json::from_slice(&wire_body).unwrap(); + let operation = target.rsplit('.').next().unwrap().to_owned(); + let mut state = worker_state.lock().unwrap(); + state.requests.push((operation.clone(), body.clone())); + let failure = state + .failure + .clone() + .filter(|(failed_target, _, _, _)| failed_target == &operation); + let result = if let Some((_, commit, status, response)) = failure { + if commit { + drop(respond(&mut state, &operation, &body)); + } + (status, response) + } else { + respond(&mut state, &operation, &body) + }; + let delay = (operation == "UpdateItem") + .then_some(state.allocation_delay) + .flatten(); + drop(state); + if let Some(wait) = delay { + thread::sleep(wait); + } + let encoded = serde_json::to_vec(&result.1).unwrap(); + let header = format!( + "HTTP/1.1 {} Response\r\ncontent-type: application/x-amz-json-1.0\r\ncontent-length: {}\r\nconnection: close\r\n\r\n", + result.0, + encoded.len() + ); + stream + .write_all(header.as_bytes()) + .and_then(|()| stream.write_all(&encoded)) + .unwrap_or_default(); + } + Err(error) if error.kind() == ErrorKind::WouldBlock => { + thread::sleep(Duration::from_millis(1)); + } + Err(error) => panic!("loopback accept: {error}"), + } + } + }); + Self { + address, + state: shared_state, + stopped, + worker: Some(worker), + } + } + + fn requests(&self) -> Vec<(String, Value)> { + self.state.lock().unwrap().requests.clone() + } + + fn fail(&self, operation: &str, commit: bool, status: u16, code: &str) { + self.state.lock().unwrap().failure = Some(( + operation.to_owned(), + commit, + status, + json!({"__type": code, "message": "PRIVATE_RECORD_SENTINEL"}), + )); + } +} + +#[expect(clippy::missing_trait_methods, reason = "Drop::pin_drop is unstable")] +impl Drop for Fixture { + fn drop(&mut self) { + self.stopped.store(true, Ordering::Release); + self.worker.take().unwrap().join().unwrap(); + } +} + +fn wire_key(item: &Value) -> (String, String) { + ( + item["pk"]["S"].as_str().unwrap().to_owned(), + item["sk"]["S"].as_str().unwrap().to_owned(), + ) +} + +fn conditional_failure() -> (u16, Value) { + (400, json!({"__type": "ConditionalCheckFailedException"})) +} + +fn respond(state: &mut State, operation: &str, body: &Value) -> (u16, Value) { + match operation { + "DescribeTable" => ( + 200, + state.describe_response.clone().unwrap_or_else(record_table), + ), + "UpdateItem" => { + let key = wire_key(&body["Key"]); + let current = state + .rows + .get(&key) + .and_then(|item| item["generation"]["N"].as_str()) + .map_or(0, |number| number.parse::().unwrap()); + let Some(next) = current.checked_add(1) else { + return conditional_failure(); + }; + let mut item = body["Key"].clone(); + item["generation"] = json!({"N": next.to_string()}); + state.rows.insert(key, item); + ( + 200, + json!({"Attributes": {"generation": {"N": next.to_string()}}}), + ) + } + "PutItem" => { + let item = &body["Item"]; + let key = wire_key(item); + let existing = state.rows.get(&key); + if body["ConditionExpression"] == "attribute_not_exists(#pk)" && existing.is_some() { + return conditional_failure(); + } + if body["ConditionExpression"] == "#generation = :expected" + && existing.map(|stored| &stored["generation"]) + != Some(&body["ExpressionAttributeValues"][":expected"]) + { + return conditional_failure(); + } + state.rows.insert(key, item.clone()); + (200, json!({})) + } + "GetItem" => ( + 200, + state + .rows + .get(&wire_key(&body["Key"])) + .map_or_else(|| json!({}), |item| json!({"Item": item})), + ), + "DeleteItem" => { + state.rows.remove(&wire_key(&body["Key"])); + (200, json!({})) + } + "Query" => ( + 200, + state + .query_pages + .pop_front() + .unwrap_or_else(|| list_response(state, body, true)), + ), + "Scan" => ( + 200, + state + .scan_pages + .pop_front() + .unwrap_or_else(|| list_response(state, body, false)), + ), + _ => panic!("unexpected operation {operation}"), + } +} + +fn record_table() -> Value { + let mut table = active_table(); + table["Table"]["TableArn"] = json!("arn:aws:dynamodb:us-east-1:111111111111:table/fixture-kv"); + table["Table"]["TableId"] = json!("fixture-table-id"); + table +} + +fn list_response(state: &State, body: &Value, query: bool) -> Value { + let after = body.get("ExclusiveStartKey").map(wire_key); + let values = &body["ExpressionAttributeValues"]; + let evaluated: Vec<_> = state + .rows + .iter() + .filter(|(key, _)| { + after.as_ref().is_none_or(|previous| *key > previous) + && (!query || key.0 == values[":pk"]["S"].as_str().unwrap()) + }) + .take(usize::try_from(body["Limit"].as_u64().unwrap()).unwrap()) + .collect(); + let items: Vec<_> = evaluated + .iter() + .filter(|(key, item)| { + item["format"]["S"] == FORMAT + && item["kind"]["S"] == "r" + && (query + || (key.0.starts_with(values[":binding"]["S"].as_str().unwrap()) + && key.1.starts_with(values[":prefix"]["S"].as_str().unwrap()))) + }) + .map(|(_, item)| (*item).clone()) + .collect(); + let mut response = json!({"Items": items}); + if evaluated.len() == usize::try_from(body["Limit"].as_u64().unwrap()).unwrap() + && let Some((key, _)) = evaluated.last() + { + response["LastEvaluatedKey"] = json!({"pk": {"S": key.0}, "sk": {"S": key.1}}); + } + response +} + +fn settings() -> DynamoDbKvSettings { + DynamoDbKvSettings { + region: "us-east-1".to_owned(), + table: "fixture-kv".to_owned(), + namespace: "records-wire".to_owned(), + limits: DynamoDbKvLimits::default(), + } +} + +fn runtime() -> Runtime { + RuntimeBuilder::new_current_thread() + .enable_all() + .build() + .unwrap() +} + +fn store( + runtime: &Runtime, + fixture: &Fixture, + settings: &DynamoDbKvSettings, +) -> DynamoDbRecordStore { + let client = loopback_client( + runtime, + &format!("http://{}", fixture.address), + settings.limits, + ); + let layout = RecordLayout::new(Groups).unwrap(); + let base = record_partition_base(settings, &layout).unwrap(); + let inner = runtime.block_on(async { + let mut preparation = + AwsPreparation::new(MonotonicClock::default(), AwsPreparationLimits::default()) + .unwrap(); + preparation.validate_dynamodb_mapping(settings).unwrap(); + preparation + .finish_dynamodb_preparation(settings, client) + .await + .unwrap() + }); + let store = DynamoDbRecordStore::new(inner, layout, base).unwrap(); + fixture.state.lock().unwrap().requests.clear(); + store +} + +fn write(body: &[u8], mode: RecordWriteMode) -> RecordWrite<'_> { + RecordWrite { + body, + metadata: Some("m\u{e9}tadata"), + ttl: None, + mode, + } +} + +#[test] +fn resolved_endpoint_and_table_account_bind_native_cursors() { + let runtime = runtime(); + let first_fixture = Fixture::start(); + let second_fixture = Fixture::start(); + let first = store(&runtime, &first_fixture, &settings()); + let second = store(&runtime, &second_fixture, &settings()); + assert_ne!(first.binding.scope, second.binding.scope); + let cursor = first + .next_cursor( + "scan", + "", + None, + Some(physical_key("foreign".to_owned(), "internal".to_owned())), + None, + ) + .unwrap() + .unwrap(); + assert!(matches!( + runtime.block_on(second.page("", None, Some(&cursor), 1)), + Err(KvError::Validation(_)) + )); + assert!(second_fixture.requests().is_empty()); + let mut account_description = record_table(); + account_description["Table"]["TableArn"] = + json!("arn:aws:dynamodb:us-east-1:222222222222:table/fixture-kv"); + first_fixture.state.lock().unwrap().describe_response = Some(account_description); + let other_account = store(&runtime, &first_fixture, &settings()); + assert_ne!(first.binding.scope, other_account.binding.scope); + assert!(matches!( + runtime.block_on(other_account.page("", None, Some(&cursor), 1)), + Err(KvError::Validation(_)) + )); + assert!(first_fixture.requests().is_empty()); +} + +#[test] +fn endpoint_scope_excludes_userinfo_query_and_fragment() { + assert_eq!( + endpoint_identity( + "http://user:PRIVATE_PASSWORD@127.0.0.1:8000/base?token=PRIVATE_TOKEN#PRIVATE_FRAGMENT" + ) + .unwrap(), + "http://127.0.0.1:8000/base" + ); +} + +#[test] +fn missing_table_identity_is_rejected_only_by_record_construction() { + let runtime = runtime(); + let fixture = Fixture::start(); + fixture.state.lock().unwrap().describe_response = Some(active_table()); + let settings = settings(); + let client = loopback_client( + &runtime, + &format!("http://{}", fixture.address), + settings.limits, + ); + let inner = runtime.block_on(async { + let mut preparation = + AwsPreparation::new(MonotonicClock::default(), AwsPreparationLimits::default()) + .unwrap(); + preparation.validate_dynamodb_mapping(&settings).unwrap(); + preparation + .finish_dynamodb_preparation(&settings, client) + .await + .unwrap() + }); + let layout = RecordLayout::new(Groups).unwrap(); + assert!(matches!( + DynamoDbRecordStore::new( + inner, + layout.clone(), + record_partition_base(&settings, &layout).unwrap() + ), + Err(PreparationError::Malformed) + )); + assert_eq!(fixture.requests().len(), 1); + assert_eq!(fixture.requests()[0].0, "DescribeTable"); +} + +#[test] +fn shared_public_record_contract_uses_real_sdk_wire_requests() { + let runtime = runtime(); + let fixture = Fixture::start(); + let kv = KvHandle::new(Arc::new(store(&runtime, &fixture, &settings()))); + runtime.block_on(contract::assert_record_contract(&kv, "group/")); +} + +#[test] +fn record_preparation_shares_table_preflight_without_mutation_probe() { + let runtime = runtime(); + let fixture = Fixture::start(); + let settings = settings(); + let client = loopback_client( + &runtime, + &format!("http://{}", fixture.address), + settings.limits, + ); + runtime.block_on(async { + let mut preparation = + AwsPreparation::new(MonotonicClock::default(), AwsPreparationLimits::default()) + .unwrap(); + preparation.validate_dynamodb_mapping(&settings).unwrap(); + let inner = preparation + .finish_dynamodb_preparation(&settings, client) + .await + .unwrap(); + let layout = RecordLayout::new(Groups).unwrap(); + let kv = KvHandle::new(Arc::new( + DynamoDbRecordStore::new( + inner, + layout.clone(), + record_partition_base(&settings, &layout).unwrap(), + ) + .unwrap(), + )); + assert_eq!( + kv.require_records().unwrap().binding().max_metadata_bytes, + settings.limits.max_value_bytes + ); + }); + assert_eq!( + fixture + .requests() + .iter() + .map(|(target, _)| target.as_str()) + .collect::>(), + ["DescribeTable"] + ); +} + +#[test] +fn atomic_allocation_and_conditions_have_distinct_partition_and_row_keys() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + runtime.block_on(async { + for key in ["a/full-key", "b/full-key"] { + let partition = provider.binding.layout.resolve(key).unwrap(); + assert_eq!( + provider + .insert(key, &partition, write(&[0, 255], RecordWriteMode::Add)) + .await + .unwrap(), + RecordWriteOutcome::Written + ); + } + }); + let requests = fixture.requests(); + assert_eq!(requests.len(), 4); + assert_eq!(requests[0].0, "UpdateItem"); + assert_eq!(requests[0].1["Key"]["sk"]["S"], "g#a/full-key"); + assert_eq!(requests[1].1["Item"]["sk"]["S"], "r#a/full-key"); + assert_ne!(requests[0].1["Key"]["pk"], requests[2].1["Key"]["pk"]); + assert_eq!( + requests[0].1["ConditionExpression"], + "attribute_not_exists(#generation) OR #generation < :ceiling" + ); + assert_eq!( + requests[0].1["ExpressionAttributeValues"][":ceiling"]["N"], + u64::MAX.to_string() + ); + assert_eq!(requests[0].1["ReturnValues"], "UPDATED_NEW"); + assert!(requests[0].1.get("ttl_epoch_s").is_none()); + assert_eq!( + requests[1].1["ConditionExpression"], + "attribute_not_exists(#pk)" + ); + assert_eq!(requests[1].1["Item"]["v"]["B"], "AP8="); +} + +#[test] +fn same_row_replacement_clears_metadata_ttl_and_invalidates_old_generations() { + let runtime = runtime(); + let fixture = Fixture::start(); + let kv = KvHandle::new(Arc::new(store(&runtime, &fixture, &settings()))); + runtime.block_on(async { + let records = kv.require_records().unwrap(); + let key = "a/coherent"; + records + .insert( + key, + RecordWrite { + ttl: Some(Duration::from_secs(60)), + ..write(&[0, 255], RecordWriteMode::Add) + }, + ) + .await + .unwrap(); + let old = records.lookup(key).await.unwrap().unwrap(); + assert_eq!(old.body.as_ref(), &[0, 255]); + assert_eq!(old.metadata.as_deref(), Some("m\u{e9}tadata")); + kv.put_bytes(key, Bytes::from_static(b"ordinary")) + .await + .unwrap(); + let fresh = records.lookup(key).await.unwrap().unwrap(); + assert_eq!(fresh.body.as_ref(), b"ordinary"); + assert_eq!(fresh.metadata, None); + assert_ne!(fresh.generation, old.generation); + assert_eq!( + records + .insert( + key, + write(b"stale", RecordWriteMode::IfGenerationMatch(old.generation)) + ) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + records.delete(key).await.unwrap(); + assert!(!records.exists_strong(key).await.unwrap()); + records + .insert(key, write(b"recreated", RecordWriteMode::Add)) + .await + .unwrap(); + assert_eq!( + records + .insert( + key, + write( + b"stale", + RecordWriteMode::IfGenerationMatch(fresh.generation) + ) + ) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + }); + let requests = fixture.requests(); + let puts: Vec<_> = requests + .iter() + .filter(|(target, _)| target == "PutItem") + .collect(); + assert!(puts[0].1["Item"].get("ttl_epoch_s").is_some()); + assert!(puts[0].1["Item"].get("expires_at_ms").is_none()); + assert!(puts[1].1["Item"].get("ttl_epoch_s").is_none()); + assert!(puts[1].1["Item"].get("metadata").is_none()); + assert!( + requests + .iter() + .filter(|(target, _)| target == "GetItem") + .all(|(_, body)| body["ConsistentRead"] == true) + ); + assert!( + requests + .iter() + .filter(|(target, _)| target == "DeleteItem") + .all(|(_, body)| body["Key"]["sk"]["S"].as_str().unwrap().starts_with("r#")) + ); +} + +#[test] +fn native_expired_but_retained_values_remain_visible_and_block_add() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + runtime.block_on(async { + let partition = provider.binding.layout.resolve("a/ttl").unwrap(); + provider + .insert("a/ttl", &partition, write(b"body", RecordWriteMode::Add)) + .await + .unwrap(); + let key = ( + provider.partition_key(&partition).unwrap(), + "r#a/ttl".to_owned(), + ); + fixture.state.lock().unwrap().rows.get_mut(&key).unwrap()["ttl_epoch_s"] = + json!({"N": "1"}); + assert!( + provider + .lookup("a/ttl", &partition) + .await + .unwrap() + .is_some() + ); + assert_eq!( + provider + .insert("a/ttl", &partition, write(b"other", RecordWriteMode::Add)) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + }); +} + +#[test] +fn independent_add_and_cas_writers_have_one_acknowledged_winner() { + let runtime = runtime(); + let fixture = Fixture::start(); + let first = store(&runtime, &fixture, &settings()); + let second = store(&runtime, &fixture, &settings()); + runtime.block_on(async { + let partition = first.binding.layout.resolve("a/race").unwrap(); + let (add_first, add_second) = futures::join!( + first.insert("a/race", &partition, write(b"first", RecordWriteMode::Add)), + second.insert("a/race", &partition, write(b"second", RecordWriteMode::Add)) + ); + assert_eq!( + [add_first.unwrap(), add_second.unwrap()] + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::Written) + .count(), + 1 + ); + let generation = first + .lookup("a/race", &partition) + .await + .unwrap() + .unwrap() + .generation; + let (cas_first, cas_second) = futures::join!( + first.insert( + "a/race", + &partition, + write(b"first", RecordWriteMode::IfGenerationMatch(generation)) + ), + second.insert( + "a/race", + &partition, + write(b"second", RecordWriteMode::IfGenerationMatch(generation)) + ) + ); + assert_eq!( + [cas_first.unwrap(), cas_second.unwrap()] + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::Written) + .count(), + 1 + ); + }); +} + +#[test] +fn lost_value_acknowledgement_is_uncertain_and_never_replayed() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + fixture.fail("PutItem", true, 500, "InternalServerError"); + runtime.block_on(async { + let partition = provider.binding.layout.resolve("a/uncertain").unwrap(); + let error = provider + .insert( + "a/uncertain", + &partition, + write(b"PRIVATE_BODY", RecordWriteMode::Add), + ) + .await + .unwrap_err(); + assert!(matches!(error, KvError::MutationUncertain)); + assert!(!format!("{error:?} {error}").contains("PRIVATE")); + assert!( + provider + .lookup("a/uncertain", &partition) + .await + .unwrap() + .is_some() + ); + }); + assert_eq!( + fixture + .requests() + .iter() + .filter(|(target, _)| target == "PutItem") + .count(), + 1 + ); +} + +#[test] +fn lost_delete_acknowledgement_is_uncertain_without_deleting_allocator() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + runtime.block_on(async { + let partition = provider.binding.layout.resolve("a/delete").unwrap(); + provider + .insert("a/delete", &partition, write(b"body", RecordWriteMode::Add)) + .await + .unwrap(); + fixture.fail("DeleteItem", true, 500, "InternalServerError"); + assert!(matches!( + provider + .delete_value("a/delete", &partition) + .await + .unwrap_err(), + KvError::MutationUncertain + )); + assert!( + provider + .lookup("a/delete", &partition) + .await + .unwrap() + .is_none() + ); + }); + assert_eq!( + fixture + .requests() + .iter() + .filter(|(target, _)| target == "DeleteItem") + .count(), + 1 + ); + assert_eq!(fixture.state.lock().unwrap().rows.len(), 1); +} + +#[test] +fn allocator_failure_or_exhaustion_never_dispatches_a_value_mutation() { + for exhausted in [false, true] { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + let partition = provider.binding.layout.resolve("a/exhausted").unwrap(); + if exhausted { + let key = provider.key("a/exhausted", &partition, "g").unwrap(); + fixture.state.lock().unwrap().rows.insert((provider.partition_key(&partition).unwrap(), "g#a/exhausted".to_owned()), json!({"pk": {"S": key["pk"].as_s().unwrap()}, "sk": {"S": "g#a/exhausted"}, "generation": {"N": u64::MAX.to_string()}})); + } else { + fixture.fail("UpdateItem", true, 500, "InternalServerError"); + } + assert!(matches!( + runtime + .block_on(provider.insert( + "a/exhausted", + &partition, + write(b"body", RecordWriteMode::Add) + )) + .unwrap_err(), + KvError::Unavailable + )); + assert_eq!(fixture.requests().len(), 1); + assert_eq!(fixture.requests()[0].0, "UpdateItem"); + } +} + +#[test] +fn one_deadline_bounds_allocation_before_value_dispatch() { + let runtime = runtime(); + let fixture = Fixture::start(); + let mut settings = settings(); + settings.limits.operation_timeout_ms = 40; + settings.limits.attempt_timeout_ms = 40; + settings.limits.connect_timeout_ms = 40; + let provider = store(&runtime, &fixture, &settings); + fixture.state.lock().unwrap().allocation_delay = Some(Duration::from_millis(100)); + let partition = provider.binding.layout.resolve("a/budget").unwrap(); + assert!(matches!( + runtime + .block_on(provider.insert("a/budget", &partition, write(b"body", RecordWriteMode::Add))) + .unwrap_err(), + KvError::Unavailable + )); + assert_eq!(fixture.requests().len(), 1); +} + +#[test] +fn query_and_scan_keep_empty_filtered_pages_and_native_foreign_positions() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + let partition = provider.binding.layout.partition("a/").unwrap(); + let pk = provider.partition_key(&partition).unwrap(); + fixture.state.lock().unwrap().query_pages.push_back( + json!({"Items": [], "LastEvaluatedKey": {"pk": {"S": pk}, "sk": {"S": "g#a/internal"}}}), + ); + fixture.state.lock().unwrap().scan_pages.push_back(json!({"Items": [], "LastEvaluatedKey": {"pk": {"S": "foreign-namespace"}, "sk": {"S": "foreign-key"}}})); + runtime.block_on(async { + let query = provider + .record_list_partition(&partition, None, 1) + .await + .unwrap(); + assert!(query.keys.is_empty()); + let query_cursor = query.cursor.unwrap(); + provider + .page("a/", None, Some(&query_cursor), 1) + .await + .unwrap_err(); + provider + .record_list_partition(&partition, Some(&query_cursor), 1) + .await + .unwrap(); + let scan = provider.page("", None, None, 1).await.unwrap(); + assert!(scan.keys.is_empty()); + let scan_cursor = scan.cursor.unwrap(); + provider + .record_list_partition(&partition, Some(&scan_cursor), 1) + .await + .unwrap_err(); + provider + .page("", None, Some(&scan_cursor), 1) + .await + .unwrap(); + }); + let requests = fixture.requests(); + assert_eq!(requests.len(), 4); + assert_eq!(requests[0].1["ConsistentRead"], true); + assert_eq!(requests[0].1["ExpressionAttributeValues"][":pk"]["S"], pk); + assert_eq!( + requests[1].1["ExclusiveStartKey"]["sk"]["S"], + "g#a/internal" + ); + assert_eq!( + requests[3].1["ExclusiveStartKey"]["pk"]["S"], + "foreign-namespace" + ); + assert_eq!( + requests[2].1["FilterExpression"], + "begins_with(#pk, :binding) AND #format = :format AND #kind = :kind AND begins_with(#sk, :prefix)" + ); + assert_eq!(requests[2].1["Limit"], 1_i32); + assert!(requests[2].1.get("IndexName").is_none()); +} + +#[test] +fn cursors_bind_scope_layout_operation_prefix_and_reject_nonadvancement() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + let position = physical_key("foreign".to_owned(), "internal".to_owned()); + let cursor = provider + .next_cursor("scan", "a", None, Some(position.clone()), None) + .unwrap() + .unwrap(); + provider + .decode_cursor("scan", "b", &cursor, None) + .unwrap_err(); + provider + .decode_cursor("query", "a", &cursor, None) + .unwrap_err(); + provider + .next_cursor("scan", "a", Some(&position), Some(position.clone()), None) + .unwrap_err(); + for field in ["format", "scope", "layout_id", "mode", "prefix"] { + let mut altered: Value = serde_json::from_str(&cursor).unwrap(); + altered[field] = json!("other"); + provider + .decode_cursor("scan", "a", &altered.to_string(), None) + .unwrap_err(); + } + let mut altered: Value = serde_json::from_str(&cursor).unwrap(); + altered["layout_version"] = json!(2_u32); + provider + .decode_cursor("scan", "a", &altered.to_string(), None) + .unwrap_err(); + assert!(fixture.requests().is_empty()); +} + +#[test] +fn validation_precedes_allocation_and_metadata_counts_utf8_bytes() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + runtime.block_on(async { + let partition = provider.binding.layout.resolve("a/limits").unwrap(); + provider + .insert( + "a/limits", + &partition, + RecordWrite { + metadata: Some( + &"\u{e9}".repeat( + settings() + .limits + .max_value_bytes + .div_ceil(2) + .checked_add(1) + .unwrap(), + ), + ), + ..write(b"body", RecordWriteMode::Add) + }, + ) + .await + .unwrap_err(); + provider + .insert( + "a/limits", + &partition, + write( + &vec![0; settings().limits.max_value_bytes + 1], + RecordWriteMode::Add, + ), + ) + .await + .unwrap_err(); + provider + .insert( + "a/limits", + &partition, + RecordWrite { + ttl: Some(Duration::ZERO), + ..write(b"body", RecordWriteMode::Add) + }, + ) + .await + .unwrap_err(); + provider.page("", None, None, 0).await.unwrap_err(); + }); + assert!(fixture.requests().is_empty()); +} + +#[test] +fn metadata_and_body_share_the_configured_native_payload_budget() { + let runtime = runtime(); + let fixture = Fixture::start(); + let mut settings = settings(); + settings.limits.max_value_bytes = 8; + let provider = store(&runtime, &fixture, &settings); + assert_eq!(provider.binding.max_metadata_bytes, 8); + let partition = provider.binding.layout.resolve("a/budget").unwrap(); + runtime.block_on(async { + assert_eq!( + provider + .insert( + "a/budget", + &partition, + RecordWrite { + body: b"1234", + metadata: Some("\u{e9}\u{e9}"), + ttl: None, + mode: RecordWriteMode::Add, + } + ) + .await + .unwrap(), + RecordWriteOutcome::Written + ); + let before = fixture.requests().len(); + provider + .insert( + "a/budget", + &partition, + RecordWrite { + body: b"12345", + metadata: Some("\u{e9}\u{e9}"), + ttl: None, + mode: RecordWriteMode::Overwrite, + }, + ) + .await + .unwrap_err(); + assert_eq!(fixture.requests().len(), before); + assert_eq!( + provider + .insert( + "a/budget", + &partition, + RecordWrite { + body: b"", + metadata: Some("\u{e9}\u{e9}\u{e9}\u{e9}"), + ttl: None, + mode: RecordWriteMode::Overwrite, + } + ) + .await + .unwrap(), + RecordWriteOutcome::Written + ); + assert_eq!( + provider + .lookup("a/budget", &partition) + .await + .unwrap() + .unwrap() + .metadata + .as_deref(), + Some("\u{e9}\u{e9}\u{e9}\u{e9}") + ); + let key = ( + provider.partition_key(&partition).unwrap(), + "r#a/budget".to_owned(), + ); + fixture.state.lock().unwrap().rows.get_mut(&key).unwrap()["v"] = json!({"B": "eA=="}); + assert!(matches!( + provider.lookup("a/budget", &partition).await.unwrap_err(), + KvError::Unavailable + )); + }); +} + +#[test] +fn multi_partition_scan_and_query_traverse_values_once_through_allocator_pages() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + let mut foreign_settings = settings(); + foreign_settings.namespace = "foreign".to_owned(); + let foreign = store(&runtime, &fixture, &foreign_settings); + runtime.block_on(async { + for key in ["a/one", "a/two", "b/one", "b/two"] { + let partition = provider.binding.layout.resolve(key).unwrap(); + provider + .insert(key, &partition, write(b"body", RecordWriteMode::Add)) + .await + .unwrap(); + } + foreign + .insert( + "a/foreign", + &foreign.binding.layout.resolve("a/foreign").unwrap(), + write(b"body", RecordWriteMode::Add), + ) + .await + .unwrap(); + let partition = provider.binding.layout.partition("a/").unwrap(); + let mut cursor = None; + let mut keys = BTreeSet::new(); + let mut empty_pages = 0_usize; + loop { + let page = provider + .record_list_partition(&partition, cursor.as_deref(), 1) + .await + .unwrap(); + if page.keys.is_empty() && page.cursor.is_some() { + empty_pages += 1; + } + for key in page.keys { + assert!(keys.insert(key)); + } + cursor = page.cursor; + if cursor.is_none() { + break; + } + } + assert_eq!(keys.into_iter().collect::>(), ["a/one", "a/two"]); + assert_eq!(empty_pages, 2); + for prefix in ["", "a", "a/o", "missing"] { + let mut scan_cursor = None; + let mut scan_keys = BTreeSet::new(); + loop { + let page = provider + .page(prefix, None, scan_cursor.as_deref(), 1) + .await + .unwrap(); + for key in page.keys { + assert!(scan_keys.insert(key)); + } + scan_cursor = page.cursor; + if scan_cursor.is_none() { + break; + } + } + let expected: BTreeSet<_> = ["a/one", "a/two", "b/one", "b/two"] + .into_iter() + .filter(|key| key.starts_with(prefix)) + .map(str::to_owned) + .collect(); + assert_eq!(scan_keys, expected); + } + }); +} + +#[test] +fn recreated_provider_and_expired_row_removal_do_not_reset_allocator() { + let runtime = runtime(); + let fixture = Fixture::start(); + let provider = store(&runtime, &fixture, &settings()); + let partition = provider.binding.layout.resolve("a/reopen").unwrap(); + let old = runtime.block_on(async { + provider + .insert("a/reopen", &partition, write(b"old", RecordWriteMode::Add)) + .await + .unwrap(); + provider + .lookup("a/reopen", &partition) + .await + .unwrap() + .unwrap() + .generation + }); + fixture.state.lock().unwrap().rows.remove(&( + provider.partition_key(&partition).unwrap(), + "r#a/reopen".to_owned(), + )); + drop(provider); + let reopened = store(&runtime, &fixture, &settings()); + runtime.block_on(async { + reopened + .insert("a/reopen", &partition, write(b"new", RecordWriteMode::Add)) + .await + .unwrap(); + assert_ne!( + reopened + .lookup("a/reopen", &partition) + .await + .unwrap() + .unwrap() + .generation, + old + ); + assert_eq!( + reopened + .insert( + "a/reopen", + &partition, + write(b"stale", RecordWriteMode::IfGenerationMatch(old)) + ) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + }); +} + +#[test] +fn partition_encoding_is_collision_free_and_preserves_full_unicode_keys() { + let runtime = runtime(); + let fixture = Fixture::start(); + let mut namespace_a = settings(); + namespace_a.namespace = "a:1:b".to_owned(); + let mut namespace_b = settings(); + namespace_b.namespace = "a".to_owned(); + let provider_a = store(&runtime, &fixture, &namespace_a); + let provider_b = store(&runtime, &fixture, &namespace_b); + let partition = provider_a.binding.layout.partition("b/").unwrap(); + assert_ne!( + provider_a.partition_key(&partition).unwrap(), + provider_b.partition_key(&partition).unwrap() + ); + let key = "\u{e9}/e\u{301}"; + let key_partition = provider_a.binding.layout.resolve(key).unwrap(); + assert_eq!( + provider_a.key(key, &key_partition, "r").unwrap()["sk"], + AttributeValue::S(format!("r#{key}")) + ); +} diff --git a/crates/edgezero-store-aws/src/dynamodb_tests.rs b/crates/edgezero-store-aws/src/dynamodb_tests.rs index 593a5fb4..861d054c 100644 --- a/crates/edgezero-store-aws/src/dynamodb_tests.rs +++ b/crates/edgezero-store-aws/src/dynamodb_tests.rs @@ -233,7 +233,7 @@ fn serve_one(stream: &mut TcpStream, state: &Arc>) { .unwrap_or_default(); } -fn read_request(stream: &mut TcpStream) -> Option<(String, Vec)> { +pub(super) fn read_request(stream: &mut TcpStream) -> Option<(String, Vec)> { let mut data = Vec::with_capacity(4096); let mut chunk = [0_u8; 2048]; let header_end = loop { @@ -396,7 +396,11 @@ fn decode_base64_known(input: &str) -> Vec { .expect("SDK Binary uses standard base64") } -fn loopback_client(runtime: &Runtime, endpoint: &str, limits: DynamoDbKvLimits) -> Client { +pub(super) fn loopback_client( + runtime: &Runtime, + endpoint: &str, + limits: DynamoDbKvLimits, +) -> Client { // Region and static credentials are explicitly injected before config load; // the service endpoint is explicit in the DynamoDB config. There is no // profile, web-identity, IMDS, environment-credential, or endpoint lookup. @@ -431,7 +435,7 @@ fn loopback_client(runtime: &Runtime, endpoint: &str, limits: DynamoDbKvLimits) Client::from_conf(config) } -fn active_table() -> Value { +pub(super) fn active_table() -> Value { json!({"Table": { "TableName": TABLE, "TableStatus": "ACTIVE", "KeySchema": [ @@ -482,6 +486,10 @@ fn settings() -> DynamoDbKvSettings { } } +#[expect( + clippy::missing_trait_methods, + reason = "ordinary provider wrapper deliberately retains Unsupported record defaults" +)] #[async_trait(?Send)] impl KvStore for RuntimeStore { async fn delete(&self, key: &str) -> Result<(), KvError> { diff --git a/crates/edgezero-store-aws/src/lib.rs b/crates/edgezero-store-aws/src/lib.rs index 160e6b39..c4df04a6 100644 --- a/crates/edgezero-store-aws/src/lib.rs +++ b/crates/edgezero-store-aws/src/lib.rs @@ -2,6 +2,18 @@ //! //! Settings remain available without either service feature. Service failures //! use closed categories; no SDK, HTTP, settings or payload source chain survives. +//! +//! The default-off `dynamodb-records` feature adds +//! `AwsPreparation::prepare_dynamodb_records` on native targets. Its explicit +//! partitioned format does not read or migrate legacy ordinary KV rows. Inserts +//! allocate a retained per-key generation with `UpdateItem`, then `PutItem` the value. +//! Allocators must survive deletion and must not have TTL. Value TTL uses native +//! `ttl_epoch_s` cleanup, without a hard expiry filter. Body and UTF-8 metadata +//! share the configured `max_value_bytes` payload budget. Partition pages use +//! strong base-table Query; ordinary global listing +//! uses unordered Scan, which reads broadly before namespace/prefix filtering. +//! Record preparation binds cursors to the resolved endpoint and `DescribeTable` +//! ARN, plus the table ID when supplied. It does not perform an IAM/write probe. #![cfg_attr( all( diff --git a/crates/edgezero-store-aws/tests/dynamodb_records_local.rs b/crates/edgezero-store-aws/tests/dynamodb_records_local.rs new file mode 100644 index 00000000..4e9db6e7 --- /dev/null +++ b/crates/edgezero-store-aws/tests/dynamodb_records_local.rs @@ -0,0 +1,586 @@ +//! Explicit `DynamoDB` Local acceptance. Invoked by the existing local AWS runner. +//! No hosted AWS credentials, IAM qualification, retry fault injection or native +//! TTL sweeper behavior is claimed by these tests. + +#![cfg(all(not(target_arch = "wasm32"), feature = "dynamodb-records"))] +#![expect( + clippy::arbitrary_source_item_ordering, + reason = "group local-service setup, provider helpers and complete test lifecycles" +)] + +#[cfg(test)] +mod tests { + use std::collections::{BTreeMap, BTreeSet, HashMap}; + use std::env; + use std::net::{Ipv4Addr, SocketAddr}; + use std::process::{self, Command}; + use std::time::{Duration, SystemTime, UNIX_EPOCH}; + + use aws_sdk_dynamodb::Client; + use aws_sdk_dynamodb::config::{Credentials, Region, retry::RetryConfig}; + use aws_sdk_dynamodb::types::{ + AttributeDefinition, AttributeValue, BillingMode, KeySchemaElement, KeyType, + ScalarAttributeType, + }; + use edgezero_core::key_value_store::{KvError, KvHandle}; + use edgezero_core::record_store::{ + RecordGeneration, RecordKeyLayout, RecordLayout, RecordWrite, RecordWriteMode, + RecordWriteOutcome, contract, + }; + use edgezero_core::time::MonotonicClock; + use edgezero_store_aws::AwsPreparation; + use edgezero_store_aws::settings::{ + AwsPreparationLimits, DynamoDbKvLimits, DynamoDbKvSettings, + }; + use tokio::runtime::Builder; + + type Item = HashMap; + + #[derive(Clone, Copy)] + struct Groups; + + impl RecordKeyLayout for Groups { + fn id(&self) -> &'static str { + "dynamodb-local-groups" + } + fn version(&self) -> u32 { + 1 + } + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + let end = key + .find('/') + .ok_or_else(|| KvError::Validation("missing group".to_owned()))?; + key.get(..=end) + .ok_or_else(|| KvError::Validation("invalid group".to_owned())) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.bytes().filter(|byte| *byte == b'/').count() == 1 { + Ok(()) + } else { + Err(KvError::Validation("incomplete group".to_owned())) + } + } + } + + fn endpoint() -> String { + let endpoint = env::var("EDGEZERO_AWS_TEST_DYNAMODB_ENDPOINT") + .expect("explicit DynamoDB Local endpoint required; use the existing local AWS runner"); + let address = endpoint + .strip_prefix("http://") + .and_then(|authority| authority.parse::().ok()) + .expect("DynamoDB Local endpoint must be literal http://127.0.0.1:"); + assert_eq!(address.ip(), Ipv4Addr::LOCALHOST); + assert_ne!(address.port(), 0); + assert_eq!(endpoint, format!("http://{address}")); + endpoint + } + + fn isolated_child(test: &str) -> bool { + let endpoint = endpoint(); + if env::var("EDGEZERO_DYNAMODB_RECORDS_CHILD").as_deref() == Ok(test) { + assert!( + env::var("AWS_ENDPOINT_URL_DYNAMODB").is_ok_and(|value| value == endpoint), + "public preparation must use the validated local endpoint" + ); + assert!( + env::var("AWS_ACCESS_KEY_ID").as_deref() == Ok("DUMMYLOCAL") + && env::var("AWS_SECRET_ACCESS_KEY").as_deref() == Ok("DUMMYLOCAL"), + "only dummy local credentials are allowed" + ); + return false; + } + let qualified_test = format!("tests::{test}"); + let status = Command::new(env::current_exe().expect("test executable")) + .args(["--ignored", "--exact", &qualified_test, "--nocapture"]) + .env("EDGEZERO_DYNAMODB_RECORDS_CHILD", test) + .env("EDGEZERO_AWS_TEST_DYNAMODB_ENDPOINT", &endpoint) + .env("AWS_ENDPOINT_URL_DYNAMODB", &endpoint) + .env("AWS_ENDPOINT_URL", &endpoint) + .env("AWS_IGNORE_CONFIGURED_ENDPOINT_URLS", "false") + .env("AWS_ACCESS_KEY_ID", "DUMMYLOCAL") + .env("AWS_SECRET_ACCESS_KEY", "DUMMYLOCAL") + .env("AWS_REGION", "us-east-1") + .env("AWS_DEFAULT_REGION", "us-east-1") + .env("AWS_EC2_METADATA_DISABLED", "true") + .env("AWS_ACCOUNT_ID_ENDPOINT_MODE", "disabled") + .env_remove("AWS_SESSION_TOKEN") + .env_remove("AWS_PROFILE") + .env_remove("AWS_WEB_IDENTITY_TOKEN_FILE") + .env_remove("AWS_ROLE_ARN") + .status() + .expect("spawn isolated dummy-credential test process"); + assert!(status.success(), "DynamoDB Local child failed"); + true + } + + async fn admin() -> Client { + let config = aws_config::defaults(aws_config::BehaviorVersion::v2026_01_12()) + .region(Region::new("us-east-1")) + .credentials_provider(Credentials::new( + "DUMMYLOCAL", + "DUMMYLOCAL", + None, + None, + "local-record-proof", + )) + .endpoint_url(endpoint()) + .load() + .await; + Client::from_conf( + aws_sdk_dynamodb::Config::from(&config) + .to_builder() + .retry_config(RetryConfig::standard().with_max_attempts(1)) + .build(), + ) + } + + async fn table(client: &Client) -> String { + let nanos = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); + let table = format!("edgezero-records-local-{}-{nanos}", process::id()); + client + .create_table() + .table_name(&table) + .billing_mode(BillingMode::PayPerRequest) + .attribute_definitions( + AttributeDefinition::builder() + .attribute_name("pk") + .attribute_type(ScalarAttributeType::S) + .build() + .unwrap(), + ) + .attribute_definitions( + AttributeDefinition::builder() + .attribute_name("sk") + .attribute_type(ScalarAttributeType::S) + .build() + .unwrap(), + ) + .key_schema( + KeySchemaElement::builder() + .attribute_name("pk") + .key_type(KeyType::Hash) + .build() + .unwrap(), + ) + .key_schema( + KeySchemaElement::builder() + .attribute_name("sk") + .key_type(KeyType::Range) + .build() + .unwrap(), + ) + .send() + .await + .expect("create unique local record table"); + table + } + + async fn prepared(table: &str, namespace: &str) -> KvHandle { + let mut preparation = + AwsPreparation::new(MonotonicClock::default(), AwsPreparationLimits::default()) + .unwrap(); + preparation + .prepare_dynamodb_records( + &DynamoDbKvSettings { + region: "us-east-1".to_owned(), + table: table.to_owned(), + namespace: namespace.to_owned(), + limits: DynamoDbKvLimits::default(), + }, + &RecordLayout::new(Groups).unwrap(), + ) + .await + .expect("prepare actual public record provider") + } + + fn write(body: &[u8], mode: RecordWriteMode) -> RecordWrite<'_> { + RecordWrite { + body, + metadata: Some("exact m\u{e9}tadata"), + ttl: None, + mode, + } + } + + async fn all_items(client: &Client, table: &str) -> Vec { + let mut after = None; + let mut items = Vec::new(); + loop { + let output = client + .scan() + .table_name(table) + .consistent_read(true) + .set_exclusive_start_key(after) + .send() + .await + .unwrap(); + items.extend(output.items.unwrap_or_default()); + after = output.last_evaluated_key.filter(|key| !key.is_empty()); + if after.is_none() { + return items; + } + } + } + + fn text<'item>(item: &'item Item, name: &str) -> &'item str { + item.get(name) + .and_then(|value| value.as_s().ok()) + .expect("physical string attribute") + } + + fn key(item: &Item) -> Item { + Item::from([ + ("pk".to_owned(), item["pk"].clone()), + ("sk".to_owned(), item["sk"].clone()), + ]) + } + + async fn global_keys(kv: &KvHandle, prefix: &str) -> (BTreeSet, usize) { + let mut cursor = None; + let mut keys = BTreeSet::new(); + let mut empty_continuations = 0_usize; + loop { + let page = kv + .list_keys_page(prefix, cursor.as_deref(), 1) + .await + .unwrap(); + assert!(page.keys.len() <= 1); + if page.keys.is_empty() && page.cursor.is_some() { + empty_continuations = empty_continuations.checked_add(1).expect("page counter"); + } + for key in page.keys { + assert!(keys.insert(key), "duplicate global Scan key"); + } + cursor = page.cursor; + if cursor.is_none() { + return (keys, empty_continuations); + } + } + } + + #[test] + #[ignore = "requires EDGEZERO_AWS_TEST_DYNAMODB_ENDPOINT from existing local AWS runner"] + #[expect( + clippy::too_many_lines, + reason = "keep the actual common contract, independent writers and physical-layout assertions together" + )] + fn dynamodb_records_local_contract() { + if isolated_child("dynamodb_records_local_contract") { + return; + } + Builder::new_current_thread() + .enable_all() + .build() + .unwrap() + .block_on(async { + let client = admin().await; + let table = table(&client).await; + let kv = prepared(&table, "contract").await; + contract::assert_record_contract(&kv, "group/").await; + let first = prepared(&table, "independent") + .await + .require_records() + .unwrap(); + let second = prepared(&table, "independent") + .await + .require_records() + .unwrap(); + let (add_first, add_second) = futures::join!( + first.insert("a/add-race", write(b"first", RecordWriteMode::Add)), + second.insert("a/add-race", write(b"second", RecordWriteMode::Add)) + ); + assert_eq!( + [add_first.unwrap(), add_second.unwrap()] + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::Written) + .count(), + 1 + ); + let generation = first + .lookup("a/add-race") + .await + .unwrap() + .unwrap() + .generation; + let (cas_first, cas_second) = futures::join!( + first.insert( + "a/add-race", + write(b"first", RecordWriteMode::IfGenerationMatch(generation)) + ), + second.insert( + "a/add-race", + write(b"second", RecordWriteMode::IfGenerationMatch(generation)) + ) + ); + assert_eq!( + [cas_first.unwrap(), cas_second.unwrap()] + .iter() + .filter(|outcome| **outcome == RecordWriteOutcome::Written) + .count(), + 1 + ); + let listing = prepared(&table, "listing").await; + let records = listing.require_records().unwrap(); + let expected: BTreeSet<_> = ["a/one", "a/two", "b/one", "b/two"] + .into_iter() + .map(str::to_owned) + .collect(); + for key in &expected { + records + .insert(key, write(key.as_bytes(), RecordWriteMode::Add)) + .await + .unwrap(); + } + let foreign = prepared(&table, "foreign").await.require_records().unwrap(); + foreign + .insert("a/foreign", write(b"foreign", RecordWriteMode::Add)) + .await + .unwrap(); + let mut cursor = None; + let mut keys = BTreeSet::new(); + let mut empty_continuations = 0_usize; + let partition = records.partition("a/").unwrap(); + loop { + let page = records + .list_partition(&partition, cursor.as_deref(), 1) + .await + .unwrap(); + if page.keys.is_empty() && page.cursor.is_some() { + empty_continuations = + empty_continuations.checked_add(1).expect("page counter"); + } + for key in page.keys { + assert!(keys.insert(key), "duplicate strong Query key"); + } + cursor = page.cursor; + if cursor.is_none() { + break; + } + } + assert_eq!( + keys, + BTreeSet::from(["a/one".to_owned(), "a/two".to_owned()]) + ); + assert!( + empty_continuations >= 2, + "native Query traverses allocator-only pages" + ); + let (global, empty) = global_keys(&listing, "").await; + assert_eq!(global, expected); + assert!( + empty > 0, + "native Scan preserves internal/foreign filtered pages" + ); + assert_eq!( + global_keys(&listing, "a/o").await.0, + BTreeSet::from(["a/one".to_owned()]) + ); + assert!(global_keys(&listing, "nonmatching").await.0.is_empty()); + let items = all_items(&client, &table).await; + let values: BTreeMap<_, _> = items + .iter() + .filter(|item| { + ["r#a/one", "r#a/two", "r#b/one", "r#b/two"].contains(&text(item, "sk")) + }) + .map(|item| (text(item, "sk"), item)) + .collect(); + assert_ne!(text(values["r#a/one"], "pk"), text(values["r#b/one"], "pk")); + assert_eq!(text(values["r#a/one"], "pk"), text(values["r#a/two"], "pk")); + for logical in &expected { + let value = values[format!("r#{logical}").as_str()]; + let allocator = items + .iter() + .find(|item| { + text(item, "pk") == text(value, "pk") + && text(item, "sk") == format!("g#{logical}") + }) + .expect("retained per-key allocator colocated with value"); + assert!(allocator.get("ttl_epoch_s").is_none()); + assert!(allocator.get("v").is_none()); + assert!( + allocator + .get("generation") + .unwrap() + .as_n() + .unwrap() + .parse::() + .unwrap() + > 0 + ); + } + assert!( + items + .iter() + .filter(|item| item.get("generation").is_some() && item.get("v").is_none()) + .all(|item| text(item, "sk").starts_with("g#")), + "no namespace-wide allocator row" + ); + client + .delete_table() + .table_name(&table) + .send() + .await + .unwrap(); + }); + } + + #[test] + #[ignore = "requires EDGEZERO_AWS_TEST_DYNAMODB_ENDPOINT from existing local AWS runner"] + #[expect( + clippy::too_many_lines, + reason = "keep retained-row visibility, simulated cleanup and provider reconstruction in one lifecycle" + )] + fn dynamodb_records_local_simulated_expiry_reopen() { + if isolated_child("dynamodb_records_local_simulated_expiry_reopen") { + return; + } + Builder::new_current_thread() + .enable_all() + .build() + .unwrap() + .block_on(async { + let client = admin().await; + let table = table(&client).await; + let kv = prepared(&table, "expiry-history").await; + let records = kv.require_records().unwrap(); + records + .insert( + "a/expiry", + RecordWrite { + ttl: Some(Duration::from_secs(60)), + ..write(b"retained", RecordWriteMode::Add) + }, + ) + .await + .unwrap(); + let old = records + .lookup("a/expiry") + .await + .unwrap() + .unwrap() + .generation; + let items = all_items(&client, &table).await; + let value = items + .iter() + .find(|item| text(item, "sk") == "r#a/expiry") + .unwrap(); + let physical = key(value); + client + .update_item() + .table_name(&table) + .set_key(Some(physical.clone())) + .update_expression("SET #ttl = :past") + .expression_attribute_names("#ttl", "ttl_epoch_s") + .expression_attribute_values(":past", AttributeValue::N("1".to_owned())) + .send() + .await + .unwrap(); + assert!( + records.exists_strong("a/expiry").await.unwrap(), + "native past-TTL retained row remains visible" + ); + assert_eq!( + records + .lookup("a/expiry") + .await + .unwrap() + .unwrap() + .generation, + old + ); + assert_eq!( + records + .insert("a/expiry", write(b"blocked", RecordWriteMode::Add)) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + assert!(global_keys(&kv, "a/").await.0.contains("a/expiry")); + // DynamoDB Local does not sweep TTL. This deletion simulates physical + // expiry cleanup of only the value row, not a native sweeper proof. + client + .delete_item() + .table_name(&table) + .set_key(Some(physical)) + .send() + .await + .unwrap(); + drop(records); + drop(kv); + let reopened = prepared(&table, "expiry-history") + .await + .require_records() + .unwrap(); + reopened + .insert("a/expiry", write(b"recreated", RecordWriteMode::Add)) + .await + .unwrap(); + assert_ne!( + reopened + .lookup("a/expiry") + .await + .unwrap() + .unwrap() + .generation, + old + ); + assert_eq!( + reopened + .insert( + "a/expiry", + write(b"stale", RecordWriteMode::IfGenerationMatch(old)) + ) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + let reopened_items = all_items(&client, &table).await; + let allocator = reopened_items + .iter() + .find(|item| text(item, "sk") == "g#a/expiry") + .unwrap(); + assert!(allocator.get("ttl_epoch_s").is_none()); + client + .update_item() + .table_name(&table) + .set_key(Some(key(allocator))) + .update_expression("SET #generation = :ceiling") + .expression_attribute_names("#generation", "generation") + .expression_attribute_values( + ":ceiling", + AttributeValue::N(u64::MAX.to_string()), + ) + .send() + .await + .unwrap(); + assert!(matches!( + reopened + .insert("a/expiry", write(b"exhausted", RecordWriteMode::Overwrite)) + .await + .unwrap_err(), + KvError::Unavailable + )); + assert_ne!( + reopened + .lookup("a/expiry") + .await + .unwrap() + .unwrap() + .generation, + RecordGeneration::from_u64(u64::MAX) + ); + client + .delete_table() + .table_name(&table) + .send() + .await + .unwrap(); + }); + } +} diff --git a/crates/edgezero-store-celld/Cargo.toml b/crates/edgezero-store-celld/Cargo.toml index 4ab788c6..8f4ab299 100644 --- a/crates/edgezero-store-celld/Cargo.toml +++ b/crates/edgezero-store-celld/Cargo.toml @@ -15,6 +15,7 @@ default = [] config = ["_transport"] kv = ["_transport", "dep:base64", "dep:sha2"] records = ["kv"] +records-v2 = ["kv"] _transport = ["dep:edgezero-core", "dep:bytes", "dep:async-trait", "dep:reqwest", "dep:tokio", "dep:futures-util", "dep:serde_json"] [dependencies] @@ -37,4 +38,6 @@ tokio = { workspace = true, optional = true } serde_json = { workspace = true } [target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies] +edgezero-core = { workspace = true, features = ["test-utils"] } +tempfile = { workspace = true } tokio-rustls = { workspace = true } diff --git a/crates/edgezero-store-celld/README.md b/crates/edgezero-store-celld/README.md new file mode 100644 index 00000000..7ca9d3b2 --- /dev/null +++ b/crates/edgezero-store-celld/README.md @@ -0,0 +1,37 @@ +# Native scoped celld stores + +All runtime features are default-off. Settings remain available without native transport dependencies. + +- `kv` prepares the existing raw `/v1/.../value` binding without a remote probe. +- `records` retains the legacy, separate positive-TTL record client and binary metadata API. +- `records-v2` enables the portable core record contract over a fresh v2 dataset. It implies `kv` for shared transport, not automatic conversion of raw bindings. + +Prepare an explicit record binding with: + +```rust,ignore +preparation + .prepare_record_store(&settings, &layout, token.as_ref()) + .await?; +``` + +The method takes `&mut CelldPreparation`, `&CelldKvSettings`, `&RecordLayout`, and `Option<&CelldAccessToken>`, returning `Result`. Application layouts must be installed before provider preparation. The initializer can acquire `KvHandle::require_records()` without changing routing or probing with a mutation. + +Preparation performs a read-only `/v2/stores/{alias}/record-capabilities` handshake and checks the exact format, keyset pagination, no-expiry support, metadata limit, and response version. An old or incompatible service fails preparation. Capability does not establish permanent availability or write authorization. There is no downgrade, data migration or raw-data fallback. + +Ordinary and record operations on this binding use the same `/v2/.../record` rows. Ordinary replacements clear metadata, allocate a fresh generation and apply the supplied expiry. Ordinary no-TTL writes send `ttl_ms: null`, which stores actual non-expiry. Raw and v1 record datasets remain separate and are not automatically visible in v2. + +Bodies remain arbitrary bytes. Portable metadata is optional UTF-8, with absent and empty normalized to None. Nonempty text is preserved exactly, invalid UTF-8 responses fail, and the actual celld limit is 1024 metadata bytes. The body cap is the configured native cap, at most 1 MiB. Existing core TTL bounds apply to the portable interface; relative TTL rounds upward to whole milliseconds. Expiry visibility and physical cleanup follow celld's native behavior. Exact HEAD existence may include expired rows pending sweep. + +Both ordinary arbitrary-prefix/global listing and record partition listing use native keyset continuation. The latter validates complete application partitions. Cursors preserve the actual service token and bind provider scope, format/layout, query mode and prefix while retaining the last observed key for progress validation. Core also binds its public record cursor. Pages are live, not a cross-page snapshot. Stop only when continuation is absent; empty or short pages may continue. + +Mutations are never automatically replayed. A dispatched v2 mutation with a lost, timed-out or untrustworthy response yields payload-free `KvError::MutationUncertain`. A definitive Add/CAS 412 is `PreconditionFailed`; an unconditional rejection is an error. Applications own reconciliation. + +The matching companion service source is the isolated local celld `feat/scoped-records-v2` worktree, based on `9350a043860c4f41dccb692ec9de98ebd9dac3ae`. Its fresh `__kv_records_v2` format retains transactional namespace-local generation allocation across delete, expiry and restart. A cached older binary does not prove compatibility. Record the rebuilt artifact/source identity before acceptance. + +## Focused proof + +Colocated `records_v2` wire tests cover protocol preparation, ordinary/record routes, no-expiry encoding, metadata validation, cursor identity/progress and lost-reply classification without retries. Run legacy/raw neighboring tests separately as well. + +`tests/records_v2_interop.rs` is an explicit ignored real-service test. Supply `CELLD_RECORDS_V2_ENDPOINT=http://127.0.0.1:` for a freshly rebuilt disposable scoped listener with an empty writable alias `identity`, then run the test with `--ignored`. It executes the common public-handle contract, independent Add/CAS races, multi-group global listing, and an acknowledgement-dropping proxy that confirms actual server commits and no mutation replay. After a lost DELETE the proxy recreates the value before closing the connection, so an unsafe replay would erase observable data. + +No hosted service, fleet isolation, publication, deployment or migration claim follows from local proof. diff --git a/crates/edgezero-store-celld/src/client.rs b/crates/edgezero-store-celld/src/client.rs index 89391225..c6b86c4b 100644 --- a/crates/edgezero-store-celld/src/client.rs +++ b/crates/edgezero-store-celld/src/client.rs @@ -43,6 +43,16 @@ impl fmt::Debug for ScopedClient { } } +#[cfg(feature = "records-v2")] +#[derive(Clone, Copy)] +pub(crate) enum RecordV2Response { + Capabilities, + Exists, + Mutation, + Page, + Value, +} + #[derive(Clone, Copy)] pub(crate) enum ResponseKind { #[cfg(feature = "kv")] @@ -63,11 +73,15 @@ pub(crate) enum ResponseKind { RecordMutation, #[cfg(feature = "records")] RecordPage, + #[cfg(feature = "records-v2")] + RecordV2(RecordV2Response), } impl ResponseKind { fn mutation(self) -> bool { match self { + #[cfg(feature = "records-v2")] + Self::RecordV2(kind) => matches!(kind, RecordV2Response::Mutation), #[cfg(feature = "kv")] Self::Mutation => true, #[cfg(feature = "records")] @@ -83,6 +97,11 @@ impl ResponseKind { fn json(self) -> bool { match self { + #[cfg(feature = "records-v2")] + Self::RecordV2(kind) => matches!( + kind, + RecordV2Response::Capabilities | RecordV2Response::Page | RecordV2Response::Value + ), #[cfg(feature = "kv")] Self::Page => true, #[cfg(feature = "records")] @@ -98,6 +117,8 @@ impl ResponseKind { fn optional(self) -> bool { match self { + #[cfg(feature = "records-v2")] + Self::RecordV2(kind) => matches!(kind, RecordV2Response::Value), #[cfg(feature = "kv")] Self::Value => true, #[cfg(feature = "records")] @@ -111,12 +132,17 @@ impl ResponseKind { } } - #[cfg(feature = "records")] - fn record(self) -> bool { - matches!( - self, - Self::Record | Self::RecordExists | Self::RecordMutation | Self::RecordPage - ) + #[cfg(any(feature = "records", feature = "records-v2"))] + fn record_version(self) -> Option<&'static [u8]> { + match self { + #[cfg(feature = "records")] + Self::Record | Self::RecordExists | Self::RecordMutation | Self::RecordPage => { + Some(b"1") + } + #[cfg(feature = "records-v2")] + Self::RecordV2(_) => Some(b"2"), + _ => None, + } } } @@ -196,6 +222,20 @@ impl ScopedClient { Ok(url) } + #[cfg(feature = "records-v2")] + pub(crate) fn record_v2_url( + &self, + store: &str, + resource: &str, + ) -> Result { + let mut url = self.endpoint.clone(); + url.path_segments_mut() + .map_err(|_error| PreparationError::InvalidEndpoint)? + .clear() + .extend(["v2", "stores", store, resource]); + Ok(url) + } + /// Covers sending, streaming and decoding with one real timer and injected /// monotonic checks. None means a validated miss, never a protocol failure. pub(crate) async fn execute( @@ -219,23 +259,25 @@ impl ScopedClient { request = request.header(AUTHORIZATION, authorization.clone()); } if let Some(payload) = body { - #[cfg(feature = "records")] - let content_type = if matches!(kind, ResponseKind::RecordMutation) { + #[cfg(any(feature = "records", feature = "records-v2"))] + let content_type = if kind.record_version().is_some() && kind.mutation() { "application/json" } else { "application/octet-stream" }; - #[cfg(not(feature = "records"))] + #[cfg(not(any(feature = "records", feature = "records-v2")))] let content_type = "application/octet-stream"; request = request.header(CONTENT_TYPE, content_type).body(payload); } - let result = timeout(budget, async { + #[cfg(feature = "records-v2")] + let mut definitive_rejection = false; + let exchange = timeout(budget, async { let response = request.send().await.map_err(http_error)?; let headers = response.headers(); validate_headers(headers)?; - #[cfg(feature = "records")] - if kind.record() { - validate_record_version(headers)?; + #[cfg(any(feature = "records", feature = "records-v2"))] + if let Some(version) = kind.record_version() { + validate_record_version(headers, version)?; } let status = response.status(); let success = status @@ -282,6 +324,10 @@ impl ScopedClient { if parsed.error != expected { return Err(PreparationError::Malformed); } + #[cfg(feature = "records-v2")] + if status.is_client_error() { + definitive_rejection = true; + } if missing && kind.optional() { return Ok(None); } @@ -289,15 +335,25 @@ impl ScopedClient { }) .await .map_err(|_elapsed| PreparationError::Deadline); - self.check(deadline)?; - result? + let result = self.check(deadline).and_then(|_remaining| exchange?); + #[cfg(feature = "records-v2")] + if matches!(kind, ResponseKind::RecordV2(RecordV2Response::Mutation)) + && !definitive_rejection + { + return result.map_err(|_error| PreparationError::MutationUncertain); + } + result } } -#[cfg(feature = "records")] -fn validate_record_version(headers: &HeaderMap) -> Result<(), PreparationError> { +#[cfg(any(feature = "records", feature = "records-v2"))] +fn validate_record_version(headers: &HeaderMap, expected: &[u8]) -> Result<(), PreparationError> { let mut versions = headers.get_all("x-celld-record-version").iter(); - if versions.next().is_none_or(|value| value.as_bytes() != b"1") || versions.next().is_some() { + if versions + .next() + .is_none_or(|value| value.as_bytes() != expected) + || versions.next().is_some() + { return Err(PreparationError::UnsupportedVersion); } Ok(()) @@ -383,7 +439,7 @@ fn status_error(status: StatusCode) -> Option<(&'static str, PreparationError)> 401 => ("unauthenticated", PreparationError::Unauthenticated), 403 => ("forbidden", PreparationError::Forbidden), 404 => ("not_found", PreparationError::Missing), - #[cfg(feature = "records")] + #[cfg(any(feature = "records", feature = "records-v2"))] 412 => ("precondition_failed", PreparationError::PreconditionFailed), 413 => ("limit_exceeded", PreparationError::SizeLimit), 426 => ("unsupported_version", PreparationError::UnsupportedVersion), diff --git a/crates/edgezero-store-celld/src/kv.rs b/crates/edgezero-store-celld/src/kv.rs index d8a89f1f..e16371b7 100644 --- a/crates/edgezero-store-celld/src/kv.rs +++ b/crates/edgezero-store-celld/src/kv.rs @@ -188,6 +188,10 @@ impl CelldKvStore { } #[async_trait::async_trait(?Send)] +#[expect( + clippy::missing_trait_methods, + reason = "ordinary-only raw KV inherits Unsupported record defaults; it must not advertise record capability" +)] impl KvStore for CelldKvStore { async fn delete(&self, key: &str) -> Result<(), KvError> { let deadline = self.client.deadline().map_err(kv_error)?; @@ -328,8 +332,10 @@ fn kv_error(error: PreparationError) -> KvError { | PreparationError::UnsupportedVersion | PreparationError::Unavailable | PreparationError::Deadline => KvError::Unavailable, - #[cfg(feature = "records")] + #[cfg(any(feature = "records", feature = "records-v2"))] PreparationError::PreconditionFailed => KvError::Unavailable, + #[cfg(feature = "records-v2")] + PreparationError::MutationUncertain => KvError::MutationUncertain, } } diff --git a/crates/edgezero-store-celld/src/lib.rs b/crates/edgezero-store-celld/src/lib.rs index 4210b913..acc79b7b 100644 --- a/crates/edgezero-store-celld/src/lib.rs +++ b/crates/edgezero-store-celld/src/lib.rs @@ -23,6 +23,8 @@ mod kv; mod preparation; #[cfg(all(not(target_arch = "wasm32"), feature = "records"))] pub mod records; +#[cfg(all(not(target_arch = "wasm32"), feature = "records-v2"))] +mod records_v2; #[cfg(test)] #[cfg(all(not(target_arch = "wasm32"), feature = "_transport"))] mod test_support; @@ -52,7 +54,7 @@ pub enum PreparationError { Forbidden, #[error("missing resource")] Missing, - #[cfg(feature = "records")] + #[cfg(any(feature = "records", feature = "records-v2"))] #[error("write precondition failed")] PreconditionFailed, #[error("size limit")] @@ -65,4 +67,7 @@ pub enum PreparationError { Unavailable, #[error("deadline exceeded")] Deadline, + #[cfg(feature = "records-v2")] + #[error("mutation may have committed")] + MutationUncertain, } diff --git a/crates/edgezero-store-celld/src/preparation.rs b/crates/edgezero-store-celld/src/preparation.rs index 7dea4798..4939550f 100644 --- a/crates/edgezero-store-celld/src/preparation.rs +++ b/crates/edgezero-store-celld/src/preparation.rs @@ -29,11 +29,15 @@ use crate::config::ConfigSnapshot; use crate::kv::CelldKvStore; #[cfg(feature = "records")] use crate::records::CelldRecordStore; +#[cfg(feature = "records-v2")] +use crate::records_v2::CelldRecordKvStore; #[cfg(feature = "config")] use crate::settings::CelldConfigSettings; #[cfg(feature = "kv")] use crate::settings::CelldKvSettings; use crate::settings::{CelldAccess, CelldAccessToken, CelldPreparationLimits, endpoint}; +#[cfg(feature = "records-v2")] +use edgezero_core::record_store::RecordLayout; /// Aggregate counters, not process memory measurements. Metadata counts bounded /// identifier storage plus conservative per-record overhead. Staging includes at @@ -163,6 +167,28 @@ impl CelldPreparation { result } + /// Prepares an explicitly selected fresh v2 record binding over one dataset. + /// A read-only capability handshake establishes pagination and no-expiry support. + /// + /// # Errors + /// Rejects old/incompatible services without writes, downgrade or migration. + /// Any failed preparation poisons the session, as with other prepared stores. + #[cfg(feature = "records-v2")] + #[inline] + pub async fn prepare_record_store( + &mut self, + settings: &CelldKvSettings, + layout: &RecordLayout, + token: Option<&CelldAccessToken>, + ) -> Result { + self.begin()?; + let result = self.record_store_inner(settings, layout, token).await; + if result.is_ok() { + self.poisoned = false; + } + result + } + /// Safe aggregate counters; no resource or authorization identifiers. #[inline] #[must_use] @@ -355,6 +381,34 @@ impl CelldPreparation { Ok(records) } + #[cfg(feature = "records-v2")] + async fn record_store_inner( + &mut self, + settings: &CelldKvSettings, + layout: &RecordLayout, + token: Option<&CelldAccessToken>, + ) -> Result { + settings.validate()?; + self.charge_mapping(&[ + settings.endpoint.len(), + settings.store.len(), + layout.id().len(), + ])?; + let (_, client) = self.client(&settings.endpoint, &settings.access, token)?; + let endpoint = endpoint(&settings.endpoint, &settings.access)?; + self.check()?; + let store = CelldRecordKvStore::new( + client, + &endpoint, + settings.store.clone(), + self.limits, + layout, + )?; + store.probe(self.deadline).await?; + self.check()?; + Ok(KvHandle::new(Arc::new(store))) + } + #[cfg(feature = "kv")] fn kv_inner( &mut self, diff --git a/crates/edgezero-store-celld/src/records_v2.rs b/crates/edgezero-store-celld/src/records_v2.rs new file mode 100644 index 00000000..dfe587a4 --- /dev/null +++ b/crates/edgezero-store-celld/src/records_v2.rs @@ -0,0 +1,613 @@ +//! Explicit fresh-format records. Ordinary and record operations share v2 rows. + +#![expect( + clippy::arbitrary_source_item_ordering, + reason = "keep the prepared store and native operations before wire models and validation" +)] + +use std::sync::Arc; +use std::time::Duration; + +use base64::Engine as _; +use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD}; +use bytes::Bytes; +use edgezero_core::key_value_store::{KvError, KvHandle, KvPage, KvStore}; +use edgezero_core::record_store::{ + Record, RecordBinding, RecordGeneration, RecordLayout, RecordPage, RecordPartition, + RecordWrite, RecordWriteMode, RecordWriteOutcome, +}; +use edgezero_core::time::Deadline; +use reqwest::Method; +use serde::{Deserialize, Deserializer, Serialize}; +use sha2::{Digest as _, Sha256}; +use url::Url; + +use crate::PreparationError; +use crate::client::{RecordV2Response, ResponseKind, ScopedClient}; +use crate::settings::{CelldPreparationLimits, valid_key}; + +const MAX_METADATA_BYTES: usize = 1_024; +const MAX_PROVIDER_CURSOR: usize = 8_192; +const MAX_SERVICE_CURSOR: usize = 2_048; + +pub(crate) struct CelldRecordKvStore { + binding: RecordBinding, + client: Arc, + limits: CelldPreparationLimits, + store: String, +} + +impl CelldRecordKvStore { + pub(crate) fn new( + client: Arc, + endpoint: &Url, + store: String, + limits: CelldPreparationLimits, + layout: &RecordLayout, + ) -> Result { + let scope = fingerprint(&["celld-records-v2", endpoint.as_str(), &store])?; + Ok(Self { + binding: RecordBinding { + layout: layout.clone(), + scope, + format: "celld-records-v2".to_owned(), + max_metadata_bytes: MAX_METADATA_BYTES, + }, + client, + limits, + store, + }) + } + + pub(crate) async fn probe( + &self, + preparation_deadline: Deadline, + ) -> Result<(), PreparationError> { + let deadline = Deadline::at_instant( + preparation_deadline + .instant() + .min(self.client.deadline()?.instant()), + ); + let bytes = self + .client + .execute( + Method::GET, + self.client + .record_v2_url(&self.store, "record-capabilities")?, + None, + usize::try_from(self.limits.max_page_bytes) + .map_err(|_error| PreparationError::InvalidLimits)?, + deadline, + ResponseKind::RecordV2(RecordV2Response::Capabilities), + ) + .await? + .ok_or(PreparationError::Malformed)?; + let capabilities: Capabilities = + serde_json::from_slice(&bytes).map_err(|_error| PreparationError::Malformed)?; + if capabilities.version != 2 + || capabilities.format != "records-v2" + || capabilities.pagination != "keyset" + || !capabilities.no_expiry + || capabilities.max_metadata_bytes != MAX_METADATA_BYTES + { + return Err(PreparationError::UnsupportedVersion); + } + self.client.check(deadline)?; + Ok(()) + } + + fn point_url(&self, key: &str) -> Result { + self.binding.layout.resolve(key)?; + let mut url = self + .client + .record_v2_url(&self.store, "record") + .map_err(kv_error)?; + url.query_pairs_mut().append_pair("key", key); + Ok(url) + } + + fn check_partition(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + if self.binding.layout.resolve(key)? != *partition { + return Err(invalid("record partition mismatch")); + } + Ok(()) + } + + fn record_cap(&self) -> Result { + let value = + usize::try_from(self.limits.max_value_bytes).map_err(|_error| KvError::Unavailable)?; + encoded_len(value) + .and_then(|size| size.checked_add(encoded_len(MAX_METADATA_BYTES)?)) + .and_then(|size| size.checked_add(256)) + .ok_or(KvError::Unavailable) + } + + async fn lookup(&self, key: &str) -> Result, KvError> { + let deadline = self.client.deadline().map_err(kv_error)?; + let Some(bytes) = self + .client + .execute( + Method::GET, + self.point_url(key)?, + None, + self.record_cap()?, + deadline, + ResponseKind::RecordV2(RecordV2Response::Value), + ) + .await + .map_err(kv_error)? + else { + return Ok(None); + }; + let wire: WireRecord = + serde_json::from_slice(&bytes).map_err(|_error| KvError::Unavailable)?; + let body = decode( + &wire.body, + usize::try_from(self.limits.max_value_bytes).map_err(|_error| KvError::Unavailable)?, + )?; + let metadata = wire + .metadata + .map(|value| { + let metadata_bytes = decode(&value, MAX_METADATA_BYTES)?; + String::from_utf8(metadata_bytes).map_err(|_error| KvError::Unavailable) + }) + .transpose()? + .filter(|text| !text.is_empty()); + let generation = wire + .generation + .parse::() + .ok() + .filter(|value| *value > 0 && value.to_string() == wire.generation) + .ok_or(KvError::Unavailable)?; + self.client.check(deadline).map_err(kv_error)?; + Ok(Some(Record { + body: Bytes::from(body), + metadata, + generation: RecordGeneration::from_u64(generation), + })) + } + + async fn insert( + &self, + key: &str, + write: RecordWrite<'_>, + ) -> Result { + let url = self.point_url(key)?; + if u64::try_from(write.body.len()) + .map_err(|_error| invalid("record value exceeds limit"))? + > self.limits.max_value_bytes + || write + .metadata + .is_some_and(|text| text.len() > MAX_METADATA_BYTES) + { + return Err(invalid("record value or metadata exceeds provider limit")); + } + let ttl_ms = write.ttl.map(ttl_ms).transpose()?; + let (mode, generation) = match write.mode { + RecordWriteMode::Add => ("add", None), + RecordWriteMode::Overwrite => ("overwrite", None), + RecordWriteMode::IfGenerationMatch(generation) if generation.as_u64() != 0 => { + ("if-generation-match", Some(generation.as_u64().to_string())) + } + RecordWriteMode::IfGenerationMatch(_) => { + return Err(invalid("invalid record generation")); + } + }; + let body = serde_json::to_vec(&WireWrite { + body: STANDARD.encode(write.body), + metadata: write + .metadata + .filter(|text| !text.is_empty()) + .map(|text| STANDARD.encode(text.as_bytes())), + ttl_ms, + mode, + generation, + }) + .map_err(|_error| invalid("invalid record write"))?; + let deadline = self.client.deadline().map_err(kv_error)?; + match self + .client + .execute( + Method::PUT, + url, + Some(Bytes::from(body)), + 0, + deadline, + ResponseKind::RecordV2(RecordV2Response::Mutation), + ) + .await + { + Ok(Some(_)) => Ok(RecordWriteOutcome::Written), + Err(PreparationError::PreconditionFailed) + if write.mode != RecordWriteMode::Overwrite => + { + Ok(RecordWriteOutcome::PreconditionFailed) + } + Ok(None) | Err(PreparationError::PreconditionFailed) => Err(KvError::Unavailable), + Err(error) => Err(kv_error(error)), + } + } + + async fn exists_value(&self, key: &str) -> Result { + self.client + .execute( + Method::HEAD, + self.point_url(key)?, + None, + 0, + self.client.deadline().map_err(kv_error)?, + ResponseKind::RecordV2(RecordV2Response::Exists), + ) + .await + .map(|body| body.is_some()) + .map_err(kv_error) + } + + async fn delete_value(&self, key: &str) -> Result<(), KvError> { + self.client + .execute( + Method::DELETE, + self.point_url(key)?, + None, + 0, + self.client.deadline().map_err(kv_error)?, + ResponseKind::RecordV2(RecordV2Response::Mutation), + ) + .await + .map(|_| ()) + .map_err(kv_error) + } + + async fn page( + &self, + prefix: &str, + cursor: Option<&str>, + limit: usize, + operation: &str, + ) -> Result { + if !valid_key(prefix, true) || !(1..=1_000).contains(&limit) { + return Err(invalid("invalid record page input")); + } + let identity = fingerprint(&[ + &self.binding.scope, + &self.binding.format, + self.binding.layout.id(), + &self.binding.layout.version().to_string(), + prefix, + operation, + ]) + .map_err(kv_error)?; + let prior = cursor + .map(|encoded| decode_cursor(encoded, &identity, prefix)) + .transpose()?; + let mut url = self + .client + .record_v2_url(&self.store, "record-keys") + .map_err(kv_error)?; + { + let mut query = url.query_pairs_mut(); + query + .append_pair("prefix", prefix) + .append_pair("limit", &limit.to_string()); + if let Some(previous) = &prior { + query.append_pair("cursor", &previous.service); + } + } + let deadline = self.client.deadline().map_err(kv_error)?; + let bytes = self + .client + .execute( + Method::GET, + url, + None, + usize::try_from(self.limits.max_page_bytes) + .map_err(|_error| KvError::Unavailable)?, + deadline, + ResponseKind::RecordV2(RecordV2Response::Page), + ) + .await + .map_err(kv_error)? + .ok_or(KvError::Unavailable)?; + let page: WirePage = + serde_json::from_slice(&bytes).map_err(|_error| KvError::Unavailable)?; + if page.keys.len() > limit { + return Err(KvError::Unavailable); + } + let mut last = prior.as_ref().and_then(|previous| previous.last.as_deref()); + for key in &page.keys { + if !valid_key(key, false) + || !key.starts_with(prefix) + || last.is_some_and(|previous_key| key.as_str() <= previous_key) + { + return Err(KvError::Unavailable); + } + last = Some(key); + } + let next_cursor = page + .cursor + .map(|service| { + if service.is_empty() + || service.len() > MAX_SERVICE_CURSOR + || prior + .as_ref() + .is_some_and(|previous| previous.service == service) + { + return Err(KvError::Unavailable); + } + let envelope = serde_json::to_vec(&PageCursor { + version: 2, + identity, + last: last.map(str::to_owned), + service, + }) + .map_err(|_error| KvError::Unavailable)?; + let encoded_cursor = URL_SAFE_NO_PAD.encode(envelope); + if encoded_cursor.len() > MAX_PROVIDER_CURSOR { + return Err(KvError::Unavailable); + } + Ok(encoded_cursor) + }) + .transpose()?; + self.client.check(deadline).map_err(kv_error)?; + Ok(RecordPage { + keys: page.keys, + cursor: next_cursor, + }) + } +} + +#[async_trait::async_trait(?Send)] +impl KvStore for CelldRecordKvStore { + async fn delete(&self, key: &str) -> Result<(), KvError> { + self.delete_value(key).await + } + async fn exists(&self, key: &str) -> Result { + self.exists_value(key).await + } + async fn get_bytes(&self, key: &str) -> Result, KvError> { + self.lookup(key) + .await + .map(|record| record.map(|value| value.body)) + } + async fn list_keys_page( + &self, + prefix: &str, + cursor: Option<&str>, + limit: usize, + ) -> Result { + self.page(prefix, cursor, limit, "ordinary") + .await + .map(|page| KvPage { + keys: page.keys, + cursor: page.cursor, + }) + } + async fn put_bytes(&self, key: &str, value: Bytes) -> Result<(), KvError> { + self.insert( + key, + RecordWrite { + body: &value, + metadata: None, + ttl: None, + mode: RecordWriteMode::Overwrite, + }, + ) + .await + .map(|_| ()) + } + async fn put_bytes_with_ttl( + &self, + key: &str, + value: Bytes, + ttl: Duration, + ) -> Result<(), KvError> { + self.insert( + key, + RecordWrite { + body: &value, + metadata: None, + ttl: Some(ttl), + mode: RecordWriteMode::Overwrite, + }, + ) + .await + .map(|_| ()) + } + fn record_binding(&self) -> Option { + Some(self.binding.clone()) + } + async fn record_lookup( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result, KvError> { + self.check_partition(key, partition)?; + self.lookup(key).await + } + async fn record_insert( + &self, + key: &str, + partition: &RecordPartition, + write: RecordWrite<'_>, + ) -> Result { + self.check_partition(key, partition)?; + self.insert(key, write).await + } + async fn record_exists_strong( + &self, + key: &str, + partition: &RecordPartition, + ) -> Result { + self.check_partition(key, partition)?; + self.exists_value(key).await + } + async fn record_delete(&self, key: &str, partition: &RecordPartition) -> Result<(), KvError> { + self.check_partition(key, partition)?; + self.delete_value(key).await + } + async fn record_list_partition( + &self, + partition: &RecordPartition, + cursor: Option<&str>, + limit: usize, + ) -> Result { + if self.binding.layout.partition(partition.prefix())? != *partition { + return Err(invalid("record partition mismatch")); + } + self.page(partition.prefix(), cursor, limit, "partition") + .await + } +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct Capabilities { + version: u8, + format: String, + pagination: String, + no_expiry: bool, + max_metadata_bytes: usize, +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct WireRecord { + body: String, + #[serde(deserialize_with = "nullable_text")] + metadata: Option, + generation: String, +} + +#[derive(Serialize)] +struct WireWrite<'mode> { + body: String, + metadata: Option, + ttl_ms: Option, + mode: &'mode str, + #[serde(skip_serializing_if = "Option::is_none")] + generation: Option, +} + +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct WirePage { + keys: Vec, + #[serde(deserialize_with = "nullable_text")] + cursor: Option, +} + +#[derive(Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +struct PageCursor { + version: u8, + identity: String, + last: Option, + service: String, +} + +fn nullable_text<'de, D>(deserializer: D) -> Result, D::Error> +where + D: Deserializer<'de>, +{ + Option::deserialize(deserializer) +} + +fn encoded_len(bytes: usize) -> Option { + bytes.div_ceil(3).checked_mul(4) +} + +fn decode(text: &str, cap: usize) -> Result, KvError> { + if text.len() > encoded_len(cap).ok_or(KvError::Unavailable)? { + return Err(KvError::Unavailable); + } + let bytes = STANDARD + .decode(text) + .map_err(|_error| KvError::Unavailable)?; + if bytes.len() > cap || STANDARD.encode(&bytes) != text { + return Err(KvError::Unavailable); + } + Ok(bytes) +} + +fn decode_cursor(text: &str, identity: &str, prefix: &str) -> Result { + if text.is_empty() || text.len() > MAX_PROVIDER_CURSOR { + return Err(invalid("invalid record cursor")); + } + let bytes = URL_SAFE_NO_PAD + .decode(text) + .map_err(|_error| invalid("invalid record cursor"))?; + let cursor: PageCursor = + serde_json::from_slice(&bytes).map_err(|_error| invalid("invalid record cursor"))?; + if cursor.version != 2 + || cursor.identity != identity + || cursor.service.is_empty() + || cursor.service.len() > MAX_SERVICE_CURSOR + || cursor + .last + .as_deref() + .is_some_and(|key| !valid_key(key, false) || !key.starts_with(prefix)) + { + return Err(invalid("foreign record cursor")); + } + Ok(cursor) +} + +#[expect( + clippy::big_endian_bytes, + reason = "length-frame native backing identity fields without delimiter collisions" +)] +fn fingerprint(parts: &[&str]) -> Result { + let mut hash = Sha256::new(); + for part in parts { + hash.update( + u64::try_from(part.len()) + .map_err(|_error| PreparationError::SizeLimit)? + .to_be_bytes(), + ); + hash.update(part.as_bytes()); + } + Ok(URL_SAFE_NO_PAD.encode(hash.finalize())) +} + +fn ttl_ms(ttl: Duration) -> Result { + if !(KvHandle::MIN_TTL..=KvHandle::MAX_TTL).contains(&ttl) { + return Err(invalid("record TTL outside supported range")); + } + let millis = u64::try_from(ttl.as_millis()) + .ok() + .and_then(|millis| { + millis.checked_add(u64::from(!ttl.subsec_nanos().is_multiple_of(1_000_000))) + }) + .ok_or_else(|| invalid("invalid record TTL"))?; + Ok(millis.to_string()) +} + +fn invalid(message: &str) -> KvError { + KvError::Validation(message.to_owned()) +} + +fn kv_error(error: PreparationError) -> KvError { + match error { + PreparationError::MutationUncertain => KvError::MutationUncertain, + PreparationError::InvalidBinding + | PreparationError::InvalidEndpoint + | PreparationError::InvalidLimits + | PreparationError::InvalidResource => invalid("invalid celld record input"), + PreparationError::SizeLimit => KvError::LimitExceeded { + message: "celld record size limit".to_owned(), + }, + PreparationError::Credentials + | PreparationError::Unauthenticated + | PreparationError::Forbidden + | PreparationError::Missing + | PreparationError::PreconditionFailed + | PreparationError::Malformed + | PreparationError::UnsupportedVersion + | PreparationError::Unavailable + | PreparationError::Deadline => KvError::Unavailable, + } +} + +#[cfg(test)] +#[path = "records_v2_tests.rs"] +mod tests; diff --git a/crates/edgezero-store-celld/src/records_v2_tests.rs b/crates/edgezero-store-celld/src/records_v2_tests.rs new file mode 100644 index 00000000..27b92c07 --- /dev/null +++ b/crates/edgezero-store-celld/src/records_v2_tests.rs @@ -0,0 +1,388 @@ +use std::sync::Arc; +use std::time::Duration; + +use bytes::Bytes; +use edgezero_core::key_value_store::{KvError, KvStore as _}; +use edgezero_core::record_store::{ + RecordKeyLayout, RecordLayout, RecordWrite, RecordWriteMode, RecordWriteOutcome, +}; +use edgezero_core::time::MonotonicClock; +use serde_json::json; +use url::Url; + +use super::CelldRecordKvStore; +use crate::client::ScopedClient; +use crate::settings::{CelldAccess, CelldKvSettings, CelldPreparationLimits}; +use crate::test_support::{Fixture, Reply}; +use crate::{CelldPreparation, PreparationError}; + +struct Layout; +impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "fixture-groups" + } + fn version(&self) -> u32 { + 1 + } + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + key.find('/') + .and_then(|index| key.get(..=index)) + .ok_or_else(|| KvError::Validation("missing fixture group".to_owned())) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.matches('/').count() == 1 && prefix.len() > 1 { + Ok(()) + } else { + Err(KvError::Validation("invalid fixture group".to_owned())) + } + } +} +fn layout() -> RecordLayout { + RecordLayout::new(Layout).unwrap() +} +fn reply(status: u16, value: &serde_json::Value) -> Reply { + Reply::custom( + status, + "application/json", + "x-celld-store-version: 1\r\nx-celld-record-version: 2\r\n", + &serde_json::to_vec(value).unwrap(), + ) +} +fn ack() -> Reply { + Reply::custom( + 204, + "application/octet-stream", + "x-celld-store-version: 1\r\nx-celld-record-version: 2\r\n", + b"", + ) +} +fn capabilities() -> Reply { + reply( + 200, + &json!({"version":2_u8,"format":"records-v2","pagination":"keyset","no_expiry":true,"max_metadata_bytes":1_024_usize}), + ) +} +fn store(fixture: &Fixture) -> CelldRecordKvStore { + store_with_limits(fixture, CelldPreparationLimits::default()) +} +fn store_with_limits(fixture: &Fixture, limits: CelldPreparationLimits) -> CelldRecordKvStore { + let endpoint = Url::parse(fixture.endpoint()).unwrap(); + let client = Arc::new( + ScopedClient::new(endpoint.clone(), None, MonotonicClock::default(), limits).unwrap(), + ); + CelldRecordKvStore::new(client, &endpoint, "state".to_owned(), limits, &layout()).unwrap() +} +fn write(mode: RecordWriteMode) -> RecordWrite<'static> { + RecordWrite { + body: b"\0\xff", + metadata: Some("\u{e9}"), + ttl: None, + mode, + } +} + +#[tokio::test(flavor = "current_thread")] +async fn preparation_requires_read_only_v2_capabilities_and_preserves_raw_setup() { + let fixture = Fixture::new(vec![capabilities()]).await; + let settings = CelldKvSettings { + endpoint: fixture.endpoint().to_owned(), + access: CelldAccess::TrustedNetwork { + allow_remote_http: false, + }, + store: "state".to_owned(), + }; + let mut preparation = + CelldPreparation::new(MonotonicClock::default(), CelldPreparationLimits::default()) + .unwrap(); + let kv = preparation + .prepare_record_store(&settings, &layout(), None) + .await + .unwrap(); + assert_eq!( + kv.require_records().unwrap().binding().format, + "celld-records-v2" + ); + let requests = fixture.requests().await; + assert_eq!(requests.len(), 1); + assert_eq!(requests[0].method, "GET"); + assert_eq!(requests[0].target, "/v2/stores/state/record-capabilities"); + assert!(requests[0].body.is_empty()); + let raw = preparation.prepare_kv(&settings, None).await.unwrap(); + raw.require_records().unwrap_err(); + assert_eq!(fixture.count().await, 0, "raw preparation stays probe-free"); + for response in [ + Reply::response(200, "application/json", br#"{"keys":[]}"#), + reply( + 200, + &json!({"version":2_u8,"format":"records-v2","pagination":"keyset","no_expiry":false,"max_metadata_bytes":1_024_usize}), + ), + reply( + 200, + &json!({"version":2_u8,"format":"records-v2","pagination":"keyset","no_expiry":true}), + ), + ] { + let incompatible_fixture = Fixture::new(vec![response]).await; + let incompatible_settings = CelldKvSettings { + endpoint: incompatible_fixture.endpoint().to_owned(), + ..settings.clone() + }; + let mut incompatible_preparation = + CelldPreparation::new(MonotonicClock::default(), CelldPreparationLimits::default()) + .unwrap(); + incompatible_preparation + .prepare_record_store(&incompatible_settings, &layout(), None) + .await + .unwrap_err(); + assert!(matches!( + incompatible_preparation + .prepare_record_store(&incompatible_settings, &layout(), None) + .await, + Err(PreparationError::InvalidBinding) + )); + assert_eq!(incompatible_fixture.count().await, 1); + } +} + +#[tokio::test(flavor = "current_thread")] +async fn ordinary_and_record_operations_use_only_v2_rows_and_native_no_expiry() { + let fixture = Fixture::new(vec![ + ack(), + ack(), + reply( + 200, + &json!({"body":"AP8=","metadata":"w6k=","generation":"18446744073709551615"}), + ), + ack(), + ]) + .await; + let kv = store(&fixture); + kv.put_bytes("p/key", Bytes::from_static(b"\0\xff")) + .await + .unwrap(); + kv.put_bytes_with_ttl( + "p/key", + Bytes::new(), + Duration::from_secs(60) + Duration::from_nanos(1), + ) + .await + .unwrap(); + let partition = layout().partition("p/").unwrap(); + let record = kv + .record_lookup("p/key", &partition) + .await + .unwrap() + .unwrap(); + assert_eq!(record.body.as_ref(), b"\0\xff"); + assert_eq!(record.metadata.as_deref(), Some("\u{e9}")); + assert_eq!(record.generation.as_u64(), u64::MAX); + kv.record_delete("p/key", &partition).await.unwrap(); + let requests = fixture.requests().await; + assert!( + requests + .iter() + .all(|request| request.target.starts_with("/v2/stores/state/record?key=")) + ); + let plain: serde_json::Value = serde_json::from_slice(&requests[0].body).unwrap(); + assert!(plain["ttl_ms"].is_null()); + assert!(plain["metadata"].is_null()); + assert_eq!(plain["mode"], "overwrite"); + assert_eq!(plain["body"], "AP8="); + let ttl: serde_json::Value = serde_json::from_slice(&requests[1].body).unwrap(); + assert_eq!(ttl["ttl_ms"], "60001"); +} + +#[tokio::test(flavor = "current_thread")] +async fn uncertain_mutations_are_payload_free_and_never_replayed() { + for delete in [false, true] { + for stall in [false, true] { + let fixture = Fixture::new(vec![if stall { Reply::Stall } else { Reply::Drop }]).await; + let kv = store_with_limits( + &fixture, + CelldPreparationLimits { + operation_timeout_ms: 40, + connect_timeout_ms: 10, + ..CelldPreparationLimits::default() + }, + ); + let error = if delete { + kv.delete("p/private-key").await.unwrap_err() + } else { + kv.put_bytes("p/private-key", Bytes::from_static(b"private-body")) + .await + .unwrap_err() + }; + assert!(matches!(error, KvError::MutationUncertain)); + assert!(!format!("{error:?} {error}").contains("private")); + assert_eq!(fixture.count().await, 1); + } + } + let fixture = Fixture::new(vec![Reply::response(204, "application/octet-stream", b"")]).await; + assert!( + matches!( + store(&fixture).delete("p/key").await, + Err(KvError::MutationUncertain) + ), + "missing v2 response header after dispatch is uncertain" + ); +} + +#[tokio::test(flavor = "current_thread")] +async fn only_definitive_add_or_cas_failure_is_contention() { + let fixture = Fixture::new(vec![ + reply(412, &json!({"error":"precondition_failed"})), + reply(412, &json!({"error":"precondition_failed"})), + reply(412, &json!({"error":"precondition_failed"})), + ]) + .await; + let kv = store(&fixture); + let partition = layout().partition("p/").unwrap(); + assert_eq!( + kv.record_insert("p/key", &partition, write(RecordWriteMode::Add)) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + assert_eq!( + kv.record_insert( + "p/key", + &partition, + write(RecordWriteMode::IfGenerationMatch( + edgezero_core::RecordGeneration::from_u64(1) + )) + ) + .await + .unwrap(), + RecordWriteOutcome::PreconditionFailed + ); + assert!(matches!( + kv.record_insert("p/key", &partition, write(RecordWriteMode::Overwrite)) + .await, + Err(KvError::Unavailable) + )); + assert_eq!(fixture.count().await, 3); +} + +#[tokio::test(flavor = "current_thread")] +async fn validated_rejections_are_not_ambiguous_but_untrusted_failures_are() { + let fixture = Fixture::new(vec![ + reply(413, &json!({"error":"limit_exceeded"})), + reply(403, &json!({"error":"forbidden"})), + reply(503, &json!({"error":"unavailable"})), + reply(413, &json!({"error":"wrong_error_body"})), + ]) + .await; + let kv = store(&fixture); + assert!(matches!( + kv.delete("p/key").await, + Err(KvError::LimitExceeded { .. }) + )); + assert!(matches!( + kv.delete("p/key").await, + Err(KvError::Unavailable) + )); + assert!(matches!( + kv.delete("p/key").await, + Err(KvError::MutationUncertain) + )); + assert!(matches!( + kv.delete("p/key").await, + Err(KvError::MutationUncertain) + )); + assert_eq!(fixture.count().await, 4); +} + +#[tokio::test(flavor = "current_thread")] +async fn metadata_is_strict_utf8_and_empty_or_missing_is_none() { + let fixture = Fixture::new(vec![ + reply(200, &json!({"body":"","metadata":"/w==","generation":"1"})), + reply(200, &json!({"body":"","metadata":"","generation":"2"})), + reply(200, &json!({"body":"","metadata":null,"generation":"3"})), + ]) + .await; + let kv = store(&fixture); + let partition = layout().partition("p/").unwrap(); + assert!(matches!( + kv.record_lookup("p/key", &partition).await, + Err(KvError::Unavailable) + )); + assert_eq!( + kv.record_lookup("p/key", &partition) + .await + .unwrap() + .unwrap() + .metadata, + None + ); + assert_eq!( + kv.record_lookup("p/key", &partition) + .await + .unwrap() + .unwrap() + .metadata, + None + ); +} + +#[tokio::test(flavor = "current_thread")] +async fn pages_bind_scope_prefix_operation_and_progress_with_native_continuation() { + let fixture = Fixture::new(vec![ + reply(200, &json!({"keys":["p/a"],"cursor":"native-one"})), + reply(200, &json!({"keys":[],"cursor":"native-two"})), + reply(200, &json!({"keys":["p/a"],"cursor":null})), + ]) + .await; + let kv = store(&fixture); + let first = kv.list_keys_page("p/", None, 2).await.unwrap(); + let token = first.cursor.unwrap(); + let partition = layout().partition("p/").unwrap(); + kv.record_list_partition(&partition, Some(&token), 2) + .await + .unwrap_err(); + kv.list_keys_page("q/", Some(&token), 2).await.unwrap_err(); + let other = Fixture::new(vec![]).await; + store(&other) + .list_keys_page("p/", Some(&token), 2) + .await + .unwrap_err(); + assert_eq!(other.count().await, 0); + let second = kv.list_keys_page("p/", Some(&token), 2).await.unwrap(); + assert!(second.keys.is_empty()); + assert!(second.cursor.is_some()); + kv.list_keys_page("p/", second.cursor.as_deref(), 2) + .await + .expect_err("repeated prior key after an empty page must fail"); + let requests = fixture.requests().await; + assert_eq!(requests.len(), 3); + assert!(requests[1].target.contains("cursor=native-one")); + assert!(requests[2].target.contains("cursor=native-two")); + assert!( + requests + .iter() + .all(|request| request.target.starts_with("/v2/stores/state/record-keys?")) + ); +} + +#[tokio::test(flavor = "current_thread")] +async fn invalid_direct_inputs_fail_before_transport() { + let fixture = Fixture::new(vec![]).await; + let kv = store(&fixture); + let partition = layout().partition("p/").unwrap(); + kv.put_bytes("not-declared", Bytes::new()) + .await + .unwrap_err(); + kv.put_bytes_with_ttl("p/key", Bytes::new(), Duration::from_secs(59)) + .await + .unwrap_err(); + kv.list_keys_page("", None, 0).await.unwrap_err(); + kv.list_keys_page("", None, 1001).await.unwrap_err(); + let mut oversized = write(RecordWriteMode::Overwrite); + let metadata = "\u{e9}".repeat(513); + oversized.metadata = Some(&metadata); + kv.record_insert("p/key", &partition, oversized) + .await + .unwrap_err(); + assert_eq!(fixture.count().await, 0); +} diff --git a/crates/edgezero-store-celld/tests/records_v2_interop.rs b/crates/edgezero-store-celld/tests/records_v2_interop.rs new file mode 100644 index 00000000..6249790f --- /dev/null +++ b/crates/edgezero-store-celld/tests/records_v2_interop.rs @@ -0,0 +1,440 @@ +//! Explicit v2 public-handle proof against a disposable rebuilt service. +#![cfg(all(not(target_arch = "wasm32"), feature = "records-v2"))] + +#[cfg(test)] +mod tests { + #![expect( + clippy::arbitrary_source_item_ordering, + reason = "keep local service and lost-reply fixture lifecycles before public contract assertions" + )] + + use std::env; + use std::fs::{self, File}; + use std::path::PathBuf; + use std::process::{Child, Command, Stdio}; + use std::sync::Arc; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::time::{Duration, Instant}; + + use bytes::Bytes; + use edgezero_core::key_value_store::{KvError, KvHandle}; + use edgezero_core::record_store::{ + RecordKeyLayout, RecordLayout, RecordWrite, RecordWriteMode, RecordWriteOutcome, + contract::assert_record_contract, + }; + use edgezero_core::time::MonotonicClock; + use edgezero_store_celld::CelldPreparation; + use edgezero_store_celld::settings::{CelldAccess, CelldKvSettings, CelldPreparationLimits}; + use serde_json::json; + use tempfile::TempDir; + use tokio::io::{AsyncBufReadExt as _, AsyncReadExt as _, AsyncWriteExt as _, BufReader}; + use tokio::net::TcpListener; + use tokio::task::JoinHandle; + use tokio::time::sleep; + use url::Url; + + struct LocalService { + child: Child, + _directory: TempDir, + } + + #[expect( + clippy::missing_trait_methods, + reason = "`Drop::pin_drop` is unstable and conflicts with the required drop method" + )] + impl Drop for LocalService { + fn drop(&mut self) { + let _killed = self.child.kill(); + let _reaped = self.child.wait(); + } + } + + async fn service_fixture() -> (String, Option) { + if let Some(endpoint) = env::var_os("CELLD_RECORDS_V2_ENDPOINT") { + return ( + endpoint + .into_string() + .expect("fixture endpoint must be UTF-8"), + None, + ); + } + let binary = env::var_os("CELLD_BIN").map(PathBuf::from).expect( + "explicit CELLD_RECORDS_V2_ENDPOINT or rebuilt CELLD_BIN required; never a passing skip", + ); + assert!( + binary.is_absolute() && binary.is_file(), + "CELLD_BIN must select an absolute executable" + ); + let directory = TempDir::new().expect("disposable celld fixture"); + let policy = directory.path().join("policy.json"); + fs::write(&policy, serde_json::to_vec(&json!({ + "version":1_u8,"listen":"127.0.0.1:0", + "stores":[{"alias":"identity","namespace":"edgezero-records-v2-interop"}], + "access":{"mode":"trusted-network","grants":[{"store":"identity","operations":["read","write","delete","list"]}]} + })).unwrap()).unwrap(); + let log = directory.path().join("celld.log"); + let output = File::create(&log).unwrap(); + let child = Command::new(binary) + .current_dir(directory.path()) + .env_clear() + .env("TMPDIR", env::temp_dir()) + .env("CELLD_TEST_DATA_DIR", directory.path().join("data")) + .args([ + "--listen", + "127.0.0.1:0", + "--internal-listen", + "127.0.0.1:0", + "--store-api-config", + ]) + .arg(policy) + .stdout(Stdio::from(output.try_clone().unwrap())) + .stderr(Stdio::from(output)) + .spawn() + .expect("start explicit rebuilt celld"); + let mut service = LocalService { + child, + _directory: directory, + }; + let end = Instant::now().checked_add(Duration::from_secs(30)).unwrap(); + loop { + let text = fs::read_to_string(&log).unwrap(); + if let Some(address) = text.lines().find_map(|line| { + line.split_once("celld scoped stores listening on ") + .map(|(_, address)| address.trim()) + }) { + return (format!("http://{address}"), Some(service)); + } + if let Some(status) = service.child.try_wait().unwrap() { + panic!("celld startup {status}: {text}"); + } + assert!( + Instant::now() < end, + "celld fixture startup deadline: {text}" + ); + sleep(Duration::from_millis(10)).await; + } + } + + struct Layout; + impl RecordKeyLayout for Layout { + fn id(&self) -> &'static str { + "celld-v2-interop" + } + fn version(&self) -> u32 { + 1 + } + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + key.find('/') + .and_then(|index| key.get(..=index)) + .ok_or_else(|| KvError::Validation("missing interop group".to_owned())) + } + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.matches('/').count() == 1 && prefix.len() > 1 { + Ok(()) + } else { + Err(KvError::Validation("invalid interop group".to_owned())) + } + } + } + + async fn prepare(endpoint: &str) -> KvHandle { + let mut preparation = + CelldPreparation::new(MonotonicClock::default(), CelldPreparationLimits::default()) + .unwrap(); + preparation + .prepare_record_store( + &CelldKvSettings { + endpoint: endpoint.to_owned(), + access: CelldAccess::TrustedNetwork { + allow_remote_http: false, + }, + store: "identity".to_owned(), + }, + &RecordLayout::new(Layout).unwrap(), + None, + ) + .await + .unwrap() + } + + fn write(body: &[u8], mode: RecordWriteMode) -> RecordWrite<'_> { + RecordWrite { + body, + metadata: None, + ttl: None, + mode, + } + } + + struct LoseMutationReply { + endpoint: String, + mutations: Arc, + task: JoinHandle<()>, + } + #[expect( + clippy::missing_trait_methods, + reason = "`Drop::pin_drop` is unstable and conflicts with the required drop method" + )] + impl Drop for LoseMutationReply { + fn drop(&mut self) { + self.task.abort(); + } + } + + impl LoseMutationReply { + async fn start(upstream: &str) -> Self { + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let endpoint = format!("http://{}", listener.local_addr().unwrap()); + let destination = upstream.to_owned(); + let mutations = Arc::new(AtomicUsize::new(0)); + let seen = Arc::clone(&mutations); + let task = tokio::spawn(async move { + let client = reqwest::Client::builder().no_proxy().build().unwrap(); + loop { + let (stream, _) = listener.accept().await.unwrap(); + let mut socket = BufReader::new(stream); + let mut line = String::new(); + if socket.read_line(&mut line).await.unwrap() == 0 { + continue; + } + let mut parts = line.split_whitespace(); + let method = + reqwest::Method::from_bytes(parts.next().unwrap().as_bytes()).unwrap(); + let target = parts.next().unwrap().to_owned(); + let mut length = 0; + loop { + line.clear(); + socket.read_line(&mut line).await.unwrap(); + if line == "\r\n" { + break; + } + if let Some((name, value)) = line.split_once(':') + && name.eq_ignore_ascii_case("content-length") + { + length = value.trim().parse().unwrap(); + } + } + let mut body = vec![0; length]; + socket.read_exact(&mut body).await.unwrap(); + let reply = client + .request(method.clone(), format!("{destination}{target}")) + .header("content-type", "application/json") + .body(body) + .send() + .await + .unwrap(); + if method == reqwest::Method::PUT || method == reqwest::Method::DELETE { + assert_eq!( + reply.status(), + 204, + "upstream must acknowledge commit before dropping response" + ); + seen.fetch_add(1, Ordering::SeqCst); + if method == reqwest::Method::DELETE { + // A replayed delete would erase this independently recreated value. + let recreated = client.put(format!("{destination}{target}")).json(&json!({"body":"cmVjcmVhdGVk","metadata":null,"ttl_ms":null,"mode":"overwrite"})).send().await.unwrap(); + assert_eq!(recreated.status(), 204); + } + continue; + } + let status = reply.status().as_u16(); + let content_type = reply.headers()["content-type"].to_str().unwrap().to_owned(); + let store_version = reply.headers()["x-celld-store-version"] + .to_str() + .unwrap() + .to_owned(); + let record_version = reply.headers()["x-celld-record-version"] + .to_str() + .unwrap() + .to_owned(); + let response_body = reply.bytes().await.unwrap(); + let head = format!( + "HTTP/1.1 {status} Fixture\r\ncontent-type: {content_type}\r\nx-celld-store-version: {store_version}\r\nx-celld-record-version: {record_version}\r\ncontent-length: {}\r\nconnection: close\r\n\r\n", + response_body.len() + ); + socket.get_mut().write_all(head.as_bytes()).await.unwrap(); + socket.get_mut().write_all(&response_body).await.unwrap(); + } + }); + Self { + endpoint, + mutations, + task, + } + } + } + + async fn maximum_length_key_pages(kv: &KvHandle) { + let records = kv.require_records().unwrap(); + for prefix in [ + format!("{}ab/", "\u{e9}\"\\".repeat(127)), + format!("{}/", "\"".repeat(510)), + ] { + assert_eq!( + prefix.len(), + 511, + "prefix length is measured in UTF-8 bytes" + ); + let expected = [format!("{prefix}a"), format!("{prefix}b")]; + for key in &expected { + assert_eq!(key.len(), 512); + kv.put_bytes(key, Bytes::from_static(b"long-key")) + .await + .unwrap(); + } + let partition = records.partition(&prefix).unwrap(); + let first = records.list_partition(&partition, None, 1).await.unwrap(); + assert_eq!(first.keys, [expected[0].clone()]); + assert!( + first.cursor.is_some(), + "full first page must preserve native continuation" + ); + let last = records + .list_partition(&partition, first.cursor.as_deref(), 1) + .await + .unwrap(); + assert_eq!(last.keys, [expected[1].clone()]); + assert!( + last.cursor.is_none(), + "lookahead must establish the exact-multiple end" + ); + let ordinary_first = kv.list_keys_page(&prefix, None, 1).await.unwrap(); + assert_eq!(ordinary_first.keys, [expected[0].clone()]); + assert!(ordinary_first.cursor.is_some()); + let ordinary_last = kv + .list_keys_page(&prefix, ordinary_first.cursor.as_deref(), 1) + .await + .unwrap(); + assert_eq!(ordinary_last.keys, [expected[1].clone()]); + assert!(ordinary_last.cursor.is_none()); + for key in &expected { + kv.delete(key).await.unwrap(); + } + } + } + + async fn independent_races(kv: &KvHandle, endpoint: &str) { + let independent = prepare(endpoint).await; + let records = kv.require_records().unwrap(); + let other = independent.require_records().unwrap(); + let (add_left, add_right) = tokio::join!( + records.insert("race/key", write(b"a", RecordWriteMode::Add)), + other.insert("race/key", write(b"b", RecordWriteMode::Add)) + ); + let outcomes = [add_left.unwrap(), add_right.unwrap()]; + assert_eq!( + outcomes + .iter() + .filter(|value| **value == RecordWriteOutcome::Written) + .count(), + 1 + ); + assert_eq!( + outcomes + .iter() + .filter(|value| **value == RecordWriteOutcome::PreconditionFailed) + .count(), + 1 + ); + let generation = records + .lookup("race/key") + .await + .unwrap() + .unwrap() + .generation; + let mode = RecordWriteMode::IfGenerationMatch(generation); + let (cas_left, cas_right) = tokio::join!( + records.insert("race/key", write(b"a", mode)), + other.insert("race/key", write(b"b", mode)) + ); + let cas_outcomes = [cas_left.unwrap(), cas_right.unwrap()]; + assert_eq!( + cas_outcomes + .iter() + .filter(|value| **value == RecordWriteOutcome::Written) + .count(), + 1 + ); + assert_eq!( + cas_outcomes + .iter() + .filter(|value| **value == RecordWriteOutcome::PreconditionFailed) + .count(), + 1 + ); + records.delete("race/key").await.unwrap(); + } + + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires explicit rebuilt CELLD_BIN or fresh CELLD_RECORDS_V2_ENDPOINT, alias identity"] + async fn real_celld_records_v2_public_contract_races_and_lost_acknowledgement() { + let (endpoint, _service) = service_fixture().await; + let url = Url::parse(&endpoint).unwrap(); + assert_eq!(url.scheme(), "http"); + assert_eq!(url.host_str(), Some("127.0.0.1")); + assert!(url.port().is_some()); + let kv = prepare(&endpoint).await; + assert_record_contract(&kv, "contract/").await; + independent_races(&kv, &endpoint).await; + kv.put_bytes("left/key", Bytes::from_static(b"left")) + .await + .unwrap(); + kv.put_bytes("right/key", Bytes::from_static(b"right")) + .await + .unwrap(); + let mut cursor = None; + let mut keys = Vec::new(); + loop { + let page = kv.list_keys_page("", cursor.as_deref(), 1).await.unwrap(); + keys.extend(page.keys); + cursor = page.cursor; + if cursor.is_none() { + break; + } + } + keys.sort(); + assert_eq!(keys, ["left/key", "right/key"]); + for key in keys { + kv.delete(&key).await.unwrap(); + } + let proxy = LoseMutationReply::start(&endpoint).await; + let through_proxy = prepare(&proxy.endpoint).await; + assert!(matches!( + through_proxy + .put_bytes("uncertain/key", Bytes::from_static(b"committed")) + .await, + Err(KvError::MutationUncertain) + )); + assert_eq!( + kv.get_bytes("uncertain/key") + .await + .unwrap() + .unwrap() + .as_ref(), + b"committed" + ); + assert_eq!(proxy.mutations.load(Ordering::SeqCst), 1); + assert!(matches!( + through_proxy.delete("uncertain/key").await, + Err(KvError::MutationUncertain) + )); + assert_eq!( + kv.get_bytes("uncertain/key") + .await + .unwrap() + .unwrap() + .as_ref(), + b"recreated" + ); + assert_eq!( + proxy.mutations.load(Ordering::SeqCst), + 2, + "one original PUT and one original DELETE, no replay" + ); + kv.delete("uncertain/key").await.unwrap(); + maximum_length_key_pages(&kv).await; + } +} diff --git a/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/Cargo.toml b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/Cargo.toml index 3d61001e..278c678a 100644 --- a/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/Cargo.toml +++ b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/Cargo.toml @@ -8,6 +8,7 @@ publish.workspace = true qualification = ["progress"] progress = ["fixture-core/qualification", "fixture-core/native-body-witness", "edgezero-adapter-fastly/test-utils"] readiness = ["dep:fastly-sys"] +records = ["edgezero-core/test-utils"] [dependencies] edgezero-core.workspace = true @@ -83,5 +84,10 @@ path = "src/bin/limit_memory.rs" name = "fixture-fastly-faults" path = "src/bin/faults.rs" +[[bin]] +name = "fixture-fastly-record-contract" +path = "src/bin/record_contract.rs" +required-features = ["records"] + [lints] workspace = true diff --git a/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/fastly.toml b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/fastly.toml index 7165a154..81247391 100644 --- a/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/fastly.toml +++ b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/fastly.toml @@ -9,6 +9,9 @@ language = "rust" format = "inline-toml" [local_server.config_stores.fixture_config.contents] marker = "local-fixture" +[local_server.kv_stores] +fixture_records = [] + [[local_server.kv_stores.fixture_kv]] key = "__init__" data = "" diff --git a/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/bin/record_contract.rs b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/bin/record_contract.rs new file mode 100644 index 00000000..2778d412 --- /dev/null +++ b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/bin/record_contract.rs @@ -0,0 +1,99 @@ +//! Run the shared public-handle contract against Viceroy's native KV operations. +//! Use the fixture's empty fixture_records store, not SDK native-test stubs. + +use std::sync::Arc; + +use edgezero_adapter_fastly::key_value_store::FastlyKvStore; +use edgezero_core::key_value_store::{KvError, KvHandle}; +use edgezero_core::record_store::{ + RecordKeyLayout, RecordLayout, contract::assert_record_contract, +}; +use fastly::{Error, Request, Response}; +use futures::executor::block_on; + +#[path = "../record_cases.rs"] +mod record_cases; + +struct FixtureLayout; + +impl RecordKeyLayout for FixtureLayout { + fn id(&self) -> &str { + "fastly-record-contract" + } + + fn version(&self) -> u32 { + 1 + } + + fn validate(&self) -> Result<(), KvError> { + Ok(()) + } + + fn partition_prefix<'a>(&self, key: &'a str) -> Result<&'a str, KvError> { + key.find('/') + .filter(|index| *index > 0 && *index + 1 < key.len()) + .map(|index| &key[..=index]) + .ok_or_else(|| KvError::Validation("invalid fixture record key".to_owned())) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.ends_with('/') && prefix.len() > 1 && prefix.matches('/').count() == 1 { + Ok(()) + } else { + Err(KvError::Validation("invalid fixture partition".to_owned())) + } + } +} + +fn main() -> Result<(), Error> { + fixture_fastly::initialize_launch(env!("CARGO_BIN_NAME")); + fastly::init(); + let request = Request::from_client(); + if request.get_path() == "/record-ready" { + send_receipt(serde_json::json!({ + "record_ready": true, + "provider": "fastly-native", + })); + return Ok(()); + } + let path = request.get_path(); + if path != "/record-contract" + && !path.starts_with("/record-race/") + && !path.starts_with("/record-existence/") + && path != "/record-cases/clean" + { + Response::from_status(404).send_to_client(); + return Ok(()); + } + + let layout = RecordLayout::new(FixtureLayout)?; + let store = FastlyKvStore::open_records("fixture_records", layout)?; + let kv = KvHandle::new(Arc::new(store)); + if path != "/record-contract" { + send_receipt(block_on(record_cases::run(&kv, path))?); + return Ok(()); + } + block_on(assert_record_contract(&kv, "group/")); + + // A failed assertion traps before this receipt. A successful receipt proves + // this suite executed, not races, expiry timing, restart or lost acknowledgements. + let receipt = serde_json::json!({ + "record_contract": "passed", + "provider": "fastly-native", + "executed_suites": 1, + }); + println!("{receipt}"); + send_receipt(receipt); + Ok(()) +} + +fn send_receipt(receipt: serde_json::Value) { + Response::from_status(200) + .with_header( + fixture_core::launch::HEADER, + fixture_core::launch::value().unwrap_or("unconfigured"), + ) + .with_header("content-type", "application/json") + .with_body(receipt.to_string()) + .send_to_client(); +} diff --git a/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/record_cases.rs b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/record_cases.rs new file mode 100644 index 00000000..e4f430dd --- /dev/null +++ b/crates/edgezero-test-harness/fixtures/reusable-app/crates/fixture-fastly/src/record_cases.rs @@ -0,0 +1,136 @@ +use std::collections::HashSet; + +use edgezero_core::key_value_store::{KvError, KvHandle}; +use edgezero_core::record_store::{ + RecordGeneration, RecordWrite, RecordWriteMode, RecordWriteOutcome, +}; +use serde_json::{Value, json}; + +const DESCENDANTS: usize = 1001; +const EXACT: &str = "pages/needle"; + +fn write(body: &[u8], mode: RecordWriteMode) -> RecordWrite<'_> { + RecordWrite { + body, + metadata: None, + ttl: None, + mode, + } +} + +fn invalid_route() -> KvError { + KvError::Validation("invalid native record case".to_owned()) +} + +fn descendant(index: usize) -> String { + format!("{EXACT}/{index:04}") +} + +pub async fn run(kv: &KvHandle, path: &str) -> Result { + let records = kv.require_records()?; + let parts: Vec<_> = path.trim_start_matches('/').split('/').collect(); + match parts.as_slice() { + ["record-race", "prepare"] => { + records.delete("race/add").await?; + records.delete("race/cas").await?; + assert_eq!( + records + .insert("race/cas", write(b"seed", RecordWriteMode::Add)) + .await?, + RecordWriteOutcome::Written + ); + let generation = records.lookup("race/cas").await?.unwrap().generation; + Ok(json!({"record_race":"prepared","generation":generation.as_u64()})) + } + ["record-race", mode, writer, rest @ ..] if ["a", "b"].contains(writer) => { + let condition = match (*mode, rest) { + ("add", []) => RecordWriteMode::Add, + ("cas", [generation]) => RecordWriteMode::IfGenerationMatch( + RecordGeneration::from_u64(generation.parse().map_err(|_| invalid_route())?), + ), + _ => return Err(invalid_route()), + }; + let outcome = records + .insert(&format!("race/{mode}"), write(writer.as_bytes(), condition)) + .await?; + Ok( + json!({"record_race":mode,"writer":writer,"outcome":match outcome { + RecordWriteOutcome::Written => "written", + RecordWriteOutcome::PreconditionFailed => "precondition-failed", + }}), + ) + } + ["record-race", "result", mode] if ["add", "cas"].contains(mode) => { + let record = records.lookup(&format!("race/{mode}")).await?.unwrap(); + Ok(json!({"record_race":mode,"body":std::str::from_utf8(&record.body).unwrap()})) + } + ["record-race", "cleanup"] => { + records.delete("race/add").await?; + records.delete("race/cas").await?; + Ok(json!({"record_race":"cleaned"})) + } + ["record-existence", operation, start, end] if ["seed", "cleanup"].contains(operation) => { + let start: usize = start.parse().map_err(|_| invalid_route())?; + let end: usize = end.parse().map_err(|_| invalid_route())?; + if start >= end || end > DESCENDANTS { + return Err(invalid_route()); + } + for index in start..end { + let key = descendant(index); + if *operation == "seed" { + assert_eq!( + records + .insert(&key, write(b"descendant", RecordWriteMode::Add)) + .await?, + RecordWriteOutcome::Written + ); + } else { + records.delete(&key).await?; + } + } + Ok(json!({"record_existence":operation,"count":end-start})) + } + ["record-existence", "check"] => { + let partition = records.partition("pages/")?; + let mut cursor = None; + let mut keys = HashSet::new(); + let mut cursors = HashSet::new(); + let mut pages = 0; + loop { + let page = records + .list_partition(&partition, cursor.as_deref(), 1000) + .await?; + pages += 1; + assert!(page.keys.len() <= 1000); + for key in page.keys { + assert!(keys.insert(key)); + } + cursor = page.cursor; + let Some(next) = &cursor else { break }; + assert!(cursors.insert(next.clone())); + } + assert!(pages >= 2); + assert_eq!(keys, (0..DESCENDANTS).map(descendant).collect()); + assert!(!records.exists_strong(EXACT).await?); + assert_eq!( + records + .insert(EXACT, write(b"exact", RecordWriteMode::Add)) + .await?, + RecordWriteOutcome::Written + ); + assert!(records.exists_strong(EXACT).await?); + records.delete(EXACT).await?; + assert!(!records.exists_strong(EXACT).await?); + Ok( + json!({"record_existence":"passed","descendants":DESCENDANTS,"native_pages":pages, + "exact_absent_before":true,"exact_present":true,"exact_absent_after":true}), + ) + } + ["record-cases", "clean"] => { + let page = kv.list_keys_page("", None, 1).await?; + assert!(page.keys.is_empty() && page.cursor.is_none()); + Ok(json!({"record_cases":"clean","remaining_keys":0})) + } + _ => Err(invalid_route()), + } +} diff --git a/crates/edgezero-test-harness/src/main.rs b/crates/edgezero-test-harness/src/main.rs index 9a2e711f..70f5db99 100644 --- a/crates/edgezero-test-harness/src/main.rs +++ b/crates/edgezero-test-harness/src/main.rs @@ -58,7 +58,7 @@ impl Args { while let Some(key) = args.next() { if key == "--help" || key == "-h" { println!( - "edgezero-test-harness [--adapter fastly|cloudflare|spin|axum|all] [--suite smoke|benchmark|assembly|outbound-readiness|outbound-client-progress|outbound-progress|outbound-overflow|outbound-wire|outbound-lifetimes|response-framing|synthesized-headers|ingress-fidelity|ci] [--output EMPTY_DIR] [--requests 100] [--repetitions 3] [--seed 856] [--construction-rounds 0] [--max-requests 10] [--readiness-timeout 40] [--record-observations]" + "edgezero-test-harness [--adapter fastly|cloudflare|spin|axum|all] [--suite smoke|benchmark|assembly|record-contract|outbound-readiness|outbound-client-progress|outbound-progress|outbound-overflow|outbound-wire|outbound-lifetimes|response-framing|synthesized-headers|ingress-fidelity|ci] [--output EMPTY_DIR] [--requests 100] [--repetitions 3] [--seed 856] [--construction-rounds 0] [--max-requests 10] [--readiness-timeout 40] [--record-observations]" ); std::process::exit(0); } @@ -94,6 +94,7 @@ impl Args { "smoke", "benchmark", "assembly", + "record-contract", "outbound-readiness", "outbound-client-progress", "outbound-progress", @@ -108,6 +109,10 @@ impl Args { .contains(&a.suite.as_str()), "unknown suite", )?; + require( + a.suite != "record-contract" || a.adapter == "fastly", + "record contract requires --adapter fastly", + )?; require( !matches!( a.suite.as_str(), @@ -1042,6 +1047,16 @@ mod argument_tests { assert!(Args::parse_from(["--suite", "outbound-wire"].map(str::to_owned)).is_err()); } + #[test] + fn record_contract_suite_selects_only_fastly() { + for adapter in ["fastly", "axum", "cloudflare", "spin", "all"] { + let result = Args::parse_from( + ["--adapter", adapter, "--suite", "record-contract"].map(str::to_owned), + ); + assert_eq!(result.is_ok(), adapter == "fastly", "{adapter}"); + } + } + #[test] fn client_progress_suite_is_explicit_and_fastly_only() { for adapter in ["fastly", "axum", "cloudflare", "spin", "all"] { diff --git a/crates/edgezero-test-harness/src/runners.rs b/crates/edgezero-test-harness/src/runners.rs index 09d16042..498e23a2 100644 --- a/crates/edgezero-test-harness/src/runners.rs +++ b/crates/edgezero-test-harness/src/runners.rs @@ -7,6 +7,7 @@ mod lifetimes; mod overflow; mod progress; mod readiness; +mod record_contract; mod response_headers; mod reuse; mod wire; @@ -361,6 +362,9 @@ pub fn fastly(args: &Args, run: &Path, records: &mut Records) -> Result<()> { // Keep going: an unpinned Viceroy still gets signal, and a failure still exits 1. records.push(json!({"status":"unverified","adapter":"fastly","reason":format!("Viceroy version {version} does not match pinned {pin}")}))?; } + if args.suite == "record-contract" { + return record_contract::run(&exe, run, records); + } if args.suite == "outbound-wire" { return wire::run("fastly", run, records); } diff --git a/crates/edgezero-test-harness/src/runners/ci.rs b/crates/edgezero-test-harness/src/runners/ci.rs index 5e703b5b..95ee1175 100644 --- a/crates/edgezero-test-harness/src/runners/ci.rs +++ b/crates/edgezero-test-harness/src/runners/ci.rs @@ -234,6 +234,26 @@ fn pinned_tool( pub(crate) fn run(args: &Args, run: &Path, records: &mut Records) -> Result<()> { if args.adapter != "spin" { + if args.adapter == "fastly" { + let output = run.join("record-contract"); + phase( + "fastly", + "record-contract", + Command::new(std::env::current_exe()?) + .args([ + "--adapter", + "fastly", + "--suite", + "record-contract", + "--output", + ]) + .arg(&output), + &repository_root(), + run, + records, + )?; + super::record_contract::validate_events(&read_events(&output.join("events.jsonl"))?)?; + } return platforms::run(args, run, records); } require( diff --git a/crates/edgezero-test-harness/src/runners/record_contract.rs b/crates/edgezero-test-harness/src/runners/record_contract.rs new file mode 100644 index 00000000..a2570508 --- /dev/null +++ b/crates/edgezero-test-harness/src/runners/record_contract.rs @@ -0,0 +1,387 @@ +//! Target-matched public-handle Fastly record conformance, without native SDK stubs. + +use super::*; +use std::sync::{Arc, Barrier}; + +const BIN: &str = "fixture-fastly-record-contract"; + +fn validate_receipt(receipt: &Value) -> Result<()> { + require( + receipt["record_contract"] == "passed" + && receipt["provider"] == "fastly-native" + && receipt["executed_suites"] == 1, + "missing or invalid native record contract receipt", + ) +} + +fn validate_reply(reply: &Value) -> Result { + let receipt = guest(reply)?; + validate_receipt(&receipt)?; + Ok(receipt) +} + +fn validate_ready(reply: &Value) -> Result<()> { + let ready = guest(reply)?; + require( + ready["record_ready"] == true + && ready["provider"] == "fastly-native" + && ready.get("record_contract").is_none() + && ready.get("executed_suites").is_none(), + "invalid read-only record readiness response", + ) +} + +pub(super) fn validate_events(events: &[Value]) -> Result<()> { + require( + !events.iter().any(|event| event["status"] == "fail"), + "failed record contract evidence", + )?; + let suites = events + .iter() + .filter(|event| event["event"] == "fastly-record-contract") + .collect::>(); + require( + suites.len() == 1 && suites[0]["status"] == "pass" && suites[0]["executed_suites"] == 1, + "missing or duplicate record contract suite", + )?; + validate_receipt(&suites[0]["receipt"])?; + for mode in ["add", "cas"] { + let races = events + .iter() + .filter(|event| event["event"] == "fastly-record-race" && event["mode"] == mode) + .collect::>(); + require( + races.len() == 1 + && races[0]["status"] == "pass" + && races[0]["independent_writers"] == 2 + && races[0]["written"] == 1 + && races[0]["precondition_failed"] == 1, + "missing native record race evidence", + )?; + } + let pages = events + .iter() + .filter(|event| event["event"] == "fastly-record-existence") + .collect::>(); + require( + pages.len() == 1 + && pages[0]["status"] == "pass" + && pages[0]["receipt"]["descendants"] == 1001 + && pages[0]["receipt"]["native_pages"] + .as_u64() + .is_some_and(|count| count >= 2) + && pages[0]["receipt"]["exact_absent_before"] == true + && pages[0]["receipt"]["exact_present"] == true + && pages[0]["receipt"]["exact_absent_after"] == true, + "missing native exact-existence continuation evidence", + )?; + let cleanup = events + .iter() + .filter(|event| event["event"] == "fastly-record-cleanup") + .collect::>(); + require( + cleanup.len() == 1 + && cleanup[0]["status"] == "pass" + && cleanup[0]["receipt"]["remaining_keys"] == 0, + "missing native record cleanup evidence", + ) +} + +fn race_clients(port: u16, paths: [String; 2]) -> Result> { + let barrier = Arc::new(Barrier::new(3)); + thread::scope(|scope| { + let clients = paths + .into_iter() + .map(|path| { + let barrier = barrier.clone(); + scope.spawn(move || { + barrier.wait(); + guest(&req(port, &path)?) + }) + }) + .collect::>(); + barrier.wait(); + clients + .into_iter() + .map(|client| { + client + .join() + .map_err(|_| "native record race client panicked")? + }) + .collect() + }) +} + +fn run_native_cases(port: u16, records: &mut Records) -> Result<()> { + let prepared = guest(&req(port, "/record-race/prepare")?)?; + require( + prepared["record_race"] == "prepared", + "native race preparation failed", + )?; + let generation = prepared["generation"] + .as_u64() + .ok_or("missing native race generation")?; + for mode in ["add", "cas"] { + let path = |writer| { + if mode == "add" { + format!("/record-race/add/{writer}") + } else { + format!("/record-race/cas/{writer}/{generation}") + } + }; + let replies = race_clients(port, [path("a"), path("b")])?; + require( + replies.iter().all(|reply| reply["record_race"] == mode) + && replies + .iter() + .filter(|reply| reply["outcome"] == "written") + .count() + == 1 + && replies + .iter() + .filter(|reply| reply["outcome"] == "precondition-failed") + .count() + == 1, + "native record race did not have exactly one winner", + )?; + let winner = replies + .iter() + .find(|reply| reply["outcome"] == "written") + .unwrap(); + let stored = guest(&req(port, &format!("/record-race/result/{mode}"))?)?; + require( + stored["body"] == winner["writer"], + "native race stored the losing value", + )?; + records.push( + json!({"event":"fastly-record-race","status":"pass","mode":mode, + "independent_writers":2,"written":1,"precondition_failed":1, + "winner":winner["writer"],"clients":replies,"stored":stored,"qualification":false}), + )?; + } + let cleaned = guest(&req(port, "/record-race/cleanup")?)?; + require( + cleaned["record_race"] == "cleaned", + "native race cleanup failed", + )?; + + // Batch test-fixture setup and cleanup across requests. This is not a + // provider operation-budget claim or a production mutation retry policy. + for operation in ["seed", "cleanup"] { + for start in (0..1001).step_by(100) { + let end = (start + 100).min(1001); + let reply = guest(&req( + port, + &format!("/record-existence/{operation}/{start}/{end}"), + )?)?; + require( + reply["record_existence"] == operation && reply["count"] == end - start, + "native existence batch incomplete", + )?; + } + if operation == "seed" { + let receipt = guest(&req(port, "/record-existence/check")?)?; + require( + receipt["record_existence"] == "passed", + "native exact existence failed", + )?; + records.push(json!({"event":"fastly-record-existence","status":"pass","receipt":receipt,"qualification":false}))?; + } + } + let receipt = guest(&req(port, "/record-cases/clean")?)?; + require( + receipt["record_cases"] == "clean" && receipt["remaining_keys"] == 0, + "native fixture cleanup failed", + )?; + records.push(json!({"event":"fastly-record-cleanup","status":"pass","receipt":receipt,"qualification":false})) +} + +pub(super) fn run(exe: &Path, parent: &Path, records: &mut Records) -> Result<()> { + let run = parent.join("record-contract"); + fs::create_dir_all(&run)?; + + let output = Command::new(exe).arg("--version").output()?; + require(output.status.success(), "Viceroy version command failed")?; + let version = String::from_utf8(output.stdout)?.trim().to_owned(); + let pins = fs::read_to_string(repository_root().join(".tool-versions"))?; + let pin = pins + .lines() + .find_map(|line| line.strip_prefix("viceroy ")) + .map(str::trim) + .ok_or("missing Viceroy pin")?; + require( + version.split_whitespace().nth(1) == Some(pin), + "record contract requires the pinned Viceroy runtime", + )?; + + let mut build = cargo_build()?; + // Keep explicit caller target ownership instead of silently using the + // fixture workspace's historical target directory. + if let Some(target) = std::env::var_os("CARGO_TARGET_DIR") { + build.env("CARGO_TARGET_DIR", target); + } + let output = build + .args([ + "build", + "--locked", + "--offline", + "--release", + "-p", + "fixture-fastly", + "--features", + "records", + "--bin", + BIN, + "--target", + "wasm32-wasip1", + "--message-format=json-render-diagnostics", + ]) + .stderr(Stdio::from(fs::File::create(run.join("build.log"))?)) + .output()?; + fs::write(run.join("build.jsonl"), &output.stdout)?; + require( + output.status.success(), + "Fastly record fixture build failed", + )?; + let wasm = stage(&built_executable(&output.stdout, BIN)?, &run)?; + let config = run.join("fastly.toml"); + fs::copy( + fixture_root().join("crates/fixture-fastly/fastly.toml"), + &config, + )?; + records.push(json!({ + "event":"build", "adapter":"fastly", "suite":"record-contract", + "variant":"fastly-record-contract", + "target":"wasm32-wasip1", "artifact":artifact(&wasm)?, + "config":artifact(&config)?, "runtime":artifact(exe)?, "runtime_version":version, + }))?; + + let log = run.join("runtime.log"); + let outcome = (|| -> Result<()> { + let port = net::port()?; + let mut process = Process::start( + &mut fastly_command(exe, &config, &run, "record-contract", port)?, + &log, + port, + )?; + // Attest the launch through a route that never opens KV. The mutating + // contract is dispatched once only after startup identity is verified. + let ready = req(port, "/record-ready")?; + validate_ready(&ready)?; + records.push(json!({ + "event":"fastly-record-ready", "status":"pass", "client":ready, + "qualification":false, + }))?; + let reply = req(port, "/record-contract")?; + let receipt = validate_reply(&reply)?; + run_native_cases(port, records)?; + process.finish()?; + records.push(json!({ + "event":"fastly-record-contract", "adapter":"fastly", "status":"pass", + "executed_suites":1, "receipt":receipt, "client":reply, + "qualification":false, + }))?; + Ok(()) + })(); + settle( + records, + &log, + json!({"adapter":"fastly","suite":"record-contract","qualification":false}), + json!({"phase":"record-contract"}), + outcome, + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn reply(status: u16, receipt: Value) -> Value { + json!({"status":status,"body":receipt.to_string()}) + } + + #[test] + fn readiness_is_distinct_from_mutating_contract_execution() { + let ready = reply(200, json!({"record_ready":true,"provider":"fastly-native"})); + assert!(validate_ready(&ready).is_ok()); + assert!(validate_reply(&ready).is_err()); + assert!( + validate_ready(&reply( + 200, + json!({ + "record_ready":true,"provider":"fastly-native","executed_suites":1, + }) + )) + .is_err() + ); + assert!( + validate_ready(&reply( + 500, + json!({"record_ready":true,"provider":"fastly-native"}) + )) + .is_err() + ); + } + + #[test] + fn persisted_record_evidence_requires_one_native_suite() { + let event = json!({ + "event":"fastly-record-contract", "status":"pass", "executed_suites":1, + "receipt":{"record_contract":"passed","provider":"fastly-native","executed_suites":1}, + }); + let mut events = vec![event]; + for mode in ["add", "cas"] { + events.push( + json!({"event":"fastly-record-race","status":"pass","mode":mode, + "independent_writers":2,"written":1,"precondition_failed":1}), + ); + } + events.push( + json!({"event":"fastly-record-existence","status":"pass","receipt":{ + "descendants":1001,"native_pages":2,"exact_absent_before":true, + "exact_present":true,"exact_absent_after":true, + }}), + ); + events.push( + json!({"event":"fastly-record-cleanup","status":"pass","receipt":{"remaining_keys":0}}), + ); + assert!(validate_events(&events).is_ok()); + assert!(validate_events(&[]).is_err()); + for index in 0..events.len() { + let mut missing = events.clone(); + missing.remove(index); + assert!(validate_events(&missing).is_err()); + let mut duplicate = events.clone(); + duplicate.push(events[index].clone()); + assert!(validate_events(&duplicate).is_err()); + } + let mut failed = events.clone(); + failed.push(json!({"status":"fail"})); + assert!(validate_events(&failed).is_err()); + let mut wrong = events; + wrong[0]["receipt"]["executed_suites"] = json!(0); + assert!(validate_events(&wrong).is_err()); + } + + #[test] + fn receipt_requires_successful_native_nonzero_execution() { + let receipt = json!({ + "record_contract":"passed", "provider":"fastly-native", "executed_suites":1, + }); + assert!(validate_reply(&reply(200, receipt.clone())).is_ok()); + assert!(validate_reply(&reply(500, receipt.clone())).is_err()); + for (field, wrong) in [ + ("record_contract", json!("skipped")), + ("provider", json!("stub")), + ("executed_suites", json!(0)), + ("executed_suites", json!(2)), + ("executed_suites", json!("1")), + ] { + let mut invalid = receipt.clone(); + invalid[field] = wrong; + assert!(validate_reply(&reply(200, invalid)).is_err()); + } + assert!(validate_reply(&reply(200, json!({}))).is_err()); + assert!(validate_reply(&json!({"status":200,"body":"not json"})).is_err()); + } +} diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index 894c351c..6a9185c4 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -40,6 +40,7 @@ export default defineConfig({ { text: 'Streaming', link: '/guide/streaming' }, { text: 'Outbound HTTP', link: '/guide/proxying' }, { text: 'KV Storage', link: '/guide/kv' }, + { text: 'Conditional records', link: '/guide/kv-records' }, ], }, { diff --git a/docs/guide/dynamodb-kv.md b/docs/guide/dynamodb-kv.md index f4a1603d..9cbabcbf 100644 --- a/docs/guide/dynamodb-kv.md +++ b/docs/guide/dynamodb-kv.md @@ -1,5 +1,9 @@ # Native DynamoDB KV +This page describes the ordinary `aws-dynamodb` format. For native conditional +records, partitioned storage and global Scan, see [conditional KV records](./kv-records). +The two modes are explicit and do not migrate each other's rows. + DynamoDB is an optional live KV provider for native Axum apps. It uses the existing `Kv` extractor, `KvHandle` and per-ID registry. Config and Secrets Manager stores remain independent startup snapshots. DynamoDB calls happen during KV operations, diff --git a/docs/guide/kv-records.md b/docs/guide/kv-records.md new file mode 100644 index 00000000..4e8398d1 --- /dev/null +++ b/docs/guide/kv-records.md @@ -0,0 +1,257 @@ +# Conditional KV records + +A record-capable KV binding adds generations, atomic Add/CAS, exact authoritative +existence and strongly consistent partition pages. It uses the same value rows as +ordinary `KvHandle` operations. It is not a second store or registry. + +Built-in implementations are Fastly KV and Axum's persistent redb, DynamoDB and +celld record modes. Cloudflare and Spin built-ins reject record requirements during +application construction. Ordinary-only KV bindings keep their existing behavior. + +## Declare grouping before preparation + +Declare the logical KV ID in `edgezero.toml`, then install its layout in the +portable application's configure hook: + +```toml +[stores.kv] +ids = ["sessions"] +``` + +The application defines key meanings. This example puts `session/42` in the +complete `session/` partition: + +```rust +use edgezero_core::key_value_store::KvError; +use edgezero_core::record_store::{RecordKeyLayout, RecordLayout}; + +struct FirstSegmentLayout; + +impl RecordKeyLayout for FirstSegmentLayout { + fn id(&self) -> &str { "example-first-segment" } + fn version(&self) -> u32 { 1 } + + fn validate(&self) -> Result<(), KvError> { + // Nonempty first segments followed by '/' form disjoint partitions. + Ok(()) + } + + fn partition_prefix<'key>(&self, key: &'key str) -> Result<&'key str, KvError> { + let slash = key.find('/') + .ok_or_else(|| KvError::Validation("missing key group".to_owned()))?; + if slash == 0 || slash + 1 == key.len() { + return Err(KvError::Validation("invalid grouped key".to_owned())); + } + Ok(&key[..=slash]) + } + + fn validate_partition(&self, prefix: &str) -> Result<(), KvError> { + if prefix.strip_suffix('/').is_some_and(|group| { + !group.is_empty() && !group.contains('/') + }) { + Ok(()) + } else { + Err(KvError::Validation("incomplete partition".to_owned())) + } + } +} +``` + +In your existing `Hooks::configure` implementation: + +```rust +app.require_record_store("sessions", RecordLayout::new(FirstSegmentLayout)?)?; +``` + +Keep the layout's ID/version and placement rules stable. Changing placement needs +an explicit format/cutover decision, not different rules under the same identity. +Full keys remain the identity of point operations; grouping does not replace or +shorten them. + +Declarations are sealed after configure. The initializer may call +`kv.require_records()?`, but cannot change routing. That acquisition performs no +I/O and does not upgrade an ordinary-only provider. Supplied ready/deferred handles +must already match. Local IDs sharing one backing name must request the same +layout and format; an ordinary alias cannot silently switch to the record table. + +```mermaid +flowchart TD + A[Configure layout] --> B[Validate source and format] + B --> C[Prepare provider] + C --> D[Existing KvHandle] + D --> E[Ordinary operations] + D --> F[require_records view] + F --> G[Record operations] + E --> H[Same value rows] + G --> H +``` + +## Read and conditionally replace + +Given a prepared `KvHandle` named `kv`: + +```rust +use edgezero_core::record_store::{RecordWrite, RecordWriteMode, RecordWriteOutcome}; + +let records = kv.require_records()?; +let outcome = records.insert("session/42", RecordWrite { + body: b"hello\0world", + metadata: Some("application text"), + ttl: None, + mode: RecordWriteMode::Add, +}).await?; + +if outcome == RecordWriteOutcome::Written { + let current = records.lookup("session/42").await?; + // A later lookup can observe another writer, not necessarily this write. +} +``` + +- `lookup` returns exact body bytes, optional UTF-8 metadata and a generation. + Its consistency and TTL visibility remain provider-specific. A lookup miss is + not necessarily authoritative absence. +- `Add` atomically rejects an existing value under the provider's expiry rules. +- `IfGenerationMatch(generation)` atomically checks that same key's generation. + Generations support equality, not numeric ordering or business timestamps. +- `Overwrite` has no generation precondition. A rejection is an error, not normal + Add/CAS contention. +- `exists_strong` checks exact authoritative existence, not prefix-neighbor + presence. Ordinary `kv.exists` does not acquire that promise. +- `delete` succeeds for a missing value. It does not erase generation history. + +An acknowledged insert returns `Written`, not a fabricated post-write generation. +`PreconditionFailed` means a definitive Add/CAS rejection. Reloading, merging, +retrying or deferring is application policy. + +`KvError::MutationUncertain` means a mutation may have committed. EdgeZero does not +replay it. Reconcile with storage before deciding what to do next; blindly retrying +an overwrite or delete can erase another writer's work. + +## Ordinary access and TTL + +Ordinary get/put/TTL-put/delete on this binding use the same record rows. Every +replacement changes the generation. Ordinary puts clear metadata; a no-TTL put +also clears the prior expiration and creates a non-expiring value. Empty metadata +and absent metadata both read as `None`. Metadata is text, while bodies may contain +arbitrary bytes. + +TTL uses native provider behavior. There is no portable exact visibility/deletion +deadline or added expiry envelope. In particular, DynamoDB record values remain +visible and block Add while their expired TTL attribute awaits native deletion. +redb and celld can hide expired rows before physical cleanup. The existing portable +TTL input bounds remain 60 seconds through one year. + +## Complete pagination + +```rust +let partition = records.partition("session/")?; +let mut cursor = None; +loop { + let page = records.list_partition(&partition, cursor.as_deref(), 100).await?; + for key in page.keys { + // Process the full logical key. + } + cursor = page.cursor; + if cursor.is_none() { break; } +} +``` + +Each record partition page is strong. The traversal is live, unordered and not a +frozen snapshot. Continue through short or empty pages when a cursor is present. +Cursors belong to their binding, layout, operation and query; pass them back +unchanged. They are not authorization tokens and must not be logged. + +Ordinary `kv.list_keys_page(prefix, cursor, limit)` still accepts arbitrary and +empty prefixes. On DynamoDB records it uses paginated **Scan**, separate from the +strong partition **Query**. Scan reads broadly before filtering namespace, format, +value kind and logical prefix. Internal allocator or foreign rows can consume a +page without producing keys. There is no global index or complete-store sort per +page. + +## Select the native format + +Fastly uses its native KV record operations. Axum's local provider selects an +explicit redb record table when the configured ID requires records. Old ordinary +tables are not converted or silently read as records. + +For DynamoDB, opt in to the generated Axum feature and matching bindings tag: + +```toml +# edgezero.toml +[adapters.axum.build] +features = ["aws-dynamodb-records"] +``` + +```toml +# Native bindings file selected by EDGEZERO__STORE_BINDINGS_FILE +version = 1 + +[kv.sessions] +provider = "aws-dynamodb-records" +region = "us-east-1" +table = "edgezero-records" +namespace = "example-prod/sessions" +``` + +Use an operator-owned ACTIVE table with String `pk`/`sk` keys and the supported +shape described in [DynamoDB KV](./dynamodb-kv). Record preparation also needs the +DescribeTable ARN to identify the backing account/table. It does not create a +table or certify write permissions. Enable native TTL on `ttl_epoch_s`; allocator +items have no TTL and must not be removed with values. + +Record PKs encode format, namespace, layout identity and declared partition. Value +and retained allocator SKs contain the unchanged full key under separate `r#` and +`g#` kinds. Every attempted insert allocates a per-key u64 marker with UpdateItem, +then uses conditional or unconditional PutItem. Gaps and out-of-allocation-order +commits are valid. No namespace-wide item is updated by every insert. + +The runtime needs DescribeTable, GetItem, UpdateItem, PutItem, DeleteItem, Query +and Scan on the reviewed table. Scan's broad read access differs from a +partition-only policy. This is not hosted IAM or capacity qualification. + +For celld, select `celld-records` instead: + +```toml +[adapters.axum.build] +features = ["celld-records"] +``` + +```toml +version = 1 + +[kv.sessions] +provider = "celld-records" +endpoint = "http://127.0.0.1:8090" +store = "sessions" +access = { mode = "trusted-network", allow_remote_http = false } +``` + +The matching service must implement record protocol v2, keyset continuation and +nullable expiry. Preparation checks that capability with a read-only request; +old services fail rather than downgrade. All operations use the fresh v2 record +dataset, never raw `/value`. Existing raw KV and v1 record data stay unchanged. +See [native celld stores](./native-celld-stores) for access and hosting setup. + +Both native record features are off by default. `aws-dynamodb` and `celld-kv` +remain ordinary-only modes and are not automatically converted when an app +requires records. The standalone provider features are `dynamodb-records` and +`records-v2`; legacy celld `records` remains a separate v1 API. Settings-only, +ordinary-only and WASM builds do not gain native SDK/runtime dependencies. + +## Binding limits + +Existing portable key, body, TTL and page validation still applies, followed by +provider limits. `records.binding().max_metadata_bytes` reports the supported +metadata byte maximum, excluding empty/absent metadata. + +| Binding | Metadata supported by this binding | +| ---------------- | ------------------------------------------------------------------------------------------------------------------------------ | +| Fastly native | 2000 UTF-8 bytes | +| redb records | Existing 25 MiB value ceiling | +| DynamoDB records | Configured payload budget, default 384 KiB; body plus metadata must fit together and the complete native item must fit 400 KiB | +| celld v2 | 1024 UTF-8 bytes | + +Generation history survives value overwrite, expiry, deletion/recreation and +provider reopen. It is not safe to delete allocator state while continuing to +use the same generation domain. No data migration, service image publication, +fleet failover or rolling-upgrade qualification is supplied by this feature. diff --git a/docs/guide/kv.md b/docs/guide/kv.md index 95608bb1..4ad634c3 100644 --- a/docs/guide/kv.md +++ b/docs/guide/kv.md @@ -104,11 +104,20 @@ lost writes. For example, two requests that both read `counter = 5` and write `6 will end with `counter = 6` instead of `7`. Use it only when approximate values are acceptable (e.g. visit counters, feature flags). -For strict correctness, use a transactional data store. +For atomic Add/CAS on supported bindings, use [conditional records](./kv-records). +The read/merge/conflict policy remains application-owned. ::: Key listing is paginated by design. This avoids buffering an unbounded number of keys in memory and matches the underlying provider APIs. The Spin adapter materialises `Store::get_keys()` and pages client-side; a `max_list_keys` cap (configurable via `EDGEZERO__STORES__KV____MAX_LIST_KEYS`, default `1000`) guards against runaway lists and yields `KvError::LimitExceeded` when exceeded. +## Conditional records + +[Record-capable bindings](./kv-records) add generation lookup, atomic conditional +replacement, exact authoritative existence and strong partition pages. Declare the +application's key layout before provider preparation. Ordinary and record methods +then access the same value rows; `require_records()` cannot upgrade an already +prepared ordinary-only provider. + ## Declared Store Failures Fastly, Cloudflare and Spin install every declared logical KV ID without opening its @@ -195,7 +204,9 @@ provision --adapter fastly` creates the remote store and writes only the ### Consistency -Both Fastly and Cloudflare KV stores are **eventually consistent**. +Ordinary Fastly and Cloudflare value lookups are **eventually consistent**. +[Record operations](./kv-records) make separate, explicit promises for authoritative +existence and partition pages on supported bindings. - A value written at one edge location may not be immediately visible at another. - `read_modify_write()` is **not atomic**. Concurrent updates to the same key may result in lost writes. @@ -203,13 +214,14 @@ Both Fastly and Cloudflare KV stores are **eventually consistent**. ## Limits & Validation -To ensure portability across all providers, `KvHandle` enforces the -strictest common limits: +`KvHandle` enforces the following shared validation. Providers may impose smaller +limits, such as DynamoDB's 384 KiB payload budget. Record metadata limits are +listed in [conditional records](./kv-records). | Rule | Limit | | ------------- | ------------------------------------ | | Key size | Max **512 bytes** (Cloudflare limit) | -| Value size | Max **25 MB** | +| Value size | Max **25 MiB** | | TTL minimum | **60 seconds** | | TTL maximum | **1 year** | | List page max | **1000 keys** | diff --git a/docs/guide/native-celld-stores.md b/docs/guide/native-celld-stores.md index 61640978..3f138270 100644 --- a/docs/guide/native-celld-stores.md +++ b/docs/guide/native-celld-stores.md @@ -2,11 +2,11 @@ The native provider reads complete configuration documents at startup and exposes live KV operations through a scoped listener **inside celld**. There is no gateway, sidecar, Worker bridge or celld HTTP adapter. -This is an unreleased integration. It requires the modified celld revision `f72ae21a515859570622ac9d6ca9c19cf6870b81`. Upstream celld v0.6.2 does not have this API. Local tests do not establish fleet failover, deployment security or production readiness. +This is an unreleased integration. The original config/raw KV API requires the modified celld revision `f72ae21a515859570622ac9d6ca9c19cf6870b81`. Legacy record clients need the matching record service extension. The new [record-capable KV binding](./kv-records) requires the matching v2 service extension instead; that older revision cannot supply it. Upstream celld v0.6.2 does not have these scoped APIs. Local tests do not establish fleet failover, deployment security or production readiness. ## Build only the providers you use -Generated Axum targets forward two independent, default-off features: +The config/raw KV provider features remain independent and off by default: ```toml [adapters.axum.build] diff --git a/docs/superpowers/plans/2026-11-13-kv-record-contract.md b/docs/superpowers/plans/2026-11-13-kv-record-contract.md index e6014311..fcbd8c52 100644 --- a/docs/superpowers/plans/2026-11-13-kv-record-contract.md +++ b/docs/superpowers/plans/2026-11-13-kv-record-contract.md @@ -1,6 +1,6 @@ # Portable KV records implementation plan -Status: implementation authorized and started on 2026-11-13. The user reviewed the [spec](../specs/2026-11-13-kv-record-contract-design.md) and requested this execution checklist. Focused core tests and the shared suite on real redb and Fastly/Viceroy have passed. DynamoDB SDK-wire and companion celld service tests have passed. Combined construction, DynamoDB Local, the EdgeZero celld provider and lint gates remain incomplete. +Status: implementation and required workspace, provider, generated-consumer and selected Fastly CI checks passed on 2026-11-13. Local risk-based review is underway. The pre-existing Cloudflare SDK dependency-graph question remains open. TS adoption and public changes have not started. Worktree: `/home/pav/worktrees/feat-kv-record-contract`, branch `feat/kv-record-contract`, initially based on #439 at `645703bf39f5965dad5b54d7907e28c9197a7034`. @@ -80,11 +80,11 @@ Files: Work: - [x] Register immutable per-ID record requirements through App configure. Resolve full keys to stable enumerable prefixes without embedding TS key meanings in core. -- [ ] Validate declarations after App construction and before `build_production_stores`. Initializer acquisition must not change routing after handles exist. -- [ ] Thread the same declaration through production, development and provider constructors. Ready/deferred handles must already match the required layout/mode. -- [ ] Handle IDs that resolve to a shared backing name consistently; reject conflicting declarations before publishing stores. -- [ ] Keep ordinary arbitrary-prefix/global listing distinct from complete-partition validation. An empty global prefix must not fail merely because it is not a partition. -- [ ] Preserve existing source-selection validation, preparation ordering, immutable snapshots and no-fallback behavior. Do not rewrite the hosting lifecycle. +- [x] Validate declarations after App construction and before `build_production_stores`. Initializer acquisition must not change routing after handles exist. +- [x] Thread the same declaration through production, development and provider constructors. Ready/deferred handles must already match the required layout/mode. +- [x] Handle IDs that resolve to a shared backing name consistently; reject conflicting declarations before publishing stores. +- [x] Keep ordinary arbitrary-prefix/global listing distinct from complete-partition validation. An empty global prefix must not fail merely because it is not a partition. +- [x] Preserve existing source-selection validation, preparation ordering, immutable snapshots and no-fallback behavior. Do not rewrite the hosting lifecycle. Checks: core layout tests and focused Axum startup tests prove configure-before-prepare, acquisition without mutation, mismatched supplied handles, shared-name declarations, ordinary-only compatibility and unsupported providers. Run `cargo test -p edgezero-adapter-axum` with the selected relevant features after wiring. @@ -106,10 +106,10 @@ Check: `cargo test -p edgezero-adapter-axum --lib key_value_store` plus the comm Files: `crates/edgezero-adapter-fastly/src/key_value_store.rs`, `tests/contract.rs` and the existing Fastly integration fixture paths. - [x] Use `current_generation()`, native Add and `if_generation_match`. Never use the deprecated generation accessor or get-then-put CAS. -- [ ] Normalize native metadata into optional UTF-8 text and preserve native TTL behavior. Do not return a fabricated post-write generation. -- [ ] Implement authoritative exact existence with Strong prefix listing and exact key equality. Follow continuation; descendants and truncated pages cannot prove presence/absence. -- [ ] Preserve native Strong list cursors for partition pages. Keep groups as prefixes in the existing physical store. -- [ ] Map only definitive Add/CAS rejection to normal contention. Surface uncertain mutation completion without library replay. +- [x] Normalize native metadata into optional UTF-8 text and preserve native TTL behavior. Do not return a fabricated post-write generation. +- [x] Implement authoritative exact existence with Strong prefix listing and exact key equality. Follow continuation; descendants and truncated pages cannot prove presence/absence. +- [x] Preserve native Strong list cursors for partition pages. Keep groups as prefixes in the existing physical store. +- [x] Map only definitive Add/CAS rejection to normal contention. Surface uncertain mutation completion without library replay. Check: add and execute the common suite under Viceroy using `scripts/test_integration.sh --adapter fastly --suite ci` with a nonzero selector for the new cases. SDK compilation/native Rust tests do not replace runtime proof. Keep current runtime/SDK pins unless an independently reviewed prerequisite requires change. @@ -122,13 +122,13 @@ Files: Work: -- [ ] Add explicit record-format preparation using existing native SDK clients, deadlines, table-key validation and redacted error handling. -- [ ] Encode format/namespace/group into distinct PK values. Separate value and per-key allocator SK kinds without changing the full logical key returned to callers. -- [ ] Allocate u64 markers atomically per key, then use native conditional PutItem for Add/CAS or unconditional replacement. No namespace-wide item is mutated by every insert. -- [ ] Retain allocator state across value delete/TTL; exclude it from user reads/listing. Prevent wrap/reset. Gaps and out-of-allocation-order unconditional commits are valid. -- [ ] Apply one absolute operation budget across allocation and value write. Preserve single-attempt SDK mutations. An allocator-only interruption must not be confused with an acknowledged value write. -- [ ] Implement authoritative strong GetItem and strong base-table Query for a declared partition. Carry LastEvaluatedKey through internal/filtered rows, even for an empty returned page. -- [ ] Keep ordinary point operations coherent with record rows. Use native TTL on the explicit new format and preserve the legacy ordinary binding's existing format/expiry behavior. +- [x] Add explicit record-format preparation using existing native SDK clients, deadlines, table-key validation and redacted error handling. +- [x] Encode format/namespace/group into distinct PK values. Separate value and per-key allocator SK kinds without changing the full logical key returned to callers. +- [x] Allocate u64 markers atomically per key, then use native conditional PutItem for Add/CAS or unconditional replacement. No namespace-wide item is mutated by every insert. +- [x] Retain allocator state across value delete/TTL; exclude it from user reads/listing. Prevent wrap/reset. Gaps and out-of-allocation-order unconditional commits are valid. +- [x] Apply one absolute operation budget across allocation and value write. Preserve single-attempt SDK mutations. An allocator-only interruption must not be confused with an acknowledged value write. +- [x] Implement authoritative strong GetItem and strong base-table Query for a declared partition. Carry LastEvaluatedKey through internal/filtered rows, even for an empty returned page. +- [x] Keep ordinary point operations coherent with record rows. Use native TTL on the explicit new format and preserve the legacy ordinary binding's existing format/expiry behavior. Tests: different groups produce different actual PKs; no global allocator traffic; independent Add/CAS races; stale-token rejection after overwrite/delete/expiry/restart; lost acknowledgement; allocation-only failure; exhaustion; byte/text limits; Query continuation and namespace isolation. @@ -138,12 +138,12 @@ Check: extend existing SDK loopback fixtures and DynamoDB Local proof. The curre Files: DynamoDB record provider, cursor encoding/validation, provider wire tests and local generated-consumer proof. -- [ ] Implement ordinary `list_keys_page` across all groups and arbitrary/empty prefixes with native paginated Scan. Do not reject cross-group requests. -- [ ] Filter by the binding's versioned namespace, value-row kind and requested logical prefix. Return logical keys only, never allocator rows or foreign namespace values. -- [ ] Preserve native LastEvaluatedKey. The Scan limit counts evaluated rows, so short/empty output with continuation is valid. -- [ ] Allow a native continuation position that belongs to a filtered physical row. Validate cursor structure and bind the outer token to operation, binding, layout/format and requested prefix; do not treat the native position as a returned value. -- [ ] Reject Scan/Query cursor interchange. Keep strong partition listing and authoritative exact existence off this global Scan path. -- [ ] Do not sort all groups, rescan the complete namespace per output page or create a global index/catalog. Document broad read cost and the required Scan operation without adding hosted IAM acceptance. +- [x] Implement ordinary `list_keys_page` across all groups and arbitrary/empty prefixes with native paginated Scan. Do not reject cross-group requests. +- [x] Filter by the binding's versioned namespace, value-row kind and requested logical prefix. Return logical keys only, never allocator rows or foreign namespace values. +- [x] Preserve native LastEvaluatedKey. The Scan limit counts evaluated rows, so short/empty output with continuation is valid. +- [x] Allow a native continuation position that belongs to a filtered physical row. Validate cursor structure and bind the outer token to operation, binding, layout/format and requested prefix; do not treat the native position as a returned value. +- [x] Reject Scan/Query cursor interchange. Keep strong partition listing and authoritative exact existence off this global Scan path. +- [x] Do not sort all groups, rescan the complete namespace per output page or create a global index/catalog. Document broad read cost and the required Scan operation without adding hosted IAM acceptance. Check: local DynamoDB traversal over multiple groups/namespaces with allocator-only pages, empty prefixes, nonmatching prefixes, exact page multiples and live changes. In a quiescent dataset compare complete key sets and duplicate counts, not order. Verify page-sized work uses native continuation rather than restarting at the beginning. @@ -158,10 +158,10 @@ Files: Work: -- [ ] Add strict, explicit record-mode selection and constructor arguments. Do not silently convert a raw KV binding or fall back to its old format. -- [ ] Add default-off record feature forwarding. Proposed names are `dynamodb-records` in the AWS crate, `aws-dynamodb-records` and `celld-records` in Axum; verify the final names and update commands/tests together. -- [ ] Preserve settings-only/default-off builds and current AWS/celld config/secret independence. Do not add native dependencies to core or WASM graphs. -- [ ] Extend generated consumer templates, strict validation and negative dependency tests. Record capability does not mean offline CLI can certify runtime permissions/connectivity. +- [x] Add strict, explicit record-mode selection and constructor arguments. Do not silently convert a raw KV binding or fall back to its old format. +- [x] Add default-off record feature forwarding. Proposed names are `dynamodb-records` in the AWS crate, `aws-dynamodb-records` and `celld-records` in Axum; verify the final names and update commands/tests together. +- [x] Preserve settings-only/default-off builds and current AWS/celld config/secret independence. Do not add native dependencies to core or WASM graphs. +- [x] Extend generated consumer templates, strict validation and negative dependency tests. Record capability does not mean offline CLI can certify runtime permissions/connectivity. Checks: selected feature tests, generated Cargo checks, startup suites and dependency graph script, independently disabled and enabled. Broad all-features success alone is insufficient. @@ -171,12 +171,12 @@ EdgeZero files: `crates/edgezero-store-celld/src/records.rs`, `client.rs`, setti The service change is mandatory companion work, not another EdgeZero PR or a deferred follow-up. Its implementation/publication is coordinated in Obsidian Phase 4. The available old service has neither record continuation nor ordinary no-expiry record writes. -- [ ] Agree and implement the matching versioned protocol for record pagination and no-expiry. Coordinate server `scoped_record.rs`, service `kv.js` and required client/CLI paths in the celld repository; preserve full extension source, not just a pagination patch against an unpublished base. -- [ ] Prepare the EdgeZero binding with a read-only capability/version check for the required protocol. Old services must fail preparation, not silently downgrade. -- [ ] Route all ordinary methods on this binding through the record dataset, never `/value` raw KV. Keep raw-only bindings unchanged. -- [ ] Support actual non-expiring records, not a fabricated long TTL. Apply the reviewed fresh-format/no-migration approach consistently to read, write, Add/CAS, list, sweep and alarm behavior. -- [ ] Implement record keyset continuation, scope/prefix/mode-bound cursors, end detection and text metadata normalization. -- [ ] Rebuild the matching service from exact source/lockfile and record source/tree/toolchain/build command/binary hash through the coordination plan. Do not use the cached old binary as new-protocol evidence. +- [x] Agree and implement the matching versioned protocol for record pagination and no-expiry. Coordinate server `scoped_record.rs`, service `kv.js` and required client/CLI paths in the celld repository; preserve full extension source, not just a pagination patch against an unpublished base. +- [x] Prepare the EdgeZero binding with a read-only capability/version check for the required protocol. Old services must fail preparation, not silently downgrade. +- [x] Route all ordinary methods on this binding through the record dataset, never `/value` raw KV. Keep raw-only bindings unchanged. +- [x] Support actual non-expiring records, not a fabricated long TTL. Apply the reviewed fresh-format/no-migration approach consistently to read, write, Add/CAS, list, sweep and alarm behavior. +- [x] Implement record keyset continuation, scope/prefix/mode-bound cursors, end detection and text metadata normalization. +- [x] Rebuild the matching service from exact source/lockfile and record source/tree/toolchain/build command/binary hash through the coordination plan. Do not use the cached old binary as new-protocol evidence. Checks: wire fixtures for old/malformed protocols, uncertainty and cursor validation; then common public-handle tests against the actual rebuilt local service. Prove same-row ordinary access, metadata clearing, no-expiry across restart/sweep and complete pagination. No claim of fleet/rolling-upgrade or production isolation qualification. @@ -189,12 +189,12 @@ Existing files to extend: - `crates/edgezero-adapter-axum/tests/native_aws_startup.rs` and `native_celld_startup.rs`. - `scripts/run_native_aws_integration.mjs`, `run_native_celld_integration.mjs` and their existing Node tests. -- [ ] Run the same semantic assertions on Fastly, persistent redb, DynamoDB Local and real local celld. Independent writers must share the backing store; a client mutex is not an atomicity proof. -- [ ] Require nonzero execution of selected generated/service tests. Missing services or ignored tests cannot count as passes. +- [x] Run the same semantic assertions on Fastly, persistent redb, DynamoDB Local and real local celld. Independent writers must share the backing store; a client mutex is not an atomicity proof. +- [x] Require nonzero execution of selected generated/service tests. Missing services or ignored tests cannot count as passes. - [ ] Exercise same-row get/write/delete, ordinary TTL/no-TTL replacement, metadata, stale tokens and reopen/recreate behavior on actual providers. - [ ] Fault-inject a lost response after value commit and prove no hidden replay. For the two-stage DynamoDB design, distinguish one allocation plus one value mutation from retrying either request. -- [ ] Exercise page continuation and live changes without asserting a frozen snapshot or alphabetical order. Include separate strong Query and ordinary global Scan cases. -- [ ] Retain the exact rebuilt celld artifact and source identity for TS adoption. Record local DynamoDB image/configuration and dummy endpoint/credential use. +- [x] Exercise page continuation and live changes without asserting a frozen snapshot or alphabetical order. Include separate strong Query and ordinary global Scan cases. +- [x] Retain the exact rebuilt celld artifact and source identity for TS adoption. Record local DynamoDB image/configuration and dummy endpoint/credential use. Existing runner entry points, to be extended before they count as new record acceptance: @@ -207,10 +207,10 @@ scripts/test_integration.sh --adapter fastly --suite ci ## 10. Stable checks and handoff -- [ ] Run focused tests after each code change and the full required gates on a stable combined state. +- [x] Run focused tests after each code change and the full required gates on a stable combined state. - [ ] Run default-off/enabled feature graphs and target-matched SDK/runtime checks. Add the final record feature selections to the command record. - [ ] Review the entire feature diff against the refreshed #439 parent. No #454 prototype, unrelated prerequisite repair or TS business policy belongs in this PR. -- [ ] Document record binding selection, native TTL/metadata limits, generation allocation cost, Scan versus Query, protocol/artifact requirements and migration exclusions. +- [x] Document record binding selection, native TTL/metadata limits, generation allocation cost, Scan versus Query, protocol/artifact requirements and migration exclusions. - [ ] Give the Obsidian coordination plan an exact verified EdgeZero source/dependency/service checkpoint for TS adoption. Do not claim runtime readiness from the spec or historical receipts. - [ ] Use the coordination plan for owned TS PR reconciliation, preserving old public heads, forward merge into #454 and official stack recreation. Do not execute publication, deployment or main merges from this checklist alone. @@ -241,43 +241,31 @@ Also run affected docs, demo/generated-consumer and other required WASM checks f ## Progress and evidence -Implementation is in progress. Checkmarks record implemented behaviors with focused evidence, not a completed provider/startup qualification. Broad lint, generated-consumer and real-service gates remain separate. +All commands use Rust `+1.98.1`, `--locked --offline`, task-local TMPDIR and isolated local services. This is executed local acceptance, not hosted deployment qualification. No publication or TS source edits have occurred. -For each completed behavior, record the exact source/dependency/service inputs, command, result, elapsed time and log. Reuse results only while relevant inputs remain unchanged. Report baseline failures separately and do not relax bounds to turn them green. - -| Checkpoint | State | Evidence | +| Checkpoint | Current result | Evidence under `/home/pav/.cache/edgezero-kv-record-contract/` | | --- | --- | --- | -| Core contract and layout | Functional checks passed; lint pending | `record_api`: 12 passed; `record_configuration`: 4 passed. Full core tests reached the subsequent Clippy command with task-local TMPDIR. Clippy failed on new-code conventions and ordinary-only default-method lint sites. | -| Persistent records | Focused checks and lib lint passed | 36 selected KV tests passed, including 9 record cases, the public shared suite, independent Add/CAS writers, delete/reopen, missing allocator state, delayed cleanup and expiry/recreation. A reproduced strong-existence expiry mismatch was fixed. Axum lib Clippy passed after source organization/style fixes. | -| Fastly records | Basic real-path suite passed; driver/edge cases pending | Release WASM fixture built; explicit repaired Viceroy 0.21.0 returned HTTP 200 and `executed_suites=1` after the public shared assertions. No native race/restart/ack-loss claim. | -| DynamoDB writes/Query/global Scan | SDK-wire and Local checks passed; lint/generated proof pending | Updated crate: 66 passed after endpoint/ARN/table identity correction and 3 scope regressions. Both explicitly selected Local tests passed against the existing pinned DynamoDB Local 3.3.1 image, including real group/allocator rows, shared suite, independent clients, complete Query/Scan and simulated physical expiry removal/reopen. | -| Feature/binding selection | Default-native checks passed; broader selections pending | Default Axum `record_`: 13 passed, `native_`: 63 passed. Explicit mode/layout/shared-name/ready/deferred checks executed. Platform/CLI/template and selected native-feature checks are still being collected. | -| Matching celld protocol and provider | Service tested; EdgeZero client in progress | 12 HTTP/protocol cases and 4 new real-process v2 cases passed. Startup neighbors: 10 default cases plus both explicitly selected TLS/60-second cases passed. EdgeZero records-v2 wire unit: 8 passed; real client/service interop remains pending. | -| Four-provider real-path conformance | Basic shared suite passed on all four; combined acceptance pending | Real redb, Fastly/Viceroy, DynamoDB Local and the EdgeZero client against rebuilt celld have executed the shared assertions. Celld additionally passed independent native races/global listing and committed PUT/DELETE dropped-reply proof. New automatic Fastly race/continuation/CI, generated consumers, broad gates and review remain pending. | -| Stable integration and coordination handoff | Pending | No TS adoption or public mutation yet. | - -Checkpoint logs and inputs live under `/home/pav/.cache/edgezero-kv-record-contract/`: - -- `core-redb-expanded.log`: API/configuration and persistent focused commands; builds 1.89s and 12.57s, persistent tests 0.24s. `first-record-checkpoints.sha256` records relevant inputs before the later redb cleanup/expiry test additions. -- `core-neighbors-lint.log`: first broad core attempt, 11 existing temporary-file writes failed with EDQUOT on `/tmp`. No application limit or test was weakened. Scoped TMPDIR rerun proceeded to Clippy; `core-neighbors-lint-disk-tmp.log` retains its lint diagnostics. Subsequent checks use the task-local disk-backed `tmp/` directory. -- `dynamodb-records-unit.log`: 63 SDK-wire/settings cases passed, build 27.75s and tests 3.07s. A neighboring ordinary-only feature run, crate lint and real DynamoDB Local proof are still required. -- `fastly-fixture-build.log` and `fastly-runtime-receipt.json`: build 25.35s; one real shared suite passed. Guest SHA-256 `e5fc4beb55042df62504d60e76db8475bf9429de9d1bd931e2fae90635128ddd`; selected runtime SHA-256 `c846e8db180beeb1477fb504ec1d89c09251203633cd98c7fd9d97abbd7b9755` at `/home/pav/.cache/rc-refresh-20261110/target-viceroy-repair/debug/viceroy`. The global 0.17.0 binary was not used. The earlier native Fastly lib-test attempt failed on unresolved SDK host imports and is not a test pass or established baseline regression. -- `celld-protocol-unit.log`, `celld-v2-runtime.log` and `celld-startup-neighbors.log`: 12 HTTP cases, 4 v2 real-process cases and 10 nonignored startup neighbors passed. Builds took 77s and 54.23s; v2 runtime tests took 0.77s. Both explicit ignored neighbors passed in `celld-startup-ignored.log`, 2 cases in 61.20s. -- `artifacts/celld-v2-f345ffd84455`: retained rebuilt service, SHA-256 `f345ffd844551c6dd3780cd2b8a5b9d2c0dc5ecd33522e17edea7ca5d2c53e39`, from companion base `9350a043860c4f41dccb692ec9de98ebd9dac3ae` plus `artifacts/celld-v2-source.patch`, SHA-256 `0427280d95893fb45aa28d085a42e75ab06e1de39b40cc676411234a11494980`. Source fingerprints are in `artifacts/celld-v2-source.sha256`; Cargo.lock SHA-256 is `d6a17422788aa910ba76bed9cf59ac15767f93fe107ebdb336954d0e9efe052b`. Rust 1.98.1 built the binary through the focused service integration command. Full source publication must include the base extension, not only this v2 delta. - -Additional current receipts: - -- `redb-native-check.log`: 36 persistent, 13 record-filtered and 63 native-filtered cases passed. `redb-expiry-red2.log` is the intentional failing expiry-consistency regression before the fix. `redb-style-tests.log` passed 36 cases again after moving the private implementation to `src/key_value_store_records.rs`; `axum-clippy-recheck.log` passed lib Clippy. -- `core-tests-recheck.log`: all core targets passed, including 1147 unit, 12 API, 4 configuration and 15 doc tests. `core-clippy-recheck.log` reduced new-code lint failures from 148 to 10; those last fixes and the generic pagination test-loop correction remain pending. -- `fastly-wasi-unit.log`: 9 provider tests passed on wasm32-wasip1 with the explicit repaired runtime, including 6 new cases. `fastly-harness-unit.log`: 4 nonzero/readiness/evidence validator tests passed. Expanded native races/1001-descendant proof still needs its new runtime receipt. -- `dynamodb-scope-test.log`: 66 SDK-wire/settings cases passed. `dynamodb-local-records.log`: 2 explicitly ignored Local cases and their isolated exact-test children passed, build 3.40s and execution 1.26s. The expiry case labels manual physical value-row deletion as simulation, not a Local/native AWS TTL sweep. `dynamodb-local-container.json` records the task-owned loopback container and pinned image digest. -- `celld-client-unit.log`: 8 wire cases passed. `celld-client-interop.log`: 8 current wire cases and 1 explicitly selected real-service case passed, the latter in 0.26s. The service used the retained f345 artifact and a fresh writable `identity` alias; both parent and worker temporary listeners were subsequently stopped. -- `record-runner-node-tests.log`: 18 tests passed, including mandatory old/new generated sentinels, both Local provider cases, zero-test rejection, interruption cleanup and credential/trust-input isolation. Existing runners are being extended, not replaced with another emulator framework. -- Public `docs/guide/kv-records.md` and links now document the actual API, explicit formats, native TTL/metadata, generation cost and Scan/Query distinction. Prettier 3.9.10 check passed for the five touched published-doc files. +| Workspace | Strict all-target/all-feature Clippy and full all-target tests passed | `noninteractive-clippy.log`, `noninteractive-tests.log`. Workspace feature unification ran 1174 core, 503 Axum and 261 CLI unit tests, plus other workspace targets. Separate core run included 15 doctests. | +| Persistent records | Transactional contract, lifetime, native races and expiry cleanup passed | 36 selected KV cases, including 9 record cases. Final workspace rerun covers the test-only convention fixes. | +| Fastly records | Target-matched provider tests/lint and full selected CI passed | `fastly-style-all-targets-clippy-final.log`, `fastly-style-all-targets-unit.log`; `fastly-ci-20261113/` records all ten CI stages. Independent Add/CAS each had one winner, 1001 descendants crossed two pages, exact existence and cleanup passed, shared suite executed exactly once. | +| DynamoDB | Record66, legacy46, settings11, strict lint and Local2 passed | `aws-verify-*.log`. Scope includes resolved endpoint, table ARN and optional TableId. Expiry/reopen proof simulates physical value-row removal, not an AWS TTL sweep. | +| Native/platform selection | Combined features, native startup, unsupported providers and WASM checks passed | `final-check.log`, `final-native-records.log`, `final-spin-wasm.log`; both-native `record_`13 and `native_`63, plus earlier platform/CLI/template selections. | +| celld service/client | Protocol12, process4, startup10 plus explicit TLS/60s2; client wire8, neighbors64, default5 and strict lint passed | `celld-worker-*-current.log`, service receipts in the cross-repository plan. Real interop includes independent races, post-commit dropped PUT/DELETE replies and 512-byte escaped/Unicode keys at limit1. | +| Generated consumers | Both old/new AWS consumers2 plus Local2; both old/new celld TLS consumers2 plus real interop1 passed | `generated-aws-runner.log`, `generated-celld-runner.log`. Existing runners require exact nonzero sentinels. | +| Runner/docs | Node18 and affected published docs passed | `record-runner-node-tests.log`, `docs-build.log`, `docs-lint.log`; Prettier3.9.10 checked the five published-doc files. | + +### Inputs and retained limitations + +- Working-tree review fingerprint `eb6310f2b68da5f09d4ae5b7cd13de3d31d0a30ae830ea70ad24cc86e38c2fae` covers both repositories, tracked changes and untracked source. Recompute with `bun /home/pav/.cache/edgezero-kv-record-contract/reviews/review-target.mjs`. It is not an immutable published dependency revision. +- Explicit repaired Viceroy0.21.0 is `/home/pav/.cache/rc-refresh-20261110/target-viceroy-repair/debug/viceroy`, SHA256 `c846e8db180beeb1477fb504ec1d89c09251203633cd98c7fd9d97abbd7b9755`. The global0.17 binary was not used; SDK pins and host ABI were not changed. +- Retained service `artifacts/celld-v2-f345ffd84455`, SHA256 `f345ffd844551c6dd3780cd2b8a5b9d2c0dc5ecd33522e17edea7ca5d2c53e39`, comes from base9350a043 plus `artifacts/celld-v2-source.patch`. Exact source/lock/build manifest remains in `artifacts/celld-v2-checkpoint.json`. Source publication must include the full base extension. +- Native Fastly lost-ack, TTL-after-wait and restart token lifetime remain unproven. No delegating fault wrapper or host stub is offered as native proof. DynamoDB wire uncertainty and celld actual post-commit reply-loss proofs have their stated scopes. +- The first parallel workspace attempt exceeded an unchanged TCP test's205ms ceiling by0.263ms. Its isolated rerun and final serial workspace run passed without changing the timing bound. A later CLI test assumed non-TTY stdin; the corrected full invocation closes stdin with ` worker -> edgezero-adapter-cloudflare`; `cloudflare-sdk-tokio-baseline.log` records selected features. The user was asked whether this pre-existing SDK dependency is outside the record work or a separate prerequisite. No SDK pin, global lint or dependency policy has been relaxed. The stricter graph gate remains incomplete while that scope question is open. +Pinned Cloudflare `worker =0.8.5` already brings featureless Tokio1.52.3 into WASM. No Tokio runtime or features are activated. Core/Cloudflare dependency pins remain unchanged from439; the only root lock change is celld's native test `tempfile` dependency. All new record graph checks pass before the final absolute no-Tokio check. That pre-existing edge still fails the script pending the user's scope decision. No SDK pin or dependency policy has been relaxed. ## References diff --git a/scripts/check_native_store_dependency_contract.sh b/scripts/check_native_store_dependency_contract.sh index 7eb365d0..41a4e2ad 100755 --- a/scripts/check_native_store_dependency_contract.sh +++ b/scripts/check_native_store_dependency_contract.sh @@ -11,6 +11,21 @@ assert_no_sdk() { fi } +assert_no_native_runtime() { + local graph + graph="$(cargo tree --offline --locked --edges normal --prefix none "$@")" + if grep -Eq '^(aws-(config|sdk-[^ ]+)|reqwest|tokio|celld(-[^ ]+)?|v8) v' <<< "$graph"; then + printf 'Native runtime/client dependency leaked into core or WASM: %s\n' "$*" >&2 + exit 1 + fi +} + +assert_no_native_runtime -p edgezero-core --all-features +assert_no_native_runtime -p edgezero-adapter-fastly --target wasm32-wasip1 --features fastly +assert_no_native_runtime -p edgezero-adapter-spin --target wasm32-wasip2 --features spin +assert_no_native_runtime -p edgezero-store-aws --target wasm32-wasip1 --no-default-features --features dynamodb-records +assert_no_native_runtime -p edgezero-store-celld --target wasm32-wasip1 --no-default-features --features records-v2 + assert_no_sdk -p edgezero-cli --no-default-features --features cli,edgezero-adapter-axum assert_no_sdk -p edgezero-store-aws --no-default-features assert_no_sdk -p edgezero-store-aws --no-default-features --features appconfig-agent @@ -19,26 +34,28 @@ assert_no_sdk -p edgezero-adapter-cloudflare --target wasm32-unknown-unknown --f assert_no_sdk -p edgezero-adapter-fastly --target wasm32-wasip1 --features fastly assert_no_sdk -p edgezero-adapter-spin --target wasm32-wasip2 --features spin dynamodb_graph="$(cargo tree --offline --locked --edges normal --prefix none -p edgezero-store-aws --no-default-features --features dynamodb-kv)" +dynamodb_records_graph="$(cargo tree --offline --locked --edges normal --prefix none -p edgezero-store-aws --no-default-features --features dynamodb-records)" secrets_graph="$(cargo tree --offline --locked --edges normal --prefix none -p edgezero-store-aws --no-default-features --features secrets-manager)" stores_graph="$(cargo tree --offline --locked --edges normal --prefix none -p edgezero-adapter-axum --no-default-features --features aws-stores)" -if ! grep -Eq '^aws-sdk-dynamodb v' <<< "$dynamodb_graph" || ! grep -Eq '^aws-sdk-secretsmanager v' <<< "$secrets_graph"; then +if ! grep -Eq '^aws-sdk-dynamodb v' <<< "$dynamodb_graph" || ! grep -Eq '^aws-sdk-dynamodb v' <<< "$dynamodb_records_graph" || ! grep -Eq '^aws-sdk-secretsmanager v' <<< "$secrets_graph"; then printf 'Selected AWS service SDK is missing.\n' >&2 exit 1 fi -if grep -Eq '^aws-sdk-secretsmanager v' <<< "$dynamodb_graph" || grep -Eq '^aws-sdk-dynamodb v' <<< "$secrets_graph" || grep -Eq '^aws-sdk-dynamodb v' <<< "$stores_graph"; then +if grep -Eq '^aws-sdk-secretsmanager v' <<< "$dynamodb_graph" || grep -Eq '^aws-sdk-secretsmanager v' <<< "$dynamodb_records_graph" || grep -Eq '^aws-sdk-dynamodb v' <<< "$secrets_graph" || grep -Eq '^aws-sdk-dynamodb v' <<< "$stores_graph"; then printf 'AWS service SDK leaked into an independently selected provider.\n' >&2 exit 1 fi # Celld settings are present in bindings-only builds, but live transport is not. -celld_features() { - local graph +provider_features() { + local package="$1" graph + shift graph="$(cargo tree --offline --locked --edges normal --prefix none --format '{p}|{f}' "$@")" - awk -F'|' '/^edgezero-store-celld v/ { sub(/ [(][*][)]$/, "", $2); print $2; exit }' <<< "$graph" + awk -F'|' -v package="$package" '$1 ~ ("^" package " v") { sub(/ [(][*][)]$/, "", $2); print $2; exit }' <<< "$graph" } assert_celld_off() { local selected - selected="$(celld_features "$@")" + selected="$(provider_features edgezero-store-celld "$@")" if [[ -n "$selected" ]]; then printf 'Live celld features leaked into isolated graph: %s: %s\n' "$*" "$selected" >&2 exit 1 @@ -47,25 +64,48 @@ assert_celld_off() { assert_celld_off -p edgezero-cli --no-default-features --features cli,edgezero-adapter-axum assert_celld_off -p edgezero-adapter-axum --no-default-features --features axum assert_celld_off -p edgezero-adapter-axum --no-default-features --features aws-stores,aws-dynamodb +assert_celld_off -p edgezero-adapter-axum --no-default-features --features aws-dynamodb-records +assert_no_sdk -p edgezero-adapter-axum --no-default-features --features celld-records + +aws_raw_features="$(provider_features edgezero-store-aws -p edgezero-adapter-axum --no-default-features --features aws-dynamodb)" +aws_record_features="$(provider_features edgezero-store-aws -p edgezero-adapter-axum --no-default-features --features aws-dynamodb-records)" +celld_raw_features="$(provider_features edgezero-store-celld -p edgezero-adapter-axum --no-default-features --features celld-kv)" +celld_record_features="$(provider_features edgezero-store-celld -p edgezero-adapter-axum --no-default-features --features celld-records)" +if grep -Eq '(^|,)dynamodb-records(,|$)' <<< "$aws_raw_features" || ! grep -Eq '(^|,)dynamodb-records(,|$)' <<< "$aws_record_features"; then + printf 'Explicit DynamoDB record feature forwarding failed.\n' >&2 + exit 1 +fi +if grep -Eq '(^|,)records-v2(,|$)' <<< "$celld_raw_features" || ! grep -Eq '(^|,)records-v2(,|$)' <<< "$celld_record_features"; then + printf 'Explicit celld record feature forwarding failed.\n' >&2 + exit 1 +fi celld_settings="$(cargo tree --offline --locked --edges normal --prefix none -p edgezero-store-celld --no-default-features)" if grep -Eq '^(reqwest|tokio|edgezero-core|aws-config|aws-sdk-[^ ]+) v' <<< "$celld_settings"; then printf 'Runtime/client dependency leaked into settings-only celld.\n' >&2 exit 1 fi -for feature in config kv; do +for feature in config kv records records-v2; do assert_no_sdk -p edgezero-store-celld --no-default-features --features "$feature" graph="$(cargo tree --offline --locked --edges normal --prefix none -p edgezero-store-celld --no-default-features --features "$feature")" if grep -Eq '^(celld(-[^ ]+)?|v8|worker|worker-build|edgezero-adapter-[^ ]+) v' <<< "$graph"; then printf 'Server/Worker/adapter dependency leaked into celld provider: %s\n' "$feature" >&2 exit 1 fi - selected="$(celld_features -p edgezero-store-celld --no-default-features --features "$feature")" - other=config - if [[ "$feature" == config ]]; then other=kv; fi - if ! grep -Eq "(^|,)$feature(,|$)" <<< "$selected" || grep -Eq "(^|,)$other(,|$)" <<< "$selected"; then + selected="$(provider_features edgezero-store-celld -p edgezero-store-celld --no-default-features --features "$feature")" + forbidden=config + case "$feature" in + config) forbidden='kv|records|records-v2' ;; + kv) forbidden='config|records|records-v2' ;; + records) forbidden='config|records-v2' ;; + records-v2) forbidden='config|records' ;; + esac + if ! grep -Eq "(^|,)$feature(,|$)" <<< "$selected" || grep -Eq "(^|,)($forbidden)(,|$)" <<< "$selected"; then printf 'Celld service feature isolation failed: %s: %s\n' "$feature" "$selected" >&2 exit 1 fi done +# Run the absolute Cloudflare no-Tokio requirement last so an existing SDK +# dependency does not hide coverage of the newly added record feature graphs. +assert_no_native_runtime -p edgezero-adapter-cloudflare --target wasm32-unknown-unknown --features cloudflare printf 'Native store SDK and celld feature isolation graphs passed.\n' diff --git a/scripts/run_native_aws_integration.mjs b/scripts/run_native_aws_integration.mjs index 42c79702..abc22b5b 100755 --- a/scripts/run_native_aws_integration.mjs +++ b/scripts/run_native_aws_integration.mjs @@ -142,6 +142,8 @@ export async function main() { console.log("Pinned local AWS services ready; running generated consumer contracts"); const env = { ...process.env, + // Generated builds and service tables have one command owner per suite. + RUST_TEST_THREADS: "1", TMPDIR: work, EDGEZERO_AWS_TEST_DYNAMODB_ENDPOINT: endpoints.dynamo, EDGEZERO_AWS_TEST_SECRETS_ENDPOINT: endpoints.moto, @@ -151,22 +153,40 @@ export async function main() { EDGEZERO_AWS_TEST_AGENT_TOKEN: "local-agent-token", }; // The nonzero runner rejects misspelled filters and accidental cfg exclusion. - await new Promise((resolve, reject) => { - testProcess = spawn(join(root, "scripts/run_test_nonzero.sh"), [ - "--ignored", "generated_native_aws_emulator_contracts", "cargo", "test", - "--offline", "--locked", "-p", "edgezero-cli", "--test", "native_aws_emulator_proof", - ], { cwd: root, env, stdio: "inherit", detached: true }); - testGroup = testProcess.pid; - let timedOut = false; - const timer = setTimeout(() => { timedOut = true; signalTest("SIGKILL"); }, 20 * 60_000); - testProcess.once("error", (error) => { clearTimeout(timer); reject(error); }); - testProcess.once("close", (code, signal) => { - clearTimeout(timer); - testProcess = undefined; - if (code === 0 && !timedOut && !interrupted) resolve(); - else reject(new Error(`Native AWS integration failed: ${timedOut ? "20 minute timeout" : interrupted ?? signal ?? `exit ${code}`}`)); + const suites = [ + { + name: "generated ordinary and record AWS consumers", + required: ["generated_native_aws_emulator_contracts", "generated_native_aws_emulator_contracts_records"], + args: ["--offline", "--locked", "-p", "edgezero-cli", "--test", "native_aws_emulator_proof"], + }, + { + name: "record provider against DynamoDB Local", + required: ["dynamodb_records_local_contract", "dynamodb_records_local_simulated_expiry_reopen"], + args: ["--offline", "--locked", "-p", "edgezero-store-aws", "--no-default-features", + "--features", "dynamodb-records", "--test", "dynamodb_records_local"], + }, + ]; + for (const suite of suites) { + checkInterrupted(); + console.log(`Running ${suite.name}`); + await new Promise((resolve, reject) => { + testProcess = spawn(join(root, "scripts/run_test_nonzero.sh"), [ + "--ignored", suite.required.join("\n"), "cargo", "test", ...suite.args, + ], { cwd: root, env, stdio: "inherit", detached: true }); + testGroup = testProcess.pid; + let timedOut = false; + const timer = setTimeout(() => { timedOut = true; signalTest("SIGKILL"); }, 20 * 60_000); + testProcess.once("error", (error) => { clearTimeout(timer); reject(error); }); + testProcess.once("close", (code, signal) => { + clearTimeout(timer); + testProcess = undefined; + if (code === 0 && !timedOut && !interrupted) resolve(); + else reject(new Error(`Native AWS integration failed: ${timedOut ? "20 minute timeout" : interrupted ?? signal ?? `exit ${code}`}`)); + }); }); - }); + signalTest("SIGKILL"); + testGroup = undefined; + } } catch (error) { failure = error; for (const container of Object.values(names)) { diff --git a/scripts/run_native_aws_integration.test.mjs b/scripts/run_native_aws_integration.test.mjs index 75b0ace6..0483079b 100644 --- a/scripts/run_native_aws_integration.test.mjs +++ b/scripts/run_native_aws_integration.test.mjs @@ -32,7 +32,7 @@ test("missing Docker fails instead of silently skipping the suite", async () => async function fixture(t, { failPull = false, port = 8000, emptySuite = false, hang = false, - suiteCode = 17, slowCleanup = false, + suiteCode = 17, slowCleanup = false, missingRecordSuite = false, missingLocalSuite = false, } = {}) { const directory = await mkdtemp(join(tmpdir(), "edgezero-runner-unit-")); t.after(() => rm(directory, { recursive: true, force: true })); @@ -51,9 +51,17 @@ case "$1" in esac `, { mode: 0o755 }); await writeFile(join(directory, "cargo"), `#!/bin/sh +printf 'cargo %s\\n' "$*" >> "$MOCK_DOCKER_LOG" case "$*" in - *--list*) ${emptySuite ? ":" : "printf 'generated_native_aws_emulator_contracts: test\\n'"} ;; - *) ${hang ? "mkdir \"$TMPDIR/generated-project\"; printf log > \"$TMPDIR/generated-log\"; printf 'runtime-start\\n' >> \"$MOCK_DOCKER_LOG\"; sleep 60" : `exit ${suiteCode}`} ;; + *native_aws_emulator_proof*) names='generated_native_aws_emulator_contracts${missingRecordSuite ? "" : " generated_native_aws_emulator_contracts_records"}' ;; + *dynamodb_records_local*) names='dynamodb_records_local_contract${missingLocalSuite ? "" : " dynamodb_records_local_simulated_expiry_reopen"}' ;; + *) names='' ;; +esac +case "$*" in + *--list*) ${emptySuite ? ":" : "for name in $names; do printf '%s: test\\n' \"$name\"; done"} ;; + *) ${hang ? "mkdir \"$TMPDIR/generated-project\"; printf log > \"$TMPDIR/generated-log\"; printf 'runtime-start\\n' >> \"$MOCK_DOCKER_LOG\"; sleep 60" : `if [ ${suiteCode} -ne 0 ]; then exit ${suiteCode}; fi +for name in $names; do printf 'test %s ... ok\\n' "$name"; done +printf 'test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s\\n'`} ;; esac `, { mode: 0o755 }); return { @@ -107,6 +115,25 @@ test("a zero-test suite fails the nonzero guard", async (t) => { assert.deepEqual(await readdir(setup.temporary), []); }); +test("generated record and Local provider cases are both mandatory", async (t) => { + for (const option of ["missingRecordSuite", "missingLocalSuite"]) { + const setup = await fixture(t, { port: await readyServer(t), suiteCode: 0, [option]: true }); + await assert.rejects(exec(process.execPath, [runner], setup.options), (error) => { + assert.match(error.stderr, /required sentinel test not listed/); + return true; + }); + } +}); + +test("successful runner executes generated consumers and both Local record cases", async (t) => { + const setup = await fixture(t, { port: await readyServer(t), suiteCode: 0 }); + await exec(process.execPath, [runner], setup.options); + const log = await readFile(setup.log, "utf8"); + assert.match(log, /cargo .*--test native_aws_emulator_proof/); + assert.match(log, /cargo .*--features dynamodb-records --test dynamodb_records_local/); + assert.deepEqual(await readdir(setup.temporary), []); +}); + async function interruptAt(setup, marker) { const child = spawn(process.execPath, [runner], { env: setup.options.env, stdio: "pipe" }); child.stdout.resume(); diff --git a/scripts/run_native_celld_integration.mjs b/scripts/run_native_celld_integration.mjs index a722633c..a236ae24 100644 --- a/scripts/run_native_celld_integration.mjs +++ b/scripts/run_native_celld_integration.mjs @@ -8,13 +8,14 @@ import { fileURLToPath, pathToFileURL } from "node:url"; const root = fileURLToPath(new URL("../", import.meta.url)); const sentinel = "generated_native_consumer_uses_embedded_celld_over_tls"; +const recordSentinel = "generated_native_consumer_uses_embedded_celld_over_tls_records_v2"; const inherited = [ "PATH", "HOME", "CARGO_HOME", "RUSTUP_HOME", "RUSTUP_TOOLCHAIN", "RUSTC", "RUSTDOC", - "RUSTFLAGS", "CARGO_BUILD_JOBS", "CARGO_TARGET_DIR", "EDGEZERO_GENERATED_TEST_TARGET_DIR", + "RUSTFLAGS", "CARGO_BUILD_JOBS", "CARGO_TARGET_DIR", "EDGEZERO_GENERATED_TEST_TARGET_DIR", "TMPDIR", ]; export function fixtureEnvironment(source, binary) { - const env = {}; + const env = { RUST_TEST_THREADS: "1" }; for (const name of inherited) { if (source[name] !== undefined) env[name] = source[name]; } @@ -103,12 +104,17 @@ export async function runIntegration({ log(`celld executable SHA-256 ${fingerprint}`); await command("bash", ["scripts/check_celld_scoped_isolation.sh", binary], { cwd, env: childEnv, timeoutMs: 60_000 }); await command("scripts/run_test_nonzero.sh", [ - "--ignored", sentinel, + "--ignored", [sentinel, recordSentinel].join("\n"), "cargo", "test", "--offline", "--locked", "-p", "edgezero-cli", "--no-default-features", "--features", "cli,edgezero-adapter-axum", "--test", "celld_generated_proof", sentinel, ], { cwd, env: childEnv, timeoutMs: 600_000 }); + await command("scripts/run_test_nonzero.sh", [ + "--ignored", "real_celld_records_v2_public_contract_races_and_lost_acknowledgement", + "cargo", "test", "--offline", "--locked", "-p", "edgezero-store-celld", "--no-default-features", + "--features", "records-v2", "--test", "records_v2_interop", + ], { cwd, env: childEnv, timeoutMs: 600_000 }); if (await digest(binary) !== fingerprint) throw new Error("celld executable changed during proof"); - log("Native generated celld TLS and disposable isolation proofs passed. Fleet acceptance remains separate."); + log("Native generated celld TLS, v2 records and disposable isolation proofs passed. Fleet acceptance remains separate."); } if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { diff --git a/scripts/run_native_celld_integration.test.mjs b/scripts/run_native_celld_integration.test.mjs index 81aa5d23..c46f3de4 100644 --- a/scripts/run_native_celld_integration.test.mjs +++ b/scripts/run_native_celld_integration.test.mjs @@ -25,9 +25,9 @@ function fixture(extra = {}) { test("fixture environment excludes deployment credentials, proxies and inherited trust inputs", () => { assert.deepEqual(fixtureEnvironment({ - PATH: "path", HOME: "home", CARGO_HOME: "cargo", RUSTUP_TOOLCHAIN: "1.95.0", CELLD_BIN: "ignored", + PATH: "path", HOME: "home", CARGO_HOME: "cargo", RUSTUP_TOOLCHAIN: "1.95.0", CELLD_BIN: "ignored", TMPDIR: "/fixture/tmp", RUST_TEST_THREADS: "8", AWS_ACCESS_KEY_ID: "secret", CELLD_APP_TOKEN: "secret", ALL_PROXY: "proxy", SSL_CERT_FILE: "ca", - }, binary), { PATH: "path", HOME: "home", CARGO_HOME: "cargo", RUSTUP_TOOLCHAIN: "1.95.0", CELLD_BIN: binary }); + }, binary), { RUST_TEST_THREADS: "1", PATH: "path", HOME: "home", CARGO_HOME: "cargo", RUSTUP_TOOLCHAIN: "1.95.0", TMPDIR: "/fixture/tmp", CELLD_BIN: binary }); }); test("missing binary is a failed prerequisite, never a passing skip", async () => { @@ -50,19 +50,23 @@ test("vanilla celld cannot masquerade as embedded API acceptance", async () => { test("isolation precedes the exact nonzero generated TLS proof", async () => { const { calls, options } = fixture(); await runIntegration(options); - assert.equal(calls.length, 3); + assert.equal(calls.length, 4); assert.equal(calls[0].command, binary); assert.deepEqual(calls[1].args, ["scripts/check_celld_scoped_isolation.sh", binary]); assert.equal(calls[2].command, "scripts/run_test_nonzero.sh"); assert.equal(calls[2].args[0], "--ignored"); const sentinel = "generated_native_consumer_uses_embedded_celld_over_tls"; - assert.equal(calls[2].args[1], sentinel); + assert.equal(calls[2].args[1], [sentinel, `${sentinel}_records_v2`].join("\n"), "both exact test sentinels are mandatory"); assert.equal(calls[2].args.at(-1), sentinel, "test filter must exclude the independent deterministic suite"); assert.ok(calls[2].args.includes("--offline")); + assert.equal(calls[3].command, "scripts/run_test_nonzero.sh"); + assert.equal(calls[3].args[1], "real_celld_records_v2_public_contract_races_and_lost_acknowledgement"); + assert.ok(calls[3].args.includes("records_v2_interop"), "native commit/lost-reply proof is mandatory"); for (const call of calls) { assert.equal(call.options.env.AWS_SECRET_ACCESS_KEY, undefined); assert.equal(call.options.env.HTTP_PROXY, undefined); assert.equal(call.options.env.SSL_CERT_FILE, undefined); + assert.equal(call.options.env.RUST_TEST_THREADS, "1", "generated builds and fixture state are serialized"); } }); From 1118aa1cdca9a1ea25c1b8225125965d29ad2582 Mon Sep 17 00:00:00 2001 From: Christian Date: Fri, 13 Nov 2026 20:05:05 -0600 Subject: [PATCH 6/9] Expose registry-aware Fastly delivery test seam --- crates/edgezero-adapter-fastly/src/request.rs | 38 ++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/crates/edgezero-adapter-fastly/src/request.rs b/crates/edgezero-adapter-fastly/src/request.rs index 9fb2eb44..454728e3 100644 --- a/crates/edgezero-adapter-fastly/src/request.rs +++ b/crates/edgezero-adapter-fastly/src/request.rs @@ -616,6 +616,42 @@ pub fn send_request_with_registries_and_hooks( where Prepare: FnOnce(&mut FastlyRequest, &mut Extensions), Finalize: FnOnce(&mut Response) -> State, +{ + request_with_registries_and_delivery(app, stores, req, prepare, finalize, send_egress_response) +} + +/// Exercises production registry construction with an injected delivery boundary. +#[cfg(any(test, feature = "test-utils"))] +#[doc(hidden)] +#[inline] +pub fn run_request_with_registries_and_hooks_for_test( + app: &App, + stores: StoresMetadata, + req: FastlyRequest, + prepare: Prepare, + finalize: Finalize, + deliver: Deliver, +) -> FastlyRequestDelivery +where + Prepare: FnOnce(&mut FastlyRequest, &mut Extensions), + Finalize: FnOnce(&mut Response) -> State, + Deliver: FnOnce(ResponseEgressEnvelope) -> FastlyResponseDelivery, +{ + request_with_registries_and_delivery(app, stores, req, prepare, finalize, deliver) +} + +fn request_with_registries_and_delivery( + app: &App, + stores: StoresMetadata, + req: FastlyRequest, + prepare: Prepare, + finalize: Finalize, + deliver: Deliver, +) -> FastlyRequestDelivery +where + Prepare: FnOnce(&mut FastlyRequest, &mut Extensions), + Finalize: FnOnce(&mut Response) -> State, + Deliver: FnOnce(ResponseEgressEnvelope) -> FastlyResponseDelivery, { with_callback(|| { init_fastly_abi(); @@ -636,7 +672,7 @@ where request_start, prepare, ); - finish_request(outcome, finalize, send_egress_response) + finish_request(outcome, finalize, deliver) }) } From 5e0e34ea95d24ef4af0e88c2dc70bb05499f4462 Mon Sep 17 00:00:00 2001 From: Christian Date: Mon, 16 Nov 2026 10:26:02 -0600 Subject: [PATCH 7/9] Check Tokio runtime features in portable dependency graphs --- .github/workflows/test.yml | 2 +- scripts/check_geo_dependency_contract.sh | 24 +--- .../check_native_store_dependency_contract.sh | 33 +++-- scripts/portable_dependency_checks.sh | 23 ++++ scripts/portable_dependency_checks.test.mjs | 127 ++++++++++++++++++ 5 files changed, 179 insertions(+), 30 deletions(-) create mode 100644 scripts/portable_dependency_checks.sh create mode 100644 scripts/portable_dependency_checks.test.mjs diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8c51a8b0..c53e7482 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -241,7 +241,7 @@ jobs: - name: Bundled demo logging and fallback TCP contracts if: matrix.suite == 'cli' run: | - node --test scripts/run_test_nonzero.test.mjs scripts/check_outbound_legacy_api.test.mjs + node --test scripts/run_test_nonzero.test.mjs scripts/check_outbound_legacy_api.test.mjs scripts/portable_dependency_checks.test.mjs scripts/run_test_nonzero.sh --include-ignored demo_fallbacks_enforce_body_cap_over_tcp cargo test --offline --locked -p edgezero-cli --features demo-example --test demo_logging - name: Run EdgeZero CLI tests diff --git a/scripts/check_geo_dependency_contract.sh b/scripts/check_geo_dependency_contract.sh index 2b3f89ba..a5939499 100755 --- a/scripts/check_geo_dependency_contract.sh +++ b/scripts/check_geo_dependency_contract.sh @@ -2,6 +2,8 @@ # Check isolated normal graphs, not workspace-unified or test dependencies. set -euo pipefail export LC_ALL=C +# shellcheck source=scripts/portable_dependency_checks.sh +source "$(dirname "${BASH_SOURCE[0]}")/portable_dependency_checks.sh" scratch="$(mktemp -d)" trap 'rm -rf "$scratch"' EXIT @@ -65,26 +67,14 @@ assert_portable() { local name="$1" root="$2" shift 2 graph "$name" "$root" "$@" - if grep -Eq '^(edgezero-adapter-axum|edgezero-store-(aws|celld)|axum(-[^ ]+)?|tokio-[^ ]+|reqwest|hyper(-[^ ]+)?|mio|socket2|ipnet|aws-config|aws-sdk-[^ ]+) v' "$scratch/$name"; then + if grep -Eq '^(edgezero-adapter-axum|edgezero-store-(aws|celld)|axum(-[^ ]+)?|reqwest|hyper(-[^ ]+)?|mio|socket2|ipnet|aws-config|aws-sdk-[^ ]+) v' "$scratch/$name"; then fail "native runtime/store/SDK leaked into $name" fi - # Pinned worker 0.8.5 already uses tokio 1.52.3 with default-features=false - # and no selected features on wasm32-unknown-unknown. That empty-feature - # dependency is not the native executor. Keep the exception exact so a worker - # upgrade or enabling runtime/net features requires a fresh isolation review. - if [[ "$name" == cloudflare ]]; then - grep -Eq '^worker v0[.]8[.]5[|]' "$scratch/$name" \ - || fail 'Cloudflare worker pin changed; review its empty-feature tokio exception' - grep -Eq '^tokio v1[.]52[.]3[|]$' "$scratch/$name" \ - || fail 'missing expected Cloudflare empty-feature tokio dependency' + if [[ "$name" == core ]] && grep -Eq '^tokio(-[^ ]+)? v' "$scratch/$name"; then + fail 'Tokio dependency leaked into portable core' fi - if ! awk -F '|' -v name="$name" ' - /^tokio v/ { - sub(/ [(][*][)]$/, "", $2) - if (name != "cloudflare" || $1 != "tokio v1.52.3" || $2 != "") exit 1 - } - ' "$scratch/$name"; then - fail "unexpected tokio version or native runtime features in $name" + if ! assert_wasm_tokio_features < "$scratch/$name"; then + fail "Tokio runtime/native-I/O features leaked into $name" fi } assert_portable core edgezero-core --no-default-features diff --git a/scripts/check_native_store_dependency_contract.sh b/scripts/check_native_store_dependency_contract.sh index 41a4e2ad..ed7e475d 100755 --- a/scripts/check_native_store_dependency_contract.sh +++ b/scripts/check_native_store_dependency_contract.sh @@ -1,6 +1,8 @@ #!/usr/bin/env bash # Independent normal dependency graphs; workspace --all-features is not isolation proof. set -euo pipefail +# shellcheck source=scripts/portable_dependency_checks.sh +source "$(dirname "${BASH_SOURCE[0]}")/portable_dependency_checks.sh" assert_no_sdk() { local graph @@ -12,19 +14,29 @@ assert_no_sdk() { } assert_no_native_runtime() { - local graph - graph="$(cargo tree --offline --locked --edges normal --prefix none "$@")" - if grep -Eq '^(aws-(config|sdk-[^ ]+)|reqwest|tokio|celld(-[^ ]+)?|v8) v' <<< "$graph"; then + local scope="$1" graph forbidden + shift + graph="$(cargo tree --offline --locked --edges normal --prefix none --format '{p}|{f}' "$@")" + forbidden='aws-(config|sdk-[^ ]+)|reqwest|celld(-[^ ]+)?|v8' + if [[ "$scope" == core ]]; then + forbidden+='|tokio(-[^ ]+)?' + fi + if grep -Eq "^($forbidden) v" <<< "$graph"; then printf 'Native runtime/client dependency leaked into core or WASM: %s\n' "$*" >&2 exit 1 fi + if ! assert_wasm_tokio_features <<< "$graph"; then + printf 'Tokio runtime/native-I/O features leaked into WASM: %s\n' "$*" >&2 + exit 1 + fi } -assert_no_native_runtime -p edgezero-core --all-features -assert_no_native_runtime -p edgezero-adapter-fastly --target wasm32-wasip1 --features fastly -assert_no_native_runtime -p edgezero-adapter-spin --target wasm32-wasip2 --features spin -assert_no_native_runtime -p edgezero-store-aws --target wasm32-wasip1 --no-default-features --features dynamodb-records -assert_no_native_runtime -p edgezero-store-celld --target wasm32-wasip1 --no-default-features --features records-v2 +assert_no_native_runtime core -p edgezero-core --all-features +assert_no_native_runtime wasm -p edgezero-adapter-cloudflare --target wasm32-unknown-unknown --features cloudflare +assert_no_native_runtime wasm -p edgezero-adapter-fastly --target wasm32-wasip1 --features fastly +assert_no_native_runtime wasm -p edgezero-adapter-spin --target wasm32-wasip2 --features spin +assert_no_native_runtime wasm -p edgezero-store-aws --target wasm32-wasip1 --no-default-features --features dynamodb-records +assert_no_native_runtime wasm -p edgezero-store-celld --target wasm32-wasip1 --no-default-features --features records-v2 assert_no_sdk -p edgezero-cli --no-default-features --features cli,edgezero-adapter-axum assert_no_sdk -p edgezero-store-aws --no-default-features @@ -105,7 +117,4 @@ for feature in config kv records records-v2; do exit 1 fi done -# Run the absolute Cloudflare no-Tokio requirement last so an existing SDK -# dependency does not hide coverage of the newly added record feature graphs. -assert_no_native_runtime -p edgezero-adapter-cloudflare --target wasm32-unknown-unknown --features cloudflare -printf 'Native store SDK and celld feature isolation graphs passed.\n' +printf 'Native store SDK, WASM runtime features and celld feature isolation graphs passed.\n' diff --git a/scripts/portable_dependency_checks.sh b/scripts/portable_dependency_checks.sh new file mode 100644 index 00000000..47ab80d1 --- /dev/null +++ b/scripts/portable_dependency_checks.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# Input: isolated normal dependency graph from cargo tree --format '{p}|{f}'. +# Tokio's async-I/O traits need no executor; reject runtime and native-I/O features. +assert_wasm_tokio_features() { + awk -F '|' ' + /^tokio v/ { + if (NF != 2) { + print "missing or malformed Tokio feature data: " $0 + bad = 1 + next + } + sub(/ \(\*\)$/, "", $2) + count = split($2, features, ",") + for (i = 1; i <= count; i++) { + if (features[i] ~ /^(fs|full|io-std|io-uring|net|process|rt|rt-multi-thread|signal|time)$/) { + print "Tokio runtime/native-I/O feature in " $1 ": " features[i] + bad = 1 + } + } + } + END { exit bad } + ' >&2 +} diff --git a/scripts/portable_dependency_checks.test.mjs b/scripts/portable_dependency_checks.test.mjs new file mode 100644 index 00000000..4bf29c92 --- /dev/null +++ b/scripts/portable_dependency_checks.test.mjs @@ -0,0 +1,127 @@ +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import test from "node:test"; + +const helper = fileURLToPath( + new URL("./portable_dependency_checks.sh", import.meta.url), +); + +function check(graph) { + return spawnSync( + "bash", + ["-c", 'source "$1" && assert_wasm_tokio_features', "bash", helper], + { encoding: "utf8", input: graph }, + ); +} + +for (const graph of [ + "edgezero-core v0.1.0|\n", + "worker v0.8.5|\ntokio v1.52.3|\n", + "worker v0.9.0|\ntokio v1.53.0|\n", + "tokio v1.52.3|io-util,macros,sync\n", + "tokio v1.52.3| (*)\ntokio v1.53.0|sync (*)\n", +]) { + test(`accepts portable graph ${JSON.stringify(graph)}`, () => { + const result = check(graph); + assert.equal(result.status, 0, result.stderr); + assert.equal(result.stdout, ""); + assert.equal(result.stderr, ""); + }); +} + +for (const feature of [ + "fs", + "full", + "io-std", + "io-uring", + "net", + "process", + "rt", + "rt-multi-thread", + "signal", + "time", +]) { + test(`rejects Tokio ${feature} in WASM`, () => { + const result = check(`tokio v1.52.3|io-util,${feature},sync\n`); + assert.equal(result.status, 1, result.stderr); + assert.match(result.stderr, /Tokio runtime\/native-I\/O feature/); + assert.ok(result.stderr.includes(feature), result.stderr); + }); +} + +test("checks every Tokio version, including duplicate-tree annotations", () => { + const result = check("tokio v1.52.3|\ntokio v1.53.0|rt (*)\n"); + assert.equal(result.status, 1, result.stderr); + assert.match(result.stderr, /tokio v1\.53\.0: rt/); +}); + +for (const [script, dependency, diagnostic] of [ + [ + "check_native_store_dependency_contract.sh", + "tokio v1.52.3|", + /dependency leaked into core/, + ], + [ + "check_native_store_dependency_contract.sh", + "aws-sdk-dynamodb v1.0.0|", + /dependency leaked into core/, + ], + [ + "check_geo_dependency_contract.sh", + "tokio v1.52.3|", + /Tokio dependency leaked into portable core/, + ], +]) { + test(`${script} still rejects ${dependency} in core`, (t) => { + const directory = mkdtempSync(join(tmpdir(), "edgezero-portable-graph-")); + t.after(() => rmSync(directory, { recursive: true, force: true })); + writeFileSync( + join(directory, "cargo"), + `#!/usr/bin/env bash +root='' +features='' +for argument in "$@"; do + if [[ "$argument" == geo-cloudfront ]]; then features=',geo-cloudfront'; fi +done +while [[ $# -gt 0 ]]; do + if [[ "$1" == -p ]]; then root="$2"; shift; fi + shift +done +if [[ "$root" == edgezero-adapter-axum ]]; then features="axum$features"; fi +printf '%s v0.1.0|%s\\n' "$root" "$features" +if [[ "$root" == edgezero-core ]]; then + printf '%s\\n' "$FAKE_PORTABLE_DEPENDENCY" +else + printf 'edgezero-core v0.1.0|\\n' +fi +`, + { mode: 0o700 }, + ); + const result = spawnSync( + "bash", + [fileURLToPath(new URL(`./${script}`, import.meta.url))], + { + encoding: "utf8", + env: { + ...process.env, + PATH: `${directory}:${process.env.PATH}`, + FAKE_PORTABLE_DEPENDENCY: dependency, + }, + }, + ); + assert.equal(result.status, 1, result.stderr); + assert.match(result.stderr, diagnostic); + }); +} + +for (const graph of ["tokio v1.52.3\n", "tokio v1.52.3|rt|extra\n"]) { + test(`rejects missing or malformed feature data ${JSON.stringify(graph)}`, () => { + const result = check(graph); + assert.equal(result.status, 1, result.stderr); + assert.match(result.stderr, /missing or malformed Tokio feature data/); + }); +} From 1a1fb6f561c0a9c9b38e06937909111ffdd4c8d9 Mon Sep 17 00:00:00 2001 From: Christian Date: Mon, 16 Nov 2026 10:29:11 -0600 Subject: [PATCH 8/9] Record feature-aware dependency validation receipts --- docs/superpowers/plans/2026-11-13-kv-record-contract.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/plans/2026-11-13-kv-record-contract.md b/docs/superpowers/plans/2026-11-13-kv-record-contract.md index fcbd8c52..b7d33c29 100644 --- a/docs/superpowers/plans/2026-11-13-kv-record-contract.md +++ b/docs/superpowers/plans/2026-11-13-kv-record-contract.md @@ -263,9 +263,11 @@ All commands use Rust `+1.98.1`, `--locked --offline`, task-local TMPDIR and iso - The first parallel workspace attempt exceeded an unchanged TCP test's205ms ceiling by0.263ms. Its isolated rerun and final serial workspace run passed without changing the timing bound. A later CLI test assumed non-TTY stdin; the corrected full invocation closes stdin with ` Date: Mon, 16 Nov 2026 17:54:01 -0600 Subject: [PATCH 9/9] Record deferred Viceroy validation gap without blocking integration --- docs/superpowers/plans/2026-11-13-kv-record-contract.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/plans/2026-11-13-kv-record-contract.md b/docs/superpowers/plans/2026-11-13-kv-record-contract.md index 8ab93667..80fbc7a8 100644 --- a/docs/superpowers/plans/2026-11-13-kv-record-contract.md +++ b/docs/superpowers/plans/2026-11-13-kv-record-contract.md @@ -11,9 +11,10 @@ contract and Add race, but both independent CAS writers reported success. The one-winner assertion remains unchanged and fails. Viceroy's KV implementation checks the existing generation and replaces the row under separate lock acquisitions. The SDK backend and record fixture are unchanged from the earlier -record tag. This candidate is not ready for SDK publication. A runtime repair -with its own regression proof or an approved real-platform acceptance route is -still needed; the earlier favorable race receipt does not clear this defect. +record tag. The user deferred this emulator issue so SDK and TS integration can continue. +Keep the failed check as a disclosed validation gap in final acceptance and +publication review. Do not repair the runtime in this effort, weaken assertions +or treat an earlier favorable race receipt as clearing this defect. ## Stock-service reconciliation checks