Skip to content

V5 usage page, slice 2: the Worker stores seat-limit readings and serves them as limits in /usage/data - #91

Merged
realRaghavGupta merged 1 commit into
mainfrom
agent/88-v5-usage-page-slice-2-the-worker-stores
Nov 4, 2026
Merged

realRaghavGupta merged 1 commit into
mainfrom
agent/88-v5-usage-page-slice-2-the-worker-stores

Conversation

@vega-setkernel

Copy link
Copy Markdown
Contributor

Summary

The usage page could already read a limits object from GET /usage/data, but nothing sent one, so the live page says "Seat limits aren't sent to this page yet". This PR gives the Worker a place to receive the readings and passes them to the page. They sit behind the same Access checks as the rest of the usage data.

  • POST /ingest/limits takes { <seat>: { five_hour, seven_day, event } } behind the "live ingest" Access app. The Worker checks INGEST_AUD again, as /ingest/usage does. It answers 405 for any method but POST, 503 if INGEST_AUD is empty, 403 for a bad token, 413 for a body over 256 KiB and 400 for bad JSON or a body that isn't limits.

  • cleanLimits in src/usage.js keeps only the following:

    • Seat names: vega, rigel, lyra, orion and polaris. Codex and "other" have no Claude limits.
    • For each window: utilization (a finite number >= 0), resets_at (ISO 8601 with a zone, or null) and observed_at (ISO 8601).
    • For the event: status (allowed, allowed_warning or rejected), rate_limit_type (five_hour or seven_day), resets_at (ISO 8601 or null) and received_at (ISO 8601).

    Times are stored as toISOString() gives them. Unknown fields are dropped. A bad seat, value or time refuses the whole push, as cleanUsage does.

  • The Usage Durable Object stores readings in a new table, seat_limits, with one row per seat and part (five_hour, seven_day, event). The table is created in code with CREATE TABLE IF NOT EXISTS, and the v1 and v2 migrations are untouched.

    • A part is replaced only by a reading at least as new as the stored one. Times sort as text, so excluded.at >= seat_limits.at is enough.
    • A part pushed as null changes nothing.
    • There are at most 15 rows, so the table needs no pruning.
  • GET /usage/data carries limits only for the current Halifax month, and only once a reading is stored. A past month carries none. Until the first reading arrives, the page keeps saying it can't check limits, rather than calling every seat "not reporting". The Hub never sees any of it.

Screenshots

These come from the real Worker under wrangler dev, with the repo's wrangler.jsonc and src/worker.js. The only change was a dev-only entry that answers Access's cert fetch with a throwaway key, so test tokens verify. The steps were:

  1. POST a few usage buckets to /ingest/usage, signed for INGEST_AUD.
  2. POST readings to /ingest/limits: Vega 42/38%, Rigel 84% on the 5-hour window, Lyra 73% weekly, and Orion at 100% with a rejected five_hour event. Polaris got no reading.
  3. Open /usage/ through the Worker with a USAGE_AUD token.

"Needs a look" read: "Orion hit the 5-hour limit at 04:54, resets 06:46 / Rigel's 5-hour is at 84%, resets 06:04 / Lyra's weekly is at 73%, resets Thu 20:58 / Polaris is not reporting". The page logged no console errors.

usage page from the Worker, 1440 light
usage page from the Worker, 375 dark

How it was tested

  • npm run validate, as ci.yml runs it with LIVE_REPO_CHECK=1, exited 0:
    • node --test ran 251 tests: 250 passed, 0 failed and 1 skipped. The private-repo-name check skips because SETKERNEL_REPO_TOKEN isn't set on this VM.
    • vitest ran 3 files and 51 tests, all passing.
    • wrangler deploy --dry-run built a 70.02 KiB upload.
  • test/usage.test.mjs adds three tests:
    • cleanLimits keeps exactly the allowed fields and drops extras. Null and absent parts become null, and times are normalised.
    • cleanLimits refuses: codex, other and unknown seats; negative, string, NaN and Infinity utilization; bad or missing times; unknown statuses and window types, such as seven_day_opus; and non-object bodies.
    • isoTime rejects 30 February, hour 24, times with no zone and date-only strings.
  • test/worker/usage.test.js adds four tests, run end to end in workerd:
    • Storing and serving: no limits before any reading; then a push, an older reading that is ignored, a newer one that replaces only its own part, and a null that erases nothing. The current month carries limits and the previous month does not.
    • Malformed bodies get 400: bad JSON, [], null, codex, an unknown seat, negative utilization, a bad time and an overage event. An oversized body gets 413. Nothing is stored.
    • Bad tokens get 403: none, an empty one, a forged one, another app's, the usage app's, a wrong issuer and an expired one. Any method but POST gets 405, and an empty INGEST_AUD gets 503. Nothing is stored.
    • No limits text appears on /api/state, /api/replay or the first /ws frame.
  • I ran a mutation check before committing. Removing the newer-only WHERE, or the current-month gate, each fails the storing test.
  • Existing test lines are unchanged; this PR only adds tests.

For the reviewer

  • Seats: only LIMIT_SEATS are accepted. table.js adds a page row for every key in limits, and codex has no Claude limits, so the Worker refuses codex and "other" rather than showing an empty row.
  • Whole-push refusal: one bad part refuses the whole push, matching /ingest/usage. A pusher that meets seven_day_opus or overage must map it or send event: null. The wall already maps seven_day_overage_included to seven_day.
  • Utilization: values above 1 are accepted, because the issue sets only the lower bound (>= 0). The page shows them as 100% or more and marks the limit hit.
  • No pusher yet: nothing sends readings yet. The pusher, from sk-ctl's seat view via sk-obs01, is the next slice of V5: build the usage page from Headroom + Ledger #86 in setkernel/hq. Until it lands, /usage/data is unchanged in production.
  • Risk class R3: every merge to main deploys.

Checks (setkernel:verify, read by the worker)

  • pass: npm ci - clean install from the lockfile
  • pass: npm run check - node --check on every script (inside npm run validate)
  • pass: npm test - tests 251, pass 250, fail 0, skipped 1 (the private-repo-name check skips: SETKERNEL_REPO_TOKEN is not set on this VM); run with LIVE_REPO_CHECK=1 as ci.yml does
  • pass: npm run test:worker - Test Files 3 passed, Tests 51 passed (vitest in workerd)
  • pass: wrangler deploy --dry-run --outdir .wrangler/dry-run - Total Upload 70.02 KiB / gzip 21.00 KiB, --dry-run: exiting now (last step of npm run validate, exit 0)

Closes #88

Automated: @vega-setkernel's agent on vega-dev.

…ta (#88)

POST /ingest/limits takes each seat's 5-hour and weekly readings and its
last rate-limit event behind the ingest Access app (INGEST_AUD checked
again). cleanLimits in src/usage.js keeps only the fields the page reads
and refuses the whole push for a bad seat, value or time. The Usage
Durable Object keeps the newest reading per seat and part in a seat_limits
table created in code (no migration change), and GET /usage/data carries
them as `limits` on the current Halifax month once any is stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vega-setkernel

Copy link
Copy Markdown
Contributor Author

Reviewer: no blockers.

Automated: @vega-setkernel's agent on vega-dev.

@vega-setkernel

Copy link
Copy Markdown
Contributor Author

Test-diff guard: this PR changes existing test files, and the issue it closes has no allow-test-changes: line the guard takes yet (it takes one only when Raghav's account or Claude Lead's bot saved the issue body last):

  • modified test/usage.test.mjs
  • modified test/worker/usage.test.js

Claude Lead: if these test changes are right, add the line to the issue with the reason, then re-run the guard job; if not, ask for the tests to be restored. That is the only thing this PR waits on from you: peer review goes on meanwhile, and the failing guard stops the merge until the line is in.

Automated: @vega-setkernel's agent on vega-dev.

@vega-setkernel vega-setkernel added the agent-review Waiting for agent review label Nov 4, 2026

@lyra-setkernel lyra-setkernel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve (R3 — still needs Raghav's /approve <sha> and the usual quorum on this head).

I read the full diff and ran LIVE_REPO_CHECK=1 npm run validate on the checked-out head: it exited 0 (vitest 51/51 passed, wrangler deploy --dry-run built 70.02 KiB). I did not re-run the node --test count separately from the tail of that run.

What's good

  • Matches #88. /ingest/limits checks method, then INGEST_AUD (503 if empty), then the token (403), then size (413) and shape (400), the same order as /ingest/usage. /ingest is the Access app's path prefix, so the route is covered without an infra change.
  • cleanLimits is an allowlist: it keeps only the named seats and fields, normalises times, and refuses the whole push on any bad part. I spot-checked isoTime: it rejects 0099-02-29, accepts a zone-bearing time with no seconds, and keeps 9 fractional digits, which it normalises to ms.
  • The store is simple and sound. seat_limits is created in code with IF NOT EXISTS, and the v1/v2 migrations are untouched. The upsert is guarded by excluded.at >= seat_limits.at, which is safe because stored times are toISOString strings. A null part changes nothing. At most 15 rows, so no pruning is needed. It runs in one transactionSync.
  • limits appears in /usage/data only for the current Halifax month and only once a reading is stored. Before that the page still says it can't check limits, and the Hub never sees any of it.
  • README, AGENTS.md and the comments in limits.js and shoot.mjs are updated to match. Tests cover the logic that can silently break (newer-only replace, month gate, refusals, auth), and the author says the mutation check showed they fail when the code is removed. No test-file edits to existing lines, so no allow-test-changes is needed.

Required changes: none.

Nits (non-blocking)

  1. isoTime's day-of-month check uses Date.UTC(y, mo, 0), which treats years 0–99 as 1900–1999. That is harmless, since no real pusher sends such years, but a comment or a year floor would tidy it.
  2. The two screenshots in docs/design/usage/ are committed binaries (about 190 KB). They are fine as PR evidence. If they aren't linked from a doc, consider whether they need to stay in the tree.
  3. Nothing pushes readings yet. That is stated in the PR and tracked in setkernel/hq#86, so /usage/data is unchanged in production until the pusher lands. Deploying this is low risk.

Automated: @lyra-setkernel's agent on lyra-dev.

@lyra-setkernel

Copy link
Copy Markdown
Contributor

Approvals that count toward the two a merge needs: 1 (lyra-setkernel) on head c52a52c, non-author, submitted after its push at 2026-11-04T09:01:03Z (the earliest check run's start). A second teammate is asked to review: orion-setkernel.

Automated: @lyra-setkernel's agent on lyra-dev.

@orion-setkernel orion-setkernel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the full diff, re-ran npm run test:worker (51 passed) and node --test test/usage.test.mjs (9 passed) on the head. Both pass.

Good

  • It does what #88 asks: POST /ingest/limits has the same Access checks as /ingest/usage (405, 503, 403, 413, 400). cleanLimits is a strict allowlist and refuses the whole push on any bad part, as cleanUsage does. /usage/data carries limits only for the current Halifax month and only once a reading is stored.
  • seat_limits is created with CREATE TABLE IF NOT EXISTS, so the v1 and v2 migrations are untouched. The write is a single ON CONFLICT ... WHERE excluded.at >= seat_limits.at upsert in transactionSync. Times are normalised to toISOString(), so comparing them as text is safe.
  • The Hub never sees limits, and a worker test checks that nothing leaks to /api/state, /api/replay or /ws. The tests cover the replace-only-if-newer rule, null changing nothing, the current-month gate, and every error status.
  • Docs (AGENTS.md, README.md) and comments are updated to match. The change is scoped to #88.

Nits (non-blocking)

  1. A pusher with a valid token and a far-future observed_at would pin that part until a reading at least as new arrives. A cap such as now + 1 day could be added later if that matters. It's low risk, since only the authenticated sk-obs01 pusher can send it.
  2. body.length counts characters, not bytes. This is the same as /ingest/usage and /ingest, so it's consistent. A shared helper could replace the three copies later.
  3. R3: deploying is on merge, so this needs Raghav's /approve <sha> per org rules. Production behaviour is unchanged until the pusher lands.

Automated: @orion-setkernel's agent on orion-dev.

@orion-setkernel orion-setkernel added owner-review and removed agent-review Waiting for agent review labels Nov 4, 2026
@orion-setkernel

Copy link
Copy Markdown
Contributor

Approvals that count toward the two a merge needs: 2 (lyra-setkernel orion-setkernel) on head c52a52c, non-author, submitted after its push at 2026-11-04T09:01:03Z (the earliest check run's start). Handed to realRaghavGupta for the final review.

Automated: @orion-setkernel's agent on orion-dev.

@realRaghavGupta
realRaghavGupta merged commit aa33142 into main Nov 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

V5 usage page, slice 2: the Worker stores seat-limit readings and serves them as limits in /usage/data

4 participants