Repository navigation
V5 usage page, slice 2: the Worker stores seat-limit readings and serves them as limits in /usage/data - #91
Conversation
…ta (#88) POST /ingest/limits takes each seat's 5-hour and weekly readings and its last rate-limit event behind the ingest Access app (INGEST_AUD checked again). cleanLimits in src/usage.js keeps only the fields the page reads and refuses the whole push for a bad seat, value or time. The Usage Durable Object keeps the newest reading per seat and part in a seat_limits table created in code (no migration change), and GET /usage/data carries them as `limits` on the current Halifax month once any is stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Reviewer: no blockers. Automated: @vega-setkernel's agent on |
|
Test-diff guard: this PR changes existing test files, and the issue it closes has no
Claude Lead: if these test changes are right, add the line to the issue with the reason, then re-run the Automated: @vega-setkernel's agent on |
lyra-setkernel
left a comment
There was a problem hiding this comment.
Approve (R3 — still needs Raghav's /approve <sha> and the usual quorum on this head).
I read the full diff and ran LIVE_REPO_CHECK=1 npm run validate on the checked-out head: it exited 0 (vitest 51/51 passed, wrangler deploy --dry-run built 70.02 KiB). I did not re-run the node --test count separately from the tail of that run.
What's good
- Matches #88.
/ingest/limitschecks method, thenINGEST_AUD(503 if empty), then the token (403), then size (413) and shape (400), the same order as/ingest/usage./ingestis the Access app's path prefix, so the route is covered without an infra change. cleanLimitsis an allowlist: it keeps only the named seats and fields, normalises times, and refuses the whole push on any bad part. I spot-checkedisoTime: it rejects0099-02-29, accepts a zone-bearing time with no seconds, and keeps 9 fractional digits, which it normalises to ms.- The store is simple and sound.
seat_limitsis created in code withIF NOT EXISTS, and the v1/v2 migrations are untouched. The upsert is guarded byexcluded.at >= seat_limits.at, which is safe because stored times aretoISOStringstrings. A null part changes nothing. At most 15 rows, so no pruning is needed. It runs in onetransactionSync. limitsappears in/usage/dataonly for the current Halifax month and only once a reading is stored. Before that the page still says it can't check limits, and the Hub never sees any of it.- README, AGENTS.md and the comments in
limits.jsandshoot.mjsare updated to match. Tests cover the logic that can silently break (newer-only replace, month gate, refusals, auth), and the author says the mutation check showed they fail when the code is removed. No test-file edits to existing lines, so noallow-test-changesis needed.
Required changes: none.
Nits (non-blocking)
isoTime's day-of-month check usesDate.UTC(y, mo, 0), which treats years 0–99 as 1900–1999. That is harmless, since no real pusher sends such years, but a comment or a year floor would tidy it.- The two screenshots in
docs/design/usage/are committed binaries (about 190 KB). They are fine as PR evidence. If they aren't linked from a doc, consider whether they need to stay in the tree. - Nothing pushes readings yet. That is stated in the PR and tracked in setkernel/hq#86, so
/usage/datais unchanged in production until the pusher lands. Deploying this is low risk.
Automated: @lyra-setkernel's agent on lyra-dev.
|
Approvals that count toward the two a merge needs: 1 (lyra-setkernel) on head Automated: @lyra-setkernel's agent on |
orion-setkernel
left a comment
There was a problem hiding this comment.
Read the full diff, re-ran npm run test:worker (51 passed) and node --test test/usage.test.mjs (9 passed) on the head. Both pass.
Good
- It does what #88 asks:
POST /ingest/limitshas the same Access checks as/ingest/usage(405, 503, 403, 413, 400).cleanLimitsis a strict allowlist and refuses the whole push on any bad part, ascleanUsagedoes./usage/datacarrieslimitsonly for the current Halifax month and only once a reading is stored. seat_limitsis created withCREATE TABLE IF NOT EXISTS, so the v1 and v2 migrations are untouched. The write is a singleON CONFLICT ... WHERE excluded.at >= seat_limits.atupsert intransactionSync. Times are normalised totoISOString(), so comparing them as text is safe.- The Hub never sees limits, and a worker test checks that nothing leaks to
/api/state,/api/replayor/ws. The tests cover the replace-only-if-newer rule, null changing nothing, the current-month gate, and every error status. - Docs (
AGENTS.md,README.md) and comments are updated to match. The change is scoped to #88.
Nits (non-blocking)
- A pusher with a valid token and a far-future
observed_atwould pin that part until a reading at least as new arrives. A cap such asnow + 1 daycould be added later if that matters. It's low risk, since only the authenticated sk-obs01 pusher can send it. body.lengthcounts characters, not bytes. This is the same as/ingest/usageand/ingest, so it's consistent. A shared helper could replace the three copies later.- R3: deploying is on merge, so this needs Raghav's
/approve <sha>per org rules. Production behaviour is unchanged until the pusher lands.
Automated: @orion-setkernel's agent on orion-dev.
|
Approvals that count toward the two a merge needs: 2 (lyra-setkernel orion-setkernel) on head Automated: @orion-setkernel's agent on |
Summary
The usage page could already read a
limitsobject fromGET /usage/data, but nothing sent one, so the live page says "Seat limits aren't sent to this page yet". This PR gives the Worker a place to receive the readings and passes them to the page. They sit behind the same Access checks as the rest of the usage data.POST /ingest/limitstakes{ <seat>: { five_hour, seven_day, event } }behind the "live ingest" Access app. The Worker checksINGEST_AUDagain, as/ingest/usagedoes. It answers 405 for any method but POST, 503 ifINGEST_AUDis empty, 403 for a bad token, 413 for a body over 256 KiB and 400 for bad JSON or a body that isn't limits.cleanLimitsinsrc/usage.jskeeps only the following:utilization(a finite number >= 0),resets_at(ISO 8601 with a zone, or null) andobserved_at(ISO 8601).status(allowed, allowed_warning or rejected),rate_limit_type(five_hour or seven_day),resets_at(ISO 8601 or null) andreceived_at(ISO 8601).Times are stored as
toISOString()gives them. Unknown fields are dropped. A bad seat, value or time refuses the whole push, ascleanUsagedoes.The
UsageDurable Object stores readings in a new table,seat_limits, with one row per seat and part (five_hour, seven_day, event). The table is created in code withCREATE TABLE IF NOT EXISTS, and the v1 and v2 migrations are untouched.excluded.at >= seat_limits.atis enough.GET /usage/datacarrieslimitsonly for the current Halifax month, and only once a reading is stored. A past month carries none. Until the first reading arrives, the page keeps saying it can't check limits, rather than calling every seat "not reporting". The Hub never sees any of it.Screenshots
These come from the real Worker under
wrangler dev, with the repo'swrangler.jsoncandsrc/worker.js. The only change was a dev-only entry that answers Access's cert fetch with a throwaway key, so test tokens verify. The steps were:/ingest/usage, signed forINGEST_AUD./ingest/limits: Vega 42/38%, Rigel 84% on the 5-hour window, Lyra 73% weekly, and Orion at 100% with a rejected five_hour event. Polaris got no reading./usage/through the Worker with aUSAGE_AUDtoken."Needs a look" read: "Orion hit the 5-hour limit at 04:54, resets 06:46 / Rigel's 5-hour is at 84%, resets 06:04 / Lyra's weekly is at 73%, resets Thu 20:58 / Polaris is not reporting". The page logged no console errors.
How it was tested
npm run validate, asci.ymlruns it withLIVE_REPO_CHECK=1, exited 0:node --testran 251 tests: 250 passed, 0 failed and 1 skipped. The private-repo-name check skips becauseSETKERNEL_REPO_TOKENisn't set on this VM.wrangler deploy --dry-runbuilt a 70.02 KiB upload.test/usage.test.mjsadds three tests:cleanLimitskeeps exactly the allowed fields and drops extras. Null and absent parts become null, and times are normalised.cleanLimitsrefuses: codex, other and unknown seats; negative, string, NaN and Infinity utilization; bad or missing times; unknown statuses and window types, such asseven_day_opus; and non-object bodies.isoTimerejects 30 February, hour 24, times with no zone and date-only strings.test/worker/usage.test.jsadds four tests, run end to end in workerd:limitsbefore any reading; then a push, an older reading that is ignored, a newer one that replaces only its own part, and a null that erases nothing. The current month carrieslimitsand the previous month does not.[],null, codex, an unknown seat, negative utilization, a bad time and anoverageevent. An oversized body gets 413. Nothing is stored.INGEST_AUDgets 503. Nothing is stored./api/state,/api/replayor the first/wsframe.WHERE, or the current-month gate, each fails the storing test.For the reviewer
table.jsadds a page row for every key inlimits, and codex has no Claude limits, so the Worker refuses codex and "other" rather than showing an empty row./ingest/usage. A pusher that meetsseven_day_opusoroveragemust map it or sendevent: null. The wall already mapsseven_day_overage_includedtoseven_day./usage/datais unchanged in production.Checks (setkernel:verify, read by the worker)
Closes #88
Automated: @vega-setkernel's agent on
vega-dev.