Repository navigation
chore(release): 0.24.0 — the enforcement path stops swallowing refusals - #120
Merged
Merged
Conversation
…low (SDK-1/2/3) ## Summary Three enforcement-path defects from RELIABILITY_AUDIT_2026-11-02 (FINDINGS/EVIDENCE §SDK-1, §SDK-2, §SDK-3). Two of them let money through unbooked; one of them let a tool call run with no answer from the box at all. ### SDK-1 — the in-process gate cache made chain steps 2..N free `runtime.py` kept a module-level `_GATE_CACHE` keyed `(workflow_id, chain_id, call_model, estimated_tokens)`. Three holes in one key: * no org id, no api-key id, no tool set — the cached answer belonged to no particular tenant or principal; * it re-captured call #1's server-minted `reservation_id`, so every chain step ran under ONE execution id. `consume_v3.lua` answers `IDEMPOTENT_REPLAY` for an already-consumed execution and books nothing — steps 2..N were never billed; * the tool-block check never ran for a cached step, so a rule that was added between step 1 and step 2 was not consulted. The cache is deleted. Every step is its own `/gate` call with its own reservation. The backend's own idempotency store (keyed by the SDK's `operation_id`) is the correct retry-dedup mechanism; a client-side cache that spans calls is not, because it spans principals. `check_workflow_budget` loses the `use_gate_cache` parameter, the `_GATE_CACHE` / `_GATE_CACHE_TTL_SECONDS` / `_invalidate_gate_cache_for_chain` machinery, and the `chain_end()` cache-drop. `instrumentation/auto.py` stops passing `use_gate_cache=False`. ### SDK-2 — edge `/execute` synthesised an allow `Transport.execute`'s edge branch returned a client-side `{"decision": "allow"}` without ever asking the box. The box's `/api/v1/edge/enforce` is the enforcement point (it decides AND charges in one call); `/execute` has no box endpoint, and re-asking would double-charge. The SDK now tracks the box's answer itself: `EdgeTransport` records it per event id (`_answers`, capped at 64), `take_answer()` POPS it, and `/execute` with no recorded answer returns `not_enforced()` — a block carrying `EDGE_NOT_ENFORCED`. This mirrors the cloud's 404 EXECUTION_NOT_FOUND for a direct `/execute` with no prior `/gate`. ### SDK-3 — the edge's unreachable answer was swallowed by the fail-open The runtime's fail-open is keyed only on `is_fallback_decision_source(source)`. The edge's unreachable case used `decision_source=FALLBACK`, so the runtime fail-OPENED and returned `None` — no block was reported and the call ran unpriced. The cloud's synthetic fallbacks are ADR-008-mandated to fail open (recoverable: `/track` still books). The edge's unreachable case is NOT recoverable — there is no cloud to book through — and ADR-067 §3 requires deny. Two behaviours, so two sources: the new `EDGE_UNREACHABLE` ("edge_unreachable") is not a fallback source, and the runtime honours the block. This is a deliberate deviation from the audit's literal wording ("`decision_source == FALLBACK` propagates"). The audit's own SDK-2 sketch says "Distinguish the sources", and the runtime already has a fail-open table keyed by source — a second source is the mechanism that table exists for. ### Regression found while fixing SDK-2 With the box actually consulted, the 2nd+ MCP tool call in edge mode had no box answer and would have been blocked. The MCP pre-flight now runs per tool call in edge mode, not only when no server-minted execution id is present. ## Verification | Check | Result | |---|---| | `pytest -q` (full suite) | **1952 passed, 12 skipped** | | `ruff check src tests` | All checks passed | | `mypy src/nullrun` | 4 errors, all `auto.py` Pregel overloads — identical on pristine HEAD (same venv interpreter); pre-existing | | Mutation harness (6 mutations: M1–M3 writer call sites, M4 edge invents an allow, M5 unreachable back to FALLBACK, M6 gate cache returns) | **caught 6/6, survived 0, inconclusive 0**, byte-exact restore | New/changed tests: * `tests/test_v3_wire_contract.py` — `TestNoInProcessGateCache` (no cache attribute, no `use_gate_cache` parameter, no dead env var can bring it back) and `TestChainStepsAreAdmittedIndividually` (3 chain steps ⇒ 3 `/gate` calls, 3 distinct server-minted reservations, every step carries the tool list). * `tests/test_edge_via_edge.py` — the unreachable answer is `EDGE_UNREACHABLE` and is NOT a fallback source; `/execute` with no recorded answer blocks with `EDGE_NOT_ENFORCED`. * `tests/test_status_backend_reachable_has_a_writer.py` — the source-count pin (`test_gate_paths_call_the_writer`) counted call sites and could not tell "the guard was removed" from "a duplicated arm was removed"; replaced with five behavioural tests that assert the property (each path records True/False). A strengthening, not a weakening. * `tests/test_zero_code_preflight.py` — the `use_gate_cache` assertions became "the parameter is not passed at all". ## Law impact None in the NULLRUN backend (this repo is the SDK). The backend's laws were not touched by this change. ## Gap The edge path has no live-box integration test here: `EDGE_LEASE` / `EDGE_UNREACHABLE` behaviour is proved against a fake box, not a real edge deployment. The backend's `edge_allow_v1.lua` idempotency ledger was read, not executed.
…(DEC-5/DEC-7) The backend now refuses an approval consume whose live tool is not a member of the tool set the approval covers (audit 2026-11-02, DEC-5 / DEC-7). Before this, a money approval bound only the AMOUNT — BusinessImpact::Money carries no tool name, tool_digest is NULL for non-MCP tools — so an approval for tool A authorised tool B. New wire code APPROVAL_TOOL_MISMATCH (HTTP 409) dispatches to NullRunApprovalToolMismatchError (NR-A017). 409, not 403: the approval is valid and unconsumed, it just covers a different tool, so the remedy is a FRESH approval rather than a retry. The backend derives its error-code list from GateErrorCode::all() and its own test suite panics when a code is missing from the SDK map, so the two halves must land together.
contracts/errors.ts was deleted in the backend repo (WIRE-8 / ADR-113): it was a fourth error dialect with zero importers, 16 lowercase-slug codes with 0 overlap against the 116 machine codes, a misspelled seats_limit_reached, and a status map wrong for 400 and 422. The envelope this module actually parses is the control-plane ApiErrorResponse - backend/src/proxy/http/errors.rs, ErrorSlug values. Docstrings only. No behaviour change.
…ape (SDK-4) The LangChain callback classified a gate refusal with a three-class allow-list (NullRunBudgetError, WorkflowKilledInterrupt, WorkflowPausedException). Every OTHER refusal the gate can issue fell into the broad `except Exception` arm, was logged at DEBUG and dropped: TOOL_BLOCKED -> NullRunToolBlockedError -> NullRunDecision RATE_LIMIT_EXCEEDED -> RateLimitError -> NullRunTransportError The agent carried on after the gate had refused the call, and the refusal was invisible to the application that is supposed to honour it. Three changes: 1. The family is named by BASE class — NullRunDecision, RateLimitError, WorkflowKilledInterrupt — so a refusal subclass added later cannot rot back into the fail-OPEN arm. RateLimitError is named explicitly because it is NOT a NullRunDecision: it descends from the transport family, so naming only NullRunDecision would still swallow the 429. 2. The refusal is re-raised, and the handler sets `raise_error = True`. langchain-core swallows a handler's exception by default (`raise_error: bool = False`) and re-raises it only when the flag is set — in BOTH the sync and the async dispatch path. Without the opt-in the refusal could not leave the callback at all. 3. `_only_a_gate_refusal_escapes` guards every callback method, so a gate refusal is the ONLY thing that can escape a NullRun callback. A cosmetic failure is still swallowed at debug — the contract of these callbacks is unchanged, and opting into propagation cannot break the host chain. The deferred stash stays as the SECONDARY channel: it is the one that survives a host framework which ignores `raise_error`, and the one that can abort on the @Protect boundary's own thread. ADR-008's fail-OPEN is unchanged: a transport failure (the gate could not be reached) is still swallowed. "The gate could not be reached" and "the gate said no" stay different categories. ADR-122.
… body (SDK-5) The control-plane WebSocket HMAC covers the bytes inside `signed_payload`, NOT the flattened outer wire body. The outer body is therefore writable by anyone who can inject bytes into the socket: they rewrite `type`, `outcome`, `execution_id` while keeping a captured (signed_payload, signature) pair. The signature check still PASSES — it verifies bytes the attacker never touched. The `approval_resolved` branch was the one dispatch branch that read the outer body, and it hands that dict to `on_approval_resolved`, which pops the pending approval and sets its threading.Event — releasing a gate that was parked waiting for a human. The fix: - ONE source of truth. The branch type and every field are read from the verified payload; the outer body is a fallback only for frames that could not be authenticated at all. - A signed frame whose payload does not parse is DROPPED. The outer body is not what was signed, so it is not a fallback source. - The privileged branch requires an AUTHENTICATED frame. An unauthenticated approval is refused, counted (`ws_unverified_approval_total`) and logged at ERROR; the waiter stays parked and fails closed on the approval timeout. - The safety direction is unchanged: an unsigned state_change (kill / pause) is still delivered, so the kill switch is not weakened. Proof: 9 failed / 4 passed before the fix, 13/13 after; five mutations, each red for its own reason (dispatch the outer body 4 red; drop the authentication guard 3 red; branch type from the outer body 1 red; fall through to the outer body 1 red; locals from the outer body 1 red). The whole SDK suite: 1987 passed, 12 skipped. ruff clean. mypy clean on the two touched modules.
The runtime already does the right thing. `_route_track` re-raises a `NullRunDecision` when the backend refuses to book the money (CONSUME_OVERBUDGET 422, a budget block 402) — ADR-008's /track row is CLOSED for enforcement rejections, so the refusal propagates to the caller. The auto-instrumentation layer then caught that refusal with a blanket `except Exception` and logged it at DEBUG. The application saw a perfectly ordinary LLM call while the backend had just refused to book it: a refused track looked like a SUCCESS. The money was not billed, the reservation was not settled, and nothing told the caller. The fix: - ONE family, named by BASE class, in `nullrun.breaker.exceptions`: `ENFORCEMENT_EXCEPTIONS = (NullRunDecision, RateLimitError, WorkflowKilledInterrupt)`. The langchain module had its own copy of the same family; two hand-written copies are two places to forget a refusal subclass, so the langchain module now sources the canonical one. - Every billable emission site the auto-instrumentation layer installs re-raises that family BEFORE the broad arm: the httpx sync and async transports, the OpenAI-Agents tracer, the requests transport, the crewai / autogen / llama_index integrations, and the LangChain callback's `on_llm_end` (whose inner blanket arm defeated the SDK-4 decorator — the refusal never even reached it). - The fail-OPEN posture is UNCHANGED for everything else: a transport failure (a dead backend) is still swallowed at DEBUG and the caller still gets its response. The `on_llm_end` refusal is also recorded for the `@protect` boundary, so a host framework that ignores `raise_error` still gets the decision on the boundary's own thread. - NOT changed: `_release_reservation` (a zero-token settle inside an error handler — raising there would mask the provider's own exception) and the two streaming-skipped ghost emissions (zero tokens, no money, and they sit inside the body-read try whose blanket arm would turn a raise into a spurious reservation release). Proof: 4 failed / 5 passed before the fix, 20/20 after; ten mutations, each red for its own reason (drop the enforcement arm at a site -> that site's test red, 8 sites; widen the arm to Exception -> the fail-OPEN control red; empty the family -> all 10 refusal tests red). The end-to-end case drives the REAL runtime behind the REAL transport against a mocked wire. Whole SDK suite: 2007 passed, 12 skipped (baseline 1987 + 20 new). `ruff check src tests`: clean. `mypy src/nullrun`: 5 errors, all pre-existing in instrumentation/auto.py and identical to the HEAD baseline (checked against a pristine `git archive` copy).
…K-7) The MCP path is the SDK's second enforcement path (the first is the @protect-decorated local function), and it went to /execute with NO business_impact and NO action_digest. That is not "the approval binds the tool name but not the arguments" — on the MCP path the approval could not be spent at all. /execute with an approval_id runs Step 9.5, which SERVER-DERIVES the action digest from the live business_impact (payload_binding.rs:163). With no envelope the derive returns Ok(None) and the orchestrator fails CLOSED with BUSINESS_IMPACT_INVALID (orchestrator.rs:2325, NR-010): the row was created, the operator approved it, and the re-execute was refused. The fix: the adapter derives ONE envelope from the call it is about to make — a tool_call impact carrying the tool name and the EXACT argument dict it forwards as the canonical bag — plus that envelope's digest, and puts both on the wire. The params MUST be that same dict: the backend compares BusinessImpact::ToolCall.params with the live tool_arguments canonically, so a masked or reshaped copy would bind the operator's approval to arguments nobody approved. - The envelope is installed on the call context before the pre-flight /gate runs, so the two hops hash the SAME bytes (ADR-065). An envelope already in scope (the @Protect path) is NOT overwritten — that path's own /gate and /execute must keep agreeing. - The contextvar is released in a finally, so the envelope cannot leak into the NEXT action. - When no envelope can be built (an f64 argument, a non-ASCII tool name — the digest layer cannot round-trip them), the digest still goes on the wire over the no_impact sentinel and the envelope stays ABSENT: the backend then takes its fail-CLOSED arm ("cannot recompute") instead of the legacy "no digest at all" arm, which is a grant bound to nothing. - A caller that brings its OWN envelope keeps it verbatim — the SDK cannot reconstruct the caller's envelope, so inventing a digest for it would be a claim the backend cannot verify. Proof: 20 new behavioural tests; seven mutations, each red for its own reason (no derivation -> 13 red; wrong params -> 9 red; envelope never installed -> the /gate-vs-/execute test red; leaked envelope -> the release tests red; existing envelope overwritten -> the not-overwritten test red; fallback sends no digest -> the fail-CLOSED tests red; caller envelope replaced -> the forwarding tests red). The digest is pinned against the BACKEND's golden hex, so the two repos cannot drift apart here without this going red. Whole SDK suite: 2027 passed, 12 skipped (baseline 2007 + 20 new). ruff check src tests: clean. mypy src/nullrun: 5 errors, all pre-existing in instrumentation/auto.py; zero in the touched module.
`ruff check src tests` was RED at HEAD: UP037 twice in `toolbox/mcp.py`, both introduced by the SDK-7 commit (776eb01). The quotes are unnecessary — the module has `from __future__ import annotations` and imports `BusinessImpact` at runtime — so removing them is a no-op at runtime and turns the SDK's own lint gate green again. The SDK's release gate runs `ruff check src tests`; a red gate in the tree is a red gate for the next release too.
`NULLRUN_SENSITIVE_FAIL_OPEN` was one global boolean. It was read in
exactly ONE place -- `decorators.py::_run_tool_policy_gate`, on the
local `@protect` path -- and turning it on opened EVERY protected
tool, whatever the tool was. The audit's failure, verbatim: "enabling
it for one tool disables enforcement for every tool".
The switch is still a whole-system switch, but what it is allowed to
open is now decided per tool by an explicit classification:
runtime.register_fail_open_tools(["charge_card"])
runtime.remove_fail_open_tool("charge_card")
runtime.get_fail_open_tools()
runtime.is_fail_open_allowed("charge_card")
The set is EMPTY by default, so the switch alone opens nothing and
every tool stays fail-CLOSED. The audit's verify clause ("enable it
and check a non-sensitive tool -- it must still fail closed") is
satisfied in its strongest form.
The classification is deliberately NOT the existing sensitivity
registry. That one answers "is this tool destructive?" -- the SDK's
built-in list of financial / messaging / admin operations, four of
them plain Python identifiers. Reusing it as the fail-open reach
would make the SDK's own list of destructive operations an allowlist
for running without a policy decision, and would invert a public
API's meaning.
The classification is CODE-level on purpose: an env var can be
inherited by a deploy pipeline or a CI environment, a code-level
classification cannot.
The MCP path is unchanged and still fails CLOSED through
`fallback_mode=STRICT`; it never consults the switch.
Also in this commit: the SDK's authoritative fail-OPEN/CLOSED table
named a gate (`_enforce_sensitive_tool`) that no longer exists, and
claimed the env switch was the opt-out for the permissive fallback --
false, the switch is never read there. The table is split into the
`@protect` path and the `/execute` path, and its dangling ADR-008
citation now points at ADR-063 section 2.4, the ledger that actually
governs it. The README's "What does fail open" section moves with it.
Verified: the whole SDK suite 2043 passed / 12 skipped (was 2027 /
12); `ruff check src tests` green; `mypy src/nullrun` 5 errors, all
in `src/nullrun/instrumentation/auto.py` -- the same five the SDK-6
baseline recorded, none in the modules this change touches. Five
mutations, each red for its own reason (the pre-fix behaviour, an
always-true lookup, a global classification, a missing snapshot
initialisation, and an MCP path that falls through to the server).
No wire change, no backend change, no money change.
…K-9) The audit named a subject that no longer exists. `_GATE_CACHE`, `_GATE_CACHE_TTL_SECONDS` and `_invalidate_gate_cache_for_chain` were deleted by the SDK-1/2/3 fix (87a4c2b) six hours AFTER the audit document was written, so its line numbers describe a tree that is gone. Its *Fix* line also cited a lock "the SDK already uses for the cache elsewhere" -- no such lock ever existed for that cache. The subject survives elsewhere, and this commit fixes it where it does. `EdgeTransport._answers` is a dict shared by every thread of the agent's tool calls, and the eviction walked it LIVE: self._answers[event_id] = answer while len(self._answers) > self._answer_cap: self._answers.pop(next(iter(self._answers))) `iter(d)` and `next()` are two bytecodes apart. A thread that resizes the dict in between makes CPython raise `RuntimeError: dictionary changed size during iteration` -- and the eviction's `pop` raises `KeyError` when another thread took the key first. That is more than a crash. The exception escapes `enforce`, and the gate's fail-OPEN arm catches it and returns "no decision" -- and "no decision" is read by the caller as "no block". So the defect is a tool call that runs with NO policy decision at all, while the box has already charged the lease. The fix is the audit's Fix line, both clauses: the shared dict is read and written under a lock, and the eviction copies the key list before iterating it. self._answers_lock = threading.Lock() `take_answer` pops under the lock. `_record` inserts and evicts under it, and picks the victim from a snapshot (`keys = list(self._answers)`) rather than from a live iteration. The cap is now an EXACT promise: before, two threads could both decide to evict and drop two entries for one insert -- and an answer evicted too early is a refusal, so an over-eager eviction turns a paid-for call into a spurious block. Also measured, and left alone on purpose: `list(d.keys())` is atomic under the GIL (no GIL release inside the C call), so `runtime.py`'s poller snapshot is safe. A Python-level `for k in d:` or a split `next(iter(d))` IS racy. `actions.py::_webhooks` is a second, non-enforcement instance of the same shape -- filed as a residual, not fixed here. Verified: the new file `tests/test_sdk9_shared_state.py` 6/6; the whole SDK suite 2049 passed / 12 skipped (was 2043 / 12); `ruff check src tests` green; `mypy src/nullrun` 5 errors, all in `src/nullrun/instrumentation/auto.py` -- the same pre-existing five, none in the module this change touches. The race is a two-bytecode race, so the test has to make it land: CPython only preempts a thread at the switch interval (5 ms by default) and 160,000 records across 8 threads produced ZERO hits at that interval, while `sys.setswitchinterval(1e-5)` reproduces it in well under a second. The first version of this file was VACUOUS -- 6/6 green against the unfixed code -- until that fixture was added. Pre-fix the file is 5 red / 1 green (the green one is the audit's literal Verify clause, which is green before and after by design: the track path touches no shared state at HEAD). Four mutations, each reverted byte-identically: M1 pre-fix 5 red; M2 copy-without-lock 6 green; M3 lock-without-copy 6 green; M4 take_answer's lock removed 6 green. So the tests pin the CRASH, not the mechanism -- either clause of the audit's Fix delivers it. Both are kept, because the lock is the property that survives a future change of the eviction's shape. No wire change, no backend change, no money change.
The SDK-9 fix was committed with a docstring that named the wrong consequence: "a tool call that runs with NO policy decision". The measurement says the opposite. The crash lands in the gate's bare `except Exception` fail-OPEN arm, so a REFUSED box becomes "nothing was decided" and the caller PROCEEDS. Measured on the real LLM-span path (NullRunCallback.on_llm_start, 96 threads x 6 calls, a box that refuses every time): 35-43 of 576 LLM calls proceeded on a refused lease before the fix, 0 after. The @Protect tool path does NOT lose it — the crash happens after the answer is stored, so the /execute half reads the refusal back and blocks (576/576, before and after). The loss is only on a pre-flight with no /execute behind it. Two tests added: * test_a_refused_box_never_becomes_a_failed_open — pins the money invariant directly. Red before the fix, with the failure message naming the loss. * test_the_eviction_never_drops_a_live_answer_first — pins the eviction's ORDER. Evicting the newest instead of the oldest passed every other test in the file, so the order was unpinned; and the order decides whether a live answer is dropped before its /execute half consumes it. The module docstring is corrected to match. Verified: 8/8 in the file; 2051 passed, 12 skipped for the whole suite; ruff clean; mypy shows the same 5 pre-existing auto.py errors and none in the module this touches. ADR-127 in the NULLRUN repo records the measurements.
The capture dropped a `reservation_id` that is not a UUID with a warning and returned None. That reads "the server reserved nothing" when the truth is "the server reserved something I cannot bind" — so the call proceeded with NO reservation, the /track event had no execution_id to carry, the cost event was DROPPED, and the agent got the call for free. The backend's /gate Allow arm always carries the Step-9 reserve's server-minted uuidv7 (and /execute reuses the same id, ADR-073), so a real answer never lands here. Two changes, because one is not enough: 1. The capture REFUSES a present-but-unusable id with NullRunMalformedGateResponseError — the class the SDK already uses for "something answered, and what it said was not a verdict". An ABSENT id stays clear-and-None: a block reserves nothing, and that is a fact, not a fault. 2. That refusal must REACH the caller. The instrumentation's blanket arms re-raise the enforcement family and swallow everything else, so the malformed-answer class joins ENFORCEMENT_EXCEPTIONS. Without (2) the refusal is raised and then swallowed one layer up and the LLM call proceeds anyway — the same hole. This also closes the sibling hole: a non-JSON /gate body already raised, and was swallowed on the LLM-span path. ADR-008's fail-OPEN posture is untouched: an unreachable gate still fails open. An ANSWER the SDK cannot use is the opposite case. The two tests that pinned the old warn-and-drop behaviour are STRENGTHENED, not removed: they now assert the refusal. Proved red-then-green by two mutations: the capture dropping again (12 failures, incl. the provider request going out unbilled) and the class removed from the enforcement family (5 failures, the refusal swallowed by the langgraph blanket arm).
…on (SDK-11)
The audit's words: "the same check is implemented twice with different
handlers... the fail-open path is the simpler one, so it is the one
users hit first." The two arms were not "chain" and "single-shot" --
they were the CACHED branch's narrow arm and the UNCACHED branch's
broad one, inside ONE handler. SDK-1 deleted the cache, so the narrow
twin is gone and what is left is the BROAD arm: a bare
`except Exception` that logged, counted gate_fail_open_total, and
returned None.
None is "no decision, carry on". Every caller reads it as ALLOW.
So the call ran with NO reservation: the backend booked nothing, the
cost event was dropped, and the agent got the call for free -- the
SDK-10 hole one class away.
Three facts were conflated. They are now separated by TYPE:
* the gate NAMED a refusal -> honour it. The enforcement family is
re-raised BEFORE everything else, and the ordering is load-bearing:
RateLimitError IS a NullRunTransportError, so a later transport
arm would swallow the 429 and fail OPEN on it.
* the gate could not be REACHED -> ADR-008's fail-OPEN, and only
that (httpx.HTTPError / NullRunTransportError).
* anything else -> the SDK's OWN code faulted while asking. The
decision is INDETERMINATE, and an unanswered question is not a
yes -> NullRunGateIndeterminateError (NR-P004).
The new class joins ENFORCEMENT_EXCEPTIONS so the instrumentation's
blanket arm re-raises it instead of swallowing it. The langgraph
callback swallows everything that is not the family, so without the
membership the refusal is raised and then thrown away one layer up and
the LLM call proceeds anyway. It derives from
NullRunInfrastructureError, NOT NullRunTransportError, so ADR-126's
two-step opt-in cannot convert it into "the body runs".
A new counter, gate_indeterminate_total, because this is NOT a
fail-open: the call is stopped. Counting it as gate_fail_open_total
would send an operator to the backend dashboard when the fault is in
the SDK.
Proved red-then-green by two mutations: the arm failing open again
(10 failures, incl. the provider request going out unbilled) and the
class removed from the enforcement family (4 failures -- the refusal
swallowed by the langgraph blanket arm).
21 new tests: the refusal is typed and counted, the provider request
never leaves, the @Protect body never runs, the refusal reaches the
caller through the real LangChain callback, a transport failure still
fails OPEN, and a refusal the gate NAMED is still honoured.
New error code NR-P004 is documented per the repo's own procedure:
catalogue row plus a page under docs/errors/.
…opped (SDK-12) The action dispatcher logged an unknown action type at ERROR, recorded a forensic history event, and RETURNED. The action was then neither executed nor refused, while the caller's own bookkeeping said it had been handled: a control-plane instruction that is silently dropped is a control-plane instruction that does not exist. - Unknown action type now RAISES NullRunUnknownActionError (NR-P005) — an infrastructure error, NOT a block, so B14's anti-DoS property survives: the workflow keeps running, the action does not. - The refusal is not a transport error, so ADR-126's two-step opt-in cannot convert it into "the body runs". - `except BaseException` narrowed to `except Exception`: a Ctrl-C or a SystemExit reaches the caller again. - The handler's own refusal (kill / pause / block) is now logged at INFO, not ERROR: a delivered kill is the action working, not a fault (every server-side kill used to raise a false alarm). - The /track/batch call site reports the unknown action as a control-plane integrity event at ERROR instead of filing it under "skipped malformed action". - The false docstrings are corrected: the gate, not this handler, is what stops a killed workflow. Proof: 22 new/rewritten tests; 5 mutations (no raise / block instead / BaseException / refusal arm removed / transport arm reverted) each red for the expected reason. Suite 2121 passed / 12 skipped; ruff clean; mypy adds no new error.
…K-13) The decorator's catch-all turned a fault in the SDK's OWN code into a refusal that wore the network code NR-B001 and the policy-block type. The FastAPI integration mapped that through the decision table and rendered HTTP 429 / category "decision": the operator was told the policy refused (or rate-limited) when the truth is that the SDK has a bug. Under the two-step opt-out the same arm RETURNED, so the body ran with no policy decision at all. - The catch-all now raises NullRunGateIndeterminateError (NR-P004) -- the /execute twin of ADR-129's /gate class: an infrastructure error, not a decision, so a cookbook branch on `except NullRunDecision` can no longer read an SDK bug as a verdict. - The two-step opt-out no longer opens an SDK-internal fault. It still opens an unreachable engine (the documented dev/test bypass is untouched); what reaches the catch-all is by construction NOT a transport failure, because /execute converts every transport failure into NullRunTransportError, which the arm above owns. - The refusal is counted (`policy_gate_indeterminate_total`, a counter of its own -- it is an SDK incident, not a backend outage), logged at ERROR with the tool name, and carries the original exception as its cause. - NR-P004 gets an operator message: it had none, so the one failure that STOPS the call rendered as "Something went wrong". - Three counters that were counted but never exported in to_dict() are now exported -- a counter the operator cannot read is invisible. Proof: 17 new tests; red-before on the pristine tree (12 failed / 5 passed -- the 5 are the controls); 7 mutations (re-typed as a decision / fail-open arm restored / counter dropped / log dropped / raise->return / message dropped / export dropped) each red for the expected reason and reverted byte-identically. Suite 2138 passed / 12 skipped; ruff clean; mypy adds no new error (the 5 in instrumentation/auto.py are pre-existing).
The error-code catalogue described NR-P004 as a /gate-only refusal. After SDK-13 the same class is raised on the /execute seam, and the page would have been stale: it named only `gate_indeterminate_total`, and it did not say that the /execute seam used to wear NR-B001 and a policy -decision type (HTTP 429 / category "decision" on FastAPI). Doc-only: no code, no test, no behaviour change.
…K-14)
The SDK's own tests re-implemented the production predicate.
`tests/test_langchain_enforcement.py` spelled out its own `ENFORCEMENT`
tuple (three classes) and asserted `issubclass` against the production
family — one direction only.
Measured on the pristine tree: with the production family WIDENED by
one member, the whole suite stayed GREEN (2138 passed / 12 skipped).
The production predicate changed and the suite noticed nothing — the
hole the audit names, measured, not argued.
What the file holds now:
* the hand-written tuple is gone;
* `_CASE_BUILDERS` holds CONSTRUCTORS (each exception class has its
own signature), not the predicate;
* `TestTheFamilyAndTheCasesCorrespond` compares the production family
with the cases actually driven, in BOTH directions: a family member
nobody drives is red (widening), and a case the family no longer
covers is red (narrowing);
* the instrumentation's alias must BE the production tuple, so its
`except ImportError` fallback (an empty tuple = every refusal
downgraded to a transport error and swallowed) can never be in
effect;
* `TestEveryCaseIsDriven` drives every case through the REAL callback
(sync CallbackManager, async AsyncCallbackManager, the direct call)
and asserts the refusal ESCAPES and was RECORDED for the @Protect
boundary — `issubclass` cannot see a refusal that was swallowed;
* the transport control: a failure the family does not name must
still fail OPEN (ADR-008) — otherwise "widen the family" would
freeze every agent on a backend outage.
Mutation proof — production mutated, full suite run, file restored
byte-identically (SHA-256 verified, `git restore` not used):
M1 widen the family -> OLD suite GREEN (2138/12)
NEW suite RED (1 failed, names the
uncovered member)
M2 narrow the family -> 8 failed
M3 drop the `raise` -> 40 failed
M4 family arm -> Exception -> 2 failed
M5 alias -> () -> 46 failed
Suite: 2164 passed / 12 skipped. ruff: All checks passed. mypy: the
same 5 pre-existing errors in instrumentation/auto.py, none new.
`EdgeTransport.last_source` was the single constant `EDGE_LEASE` on BOTH
the allow path and the refusal path, so the two collapsed into one
string. A caller reading the attribute could not tell which way the box
went: it said "the lease spoke" when the truth was "the lease said no".
The audit names SDK-2 as the twin. SDK-2's first half -- the edge branch
of `Transport.execute` returning `{"decision":"allow"}` without calling
the box -- is a separate P0 and is NOT what this commit touches. This
commit is the `last_source` half: the provenance the SDK reports about
an answer, after the answer itself became correct.
The decision was never wrong. The refusal path already returned
`decision: "block"` with the box's own `error_code`. What was wrong is
the provenance reported alongside it.
What changed:
* `EDGE_LEASE_REFUSED` is a new named marker for "the box answered,
and the answer was NO";
* the refusal path writes it into `last_source`;
* the allow path keeps `EDGE_LEASE`;
* `EDGE_UNREACHABLE` (no answer) and `EDGE_NOT_ENFORCED` (nobody
asked) keep their own markers.
The wire value is deliberately UNCHANGED. `decision_source` on the
refusal path stays `EDGE_LEASE`: the lease was the authority that spoke
either way, and `decision` says which way it went. Changing it would
need a new name in the runtime's known-source set, and until the
runtime knows it, a body carrying it is a MALFORMED response -- a clean
refusal would become a crash. The new constant is a `last_source`
marker only and never reaches the wire.
The new marker is also NOT a fallback source (`is_fallback_decision_source`
is False for it): a box that said no is a VERDICT, not a degraded
transport. Reading it as synthetic would discard the refusal before the
caller saw it -- the same hole SDK-3 closed for the other edge sources.
Verification -- `tests/test_p2_17_edge_last_source.py` (9 cells):
* both paths are driven for real (a box that says yes, a box that says
no) and the attribute must tell them apart; the decisions are
asserted FIRST, so a test that only checks provenance cannot be
satisfied by an implementation that stops deciding;
* the refusal source is a NAMED exported constant, and all four edge
sources are distinct;
* the refusal still carries `EDGE_LEASE` as its wire source, still
parses through the runtime's `_require_gate_decision` as a `block`,
and is still not read as a synthetic answer;
* the neighbouring sources did not move (unreachable, not enforced).
Mutation proof -- production mutated, file restored byte-identically
(md5 `ec21ee41c232656085ec44e24a920ed1` before and after, `git
restore` not used):
M1 refusal writes EDGE_LEASE again -> 2 failed / 7 passed
(the collapse assertion and the named-constant assertion; the
other seven stayed GREEN, they assert the neighbours did not move)
Suite: 2173 passed / 12 skipped (2164 / 12 before this commit, +9).
ruff: All checks passed. mypy on the changed file: clean; the suite-wide
run reports the same 5 pre-existing errors in instrumentation/auto.py,
none new.
No wire-format change, no version bump, no release.
`MCPAdapter.call_tool` ran `runtime.check_workflow_budget()` only when
`get_server_minted_execution_id()` was None. That contextvar is set by
/gate and by /execute and is NEVER reset in `src/`, so the second and
later MCP tool calls in one scope ran NO /gate: no budget check, no
reservation, no spend reserved for those calls. Both /execute bodies
also carried the FIRST call's execution id, so the second call settled
against a reservation that was already spent.
Measured on the pristine tree (tests/test_p2_18_mcp_preflight_scope.py):
two tool calls -> ONE /gate ("the second tool call ran no /gate"), and
both /execute bodies carrying the SAME execution id.
The condition could not simply become "always pre-flight". Two cases
legitimately skip it, and both are about THIS action:
* the approval re-entry -- the backend's require_approval arm echoes
the approval row's execution id into `reservation_id`
(backend/src/enforcement/gate_wire_adapter.rs), so the ambient id IS
the approval's execution. A second /gate would mint a DIFFERENT id
and the approval's consume (`WHERE execution_id = $3`) would miss its
row (APPROVAL_REPLAY_REJECTED).
* the @Protect scope -- the scope reserved for itself, and a second
/gate inside it would reserve twice and orphan the first (NR-059).
So the pre-flight runs unless the ambient reservation is known to
cover THIS action: `approval_id is not None` (the re-entry) or an
envelope was already in scope (the @Protect scope). The envelope marker
is read BEFORE the adapter installs its own envelope -- otherwise every
call would read as "already gated".
Via-edge keeps asking the box for every tool call: there the box ask IS
the charge, and the /execute reads back the box's answer for ITS event
id.
What the tests hold now:
* two tool calls -> two /gate, two distinct execution ids, order
gate/execute/gate/execute;
* the approval re-entry and the @Protect scope still run NO second
/gate;
* the edge asks the box even when the scope is covered.
`test_the_preflight_is_skipped_when_a_gate_already_ran` pinned the
defective contract (an ambient execution id alone skipped the
pre-flight). It is replaced by three cells that state the corrected
contract and the two skips it must not lose.
Verification:
* pytest -q -> 2181 passed / 12 skipped / 0 failed;
* ruff check src tests -> clean;
* mypy src/nullrun -> the same 5 pre-existing errors in
instrumentation/auto.py, none new (measured on a pristine HEAD
copy);
* MUTATION-A (the pre-fix condition) -> the 4 defect cells go red;
* MUTATION-B (always pre-flight) -> the 5 skip cells go red.
No wire-format change, no version bump, no release.
… (P2-19)
The WebSocket receiver verified the signature only when all four of
`signature`, `timestamp`, `api_key` and `secret_key` were present. A
frame missing any one of them skipped the verification block entirely
and was then read from its outer body -- the body the server never
signed.
A client that holds the API key and the HMAC secret can verify every
frame the server sends it: the server signs every frame it sends for
such a key (`backend/src/proxy/http/ws_control.rs::send_signed_or_raw`).
A frame that arrives at such a client unsigned -- or with a signature
but no timestamp -- did not come from the server, so it is refused
before any branch reads a field out of it. The refusal is counted
(`ws_unverified_frame_total`) and logged. An approval frame keeps
bumping `ws_unverified_approval_total`, so the operator still sees that
somebody pushed an approval this client cannot verify.
A client WITHOUT a secret (a legacy key) can verify nothing, so the
unsigned path stays open for it: the kill switch must keep working
there, and the privileged approval branch already refuses
unauthenticated frames on its own.
A refused frame also cannot poison the per-workflow version dedup, so a
forged frame cannot silence the next real kill.
7 new cells drive the real `WebSocketConnection._handle_message`.
Verification:
* pytest -q -> 2188 passed / 12 skipped / 0 failed;
* ruff check src tests -> clean;
* mypy src/nullrun -> the same 5 pre-existing errors in
instrumentation/auto.py, none new (measured on a pristine HEAD
copy);
* MUTATION-A (no refusal) -> the 5 defect cells go red;
* MUTATION-B (refuse every unsigned frame, even for a client that
cannot verify) -> the kill-switch cell goes red;
* MUTATION-C (refuse every frame for a client with a secret) -> the
signed-kill canary and the dedup cell go red.
No wire-format change, no version bump, no release.
The server's own refusal frame is unsigned BY DESIGN: it is sent before the connection is authenticated, so there is no secret to sign it with yet. P2-19's refusal drops such a frame, so the operator saw the refusal but not the server's reason. The refusal log now names the frame's code as well as its type. The code is read from the UNSIGNED outer body, so it is a hint, not a source of truth -- the log line says the frame was refused, and the code only tells the operator what the server claimed. One new cell drives the real `_handle_message` and reads the log: 2189 passed / 12 skipped / 0 failed; ruff clean. No wire-format change, no version bump, no release.
Cause: `time.sleep` is ONE attribute on ONE module object. The suite patched that attribute in three places — the conftest cap, and the same attribute again through `nullrun.transport.time` and `nullrun.breaker.circuit_breaker.time`, which ARE the stdlib module (measured: `nullrun.transport.time is time` is True) — and the retry tests patched it a fourth time through `nullrun.transport.time.sleep`. A patch on that attribute captures the sleep of EVERY thread in the process, so the webhook-delivery singleton's 0.5s idle poll lands in a retry-delay list and breaks the floor assertion for a reason that has nothing to do with the retry path. The webhook tests already documented that flake for their own module. Fix: the transport gets a `_sleep` seam next to `_wait_on`, and the retry tests patch the seam instead of the global. The conftest keeps ONE patch site for the global (the 1ms cap) and stops repeating it through the module aliases: no module in src/ or tests/ does `from time import sleep`, so the single patch already reaches every call site. Verified: - new test `test_another_threads_sleep_does_not_land_in_the_retry_delays` — RED before the fix (a foreign 0.5 landed in the delays), GREEN after - mutation A (the test patches the global again) — RED - mutation B (the seam bypassed) — 8 RED in the file - pytest -q — 2190 passed, 12 skipped (baseline on HEAD: 2189 / 12) - ruff check src tests — clean - mypy src/nullrun — the same 5 pre-existing errors in src/nullrun/instrumentation/auto.py, present on HEAD Residual: two tests in the same file still patch `nullrun.transport.threading.Event.wait`, which is the global `threading.Event.wait` — the same class, a different attribute. The audit named the transport's `time.sleep`.
…ld (SDK-15)
The SDK's v3 error-code catalog lagged the backend's registry by 34 codes.
The parity guard in backend/tests/nr007_sdk_error_code_parity.rs could not
see the drift because it was wired into no workflow. Every missing code
fell through to the status-based fallback, so the refusal arrived as a
generic NullRunBackendError (or, on a 4xx, as an authentication error)
with no diagnostic class at all.
Two defects, one symptom:
1. The catalog was incomplete. 34 codes added, each mapped by its WIRE
BUCKET -- the bucket the backend's own registry (GateErrorCode::
http_status) gives the code -- so the SDK class and the backend status
cannot disagree. Money (402/429) lands on NullRunBudgetError, a 429 on
RateLimitError, a refusal on the refusal's own class, a 5xx on
NullRunBackendError.
2. The catalog named classes the dispatcher could not build. The
blocked-family arm tested the class by IDENTITY, so every typed
SUBCLASS of NullRunBlockedException fell through to the generic
fallback, which calls catalog(full_message, **forwarded) against a
constructor that requires workflow_id and reason: a TypeError, which
escaped the parser into the transport's catch-all and surfaced as a
synthetic {"decision": "block"} dict. Nineteen codes were flattened
that way (eleven approval, eight MCP). The arm now matches by
SUBCLASS. Two dispatch arms were added for the families whose
constructors the generic fallback cannot satisfy: RateLimitError
(requires source + endpoint) and NullRunChainError (drops
chain_id / backend_code).
The four source pins in tests/test_2026_09_10_toolblocked_parser.py
pinned the old arm's exact bytes. They are re-pointed at the structural
boundary (the branch matches the family by subclass; the branch's
constructor call is unchanged; the branch lives in the dispatching
function). The needle was moved twice: a class NAME is self-satisfying
(the import block names it even with the branch deleted) and the
constructor call is not unique to the branch (the NullRunBudgetError
arm builds the same argument).
New test tests/test_error_code_catalog.py holds the property over the
whole catalog: for every code, the real dispatcher returns an instance
of the catalog's class, for every status, because the code names the
refusal and the status is only its HTTP rendering. It also pins the 34
codes by name -- a removal is invisible to a test parametrized over the
map's own keys.
Verified: pytest 2431 passed / 12 skipped; ruff clean; mypy 5 errors,
all pre-existing in instrumentation/auto.py. Mutation-proved: reverting
the arm to the identity test turns 35 cells red with
TypeError: NullRunBlockedException.__init__() missing 1 required
positional argument: 'reason'; disabling the RateLimitError arm turns 5
red; deleting one map entry turns the name pin red; the re-pointed pins
turn red when the branch is gutted. The backend parity target
(nr007_sdk_error_code_parity) is green against this tree and was red
before it.
Not pushed: the release is the owner's decision.
(SDK-16) The LangChain/LangGraph callback swallowed every refusal the runtime raises on purpose. NullRunAuthenticationError, NullRunUnclassifiedRefusalError and NullRunRateLimitRedisError fell into the callback's broad arm. The same refusal stopped the call through the @Protect boundary and vanished through the callback, so the LLM call ran with no reservation and its cost event was dropped: the agent got the call for free. The callback now classifies like the @Protect boundary. The family raises, a transport failure fails OPEN (ADR-008), and every other fault fails CLOSED. The transport arm comes first, because a transport failure is also an infrastructure error and the order decides whether a dead backend freezes the agent. The runtime docstring records that the callback obeys the check_workflow_budget row. Tests: the callback's own classification, the same refusal through the real runtime and the real @Protect boundary (one outcome, two paths), the transport counter-test, an AST pin on the arm order, and a drift tripwire on the table row. The /track path is untouched. No wire change, no backend change.
The backend now refuses a chain presented by an API key of the same organization with CHAIN_KEY_MISMATCH (403). The SDK had no entry for it: the dispatcher fell through to the 403 auth bucket, so a caller branching on NullRunChainError never caught it. The code is now in the typed catalog and in the envelope parser, and the frozen-catalog test carries it so the entry cannot be dropped again. Verified: the two cells go red when the entries are removed. Full suite 2454 passed / 12 skipped. Ruff clean. Mypy reports 5 errors in src/nullrun/instrumentation/auto.py — pre-existing on master, not in this diff.
ADR-185. Security audit 2026-11-06, SDK-3 (Medium). uv.lock existed and pinned the dependencies, but no workflow consumed it. All four install sites ran `pip install -e ".[dev]"`, and the publish job ran `pip install --upgrade build`. The build backend was unpinned, and it is not part of uv's resolution, so no lock-based job could see it. The publish job holds the PyPI Trusted Publishing token (`permissions: id-token: write`). Every `uses:` in it named a branch or a tag. A moved tag executes in the job that can mint an upload token, and the wheel it produces carries the payload to every `pip install nullrun`. What changed: - every `uses:` names a 40-hex commit SHA (26 references, six actions, the old tag kept in a trailing comment) - the build backend is pinned exactly: `hatchling==1.32.4` - every install comes from the lock: `uv sync --frozen --extra dev`. `--frozen` fails the job when the lock and pyproject.toml disagree - the publish tooling moves into a PEP 735 `[dependency-groups]` entry, and the build job resolves it with `--only-group publish --frozen` from the lock, which carries the hashes - `scripts/check_supply_chain.py` lints the workflows. It runs inside the token-holding job, before the build and the upload, plus a local pre-commit hook and a `make` target. It has no third-party dependency: PyYAML when it is importable, a text scanner otherwise, and the two scanners are proven to agree The pre-commit chain could not start. It pointed at a mirror of `pre-commit/mypy` that no longer exists, and pre-commit installs every repository in the file before it runs any hook, so the whole run aborted and no hook in the file executed. The dead reference is gone. The mypy check is not lost: CI runs `mypy src/` and `make type-check` runs `mypy src/nullrun --strict`. Tests: `tests/test_supply_chain_lint.py` (new).
ADR-187. Security audit 2026-11-06, SDK-5 (Low). Three places printed a fragment of the key: - the status snapshot: `self.api_key[:10]` - the runtime's error context: the same ten characters - the transport's debug log: `api_key[:8] + "***"` A prefix of a credential is the part an attacker brute-forces the rest from. Because the three places disagreed about how much they printed, "the full key never leaves the SDK" was not a statement anyone could check. One helper now answers it. `api_key_fingerprint` returns the first eight hex characters of the SHA-256 digest of the key, or `None` for a missing key. All three sites call it, so the disagreement is gone by construction. The backend's `compute_fingerprint` is SHA-256 over the raw key truncated to 16 bytes. The SDK's eight characters are a prefix of the backend's 32, so a client log line and a backend row correlate on both sides without either side holding a fragment of the secret. The public field keeps its legacy name `api_key_prefix`. It is part of the SDK's surface. Only its rendered label changed, to `api_key_fp=`. Tests: `tests/test_api_key_fingerprint.py` (new).
ADR-184. Security audit 2026-11-06, SDK-2 (Medium). `_is_production_environment` decided whether the two fail-open guards arm by asking one question: is any marker a SUBSTRING of the hostname? `"test"` is a substring of `latest`, `contest` and `attest`. `"localhost"` is a substring of `notlocalhost.example.com`. So a host that merely CONTAINED a dev marker was classified non-prod. Worse, the operator's `NULLRUN_ENV=production` was suppressed by the `and not any(marker in host_lc ...)` clause. `NULLRUN_SKIP_BUDGET_CHECK=1` then bypassed the budget gate, the rate limit and the tool-block check, with a WARNING rather than a refusal. Four ordered rules now: 1. an exact loopback, or a `.localhost` suffix, is dev 2. the host's FIRST LABEL being `staging` or `test` is dev 3. the host being exactly `api.nullrun.io` is prod 4. `NULLRUN_ENV=production` or `prod` is prod for every other host, so the operator's declaration is no longer vetoed by a spelling Rule 2 compares label by label, never as a substring. `staging.internal` is dev and `latest.nullrun.io` is not. The documented dev escape hatch keeps working. Tests: `tests/test_production_environment_detection.py` (new).
ADR-186. Security audit 2026-11-06, SDK-4 (Low). The WAL path was `os.path.join(tempfile.gettempdir(), "nullrun.wal")`. It was the same string for every user and every org on the host, and `_persist_to_wal` writes with a tmp file, an fsync and an `os.replace` that puts only this runtime's buffer in place of the whole file. Two runtimes on one file do not queue. The second one destroys the first one's undelivered events, so spend that already happened is never booked, and the flush path's availability is coupled across tenants. The default is now `<tempdir>/nullrun-<user>-<key>.wal`. Both scope tokens are reduced to SHA-256 digests, so the API key never reaches a path every user on the host can list. The path is resolved once, in the constructor, and never moves. The org id is only known after authentication, and a path that changed there would orphan the file written before it: at every instant the path is "the right one" and the money is lost anyway. One API key belongs to exactly one org, so a key-scoped WAL is an org-scoped WAL in money terms. `NULLRUN_WAL_PATH` still wins outright, and it is documented as the operator's responsibility to keep unique per runtime. The legacy unscoped file is reported and left alone. It is never read, never migrated and never deleted, because its rows may belong to another tenant. The operator's CLI lists every scoped WAL this user owns, and `replay` refuses to guess between them. Tests: `tests/test_wal_scope.py` (new).
…SDK-1)
ADR-183. Security audit 2026-11-06, SDK-1 (High).
`/gate` RESERVEs the budget. The tool call runs and costs real money.
Then `POST /api/v1/track` carries the CONSUME.
Pre-fix, `_route_track` wrapped `track_single` in `except Exception`. A
`NullRunDecision` re-raised. Everything else was logged "event dropped"
and the function returned normally, so the caller returned
`{"allowed": True}` to the agent.
`track_single` was one POST, with no retry and no WAL, while the sibling
batch path was WAL-backed. The CONSUME never ran, so the period-bound
counter never moved and no `cost_events` row existed. The reservation
envelope was reaped later. The spend was free and the agent was told
`allowed`.
The taxonomy was the sharp edge. The SDK decided open-vs-closed by
exception CLASS and the backend by error CODE, and `NullRunDecision` and
`NullRunInfrastructureError` are SIBLINGS. `isinstance(exc,
NullRunDecision)` can never catch a backend refusal, whatever the code
maps to. A later narrowing to three classes still left nine codes
outside it, among them the three `RATE_LIMIT_*` codes, and the backend
rate-limits `/track`.
The rule now asks the WIRE one question: did the backend answer? The
HTTP status is stamped on the exception by the error-envelope parser. A
request that never REACHED the backend keeps the ADR-008 fail-OPEN row,
unchanged. A refusal raises, and the auto-instrumentation layer stops
swallowing it. The payload is written to the v3 WAL before the raise, so
a refusal is raised AND recoverable.
A determinate verdict is not persisted. It is the backend's answer, and
re-sending it produces the same answer.
The v3 WAL is a separate file, replayed to `/api/v1/track` (single). The
batch path does not run the CONSUME script, so a reservation replayed
through it is never settled. The row carries the idempotency key, and
one refused row no longer blocks the rows behind it.
Tests: `tests/test_v3_track_recovery.py` (new). Two existing tests
changed contract: a 500 that used to be swallowed now raises, and the
"transport failure" mock became a real `ConnectError`.
…r 1) ADR-188. Security audit 2026-11-06, SDK minors (no severity). The MCP adapter caches the tool annotations it stamps on /check. The gate's `mcp_destructive_policy` and `mcp_readonly_policy` umbrella policies are evaluated against them, so the cache is a policy input and not a convenience. The window was 300 seconds. A FAILED refresh was worse than the window: the handler kept the previous snapshot and the adapter kept presenting it as the server's current answer, so an upstream that went down left the client stamping a stale `destructive=False` with no bound on the staleness at all. The window is 30 seconds now, the floor the constructor already enforced. An operator who cannot accept any staleness passes `cache_seconds=0` and pays one `tools/list` round trip per call. A failed refresh no longer presents the snapshot to the gate. The gate gets the unknown shape the adapter already uses for a tool it has never seen, and falls back to its own classifier. The snapshot stays readable for diagnostics, where it is a view of the last successful answer rather than a claim about the current one. The one deliberate cost: the default window buys ten `tools/list` calls instead of one per five minutes. The alternative, no window at all, costs a round trip per tool call, and that is the operator's choice to make rather than the default. Tests: `tests/test_mcp_annotation_staleness.py` (new).
…one (SDK minor 2) ADR-189. Security audit 2026-11-06, SDK minors (no severity). `DecisionSource.CACHED` and `ExecuteConfig.cache_ttl` / `cache_max_size` sit in the public surface with no producer anywhere in `src/`. The SDK has no in-process gate cache: every call is its own `/gate` call with its own reservation. The surface reads as a live decision cache, and that is the shape of NR-116, which was closed. A future "restore the cache" change would land green, and a reader would assume a decision can be reused. The members stay: they are public and removing them breaks importers for no safety gain. What was missing was a check that they stay producer-less, so the audit's own named check now exists. An AST walk over the package proves that nothing references `DecisionSource.CACHED`, that the string "cached" appears exactly twice and both times as surface, and that nothing reads `cache_ttl` or `cache_max_size`. The pin is textual, and it is honest about that: it proves the absence of a producer in the tree, not the absence of a decision cache in some future version. Tests: `tests/test_decision_cache_surface_has_no_producer.py` (new).
…e (SDK minor 3)
ADR-190. Security audit 2026-11-06, SDK minors (no severity).
The SDK redacted arguments by NAME. `SENSITIVE_ARG_KEYS` knows which
argument names someone listed. It cannot catch a secret that arrives
under a name nobody listed: `{"note": "<api key>"}` printed the key in
full.
One value-shaped pass now runs beside it. `scrub_secret_values` matches
the credential itself, whatever key it travels under: this product's own
`nr_live_` / `nr_test_` keys, a JSON Web Token, an AWS access key id, an
OpenAI-style `sk-` key, a PEM private key block, and an HTTP bearer
credential.
The patterns are narrow on purpose. A shape that cannot be told apart
from ordinary data is a guess, and a guess that scrubs real data is a
bug of its own, so the near misses are test cases: `risk-assessment-
report` and `task-abcdef0123456789abcdef0123456789` survive, and so does
ordinary SQL.
Most of the shapes carry no leading word boundary. A boundary would make
the pattern miss a credential that sits right after a letter, and that is
exactly the leak this pass exists to close. The one exception is `sk-`,
whose boundary keeps a hyphenated word out of the scrub.
The pass runs BEFORE the truncation, so a credential that straddles the
cut is caught whole. Truncate-then-scrub would cut it in half and scrub
neither half.
Tests: `tests/test_value_level_redaction.py` (new).
The SDK had a shape it could not keep: the enforcement path swallowed
refusals. A gate that said "no" could be turned into a `None`, and
`None` is read by every caller as "yes"; a `/track` the backend
refused could be logged at DEBUG and returned as an ordinary call; a
LangChain callback dropped the refusal on the floor so the agent got
the call for free. Nine of the defects in this release are this one
defect seen from nine angles — an indeterminate gate answer, an
unbindable reservation id, a refused `/track`, a swallowed callback
refusal, an SDK-internal fault in the decorator — and each of them
ended the same way: the agent ran the call, the money was never booked,
and nobody was told.
The second theme is the control plane. The approval push read the
UNSIGNED outer body — the bytes the HMAC never covered — and handed
them to the handler that releases a gate parked waiting for a human.
The WebSocket receiver verified a signature only when all four of
`signature`, `timestamp`, `api_key` and `secret_key` were present, so
a frame missing any one of them skipped verification entirely. Both are
closed: the privileged branch now requires an AUTHENTICATED frame and
reads the verified payload only.
The third theme is honesty about the surface. The SDK's error-code
catalogue lagged the backend's registry by 34 codes, and the
blocked-family dispatch arm tested the class by IDENTITY, so every typed
SUBCLASS fell through to a generic fallback that raised `TypeError` —
nineteen codes were flattened into a synthetic `{"decision": "block"}`
dict. The surface also advertised `DecisionSource.CACHED`, `cache_ttl`
and `cache_max_size` for a cache that no longer existed. All of it is
now either true or pinned.
33 commits ahead of origin/master, organised into four themes:
1. Money that was never booked
- a4fbed2 fix(sdk): the money path asks one question: did the backend answer? (SDK-1, High). The CONSUME never ran; the spend was free and the agent was told `allowed`. The rule now asks the WIRE one question: did the backend answer?
- 87a4c2b fix(sdk): no in-process gate cache, edge /execute never invents an allow (SDK-1/2/3). The cache made chain steps 2..N free — one execution id for the whole chain, so consume_v3.lua answered IDEMPOTENT_REPLAY and booked nothing. The edge branch returned a client-side allow without ever asking the box.
- 0964705 fix(sdk): the box's answer cache is read and written under a lock (SDK-9). The eviction walked a live dict; a concurrent resize made the gate's fail-OPEN arm return "no decision", which the caller read as "no block".
- dc197a9 fix(sdk): a reservation id the SDK cannot bind is a REFUSAL (SDK-10). A present-but-unusable id was dropped with a warning; the call then ran with no reservation and the cost event was dropped.
- 9dcb22d fix(sdk): an indeterminate gate decision is a REFUSAL, not a permission (SDK-11). Three facts were conflated; now separated by TYPE. A named refusal is honoured, an unreachable gate fails OPEN per ADR-008, and anything else is NullRunGateIndeterminateError (NR-P004).
- 22f8d77 fix(sdk): a refused /track must reach the application (SDK-6). The auto-instrumentation layer caught the refusal with a blanket `except Exception` and logged it at DEBUG; the application saw an ordinary call.
- 28f1b00 fix(sdk): a gate refusal leaves the callback — only a refusal may escape (SDK-4). TOOL_BLOCKED and RATE_LIMIT_EXCEEDED fell into a three-class allow-list's broad arm. ADR-122.
- 1bb76dd fix(sdk): the callback obeys the same fail-OPEN/CLOSED table as @Protect (SDK-16). The same refusal stopped the call at the decorator boundary and vanished through the callback.
- ac62e10 fix(sdk): an SDK-internal fault in the @Protect gate is a REFUSAL (SDK-13). The catch-all wore NR-B001 and the policy-block type, so an SDK bug rendered as HTTP 429 / category "decision".
- b519481 fix(sdk): the fail-open switch is bounded per tool (SDK-8). "Enabling it for one tool disables enforcement for every tool." The classification is code-level on purpose: an env var can be inherited by a deploy pipeline, a code-level classification cannot.
- 1b9ef6d fix(sdk): every MCP tool call runs its own /gate pre-flight (P2-18). The contextvar that gated the pre-flight was never reset, so the second and later tool calls ran NO /gate.
- 776eb01 fix(sdk): the MCP adapter derives one envelope from the tool call (SDK-7). On the MCP path the approval could not be spent at all — /execute with no envelope fails CLOSED with BUSINESS_IMPACT_INVALID.
- 946a3a2 fix(sdk): the edge refusal reports its OWN last_source (P2-17). EDGE_LEASE was written on BOTH the allow and the refusal path, so a caller could not tell a box ALLOW from a box REFUSAL.
- 90bc05a fix(sdk): the write-ahead log is scoped per user and per key (SDK-4). Two runtimes on one file do not queue: the second destroys the first's undelivered events, so spend that already happened is never booked.
- 0d1186e test(sdk9): pin the money direction — a refusal is never a failed-open. Measured on the real LLM-span path: 35-43 of 576 LLM calls proceeded on a refused lease before the fix, 0 after.
2. The control plane
- c4e8c4a fix(sdk): the approval push reads the VERIFIED payload, not the outer body (SDK-5). The HMAC covers the bytes inside signed_payload, NOT the flattened outer body — so the outer body was writable by anyone who can inject bytes into the socket, and the signature check still PASSED.
- 2a79524 fix(sdk): refuse a control-plane frame the client could have verified (P2-19). Verification was skipped when any one of four fields was absent, and the frame was then read from the body the server never signed.
- b0f5645 fix(sdk): name the refused frame's code in the log (P2-19).
- 5d1e5e0 fix(sdk): an unknown control-plane action is REFUSED, not silently dropped (SDK-12). "A control-plane instruction that is silently dropped is a control-plane instruction that does not exist."
- 5adc309 fix(sdk): APPROVAL_TOOL_MISMATCH — a money approval must bind a tool (DEC-5/DEC-7). A money approval bound only the AMOUNT, so an approval for tool A authorised tool B. New wire code, HTTP 409.
- 2d19f45 fix(sdk): the chain-key refusal carries the chain class (ADR-180).
3. The surface says what the SDK does
- e1ec87c fix(sdk): the error-code catalog names a class the dispatcher can build (SDK-15). 34 codes added, each mapped by its WIRE BUCKET; the blocked-family arm now matches by SUBCLASS, so a typed subclass can no longer fall into a generic fallback that raises TypeError and surfaces as a synthetic {"decision": "block"} dict.
- c78ef32 fix(sdk): the SDK has no decision cache, and its surface says it has one (SDK minor 2). An AST walk over the package now proves DecisionSource.CACHED has no producer.
- 4d386bb test(sdk): the suite must drive the PRODUCTION enforcement family (SDK-14). Measured on the pristine tree: with the production family WIDENED by one member, the whole suite stayed GREEN.
4. Secrets, environment, and the supply chain
- 1ad95d0 fix(sdk): redaction follows the credential, not only the argument name (SDK minor 3). {"note": "<api key>"} printed the key in full; scrub_secret_values now matches the credential itself, whatever key it travels under.
- 66de8f5 fix(sdk): no character of the API key reaches a log line (SDK-5). Three places printed a fragment of the key and disagreed about how much; one helper now answers it.
- 249df35 fix(sdk): the production detector is not a substring test (SDK-2). "test" is a substring of latest, contest and attest; "localhost" is a substring of notlocalhost.example.com.
- 6ab5a2c fix(sdk): a failed tools/list refresh is not a policy input (SDK minor 1). The annotations are a POLICY INPUT; a failed refresh kept presenting the previous snapshot with no bound on the staleness at all.
- 06e8fce build(sdk): the CI installs from the lock and pins every action (SDK-3). Every `uses:` in the publish job — the job that holds the PyPI Trusted Publishing token — named a branch or a tag.
Plus cleanup and tooling:
- fc1666e style(sdk): drop the redundant quotes ruff flags (SDK-7 leftover)
- db7e776 docs(sdk): point the error-envelope notes at the real catalogue
- f49aa7c docs(sdk): NR-P004 covers the /execute seam too (SDK-13)
- e572403 fix(sdk): the retry clock is a seam, not the global time.sleep (P3-10)
Verification on release/0.24.0:
- ruff check src tests : clean
- mypy src/nullrun : 5 errors in src/nullrun/instrumentation/auto.py —
identical on a pristine origin/master checkout in the
same interpreter (5 errors in 39 files there, 5 in 41
here). Pre-existing, not introduced by this diff.
They are attr-defined / call-overload against
langgraph's Pregel overloads; langgraph is NOT in
uv.lock, so CI's `mypy src/` (which installs from the
lock) never imports it and sees none of them.
- pytest -q : 2592 passed / 12 skipped (82.18s)
(baseline on origin/master in the same interpreter:
1948 / 12 — +644 new tests)
- scratch diff : clean (dist_local/, defect* absent; both already in .gitignore)
- nullrun.__version__ : 0.24.0
- wire-format compatibility : unchanged from 0.23.0 for the SDK->backend contract.
The error-code catalog grew by 34 entries, each mapped to
a code the backend already sends. One code is NEW in both
repos and landed together: APPROVAL_TOOL_MISMATCH (409).
The SDK's request payloads are unchanged — the MCP envelope
puts business_impact + action_digest on the wire, which are
fields the backend already resolves.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This release closes the enforcement path's most expensive habit: it
swallowed refusals. A gate that said "no" could be turned into a
None, andNoneis read by every caller as "yes"; a/trackthebackend refused could be logged at DEBUG and returned as an ordinary
call; a LangChain callback dropped the refusal on the floor, so the
agent got the call for free. Nine of the defects below are this one
defect seen from nine angles — an indeterminate gate answer, an
unbindable reservation id, a refused
/track, a swallowed callbackrefusal, an SDK-internal fault in the decorator — and each of them
ended the same way: the agent ran the call, the money was never booked,
and nobody was told.
The second theme is the control plane. The approval push read the
UNSIGNED outer body — the bytes the HMAC never covered — and handed
them to the handler that releases a gate parked waiting for a human. The
WebSocket receiver verified a signature only when all four of
signature,timestamp,api_keyandsecret_keywere present, so aframe missing any one of them skipped verification entirely. Both are
closed: the privileged branch now requires an AUTHENTICATED frame and
reads the verified payload only.
The third theme is honesty about the surface. The SDK's error-code
catalogue lagged the backend's registry by 34 codes, and the
blocked-family dispatch arm tested the class by IDENTITY, so every typed
SUBCLASS fell through to a generic fallback that raised
TypeError—nineteen codes were flattened into a synthetic
{"decision": "block"}dict. The surface also advertised
DecisionSource.CACHED,cache_ttland
cache_max_sizefor a cache that no longer existed. All of it isnow either true or pinned.
33 commits, +15 295 / −1 213 across 77 files, +644 tests. There is no
wire-format change: every new error code the SDK names is a code the
backend already sends, and the one place the SDK speaks a NEW code
(
APPROVAL_TOOL_MISMATCH, HTTP 409) landed together with the backendhalf.
Migration
Nine things differ from 0.23.0. The first is breaking; the second and
third change what an operator's configuration does; the rest are
behavioural changes visible only when something goes wrong — which is
precisely when they are visible.
check_workflow_budgetlost itsuse_gate_cacheparameter. Thein-process gate cache is gone. A caller that passed
use_gate_cache=Falsenow gets aTypeError— remove the argument.There is no replacement: every call is its own
/gatecall with itsown reservation, and the backend's own idempotency store (keyed by
the SDK's
operation_id) is the correct retry dedup.NULLRUN_SENSITIVE_FAIL_OPENopens nothing on its own. The setof tools it may open is now an explicit, code-level classification,
and it is EMPTY by default:
runtime.register_fail_open_tools(["charge_card"])/remove_fail_open_tool/get_fail_open_tools/is_fail_open_allowed. Before this release, enabling the switch forone tool disabled enforcement for EVERY tool.
The family is named by BASE class (
NullRunDecision,RateLimitError,WorkflowKilledInterrupt—RateLimitErrorisnamed explicitly because it is NOT a
NullRunDecision), the refusalis re-raised, and the handler sets
raise_error = Truesolangchain-core does not swallow it. A cosmetic failure is still
swallowed at DEBUG — only a refusal may escape. ADR-008's fail-OPEN
for a transport failure is unchanged.
permission.
NullRunGateIndeterminateError(NR-P004) replaces thefail-open-and-return-
Nonearm. The new class is an infrastructureerror, NOT a transport error, so ADR-126's two-step opt-in cannot
convert it into "the body runs".
<tempdir>/nullrun-<user>-<key>.wal— both scope tokens reduced toSHA-256 digests. The legacy
<tempdir>/nullrun.walis reported andleft alone: never read, never migrated, never deleted.
api_key_prefixnow renders as a fingerprint. The public fieldkeeps its legacy name; its VALUE changes — the first eight hex
characters of the SHA-256 digest of the key — and the log label
changed from
api_key_prefix=toapi_key_fp=. Log parsers thatmatched the old raw prefix must change.
annotations are a POLICY INPUT, so a stale snapshot is a stale
policy. A FAILED refresh no longer presents the previous snapshot to
the gate at all.
cache_seconds=0buys zero staleness.NullRunGateIndeterminateError(NR-P004),
NullRunUnknownActionError(NR-P005) andNullRunApprovalToolMismatchError(NR-A017).to_dict().DecisionSource.CACHED,ExecuteConfig.cache_ttlandExecuteConfig.cache_max_sizedeliberately did NOT move — removing themwould break importers for no safety gain. An AST walk over the package
now proves they stay producer-less.
Fixed
Money that was never booked
CONSUME never ran, the period-bound counter never moved and no
cost_eventsrow existed: the spend was free and the agent was toldallowed. The rule now asks the WIRE one question: did the backendanswer? (
a4fbed2, +1090/-19; newtests/test_v3_track_recovery.py)No org id, no api-key id, no tool set in the key; one execution id
for the whole chain, so
consume_v3.luaansweredIDEMPOTENT_REPLAYand booked nothing. The edge branch of
Transport.executereturned aclient-side allow without asking the box. (
87a4c2b, +846/-648)lock. The eviction walked a live dict, so a concurrent resize made
CPython raise — and the gate's fail-OPEN arm caught it and returned
"no decision", which the caller read as "no block". (
0964705,+387/-4; new
tests/test_sdk9_shared_state.py)present-but-unusable id now raises
NullRunMalformedGateResponseErrorinstead of being dropped with a warning. (
dc197a9, +721/-44)permission. The gate NAMED a refusal → honour it; the gate could
not be REACHED → ADR-008's fail-OPEN, and only that; anything else →
the SDK's own code faulted →
NullRunGateIndeterminateError.(
9dcb22d, +884/-10)/trackmust reach the application. Theauto-instrumentation layer caught the refusal with a blanket
except Exceptionand logged it at DEBUG; the application saw an ordinarycall. One family, named by BASE class, now lives in
nullrun.breaker.exceptions. (22f8d77, +876/-12)refusal may escape. The three-class allow-list dropped
TOOL_BLOCKEDandRATE_LIMIT_EXCEEDEDinto the broad arm. ADR-122.(
28f1b00, +490/-44)@protect. The same refusal stopped the call at the decoratorboundary and vanished through the callback, so the LLM call ran with
no reservation. (
1bb76dd, +522/-13)@protectgate is aREFUSAL. The catch-all wore NR-B001 and the policy-block type, and
the FastAPI integration rendered it HTTP 429 / category "decision" —
the operator was told the policy refused when the truth is that the
SDK has a bug. (
ac62e10, +561/-20)one tool disables enforcement for every tool." The classification is
code-level on purpose: an env var can be inherited by a deploy
pipeline, a code-level classification cannot. (
b519481, +635/-28)/gatepre-flight. Thecontextvar that gated the pre-flight was never reset, so the second
and later tool calls in one scope ran NO
/gate: no reservation, nospend reserved. (
1b9ef6d, +445/-17)On the MCP path the approval could not be spent at all:
/executewith no envelope fails CLOSED with BUSINESS_IMPACT_INVALID. The
params MUST be the exact dict forwarded, or the operator's approval
binds arguments nobody approved. (
776eb01, +686/-7)last_source.EDGE_LEASEwas written on BOTH the allow and the refusal path, so acaller could not tell a box ALLOW from a box REFUSAL. (
946a3a2,+261/-1)
Two runtimes on one file do not queue: the second destroys the first's
undelivered events, so spend that already happened is never booked.
(
90bc05a, +787/-20)The control plane
payload, not the outer body. The HMAC covers the bytes inside
signed_payload, NOT the flattened outer body — so the outer body waswritable by anyone who can inject bytes into the socket, and the
signature check still PASSED. (
c4e8c4a, +698/-44)refused. The receiver verified the signature only when all four of
signature,timestamp,api_keyandsecret_keywere present, soa frame missing any one skipped verification entirely. (
2a79524,+312)
(
b0f5645, +30/-1)dropped. "A control-plane instruction that is silently dropped is a
control-plane instruction that does not exist." (
5d1e5e0, +756/-57)approval bound only the AMOUNT, so an approval for tool A authorised
tool B. New wire code
APPROVAL_TOOL_MISMATCH(HTTP 409) →NullRunApprovalToolMismatchError(NR-A017). (5adc309, +124/-4)CHAIN_KEY_MISMATCHno longer falls through to the 403 auth bucket.(
2d19f45, +35)The surface says what the SDK does
build. 34 codes added, each mapped by its WIRE BUCKET; and the
blocked-family arm now matches by SUBCLASS, so a typed subclass can
no longer fall into a generic fallback that raises
TypeErrorandsurfaces as a synthetic
{"decision": "block"}dict. (e1ec87c,+518/-65)
it has one. An AST walk over the package now proves
DecisionSource.CACHEDhas no producer. (c78ef32, +216/-2)Measured on the pristine tree: with the production family WIDENED by
one member, the whole suite stayed GREEN. (
4d386bb, +265/-31)failed-open. Measured on the real LLM-span path: 35–43 of 576 LLM
calls proceeded on a refused lease before the fix, 0 after.
(
0d1186e, +134/-5)Secrets, environment, and the supply chain
argument name.
{"note": "<api key>"}printed the key in full.scrub_secret_valuesnow matches the credential itself, whatever keyit travels under. (
1ad95d0, +154)line. Three places printed a fragment of the key, and they
disagreed about how much. One helper now answers it. (
66de8f5,+233/-21)
"test"is a substring of
latest,contestandattest;"localhost"is asubstring of
notlocalhost.example.com. And the operator'sNULLRUN_ENV=productionwas suppressed by a spelling. Four orderedrules now. (
249df35, +283/-28)tools/listrefresh is not a policy input.The window was 300 s, and a FAILED refresh kept presenting the
previous snapshot with no bound on the staleness at all. 30 s now;
a failed refresh presents the unknown shape. (
6ab5a2c, +321/-18)Every
uses:in the publish job — the job that holds the PyPITrusted Publishing token — named a branch or a tag. A moved tag
executes in the job that can mint an upload token. Every
uses:nownames a 40-hex commit SHA; the build backend is pinned exactly; every
install comes from the lock. (
06e8fce, +1988/-83)Cleanup
ruff check src testswas RED afterthe SDK-7 commit (UP037 twice in
toolbox/mcp.py). A red gate in thetree is a red gate for the next release too. (
fc1666e, +2/-2)error-envelope notes pointed at
contracts/errors.ts, deleted in thebackend repo (WIRE-8 / ADR-113). (
db7e776, +4/-3)/gate-only refusal; after SDK-13 the same class is raised on the/executeseam too. (f49aa7c, +25/-15)Tooling
time.sleep—nullrun.transport.timeIS the stdlib module, so a patch ontime.sleepcaptured the sleep of EVERY thread in the process. Thetransport now has a
_sleepseam next to_wait_on. (e572403,+82/-23)
scripts/check_supply_chain.pyhas no third-party dependency andruns in CI, in the pre-commit chain and as a
maketarget. Thepre-commit chain could not start at all: it pointed at a mirror of
pre-commit/mypythat no longer exists, and pre-commit installs everyrepository before it runs any hook — so the whole run aborted and no
hook in the file executed.
Verification
ruff check src testsmypy src/nullrunsrc/nullrun/instrumentation/auto.py— identical on a pristineorigin/mastercheckout in the same interpreter (5 errors in 39 files there, 5 in 41 here). Pre-existing, not introduced by this diff. They areattr-defined/call-overloadagainstlanggraph'sPregeloverloads;langgraphis not inuv.lock, so CI'smypy src/(which installs from the lock) never imports it and sees none of them.pytest -qorigin/masterin the same interpreter: 1948 passed, 12 skipped — +644 new tests)dist_local/,*.defect*absent — both already in.gitignore)nullrun.__version__0.24.0APPROVAL_TOOL_MISMATCH(HTTP 409). The SDK's own request payloads are unchanged — the MCP envelope putsbusiness_impact+action_digeston the wire, which are fields the backend already resolves.Commits included