▶ Watch the 2-minute demo · Try the live console
A merge blocks. fare_predictor's inputs were about to change under it — and more than 12% of its fare
predictions would have moved. Nobody would have noticed until the numbers were already wrong.
Interlock is a pull-request gate for machine-learning teams. When a data change lands in a PR — a renamed column, a retyped field, a rewritten transformation — Interlock reads the change out of the diff, follows DataHub's lineage graph to the production models downstream of it, replays the real model artifact on the data as it is now versus as the change would make it, and blocks the merge when a model's declared contract would break. Then it writes the fix as a reviewable patch and records the verdict back into the graph.
Every other tool tells you a model broke after it broke. Interlock stops the merge that would have broken it.
A dbt PR renames
airport_feetoAirport_fee. Every test is green — the rename is internally consistent. Nothing in CI knows that a feature table three hops downstream feeds a fare model, or that the model reads that column. It merges. Two weeks later the fare estimates are quietly wrong. Interlock makes that merge fail.
On the demo stack — real NYC TLC trip data,
a real LightGBM fare model, real DataHub lineage — the actual TLC rename of airport_fee →
Airport_fee moves more than 12% of fare decisions (12.4–12.6% across sample sizes, replayed
against the trained artifact). The model's contract allows 0.5%. Interlock blocks it, names the model,
shows the evidence path, and stages a validated remediation patch — restoring the model's expected
inputs — that compiles green.
That number is measured, not asserted — the model is replayed on every run and the value is checked against a seeded band, so a drift in the replay harness fails the build rather than passing silently.
The same 14-scenario suite, judged twice against a hidden answer key: once with DataHub's context graph, once blind to it (schema diffs only — a plausible baseline).
| Arm | Correct decisions | Breaking changes blocked | Traps false-blocked |
|---|---|---|---|
| With DataHub context | 13 / 14 (93%) | 8 / 9 | 0 / 4 |
| Schema heuristics only | 6 / 14 (43%) | 4 / 9 | 3 / 4 |
The graph turns 4/9 catches into 8/9 and removes 3 false alarms. The cleanest case: a target-leakage
column and a harmless new column are the same schema operation — column_added. The blind arm
treats them identically. The graph knows one reaches the model through a forbidden path and the other
reaches nothing, so it blocks one and passes the other. Lineage, not the schema, is doing the work.
The suite reports its own honest miss (13/14, not a suspicious 14/14) and states exactly what would
turn each row red. See eval/report.md and docs/evaluation.md.
Interlock is impossible without the context graph, and it writes back to it.
- Reads: lineage (
searchAcrossLineage, column-level), schema, ownership, tags, ML entities (MLFeatureProperties.sources→MLModelProperties.mlFeatures), assertions, incidents — through the DataHub MCP Server, the Agent Context Kit, and the Python SDK. - Writes back: contracts become native DataHub assertions and a
dataContractentity; a blocked merge raises a DataHub incident, tags the impacted model, and records the verdict and blast radius as structured properties and documentation. The graph is smarter after every run. - Ships its own MCP server so other agents can consult the gate, and contributes two new skills and an example fix upstream (see Open-source contribution).
One deterministic pipeline: extract change → resolve blast radius → replay artifact → evaluate contracts → verdict → write back → remediate. It fails closed — an unreachable graph, ambiguous lineage, a failed replay, or a production model with no contract all produce a BLOCK, never a silent pass. ~900 tests, a lint-clean tree, and a GitHub Action that dogfoods the gate on this repo's own PRs. The gate self-test workflow proves offline, on every push, that a real break is blocked and a safe change passes.
The field detects breakage after the fact and files an incident. Interlock prevents the merge, and
does it by replaying the model artifact to produce a decision-flip number — not by arguing about
schemas. It also gates deployments (interlock deploy) and ships a pre-commit hook, neither of
which existed elsewhere. The LLM never decides: a pure policy engine does, and the language model only
explains the verdict and drafts remediation that is then hard-validated.
Training-serving skew from an unnoticed upstream change is a named, expensive failure on every ML platform team. Interlock catches it at the one moment it is cheap to fix — in review — and hands back a patch a data engineer would merge. It runs against any DataHub instance and any model registry, not just this demo.
One command brings up the whole stack (demo/setup.py). The gate, the console, and the examples run
with zero credentials — the language layer ships recorded real responses, so no API key is needed to
see it work. A JUDGES.md walks a reviewer through every claim in five minutes with direct
links. Live console: https://interlock-ml.vercel.app. Demo video: https://youtu.be/fwc64XI4ejo.
Two new DataHub skills — datahub-impact-gate (should-I-merge-this blast-radius analysis) and
datahub-contract-author (generate a data contract from live metadata) — plus a fix to a broken SDK
example, prepared as upstream PRs. See skills/.
# 1. Bring up DataHub + Postgres + dbt + a trained model, all loaded and lineage-linked
python demo/setup.py
# 2. Install Interlock
pip install -e .
# 3. Gate a change (this repo ships six ready scenarios under demo/scenarios/)
interlock check --repo demo --base main --head rename-airport-fee --contracts demo/contracts
# -> BLOCK: airport_fee renamed; fare_predictor at risk; 12%+ of decisions flip; contract allows 0.5%
# 4. See the model coverage and graph health
interlock status --contracts demo/contractsThe gate is also a GitHub Action, a pre-commit hook, an
MCP server (interlock-mcp), and a post-merge watcher
(interlock-watch) for drift that arrives from outside a PR.
PR diff ──▶ change extraction ──▶ DataHub lineage ──▶ blast radius (production models)
(dbt manifest / sqlglot) (skip-cache) │
▼
verdict ◀── policy engine ◀── contract eval ◀── model replay (pre vs post) ── warehouse
│ (deterministic, (native DataHub │
│ fails closed) assertions) ▼
├──▶ block the merge (non-zero exit, PR comment) decision-flip rate
├──▶ write back: incident + tag + structured props + assertion result
└──▶ remediation PR (LLM-drafted, hard-validated by a real dbt compile)
Full architecture in docs/architecture.md.
| Path | What |
|---|---|
src/interlock/ |
the library: change, contracts, policy, replay, graph, agent, remediate, mcp, watch, eval |
demo/ |
the reproducible stack: Postgres + NYC TLC + dbt + LightGBM/MLflow + six scenarios |
action/ |
the GitHub Action |
eval/ |
the graded suite, hidden answer key, and the scorecard |
examples/ |
captured real gate runs: verdicts, PR comments, remediation patches |
skills/ |
the DataHub skills contributed upstream |
console/ |
the web console |
docs/ |
architecture, the gate, evaluation, MCP, the watcher |