Skip to content

LangGraph: prune finished subgraph runs, and refuse static breakpoints - #290

Open
igalshilman wants to merge 2 commits into
mainfrom
claude/langchain-agents-review-kfztgf
Open

igalshilman wants to merge 2 commits into
mainfrom
claude/langchain-agents-review-kfztgf

Conversation

@igalshilman

Copy link
Copy Markdown
Contributor

Two fixes to graphs of your own in restate.agents.langgraph (langgraph.agent(name, graph=...), and SqlCheckpointSaver), from a review of #288 and #289.

history="latest" kept every subgraph run

LangGraph stores each subgraph run in a namespace named after its task (node:task_id, one per run), and SqlCheckpointSaver(history="latest") only pruned within the namespace being written. Every subgraph run's last checkpoint, channel values and writes were kept for good: a graph agent with a sub-agent subgraph stored a full copy of the sub-agent's messages every turn, the growth latest exists to prevent.

Each aput now also deletes the subgraph runs that the graph's earlier steps started: those whose checkpoints name (in their metadata's parents) a checkpoint of that graph older than the one being stored. Checkpoint ids sort in time order. It keeps:

  • the runs of the steps from the stored checkpoint on. With durability="async", LangGraph may store a checkpoint several steps after a later step started its runs. With "exit", a paused run names the very checkpoint being stored;
  • a paused subgraph, as the graph's checkpoint stays until it resumes;
  • the runs below a subgraph with its own history (checkpointer=True, a namespace without a task id). That subgraph's own checkpoints decide them, since it can resume a paused step after the graph above it moved on.

Static breakpoints broke a key for good

A static breakpoint (interrupt_before, interrupt_after) stops a run with nothing to wait for. The agent then answered with the run half done (often "", the text of a reply that only calls tools) and left its tool calls without results. The next message came after those calls, a model provider refused that history (recorded as a 424 refusal), and every later message on the key failed the same way until clear. create_agent(..., interrupt_before=["tools"]) is a common way to get there.

  • check() now refuses a compiled graph with static breakpoints, pointing to interrupt(), which waits for a signal.
  • A subgraph's breakpoints cannot be seen up front: a run stopped at one now fails with a terminal error, which closes the run, so the next message starts afresh. A run counts as stopped at a breakpoint only when it has steps to go and no task has an interrupt.

Tests

tests/agents_langgraph_checkpointer.py:

  • No subgraph run is left once its step is done, under sync, async and exit. Before the fix, three runs left three inner:<task_id> namespaces.
  • A paused subgraph is kept until its step is done, and resumes whole (the node before its interrupt runs once, and a channel written only at its start is still there), under all three durabilities. The saver runs on a slightly slowed database, so that with async the graph's checkpoints are stored after the subgraph has started.
  • A paused run below a checkpointer=True subgraph survives the graph moving on (update_state), and is deleted once that subgraph's step is done.

tests/agents_langgraph_graph.py:

  • A graph with interrupt_before or interrupt_after is refused.
  • A turn stopped at a subgraph's breakpoint fails with a 500, and the next message is answered. Before the fix, that turn answered 200 "".
  • at_breakpoint tells a subgraph's breakpoint from an interrupt() in a node's second subgraph call.

Each test was checked against a broken variant of its fix and fails on it: keeping only runs of the stored checkpoint (!= instead of <, which loses a paused subgraph's state under async), ignoring parents, pruning below a checkpointer=True subgraph, and the paused.next-only breakpoint check.

tests/agents_langgraph*.py (62) and the LangGraph example tests pass. ruff, pyright and mypy are clean.

Not in this PR

  • A pre-existing gap from LangGraph: interrupt() as a Restate signal, langgraph.tools(), and create_agent #289: _interrupts looks for a subgraph's interrupts in the subgraph snapshots LangGraph reads (node:task_id), so an interrupt() in a node's second subgraph call (node:task_id|1) is not waited for, and the turn answers half done, as before this PR. Fixing it means counting resumes from the pending writes of every namespace.
  • In a graph of your own, an error from a plain LangChain tool can be taken for the model's provider refusing. The built-in loop marks tool errors with from_tool; the graph path doesn't.
  • context=ctx replaces a graph's own runtime context (create_agent(context_schema=...)).

Generated by Claude Code

…d steps

LangGraph stores each subgraph run in a namespace named after its task
(node:task_id), and SqlCheckpointSaver(history="latest") only pruned within
the namespace being written. Every subgraph run's last checkpoint, values
and writes were kept for good: a graph agent (langgraph.agent(graph=...))
with a sub-agent subgraph stored a full copy of its messages each turn.

Each aput now also deletes the subgraph runs that the graph's earlier steps
started: those whose checkpoints name, in their metadata's "parents", a
checkpoint of that graph older than the one stored (checkpoint ids sort in
time order). Kept are:

- the runs of the steps from the stored checkpoint on: with
  durability="async", LangGraph may store a checkpoint after a later step
  has started its runs, and with "exit" a paused run names the very
  checkpoint being stored;
- a paused subgraph, as the graph's checkpoint stays until it resumes;
- the runs below a subgraph with its own history (checkpointer=True, a
  namespace without a task id), which its own checkpoints decide: it can
  resume a paused step after the graph above it moved on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RePHkTSkJ7FFKSJs2Vsub9
A static breakpoint (interrupt_before, interrupt_after) stops a run with
nothing to wait for. langgraph.agent(graph=...) then answered with the run
half done (often "", the text of a reply that only calls tools), and left
its tool calls without results: the next message came after them, a model
provider refused that history, and every later message on the key failed
the same way until clear.

check() now refuses a compiled graph with static breakpoints (such as
create_agent(..., interrupt_before=["tools"])), pointing to interrupt(). A
run that stops at a subgraph's breakpoint, which check() cannot see, fails
with a terminal error, which closes the run so the next message starts
afresh. A run counts as stopped there only when it has steps to go and no
task has an interrupt: an interrupt() in a node's second subgraph call is
on the node's task, though not in the subgraph snapshots LangGraph reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RePHkTSkJ7FFKSJs2Vsub9

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues. Two minor suggestions on the static-breakpoint path; the subgraph pruning looks right.

Reviewed changes

Reviewed both commits: subgraph-run pruning in SqlCheckpointSaver(history="latest") and the refusal of static breakpoints in graph agents.

  • Prune finished subgraph runs — aput now also deletes node:task_id namespaces below the stored graph whose parents[ns] checkpoint is older than the one being stored. Runs started by the current step, paused subgraphs, and anything below a checkpointer=True subgraph are kept.
  • Refuse top-level static breakpoints — check() raises TypeError for a compiled graph with interrupt_before/interrupt_after.
  • Fail a run stopped at a subgraph's breakpoint — after a run stops with steps still to go and no interrupt, at_breakpoint raises a TerminalError, which answer_once uses to close the run.
  • Tests — pruning under sync/async/exit, a paused subgraph resuming whole, a checkpointer=True subgraph, the refusal, and at_breakpoint telling a breakpoint from a second-call interrupt(). The in-process tests pass locally; the Docker-harness test is CI-only.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

start: Any
if asked:
resumes = await _resumed(ctx, app)
resumes = await _resumed(ctx, app, await _paused(app))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The breakpoint check only runs inside the while loop. A run picked up again here (a retried attempt, or an earlier prompt's unfinished run that answer_once finishes first) gets resumes is None → start = None. LangGraph treats None input as "continue past the breakpoint", so if the attempt died between the subgraph stopping and close(...) committing, the retry runs the rest of the graph and answers instead of failing. I confirmed this with InMemorySaver: ainvoke(None) on a thread stopped at the test's pause subgraph finishes with hello x.

Technical details
# Retry path skips at_breakpoint and resumes past a subgraph's static breakpoint

## Affected sites
- python/restate/agents/langgraph/_graph.py:322-324 — the `asked` branch calls `_resumed` and falls back to `start = None` without checking `at_breakpoint(paused)`

## Required outcome
- A run found stopped at a subgraph's static breakpoint on entry fails with `TerminalError(BREAKPOINTS)`, the same way it does in the loop, instead of being resumed with `None`.

## Suggested approach
```python
if asked:
    paused = await _paused(app)
    resumes = await _resumed(ctx, app, paused)
    if resumes is None and at_breakpoint(paused):
        raise TerminalError(BREAKPOINTS)
    start = Command(resume=resumes) if resumes is not None else None
```

"""Raise ``TypeError`` if ``graph`` has no ``messages`` in its state, or has static breakpoints."""
if "messages" not in graph.channels:
raise TypeError("the graph's state has no messages: build it on LangGraph's MessagesState")
if isinstance(graph, Pregel) and (graph.interrupt_before_nodes or graph.interrupt_after_nodes):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: breakpoints on a subgraph that is a node can be seen up front. builder.compile().get_subgraphs(recurse=True) yields ('pause', ...) with interrupt_before_nodes == ['greet'] for the test graph. If check() walks these too (compiling a StateGraph first), the agent is refused when it is registered, not after a turn has already made model and tool calls. The runtime at_breakpoint check would then only be needed for subgraphs invoked from inside a node function.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown

Test Results

  8 files  ±0    8 suites  ±0   3m 22s ⏱️ - 1m 27s
 71 tests ±0   71 ✅ ±0  0 💤 ±0  0 ❌ ±0 
322 runs  ±0  322 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 17f8699. ± Comparison against base commit 8ad9fe2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants