Skip to content

[Jetty] Added Workflow reads of a Stream on both backends. - #353

Draft
moetemp wants to merge 27 commits into
moe/jetty/v2/j2-native-pluginfrom
moe/jetty/v2/j3-workflow-reads
Draft

moetemp wants to merge 27 commits into
moe/jetty/v2/j2-native-pluginfrom
moe/jetty/v2/j3-workflow-reads

Conversation

@moetemp

@moetemp moetemp commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

What changed?

This PR adds Workflow reads: WorkflowStream.subscribe(*topics, from_offset=None, wait_new_records=True, prefixes=(), create_wait=None) and WorkflowTopic.subscribe(...), with the S102 signature (S89).
Each is an async iterator in Workflow code that yields Record, Superseded and Closed, and decodes each body with its Topic's type. Each page is in History, so a replay reads the same items.
Native: a page is the System Nexus ReadRecords operation. At the end of the Stream the server parks the read and completes it on an append or a close.
Redis and memory: a page is the Local Activity __temporal_stream_read:<backend>, which the plugin adds to the Worker. It waits up to 10 s, and an empty page is followed by a durable timer from 1 s up to 60 s. The Local Activity runs the codec on the bodies. A run's own writes are Local Activities that ended before the read, so a run reads its own writes.
A read error comes back as the same Stream error class, by the failure type or the reason at the start of its message. An uncaught one fails the run.
tests/contrib/streams/test_native_server.py is the native lane, which runs when TEMPORAL_STREAMS_NATIVE_TARGET names a server with the Stream service.

Why?

A Workflow needs to read Streams, for example an orchestrator that follows a child agent's turns.

How did you test it?

tests/contrib/streams/test_workflow_reads.py runs on memory and Redis. It covers a read of another Workflow's Stream with its replay, one Topic, a missing Stream, an uncaught read error, a read in a query, a run that reads its own writes, and a codec.
The native lane passes against jt-srv's t5 dev server: a client round trip, a Workflow write and read with both replays, and Redis plus native in one Worker.
pytest tests/contrib/streams and poe lint pass.

…kflow-reads

# Conflicts:
#	temporalio/contrib/streams/native.py
…kflow-reads

# Conflicts:
#	temporalio/contrib/streams/native.py
…kflow-reads

# Conflicts:
#	temporalio/contrib/streams/_errors.py
#	temporalio/contrib/streams/redis.py
#	temporalio/contrib/streams/workflow.py
…kflow-reads

The read Local Activity takes the Stream runtime of the Worker's client from the attempt, since each client has its own (S123). The two-backend test marks Redis as the default, as S119 asks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant