Repository navigation
Conversation
…kflow-reads # Conflicts: # temporalio/contrib/streams/_errors.py # temporalio/contrib/streams/workflow.py # tests/contrib/streams/test_facade.py
…kflow-reads # Conflicts: # temporalio/contrib/streams/redis.py # tests/contrib/streams/conftest.py
…kflow-reads # Conflicts: # temporalio/contrib/streams/native.py # tests/contrib/streams/fake_native.py
…kflow-reads # Conflicts: # temporalio/contrib/streams/native.py # temporalio/contrib/streams/redis.py
…kflow-reads # Conflicts: # temporalio/contrib/streams/native.py
…kflow-reads # Conflicts: # temporalio/contrib/streams/native.py
…kflow-reads # Conflicts: # temporalio/contrib/streams/_errors.py # temporalio/contrib/streams/redis.py # temporalio/contrib/streams/workflow.py
…kflow-reads The read Local Activity takes the Stream runtime of the Worker's client from the attempt, since each client has its own (S123). The two-backend test marks Redis as the default, as S119 asks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
This PR adds Workflow reads:
WorkflowStream.subscribe(*topics, from_offset=None, wait_new_records=True, prefixes=(), create_wait=None)andWorkflowTopic.subscribe(...), with the S102 signature (S89).Each is an async iterator in Workflow code that yields
Record,SupersededandClosed, and decodes each body with its Topic's type. Each page is in History, so a replay reads the same items.Native: a page is the System Nexus
ReadRecordsoperation. At the end of the Stream the server parks the read and completes it on an append or a close.Redis and memory: a page is the Local Activity
__temporal_stream_read:<backend>, which the plugin adds to the Worker. It waits up to 10 s, and an empty page is followed by a durable timer from 1 s up to 60 s. The Local Activity runs the codec on the bodies. A run's own writes are Local Activities that ended before the read, so a run reads its own writes.A read error comes back as the same Stream error class, by the failure type or the reason at the start of its message. An uncaught one fails the run.
tests/contrib/streams/test_native_server.pyis the native lane, which runs whenTEMPORAL_STREAMS_NATIVE_TARGETnames a server with the Stream service.Why?
A Workflow needs to read Streams, for example an orchestrator that follows a child agent's turns.
How did you test it?
tests/contrib/streams/test_workflow_reads.pyruns on memory and Redis. It covers a read of another Workflow's Stream with its replay, one Topic, a missing Stream, an uncaught read error, a read in a query, a run that reads its own writes, and a codec.The native lane passes against jt-srv's t5 dev server: a client round trip, a Workflow write and read with both replays, and Redis plus native in one Worker.
pytest tests/contrib/streamsandpoe lintpass.