Skip to content

feat(feature-flags): Evaluate feature flags locally with a polling runtime client - #762

Closed
stanleyphu wants to merge 13 commits into
mainfrom
feat/feature-flags-runtime-client
Closed

stanleyphu wants to merge 13 commits into
mainfrom
feat/feature-flags-runtime-client

Conversation

@stanleyphu

@stanleyphu stanleyphu commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Description

The Python SDK had no way to evaluate feature flags locally. This adds the runtime client the Node and .NET SDKs already have: it polls GET /sdk/feature-flags?payload_version=2 in the background, keeps the environment's configuration in memory, and evaluates ordered targeting rules with no request per check.

flags = workos.create_feature_flags_runtime_client()
flags.start()
flags.wait_until_ready(timeout=5)
flags.is_enabled("new-checkout", {"user": "user_123", "organization": "org_123"})
  • FeatureFlagsRuntimeClient polls on a daemon thread. AsyncFeatureFlagsRuntimeClient, from AsyncWorkOSClient, polls in an asyncio task. Evaluation is synchronous in both.
  • Evaluation matches Node and .NET: a disabled flag serves its off value, otherwise the first rule whose conditions all match wins, then the default value. The context is a flat {target_type: id} mapping, so custom target types work the same way as user and organization.
  • A failed poll keeps the last good configuration and backs off. Transient failures are retried within request_timeout, up to the client's max_retries, and a 401 stops polling.
  • Also included: bootstrap from a persisted FeatureFlagsConfiguration, on_change / on_error / on_status listeners, status, get_flag(), and a manual refresh() that works without start().
  • It handles fork(). With the SDK's default HTTP client, a sync client created before the fork (gunicorn --preload) polls on new connections in each worker. Where that isn't safe, start() and refresh() raise, and the client keeps serving its last configuration. That covers macOS, and a parent that already sent requests, from the runtime client or any other SDK call, through a caller-supplied HTTP client or the AsyncWorkOSClient. The README has the details.

Where it differs from Node on purpose:

  • It reads v2 payloads only. A v1 response fails the refresh and keeps the previous configuration.
  • Change events follow .NET: one on_change per refresh that installs a different configuration, including the first load.
  • It accepts payloads Node rejects as a whole, such as a flag without a default or off value or a rule shape it doesn't know, and skips only what it can't evaluate. get_all_flags leaves out flags without a boolean answer.

All of it is hand-maintained: the runtime package and its tests are @oagen-ignore-file, and the create_feature_flags_runtime_client methods on both clients sit inside the existing @oagen-ignore fences in _client.py. There are no new dependencies. It only adds API, so it's a minor release. Squash-merge it with this title so the intermediate fix: commits don't end up in the changelog.

How I checked it:

  • I ran it side by side with the Node SDK in our feature flags runtime lab, against the same fixture responses. All 87 scenario checks pass, 47 for Python and 40 for Node. They cover rule order, contexts, disabled flags, unknown rules, non-boolean values, bootstrap, 401, 403 and 500 responses, slow responses and v1 responses.
  • I ran it live against a local WorkOS API (v2 payload, 17 flags) with both the thread and asyncio clients. I reordered rules, disabled and re-enabled a flag, added and removed an organization target, and tried a bad API key. Node and Python returned the same value for every flag at every step.
  • The fork tests run each case in a subprocess. They pass on macOS and on Linux (python:3.12-slim in Docker).

Documentation

Does this require changes to the WorkOS Docs? E.g. the API Reference or code snippets need updates.

[x] Yes

The feature flags SDK docs describe the runtime client as Node only. A docs PR adding Python is still to do.

🤖 Generated with Claude Code

stanleyphu and others added 11 commits November 2, 2026 14:57
…ntime client

Add FeatureFlagsRuntimeClient, which polls GET /sdk/feature-flags
(payload_version=2) on a daemon thread and evaluates flags from an
immutable in-memory snapshot, matching the Node and .NET runtime
clients: ordered rules where the first match wins (including rules that
serve false), off_value/default_value, custom targets, a context that
never raises, bootstrap, readiness, change/error/status listeners,
manual refresh, status with refresh counts, flag inspection,
get_all_flags and close.

Python specifics: polling restarts in forked workers on first use,
each refresh is bounded by request_timeout and retries transient
failures up to the client's max_retries, context warnings are logged
once per distinct problem, and payload validation never fails on rule
kinds or operators the SDK does not know.

The evaluation, payload, change and context cases live in JSON fixtures
shaped for the shared cross-SDK corpus (AUTH-7098).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add AsyncFeatureFlagsRuntimeClient for AsyncWorkOSClient. It shares the
parser, evaluator, status and listener handling of the sync client and
polls in an asyncio task: start(), wait_until_ready(), refresh() and
close() are coroutines, evaluation stays synchronous, coroutine
listeners are awaited, and each refresh is also bounded with
asyncio.wait_for so close() cancels a request in flight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re the change baseline

Three groups of evaluation cases used the same flag definition and differed only in the context they evaluated. Each group is now one case with several evaluations, and each evaluation keeps the original case name in an optional `name` field.

Thirteen of the eighteen change cases repeated the same `before` payload. It now lives once as the top-level `baseline`, and a case without `before` starts from it. The corpus runner rejects unknown change-case fields, as it already does for evaluation cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The corpus tests built a WorkOSClient and a runtime client for every
case and never closed either, and a few sync-client tests did the same.
Leaked clients hold HTTP pools and surface as ResourceWarnings, so each
test now closes what it creates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…able

Most polls return the configuration the client already has, yet every
poll re-serialized, re-validated and re-canonicalized it and then diffed
every flag. A snapshot now carries a fingerprint of its content (the
payload serialized with sorted keys); a poll whose payload has the same
fingerprint keeps the existing snapshot and skips parsing and the diff.
Refresh counts, timestamps and readiness update exactly as before.

FeatureFlagsConfiguration holds a MappingProxyType, which cannot be
pickled, so the snapshot, and any RuntimeStatus or FlagsChangedEvent
holding one, failed pickle and copy.deepcopy. It now pickles by
rebuilding from its JSON, and copy/deepcopy return the same immutable
instance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r per waiter

wait_until_ready() raised a bare TimeoutError when flags never arrived,
dropping the refresh error that explains why. The timeout now names the
last refresh error in its message and chains it as __cause__.

After polling stopped on a 401, every waiter re-raised the one stored
AuthenticationError, so each raise appended to that instance's shared
__traceback__ from whatever thread was waiting. Each waiter now gets its
own copy of the error (same type and attributes, empty traceback), with
the stored error as its cause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…each retry

A 429 whose Retry-After did not fit in the refresh budget failed the
refresh, and the next poll then ignored the server's request and polled
again on the normal schedule. The next poll now waits at least
Retry-After, capped at 60 seconds like the Node SDK's
MAXIMUM_RETRY_AFTER_TIME_IN_MILLISECONDS, in both clients.

A retry inside one refresh only checked that its sleep fit in the
remaining budget, so the last retry could go out with almost no time
left, time out, and be reported as WorkOSTimeoutError instead of the
server's real error. A retry is now only sent when at least half a
second of the budget is left for the request itself; otherwise the
refresh fails with the last real error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both clients called listeners while holding the refresh lock. The async
client's asyncio.Lock is not reentrant, so a listener that awaited
runtime.refresh() deadlocked the polling task; in the sync client a
listener that waited on a refresh from another thread deadlocked the
same way.

A refresh now queues its notifications while it holds the lock, which
keeps them in refresh order, and delivers them after releasing it. One
thread (or task) delivers at a time: a batch that arrives while another
is being delivered, including one from a listener's own refresh, is
delivered next by the same deliverer, so batches never interleave.
Listener errors are still logged and isolated. A sync refresh can now
return before its notifications run when another thread is already
delivering; they then run on that thread.

The async polling task now sleeps on an event that a 401 sets, instead
of being cancelled mid-delivery when refresh() stops polling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fork safety. A forked child restarted polling but kept the parent's
WorkOSClient HTTP backend, so parent and child polled over the same
pooled keep-alive connection (one client TCP port for both), and a pool
lock held at fork time could hang the child's poller for good. With the
SDK's default HTTP client, the sync runtime now polls through an httpx2
client of its own with the same settings, abandons it in a forked child
and builds a new one there. A caller-supplied HTTP client cannot be
rebuilt, so polling does not resume in the child; neither does it on
macOS, where a forked process crashes in system libraries (DNS, proxy
settings, the trust store) when it opens a connection. In both cases the
client logs a warning, refresh() and start() raise, and evaluation keeps
serving the last configuration. An async client in a forked child stops
polling the same way. is_ready, configuration and is_closed now resume
polling after a fork like every other read.

Total timeout. request_timeout was passed to httpx as a per-phase
timeout, so a server sending the body slowly could hold a refresh far
longer. The sync poll now streams the body, gives each network wait only
the time left via timeouts httpcore reads at the start of each phase,
and abandons the response once the deadline passes, so a refresh ends
within request_timeout plus at most one read. A custom non-httpx
HTTPBackend still gets the remaining time per request. The request keeps
the client's URL, headers, User-Agent, base_url and max_retries
behavior. The async client keeps asyncio.wait_for.

The fork test now polls a real local server with distinct API keys for
parent and child and fails if they share a connection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hild

A runtime client created before a fork but never started, as in an app
gunicorn preloads that starts the client in each worker, could not start
in the child: an async client always raised there, and so did a sync
client with a caller-supplied HTTP client. A child now refuses to poll
only when the client polled in the parent, through a caller-supplied
HTTP client or, for an async client, on the parent's event loop and
connections, and on macOS. A client that never polled no longer warns on
first use in the child; start() and refresh() raise instead.

On macOS, once a process has created an HTTP client, a child it forks
crashes when it opens a new connection, and every WorkOSClient has one.
The messages and README now say to create the WorkOSClient, not only the
runtime client, after forking.

close() waits up to request_timeout for an in-flight poll and abandons
one that runs longer. Its docstring now says so, and the transport no
longer opens a new HTTP client once closed, so an abandoned poll cannot
leave one behind. The change listener docstring no longer says listeners
run on the refreshing thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es and forks

- With the SDK's default HTTP client, the sync runtime client polls
  through an HTTP client of its own, so the intro now says it uses the
  WorkOSClient's API key, base URL and retry settings rather than its
  HTTP client.
- Only 429, 500, 502, 503 and 504 are retried, not every 5xx.
- A client blocked in a forked worker warns only if it was polling in
  the parent; one that only called refresh() there just raises from
  start() and refresh().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stanleyphu
stanleyphu requested review from a team as code owners November 3, 2026 23:20
@greptile-apps

greptile-apps Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The changes appear safe to merge; all three previous findings are addressed.

Summary

Adds sync and async clients that poll feature flag settings and evaluate flags locally.

  • Feature flags use ordered rules and matching target IDs.
  • Sync and async clients keep feature flags fresh in the background.
  • Forked workers follow safe rules for resuming flag polling.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Start or manual refresh] --> B[Fetch v2 settings]
  B --> C{Fetch and parsing succeed?}
  C -->|Yes| D[Install snapshot and notify listeners]
  C -->|No| E[Keep last good snapshot]
  D --> F[Evaluate flags locally]
  E --> F
  E --> G{401 response?}
  G -->|Yes| H[Stop polling]
  G -->|No| I[Wait and retry]
Loading

Reviews (2) · Last reviewed commit: "fix(feature-flags): Notify change listen..." · Reviewed by Greptile

stanleyphu and others added 2 commits November 3, 2026 17:00
…and loop

A forked async client kept the parent's polling task, so close() in the
worker cancelled it and then waited on the parent's event loop, which does
not run there, and never returned. The fork reset now drops that task and
recreates the client's asyncio primitives.

A runtime client that never polled in the parent started in the worker even
when the WorkOSClient had already sent other requests through the HTTP
client it would poll through: a caller-supplied client's connections, or
the AsyncWorkOSClient's connections on the parent's loop. The base clients
now record whether they have sent a request, and the worker refuses to poll
in that case too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A refresh that changed only fields outside the flags map installed a new
snapshot without calling on_change, so a cache persisted from change events
kept the old payload. Listeners now hear about every snapshot a refresh
installs; changed_flags is empty when no flag changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@stanleyphu

Copy link
Copy Markdown
Contributor Author

Closing in favor of a four-PR stack that's easier to review: #764 (configuration and evaluation) → #765 (sync runtime client) → #766 (async runtime client) → #767 (fork safety). #767's tree is identical to this PR's head, so the Greptile fixes resolved here are included in #767.

@stanleyphu stanleyphu closed this Nov 4, 2026
@gjtorikian
gjtorikian deleted the feat/feature-flags-runtime-client branch November 4, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant