Repository navigation
feat: report unresolved local flags on evaluate_flags snapshots - #1133
Conversation
Implements the "Snapshot unresolved flag reporting" and "Warning for flags local evaluation never resolves" requirements of the evaluate-flags spec (PostHog/sdk-specs#85). - Add `FeatureFlagEvaluations.unresolved_flags`, with an `UnresolvedFlagReason` for each flag and the spec's fixed precedence. - Report `local_evaluation_inconclusive` instead of `flag_missing` when a snapshot read hits an unresolved flag. - Do not stop a condition at the first inconclusive filter, so a filter that does not match resolves the condition locally. - Check `active` before experience continuity, also for dependencies. - Log one warning per flag that local evaluation can never resolve, repeated only when its definition changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: 39793b31-809e-4689-a5d8-b4359cbb22c1
There was a problem hiding this comment.
PostHog Review · Standard · reviewed 5572e1c: 3 Should fix. This pull request is large, so the review ran in 4 parts with less depth than usual.
This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.
See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.
posthog-python Compliance ReportDate: 2026-10-28T20:00:33.406868+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
dustinbyrne
left a comment
There was a problem hiding this comment.
The added lock-ownership regression test behaves as intended. The known diagnostic-precedence limitation remains a cross-SDK follow-up, as discussed.
AI-assisted review.
💡 Motivation and Context
Why: sdk-specs #85 defines how a server SDK reports flags that local evaluation could not resolve. Today such a flag is absent from the
evaluate_flags()snapshot, so it looks the same as a flag that does not exist, and$feature_flag_calledreportsflag_missingfor both. Withonly_evaluate_locally=Truethis is permanent. posthog-node already implements this (PostHog/posthog-js#5234).This replaces #1033, which was closed as stale. It is rebased on
mainand adds the spec parts that #1033 did not have.Changes:
FeatureFlagEvaluations.unresolved_flagsmaps each unresolved flag to anUnresolvedFlagReason(experience_continuity,unsupported_definition,unresolved_dependency,missing_context). When several causes apply, the spec's fixed precedence picks one. Snapshots fromonly()/only_accessed()have no entries, and reading the map is not an access.local_evaluation_inconclusiveinstead offlag_missing. Values, keys and capture enrichment do not change.missing_context; a bad filter value isunsupported_definition.AsyncClientevaluates only remotely, so its snapshots have no unresolved flags. This is documented.Behavior changes (in the changeset):
/flags. A static cohort still forces the fallback only when it is the first filter in its condition that cannot be checked, as in posthog-node.falselocally.Divergences from the spec or posthog-node:
false, as before, so it is not reported asmissing_context. posthog-node does the same.unsupported_definitionbut are not in the load-time warning, because the definition alone does not show them.💚 How did you test it?
Unit tests mirror every scenario tagged
@unresolved_flags_capableand@unresolvable_flag_warning_capableinacceptance/public/evaluate-flags.feature, including the device-id row, plus precedence, the condition loop, dependencies, warning pruning and unexpected errors. Run locally:pytest --timeout=30. Live AI-provider tests were run with the provider API keys unset.ruff format --check .,ruff check ., mypy with the baselinemake public_api_checkNo manual testing in an app.
📝 Checklist
If releasing new changes
sampo addto generate a changeset file🤖 Agent context
Autonomy: Human-driven (agent-assisted)
main. Then added reasons for the newer regex raise sites, the spec's reason precedence (so a later server error cannot overwrite a higher reason), the condition loop change, dependency checks, theunsupported_definitionwarning and per-flag pruning of the warning history.Created with PostHog Desktop
🤖 Generated with Claude Code