Skip to content

Reuse feature-flag responses across connections to the same host - #630

Open
vuanhphung wants to merge 5 commits into
mainfrom
perf/conn-ffcache
Open

vuanhphung wants to merge 5 commits into
mainfrom
perf/conn-ffcache

Conversation

@vuanhphung

@vuanhphung vuanhphung commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Every sequential Thrift reconnect refetches flags after the last client closes. Five rotated A/B runs measure 30 GETs per 30 reconnects and paired connect+openSession p50 186.94 -> 134.36 ms (28% faster). Skipping that GET moves a TLS handshake into the first OpenSession, so CONNECTION_OPEN operation_latency_ms p50 rises 65 -> 134 ms for sequential reconnects despite faster total setup; notify Node.js telemetry dashboard owners before release.

Keep main's sharing key unchanged (workspace org-id, otherwise normalized host), including active-request coalescing across auth types and SPOG hosts. Retain up to 100 idle successful responses for every auth type until their original monotonic expiry, without retaining credentials, connections or timers; successful responses without a valid TTL use the 15-minute default, while failures preserve shorter retry windows. The live TTL is 3600 s, so a server-side flag flip can take up to an hour to reach a reconnecting app, the same as a long-lived client on main.

Local verification: 1,836 unit tests, 29 live Thrift/kernel e2e tests plus a warm-suite isolation repro, lint/format/types/build, typed/null/limit/cancel checksums, and 1,000-cycle 64-MiB-heap lifecycle checks pass. OAuth mock runs substitute credentials, not real grants; live external-token runs use the issued token.

Connect, five-run p50 / p95 (ms) main branch Flag GETs/run
Live PAT, 30 sequential reconnects 120.74 / 148.17 0.20 / 0.33 30 -> 0
Live PAT, burst50 after idle 125.76 / 157.04 2.48 / 3.63 1 -> 0
Full mock PAT, 30 reconnects 121.26 / 122.63 0.12 / 0.23 30 -> 0
Full mock PAT, burst50 after idle 123.54 / 124.95 2.10 / 2.84 1 -> 0

Adversarial review: converged in two rounds; bot re-review approved with no issues after fixing its one Low finding and changelog nit.

This pull request and its description were written by Isaac.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@vuanhphung
vuanhphung marked this pull request as ready for review November 19, 2026 09:37

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

Verdict: 1 Low

Looks good, with 1 low concern. Idle retention, expiry on the monotonic clock, the LRU bound, and coalescing of in-flight fetches are correct, and the tests cover them. Nit: the CHANGELOG entry is a loose paragraph rather than a bullet with a PR reference like its neighbours.

Comment thread lib/FeatureFlagCache.ts Resolved
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@vuanhphung

Copy link
Copy Markdown
Collaborator Author

Addressed the review-body nit in c712c07: the CHANGELOG entry now uses a bullet with the PR reference (#630).

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).

✅ No issues identified.

This branch was successfully deployed

1 active deployment
azure-prod — c712c078 Deployed Nov 19, 2026 by vuanhphung via e2e-test (22.23.4, 22_23_4) #2147
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant