Skip to content

Avoid speculative reads past complete Thrift responses - #632

Draft
vuanhphung wants to merge 1 commit into
mainfrom
perf/stmt-cpu
Draft

vuanhphung wants to merge 1 commit into
mainfrom
perf/stmt-cpu

Conversation

@vuanhphung

@vuanhphung vuanhphung commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Every complete Thrift HTTP reply currently ends with a speculative read of another message header, constructing and catching InputBufferUnderrunError on the successful path. A warmed 20,000-statement profile attributes 232 ms to that exception's construction; an instrumented 2,000-statement run counts 2,000 exceptions on main and zero here, with identical RPC counts. This per-reply work consumes the single JS thread under concurrent short-statement load.

Check the transport's borrowed read/write positions after decoding each message and stop when the buffer is exhausted. Keep the existing underrun/rollback path for truncated replies and transports without borrow, including multi-message dispatch and callback behavior. Log output and telemetry code are unchanged. The kernel's HTTP decoder is Rust and does not use this loop.

Verification: 1,812 unit tests; lint/format/type-check/build; 26 live e2e tests, none skipped; 27 main/branch typed-value checksum cases; enabled log/telemetry parity; 64-MiB heap stress. Five fresh-process runs per side, rotated A/B order; mock adds 2 ms response delay. Mock LIMIT10 at c50/20 sessions raises QPS 8.1% and lowers p99 32.16 -> 26.89 ms. Gains vary by scenario; shared-warehouse live QPS/tails are noisy and sometimes worse, so no live latency/throughput improvement is claimed.

Scenario Main CPU ms/statement Branch Change
Mock SELECT 1, 50 concurrent, 1 session 0.263 0.247 -6.3%
Mock LIMIT 10, 50 concurrent, 20 sessions 0.338 0.317 -6.3%
Live SELECT 1, 50 concurrent, 20 sessions 1.321 1.249 -5.4%
Live LIMIT 10, 50 concurrent, 20 sessions (500 warmup) 1.114 1.076 -3.5%
Live kernel control, 50 concurrent, 1 session 0.883 0.886 +0.4%

Adversarial review: pending; this PR remains a draft.

This pull request and its description were written by Isaac.

Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>

This branch was successfully deployed

1 active deployment
azure-prod — b854e973 Deployed Nov 19, 2026 by vuanhphung via e2e-test (24.22.1, 24_22_1) #2139
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant