Skip to content

bench: AV1, HEVC and H.264 in Chrome as a stand-in for NVENC (#82, first stage) - #167

Merged
with-developer merged 3 commits into
mainfrom
bench/chrome-codecs
Nov 18, 2026
Merged

with-developer merged 3 commits into
mainfrom
bench/chrome-codecs

Conversation

@with-developer

@with-developer with-developer commented Nov 18, 2026 •

Copy link
Copy Markdown
Owner

Purpose

Issue #82 asks how emberwave compares with the streams of a PC's NVENC. The PC is not at hand, so Chrome's WebCodecs encoders on the Mac stand in: HEVC and H.264 in hardware (VideoToolbox) and software, AV1 in software, decoded by VideoDecoder. This is the first stage; #82 stays open for NVENC's own streams, moving game footage and the iPhone.

Changes

  • scripts/chrome_codecs.mjs and scripts/chrome_codecs/page.html: headless Chrome over the DevTools protocol (mock keychain, a profile of its own, the page on 127.0.0.1 only, a Chrome for each run, ended with its process group, on a timeout and on SIGINT and SIGTERM) encodes a Y4M, a key frame first and predicted frames after it, in real-time mode at a constant bit rate, decodes it three times (one chunk at a time, all at once, and once with the frames copied out for the comparison) and compares every frame with its source (PSNR per plane). emberwave-cli encode --gpu and quality make emberwave's side at the bytes each Chrome stream used.
  • scripts/chrome_codecs/make_grainy.py: a copy of the benchmark scene with fresh noise in every frame, the hard case for prediction.
  • BENCHMARKS.md, a new last section with two tables and what they say; ARCHITECTURE.md D67 gets a follow-up bullet; one sentence in each README.

What the numbers say

  • At the same bytes the codecs with predicted frames lead in PSNR, by 15.4 to 17.7 dB of luma on the benchmark scene and 3.1 to 4.6 dB on the noisy copy, with emberwave given more bytes than the predicted frames cost (the key frame is 12 to 89 % of the bytes of a stream). Both pictures are synthetic, and the scene's motion is easy to predict; its grain sets a ceiling near 46 dB, which the codecs reach.
  • Chrome's VideoDecoder took 0.9 to 5.3 ms a frame one at a time on the scene at Moonlight's rates (hardware decoders), and up to 12.6 ms at emberwave's bytes on the noisy copy.
  • The encoders did not fill the asked rate on the scene (AV1 reached 3.8 of 20 Mbit/s at 1080p).
  • The numbers say little for D67's chroma weights; they stay upstream's.
  • Six rows did not run: Chrome's hardware H.264 encoder returned no frames at 336 and 576 Mbit/s, its hardware HEVC stream at 336 Mbit/s and 1440p could not be decoded, and its software H.264 encoder is refused at those rates.

Review

A Sonnet reviewer read the harness. Its findings that held changed the code (the queued decode time, the copy during the timed pass, Chrome left behind on a failed start or a signal, no timeout for a stuck page, the flag parser, the file server's path check, a decoder failure losing the encoder's result) and the text (the key frame's share of the bytes, the chroma gap row by row, a ceiling that is the scene's grain and not the encoder's, ranges that were off). The tables were measured again with the changed harness; the bytes and the PSNR of the 50 streams that ran in both runs agree exactly, the milliseconds do not and the text says so.

Testing

node scripts/chrome_codecs.mjs all twice (2026-11-18, M3 Max, Chrome 157, load average 5 to 9, other programs running), then summary; SIGTERM during a run leaves no Chrome and no profile behind. No code of the libraries changes, so the Rust CI jobs are unaffected.

Refs #82

…rst stage)

The user's PC and its NVENC are not at hand, so Chrome's WebCodecs encoders
on the Mac stand in. scripts/chrome_codecs.mjs drives headless Chrome over
the DevTools protocol (mock keychain, a profile of its own, a page served on
127.0.0.1 only): a page encodes a Y4M with VideoEncoder in real-time mode at
a constant bit rate, a key frame first and predicted frames after it, decodes
the chunks with VideoDecoder and compares every decoded frame with its
source. emberwave encodes the same pictures at the bytes each Chrome stream
used, and emberwave-cli quality measures it.

Two sets of pictures, both synthetic: the benchmark scene, and a copy with
fresh noise in every frame (make_grainy.py). The results are in
BENCHMARKS.md: at the same bytes the codecs with predicted frames lead in
PSNR, by 15 to 18 dB on the scene and 3 to 5 dB on the noisy copy, decode in
1.0 to 5.9 ms at Moonlight's rates, and say little about D67's chroma
weights, which stay. Rows that Chrome could not make are listed.

Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
…data says

A review of the Chrome comparison found that the "queued" decode time
included the decoder's start and the key frame, that the frames were copied
out while the one-at-a-time decode was timed, that the driver could leave
Chrome and its profile behind (on a failed start, a signal, or a shutdown
that raced the profile's writes) and could wait for a stuck page for good,
and several sentences of BENCHMARKS.md and D67 that said more than the data.

The page decodes three times now: one chunk at a time, timed, frames not
copied; all at once, timed from the first frame out to the last; and once
more with the frames copied out for the comparison. A decoder that fails
keeps what the encoder made in the result. The driver gives every run a
Chrome of its own in a process group that it ends, on a timeout and on
SIGINT and SIGTERM too, serves only the Y4M files of testdata/generated,
parses its flags (--force is a flag), and does not keep a failed run.

The tables are measured again with it. The bytes and the PSNR of the 50
streams that ran twice agree exactly; the milliseconds do not, and say so.
The text now gives the key frame's share of the bytes and that emberwave
gets more bytes than the predicted frames cost, the scene's grain ceiling of
about 46 dB, the chroma gap row by row, and each range from the new data.

Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
@with-developer
with-developer merged commit a1c456f into main Nov 18, 2026
7 checks passed
@with-developer
with-developer deleted the bench/chrome-codecs branch November 18, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant