Repository navigation
bench: AV1, HEVC and H.264 in Chrome as a stand-in for NVENC (#82, first stage) - #167
Merged
Merged
Conversation
…rst stage) The user's PC and its NVENC are not at hand, so Chrome's WebCodecs encoders on the Mac stand in. scripts/chrome_codecs.mjs drives headless Chrome over the DevTools protocol (mock keychain, a profile of its own, a page served on 127.0.0.1 only): a page encodes a Y4M with VideoEncoder in real-time mode at a constant bit rate, a key frame first and predicted frames after it, decodes the chunks with VideoDecoder and compares every decoded frame with its source. emberwave encodes the same pictures at the bytes each Chrome stream used, and emberwave-cli quality measures it. Two sets of pictures, both synthetic: the benchmark scene, and a copy with fresh noise in every frame (make_grainy.py). The results are in BENCHMARKS.md: at the same bytes the codecs with predicted frames lead in PSNR, by 15 to 18 dB on the scene and 3 to 5 dB on the noisy copy, decode in 1.0 to 5.9 ms at Moonlight's rates, and say little about D67's chroma weights, which stay. Rows that Chrome could not make are listed. Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
…data says A review of the Chrome comparison found that the "queued" decode time included the decoder's start and the key frame, that the frames were copied out while the one-at-a-time decode was timed, that the driver could leave Chrome and its profile behind (on a failed start, a signal, or a shutdown that raced the profile's writes) and could wait for a stuck page for good, and several sentences of BENCHMARKS.md and D67 that said more than the data. The page decodes three times now: one chunk at a time, timed, frames not copied; all at once, timed from the first frame out to the last; and once more with the frames copied out for the comparison. A decoder that fails keeps what the encoder made in the result. The driver gives every run a Chrome of its own in a process group that it ends, on a timeout and on SIGINT and SIGTERM too, serves only the Y4M files of testdata/generated, parses its flags (--force is a flag), and does not keep a failed run. The tables are measured again with it. The bytes and the PSNR of the 50 streams that ran twice agree exactly; the milliseconds do not, and say so. The text now gives the key frame's share of the bytes and that emberwave gets more bytes than the predicted frames cost, the scene's grain ceiling of about 46 dB, the chroma gap row by row, and each range from the new data. Claude-Session: https://claude.ai/code/session_019fJGoCTvP5e7n599q3ZQ42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Issue #82 asks how emberwave compares with the streams of a PC's NVENC. The PC is not at hand, so Chrome's WebCodecs encoders on the Mac stand in: HEVC and H.264 in hardware (VideoToolbox) and software, AV1 in software, decoded by
VideoDecoder. This is the first stage; #82 stays open for NVENC's own streams, moving game footage and the iPhone.Changes
scripts/chrome_codecs.mjsandscripts/chrome_codecs/page.html: headless Chrome over the DevTools protocol (mock keychain, a profile of its own, the page on 127.0.0.1 only, a Chrome for each run, ended with its process group, on a timeout and on SIGINT and SIGTERM) encodes a Y4M, a key frame first and predicted frames after it, in real-time mode at a constant bit rate, decodes it three times (one chunk at a time, all at once, and once with the frames copied out for the comparison) and compares every frame with its source (PSNR per plane).emberwave-cli encode --gpuandqualitymake emberwave's side at the bytes each Chrome stream used.scripts/chrome_codecs/make_grainy.py: a copy of the benchmark scene with fresh noise in every frame, the hard case for prediction.What the numbers say
VideoDecodertook 0.9 to 5.3 ms a frame one at a time on the scene at Moonlight's rates (hardware decoders), and up to 12.6 ms at emberwave's bytes on the noisy copy.Review
A Sonnet reviewer read the harness. Its findings that held changed the code (the queued decode time, the copy during the timed pass, Chrome left behind on a failed start or a signal, no timeout for a stuck page, the flag parser, the file server's path check, a decoder failure losing the encoder's result) and the text (the key frame's share of the bytes, the chroma gap row by row, a ceiling that is the scene's grain and not the encoder's, ranges that were off). The tables were measured again with the changed harness; the bytes and the PSNR of the 50 streams that ran in both runs agree exactly, the milliseconds do not and the text says so.
Testing
node scripts/chrome_codecs.mjs alltwice (2026-11-18, M3 Max, Chrome 157, load average 5 to 9, other programs running), thensummary; SIGTERM during a run leaves no Chrome and no profile behind. No code of the libraries changes, so the Rust CI jobs are unaffected.Refs #82