Skip to content

Preserve container sample aspect ratio through smart rendering - #11

Merged
DeepRegular merged 2 commits into
DeepRegular:mainfrom
SteveYu000:codex/sar-propagation
Nov 2, 2026
Merged

DeepRegular merged 2 commits into
DeepRegular:mainfrom
SteveYu000:codex/sar-propagation

Conversation

@SteveYu000

@SteveYu000 SteveYu000 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

An AV1 MP4 or Matroska file can carry its sample aspect ratio on AVStream
while codecpar has none, so exports and the outline can lose its display
aspect. Conversely, a container may deliberately override an H.264/HEVC
bitstream's aspect without rewriting its pictures. Both values need to be
preserved in their respective places.

Use av_guess_sample_aspect_ratio for the outline and output display
metadata. Keep input codecpar unchanged, so boundary encoders retain the
source's coded SAR. Set the guessed value on the output AVStream and its
own codecpar copy: FFmpeg 7.1 requires these output fields to agree. This
does not change the source parameters or rewrite copied SPSs. AV1 keeps
its display aspect through the container without an extra encoder SAR setting.

This addresses the review feedback.
The reviewer's unchanged 30 s / 640x360 generator and two keep ranges now
produce container DAR 4:3, with all 12 SPSs retaining SAR 1:1. The
previous revision wrote 3:4 in the four re-encoded SPSs. Non-square coded
SAR and HEVC are covered too.

Split from #10. This PR contains only SAR propagation, a generated regression
suite, its CI hook and the test-documentation entry. AV1 configuration #12,
clock/VFR #13 and M2TS arrival times remain separate.

Validation actually run

Windows x64 / Rust 1.98 / MSVC / FFmpeg 7.1.1. The revised Rust CLI and
conformdiag were compiled locally; baseline and previous-PR executables are
the previously compiled versions. All three use the same input bytes and
the same final test script.

  • 60/60: 44 exports + 16 outline checks. All original 48 checks still
    pass. The eight new exports cover AVC/HEVC, coded SAR 1:1/4:3 with a
    container override, and MP4/MKV output. Each output must strictly decode,
    retain the display SAR, and preserve the original SAR in every SPS.
  • The previous PR revision passes 52/60 and fails all eight new exports;
    v0.8.39 passes 40/60 on these Windows fixtures.
  • All 44 outputs retain timebase, PTS/DTS, packet duration and initial
    codec configuration against their respective controls. Original-suite
    copied packets/pixels are unchanged from the previous PR. For the eight
    override exports, all video packet data and decoded pixels are identical
    to v0.8.39, while the container keeps its intended display aspect.
  • Subsequent TS remuxes retain the coded SAR in every SPS. The four AVC
    remuxes also strictly decode. HEVC remux checking is limited to SAR
    headers: a separate FFmpeg 7.1 strict-decode failure also reproduces with
    unmodified v0.8.39; this PR does not claim to fix that failure. The MP4/MKV
    outputs themselves must strictly decode for both codecs.
  • Rust workspace: 570 passed, four previously confirmed Windows baseline
    failures filtered
    . Clippy/all-targets exits 0 with existing warnings.

The building.md row is aligned and explicitly counts 44 exports +
16 outlines = 60
. Whole-file equality is not promised: required container
metadata changes, generated Matroska SegmentUIDs and possible differences
between repeated encodes remain as before. No new Linux CI or hardware-player result
is claimed here.

cd rust && cargo build --release --locked --bins --example conformdiag && cd ..
bash tests/run_sar_tests.sh
# or: bash tests/ci.sh sar

The suite records commands, hashes and results in a fresh directory.
Use --fixtures to compare another executable against identical source bytes.

AV1 MP4/Matroska display aspect can live on AVStream while codecpar has no
SAR. Read the effective ratio using av_guess_sample_aspect_ratio, keep the
existing codecpar fallback, and carry it to boundary encoders and the
output stream.

Add a standalone SAR suite covering copy, mixed and encoded ranges for
AVC/HEVC/AV1 in MP4/MKV, plus public outline checks using conformdiag.
Register only this suite in CI and document its entry point.

Validation: same-input exports improve from 30/36 to 36/36, outline checks
from 10/12 to 12/12. Workspace tests 570 passed with four known Windows
baseline failures filtered; clippy exits 0. Source clocks, VFR handling,
AV1 configuration policy and M2TS behaviour are unchanged.

SAR-only split from PR DeepRegular#10.
@SteveYu000
SteveYu000 marked this pull request as ready for review November 1, 2026 12:42
@DeepRegular

Copy link
Copy Markdown
Owner

Thank you for splitting this out. I built it on Linux (Debian 13, FFmpeg 7.1) and compared it with v0.8.39.

Confirmed

  • tests/run_sar_tests.sh: v0.8.39 42/48 (the six AV1 4:3 exports lose their SAR), this branch 48/48, on the same fixtures.
  • Real recordings are unchanged: MPEG-2 .ts and --bdav output, VC-1 .m2ts and H.264 .mp4 are byte-identical to v0.8.39. The one MP4 that differs (anamorphic MPEG-2 to .mp4) differs only in the tkhd width, 720 → 853.33, which is the correct display width.

One problem: the boundary encoder now writes the container's SAR into the bitstream

When the container's SAR differs from the one in the bitstream, open_input overwrites codecpar, so the re-encoded pictures at the seams get the container's value in their SPS while the copied pictures keep the source's. One output then switches its in-band SAR at every seam. That is the common case of an MKV whose aspect was fixed with mkvmerge (--aspect-ratio) without re-encoding:

ffmpeg -f lavfi -i testsrc2=rate=25:size=640x360 -t 30 -c:v libx264 -g 50 syn.mp4
ffmpeg -i syn.mp4 -c copy -aspect 4:3 syn43.mkv
smartcut syn43.mkv --keep 3.3-12.1 --keep 15-22.7 -o out.mkv
ffmpeg -i out.mkv -c copy -bsf:v trace_headers -f null - 2>&1 | grep -E 'aspect_ratio_idc|sar_(width|height)'

v0.8.39 writes aspect_ratio_idc = 1 in all 12 SPSs (and loses the container's 4:3). This branch keeps 4:3 on the container, but the four re-encoded SPSs say aspect_ratio_idc = 255, sar 3:4, while the eight copied ones still say 1. A player that follows the container is fine; anything that follows the bitstream, or a later remux into TS, flips the aspect at the seams.

Could you keep codecpar's own SAR for the boundary encoders, and use the guessed SAR only where the bitstream carries none? That would mean the output AVStream (as you already do), the outline, and the encoder only when the source bitstream has no SAR of its own (AV1 here). If you would rather not track "has none", giving the encoder the bitstream's SAR and the output stream the guessed one would also work for the AV1 cases in your suite, as far as I can see.

Two small things in building.md: the new row is one column off from the others, and the count says 48 while the suite reports 36 exports + 12 outlines. Writing 48 is fine if it counts both, but please align the column.

Once the SAR change is in, I will merge this.

Preserve the input codecpar SAR for boundary encoders. Use the guessed
display SAR only for the outline and the output stream's metadata. Set
both output SAR fields to satisfy libavformat's consistency check without
changing the source parameters or the copied SPSs.

Reproduce the reviewer's container override: 12 SPSs now all retain 1:1
while the output container keeps 4:3 DAR. Add AVC/HEVC regressions with
square and non-square coded SAR, MP4/MKV outputs and TS header inspection.
Correct the building.md column and state 44 exports plus 16 outlines.

Actually build with Rust 1.98/MSVC/FFmpeg 7.1.1: 60/60 media checks,
570 workspace tests passed with four known Windows baseline failures
filtered; Clippy exits zero. Timing remains unchanged in all 44 outputs,
and the eight new override exports retain all baseline video packets
and decoded pixels. Document the pre-existing HEVC TS remux decode
failure separately from the SAR assertions.

Addresses the review on PR DeepRegular#11; no AV1 configuration, VFR or M2TS changes.
@SteveYu000

Copy link
Copy Markdown
Contributor Author

Thanks for catching this. Fixed in ac593fc.

open_input now leaves the input codecpar unchanged, so boundary encoders retain the source’s coded SAR. The guessed display SAR is used for the outline and output metadata. Only the output stream’s own codecpar copy is adjusted to match its AVStream, as FFmpeg 7.1 requires those two output fields to agree.

Your exact reproducer now retains container DAR 4:3, with all 12 SPSs reporting SAR 1:1. The existing AV1 cases still pass.

I rebuilt and tested the revised Rust CLI on Windows with FFmpeg 7.1.1:

  • 60/60 checks: 44 exports and 16 outlines, including new AVC/HEVC container-override regressions.
  • 570 unit tests passed, with the same four known Windows baseline failures filtered.
  • Clippy passed with existing warnings only.

The building.md row is now aligned and explicitly states the export/outline counts.

One separate limitation is documented: the HEVC-to-TS remux strict-decode failure also occurs with unmodified v0.8.39. The new HEVC remux assertions check SPS SAR preservation; this change does not claim to fix that decoding issue.

@DeepRegular

Copy link
Copy Markdown
Owner

Thank you, this is exactly the fix I hoped for. I rebuilt ac593fc on Linux (Debian 13, FFmpeg 7.1) and checked it against v0.8.39.

  • Your reproducer: container DAR 4:3, and all 12 SPSs say aspect_ratio_idc = 1, the same as v0.8.39. No more switching at the seams.
  • tests/sar_tests.py on the same fixtures: v0.8.39 46/60, this branch 60/60.
  • Real recordings: MPEG-2 .ts, --bdav, VC-1 .m2ts and H.264 .mp4 are byte-identical to v0.8.39. The anamorphic MPEG-2 .mp4 again differs only in the tkhd width. The .mkv outputs differ only in the track header: v0.8.39 wrote just DisplayUnit = 4 (unknown), while this branch writes the 4:3 display size for an anamorphic MPEG-2 recording and leaves the element out for square pixels. Both are improvements. The decoded pictures are identical in every case.
  • Workspace unit tests and clippy pass, with the same warnings as main.

Merging now. Thanks again for splitting #10 up; it made each change much easier to check.

@DeepRegular
DeepRegular merged commit 8e44d8f into DeepRegular:main Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants