Feature Request: Add HDR Vivid (CUVA HDR Vivid) Support

[ijuniorfu]

What should the engine do?

Description

Add support for HDR Vivid (CUVA HDR Vivid) video playback in AetherEngine.

The goal is to allow HDR Vivid content to be correctly detected, decoded, and presented through both the hardware and software video decoding paths.

Requirements

• Detect HDR Vivid / CUVA HDR Vivid video content and expose it through the existing video format / HDR detection mechanism.
• HardwareVideoDecoder should be able to decode and handle HDR Vivid streams correctly.
• SoftwareVideoDecoder should be able to decode and handle HDR Vivid streams correctly.
• Preserve HDR Vivid dynamic metadata during the decoding pipeline when available.
• Make HDR Vivid distinguishable from SDR, HDR10, HDR10+, Dolby Vision, and HLG.
• Ensure HDR Vivid playback does not cause visible flashing, incorrect colors, or loss of HDR metadata.
• Support common HDR Vivid MP4/MKV sources supported by FFmpeg.

Expected Result

AetherEngine should be able to:

1. Identify a video as HDR Vivid (CUVA HDR Vivid).
2. Decode HDR Vivid through HardwareVideoDecoder when the hardware path is available.
3. Fall back to SoftwareVideoDecoder and decode the same content correctly when hardware decoding is unavailable.
4. Preserve and expose HDR Vivid metadata required by the rendering pipeline.
5. Report the detected format through the existing VideoFormat / video state mechanism.

FFmpeg Reference

FFmpeg 8.1.3 already provides HDR Vivid metadata definitions and related codec-side support. These files may be used as references for the AetherEngine implementation:

• libavutil/hdr_dynamic_vivid_metadata.h
  FFmpeg n8.1.3 — hdr_dynamic_vivid_metadata.h

• libavcodec/dynamic_hdr_vivid.h
  FFmpeg n8.1.3 — dynamic_hdr_vivid.h

Motivation

HDR Vivid is increasingly used in real-world video content. Supporting it as a first-class HDR format would allow AetherEngine-based players to correctly identify and play these files instead of treating them as generic SDR/HDR content.

The desired result is similar to the existing HDR10/HDR10+ handling, with dedicated HDR Vivid / CUVA HDR Vivid detection and support across both hardware and software decoding paths.

Acceptance Criteria

• HDR Vivid content is correctly detected.
• HardwareVideoDecoder can play HDR Vivid content.
• SoftwareVideoDecoder can play the same HDR Vivid content.
• HDR Vivid metadata is retained throughout the decoding pipeline where available.
• No visible flashing or incorrect HDR/color behavior occurs during playback.
• Existing SDR, HDR10, HDR10+, Dolby Vision, and HLG playback remains unaffected.

Apple Platform / Metal Rendering

Since Apple platforms do not provide native HDR Vivid support, would it be possible to consider using Metal for the HDR Vivid rendering/conversion stage?

For example, AetherEngine could preserve the HDR Vivid dynamic metadata from the decoder and use a Metal-based processing pipeline to convert the decoded HDR Vivid frames into a format suitable for Apple's HDR/EDR display pipeline.

The goal would be to support HDR Vivid playback on Apple platforms without relying on native system-level HDR Vivid support.

The exact implementation is open to discussion, but a Metal-based HDR Vivid conversion/rendering path may be worth considering.

Would this architecture fit the direction of AetherEngine, and would you be open to adding HDR Vivid support?

Motivating media or use case

B.1-04.mp4

Visible flashing or incorrect HDR/color behavior occurs during playback.

Area

Codec / container support

Host app / integration context

No response

Would you be willing to open a PR?

No

[superuser404notfound]

Thanks for the sample, it made this quick to pin down.

The flicker is not caused by HDR Vivid. B.1-04.mp4 is an MP4 with B pictures but no ctts (composition offset) table, so every sample reports PTS == DTS and the container hands out decode order as presentation order. The engine already repairs that shape for H.264 (#409) by reading the picture order count from the slice headers, but the repair only armed on H.264, so this HEVC file was shown in decode order. Measured on the sample: the first 88 frames stepped backwards 35 times. With the fix in #700 the served stream has 0 backward steps across all 1500 frames. That PR extends the repair to HEVC and will go out in the next release.

On HDR Vivid itself:

• The format is designed to be backward compatible: the base layer is plain HLG or PQ in BT.2020, and the CUVA dynamic metadata is an optional SEI on top. The engine already identifies this sample as HLG (BT.2020 / HLG / 10-bit), plays it on the native path with hardware decode, and the display switches into HLG mode. A Vivid-unaware display shows exactly that static HLG/PQ picture, which is the intended fallback.
• Detection fits well: FFmpeg already parses the CUVA metadata into frame side data, so the engine can report Vivid the same way it reports HDR10+ from probe (ProbeDetail). That is planned as a follow-up.
• A Metal conversion stage is not a fit, for three reasons. The native path hands the bitstream to AVPlayer, so there is no pixel stage to insert anything into without giving up hardware decode and the system's HDR output. tvOS offers no EDR API for an app (CAMetalLayer.wantsExtendedDynamicRangeContent is unavailable there), so the app cannot know the panel's peak brightness, and dynamic tone mapping needs exactly that number. And the output is an HDR signal to the TV, which does its own tone mapping. Applying Vivid metadata on the device would mean tone mapping twice.

[superuser404notfound]

Released in 7.27.3 (b27f2971, fix commit adb80da8): HEVC MP4s without composition offsets are now presented in display order, which removes the flicker on B.1-04.mp4. A retest on your side would be welcome. HDR Vivid detection follows separately, as mentioned above.

[ijuniorfu]

Released in 7.27.3 (b27f2971, fix commit adb80da8): HEVC MP4s without composition offsets are now presented in display order, which removes the flicker on B.1-04.mp4. A retest on your side would be welcome. HDR Vivid detection follows separately, as mentioned above.

After testing, the screen flickering did not disappear.

[superuser404notfound]

The detection half is released in 7.28.0 (7de2d5a0, feature commit bbdb254f).

AetherEngine.probe(url:detecting: .hdrVivid) now reports CUVA carriage as SourceProbe.carriesHDRVividMetadata. It runs in the same packet pass and budget as .hdr10Plus, and only a complete CUVA message counts (defined system_start_code, every field present, zero bits after the last one). videoFormat stays .hlg / .hdr10, since that base layer is what the display is switched to. On B.1-04.mp4:

aetherctl probe --detect-hdr-vivid B.1-04.mp4
  format:    hlg
  HDR Vivid: CUVA metadata seen

Together with the display-order fix in 7.27.3, that covers the detection and playback parts of this request. The Metal tone-mapping path stays out of scope for the reasons given above. A retest of both on your side would be welcome.

[ijuniorfu]

The detection half is released in 7.28.0 (7de2d5a0, feature commit bbdb254f).

AetherEngine.probe(url:detecting: .hdrVivid) now reports CUVA carriage as SourceProbe.carriesHDRVividMetadata. It runs in the same packet pass and budget as .hdr10Plus, and only a complete CUVA message counts (defined system_start_code, every field present, zero bits after the last one). videoFormat stays .hlg / .hdr10, since that base layer is what the display is switched to. On B.1-04.mp4:

aetherctl probe --detect-hdr-vivid B.1-04.mp4
  format:    hlg
  HDR Vivid: CUVA metadata seen

Together with the display-order fix in 7.27.3, that covers the detection and playback parts of this request. The Metal tone-mapping path stays out of scope for the reasons given above. A retest of both on your side would be welcome.

SourceProbe.carriesHDRVividMetadata

AetherEngine.probe(url: url, detecting: .hdrVivid) is already capable of detection, whereas engine.load(url: url, options: options) is not yet able to do so.

[superuser404notfound]

Thanks for testing. I went back to the sample with the engine out of the loop entirely, and the remaining flicker is in the encoded picture itself.

B.1-04.mp4 is a self-test pattern that flickers by design on any display that ignores the CUVA metadata. The caption burned into every frame reads "HDR Vivid Self-Test: Dynamic Metadata Synchronous HLG". Decoding it with plain FFmpeg (no AetherEngine involved, frames in display order) and measuring the average luma per frame gives a strict two-frame alternation between two brightness levels for the whole clip:

ffmpeg -i B.1-04.mp4 -vf signalstats,metadata=print:key=lavfi.signalstats.YAVG -f null -
182.6 197.8 197.8 182.6 197.8 197.8 182.6 182.6 197.8 197.8 182.6 182.6 ...

The CUVA dynamic metadata alternates in lockstep with it: in ffprobe -show_frames one tone-mapping curve parameter switches between 563/1023 and 510/1023, and every frame carrying 563 is a dark one, every frame carrying 510 a bright one, with no exception. So the test checks whether the display applies the per-frame metadata synchronously: a Vivid-capable pipeline that does should render a steady patch, while a pipeline that shows the HLG base layer as is (which is what Apple's display pipeline does, since it has no CUVA support) shows the alternation at 12.5 Hz. That is the intended fallback rendering of this particular clip, not a playback defect, and ordinary Vivid content does not encode a picture that alternates like this.

The ordering fix from 7.27.3 is a separate, real defect in the same file: before it, the engine presented the frames in decode order (35 backward steps in the first 88 frames). Now the decoded ladder is monotonic on both paths (aetherctl swdecode: 0 backward steps across 229 frames, and 0 across all 1500 on the served native stream), and the luma sequence the engine presents matches FFmpeg's above.

On load() reporting HDR Vivid: that is intentional, and it matches HDR10+. Neither shows up in any container header, so detecting them means reading and parsing SEI from video packets, which load() would otherwise not do. Putting that pass in front of every playback start would add open latency for information that changes nothing about how the file plays: there is no Vivid output mode on Apple platforms, the display is switched to the HLG or PQ base layer either way, and videoFormat already reports that. The intended pattern is to run the probe alongside the load (or ahead of it, for a detail page badge):

// probe(url:detecting:) is synchronous and blocking, so it runs off the main actor.
let vivid = Task.detached {
    (try? AetherEngine.probe(url: url, detecting: .hdrVivid))?.carriesHDRVividMetadata ?? false
}
try await engine.load(url: url, options: options)
let isVivid = await vivid.value

Both open the source independently, so the probe does not delay the start of playback.

[ijuniorfu]

Thanks a lot for a reply!

[cmcpherson274]

On your retest request for 7.27.3 (b27f2971, fix commit adb80da8): not a retest of B.1-04.mp4 or of HDR Vivid, but a device confirmation of the display-order fix on a different file shape, x265 encodes in MP4 whose first video packets all read PTS == DTS. On the head of the file it agrees with your reading that the decoded ladder is monotonic on both paths; the one case we found outside it is a resume onto a CRA on the software path, at the end, as a report.

The files. Of the 2,058 HEVC titles in MP4 or MOV files in the library we test against, 77 read PTS == DTS on each of their first 80 video packets while the stream declares reordering (has_b_frames 2). All 77 are SDR films; 76 are x265 10-bit encodes and one an 8-bit HEVC web encode. The one we checked at the box level, the film measured below, has no ctts. In the six we traced with trace_headers, the head is an IDR_N_LP followed by the stock x265 pyramid (POC 5 3 1 2 4 | 10 8 6 7 9 … in decode order in five of them, groups of four in the sixth) with no leading pictures, and the sync sample nearest 600 s is a CRA in all six (in one with three RASL pictures).

Setup. Apple TV 4K (3rd generation, AppleTV14,1, tvOS 26.6), Debug builds of our app, our loopback route (the engine's local HLS server feeding AVPlayer), 7.24.0 (9bd00968) and 7.27.3 in one session on 2026-10-06, the same positions and waits, one run per row. One of the 77 (1920×1040, 23.976 fps) and, as a control, an x265 10-bit MP4 of the same kind that does carry composition offsets. We kept an HDMI capture of each run (29.97 fps, about 22 s) and counted backward steps over the distinct pictures (192-px grey frames, mean absolute difference): a triple of consecutive pictures whose outer pair differs by less than 0.8 × the larger of its two steps. Synthetic clips permuted into x265's decode order read 0.21 to 0.40 on this measure, and 0.00 in display order.

run	7.24.0	7.27.3
from the head	no #409 line; decode order, 76 backward steps in 194 triples (0.39)	#409 … confirmed; display order, 0 in 177
resume at 600 s (a CRA sync sample)	no #409 line; decode order, 159 in 380 (0.42)	#409 … confirmed; display order, 0 in 387
control film, resume at 600 s	no #409 line; display order, 0 in 518	no #409 line; display order, 0 in 521

Both 7.27.3 runs of the film logged the same line:

[Demuxer] #409 missing HEVC composition offsets confirmed on stream 0: repair step=50050 lead=100099 shift=100099 pocStep=1 cadence=250248/5 phase=3 ladderAhead=2 samples=12

On this route the resume is covered because the decision reads the file's head at open, before the resume seek (HLSVideoEngine.swift:1233-1238). At twice the width (384 px) the 7.27.3 captures read 1 backward step in 400 (resume) and 3 in 196 (head), against 0.39 and 0.38 for 7.24.0, so nothing near the decode-order rate appeared at that width either. The measure cannot see a picture AVPlayer drops; our drop and stall counters did not move on any of the six runs. AVPlayer reached its first timeControlStatus=playing at t+3.00 to t+3.06 s on all six, both versions.

The software path, as a report. On the tvOS simulator, where our VOD plays on the engine's software path, the head start is repaired too, as you found with swdecode: 7.27.3 logs the same confirmed line and reads display order (0 in 130), where 7.24.0 read decode order (48 in 198) and logged [Renderer] frame at 0.000s arrived after 0.042s was handed over; dropped, reorder depth raised to 4. A resume at 600 s is not repaired: the software host seeks before its first read (SoftwarePlaybackHost.swift:995-998), so the sample starts at the CRA and the classifier declines it (H264CompositionOffsetRepair.swift:225-226):

[Demuxer] #409 composition-offset probe on stream 0 left the stream untouched: sample does not start on a picture-order origin (samples=12)

That session plays in decode order as on 7.24.0 (142 in 354, against 141 in 366), and both versions log [Renderer] frame at 599.886s arrived after 600.011s was handed over; dropped, reorder depth raised to 4. On an Apple TV our app reaches the software path for these files only through the #461 escape or a #561 escalation, not on their ordinary start. The simulator's 7.27.3 captures were taken under a system dialog that dimmed the picture; the picture underneath moved and was scored, and the two log lines do not depend on it. One run each.

Not measured: a second film of the 77, a Dolby Vision file of this shape, a Release build.

7.27.3 is our app's engine pin as of today (2026-10-06).

[superuser404notfound]

Thanks for the measurement, the software-path finding holds and the fix is up as #707.

There were two defects behind it:

1. The verdict came too late. The demuxer decided the repair at its first read, and the software host seeks to the resume position before that read, so the sample started on the CRA and was declined, exactly as your log line shows. A seek before the first read now settles the verdict at the head of the file first.
2. The landing was anchored as if it were the first picture displayed. A CRA is displayed after the leading pictures decoded behind it, so after any seek onto one with leading pictures the axis sat one slot early per leading picture, and the B pictures that then fell below their own decode time went out untouched. This one is not specific to the software path: a native-path seek onto a CRA with RASL pictures was affected too. Your measured film resumed cleanly on the native route, which fits a landing without leading pictures (five of your six traces). The session now holds from the landing to the first trailing picture, counts the leading pictures and anchors with that count.

Both are covered by an x265 open-GOP twin with four RASL pictures at its CRA, for a resume-shaped seek and a seek during playback. It ships with the next release.