From 6a5ec7da82fdd59c5e0f3d5c38be74b864b8e22b Mon Sep 17 00:00:00 2001 From: Vincent Herbst Date: Sun, 20 Sep 2026 21:12:33 +0200 Subject: [PATCH] feat(probe): a probe can identify HDR10+ before playback The container declares HDR10+ nowhere. ST 2094-40 rides an in-band ITU-T T.35 SEI that no demuxer parses (only hevcdec surfaces it, post-decode, as AV_FRAME_DATA_DYNAMIC_HDR_PLUS), and the one demuxer that produces packet side data for it is matroskadec, from a BlockAdditional. So the engine only learned HDR10+ while playing, from the segment producer's per-packet scan, and a host badging a title had to wait for that mid-session upgrade or label it HDR10. probe(url:detecting:) takes a ProbeDetail option set and runs the opt-in passes it names over one open handle: .hdr10Plus scans demuxed video packets for either carriage, .atmos is the bounded JOC decode probeDetectingAtmos already ran (kept, now a spelling of detecting: .atmos). Asking for both costs one connection. The HDR10+ scan runs first and before any seek, so the common case is answered out of the packets find_stream_info already queued, with no further I/O. SourceProbe gains carriesHDR10PlusMetadata, and .hdr10 upgrades to .hdr10Plus through the same rule the session applies, in one place so the two cannot drift. Both passes are additive: a cap leaves the base answer untouched, a negative means "not seen inside the budget", never "proven absent". The signature scan moved out of HLSSegmentProducer into HDR10PlusMetadataScan, so probe and session agree by construction, and the producer picks up the Matroska side-data carriage it could not see before. Tests: two ~1 KB HEVC/PQ fixtures that differ only in the injected SEI, the positive one carrying a payload ffprobe parses as SMPTE2094-40 (generator in Scripts/make-hdr10plus-fixture.py verifies that before it emits the file). aetherctl probe --detect-hdr10plus / --detect-atmos make both passes observable. Suggested by Geordie. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_015PM3xUJB6ZQyqnmGK1fp6F --- CHANGELOG.md | 16 +- Scripts/make-hdr10plus-fixture.py | 165 ++++++++++++++ Sources/AetherEngine/AetherEngine+Probe.swift | 111 ++++++++-- Sources/AetherEngine/PlayerState.swift | 16 +- .../Video/HDR10PlusDetectionProbe.swift | 200 +++++++++++++++++ .../Video/HDR10PlusMetadataScan.swift | 64 ++++++ .../Video/HLSSegmentProducer.swift | 15 +- Sources/aetherctl/Probe.swift | 14 +- Sources/aetherctl/main.swift | 9 +- .../HDR10PlusDetectionOptionsTests.swift | 78 +++++++ .../HDR10PlusMetadataScanTests.swift | 136 ++++++++++++ .../HDR10PlusProbeIntegrationTests.swift | 203 ++++++++++++++++++ docs/api.md | 7 +- docs/cli.md | 8 + 14 files changed, 1008 insertions(+), 34 deletions(-) create mode 100755 Scripts/make-hdr10plus-fixture.py create mode 100644 Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift create mode 100644 Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift create mode 100644 Tests/AetherEngineTests/HDR10PlusDetectionOptionsTests.swift create mode 100644 Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift create mode 100644 Tests/AetherEngineTests/HDR10PlusProbeIntegrationTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 1505cc180..b261fa09d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,21 @@ the public-API contract. ## [Unreleased] -_Nothing yet._ +### Added + +- **A probe can identify HDR10+ before playback.** `AetherEngine.probe(url:detecting:)` takes a + `ProbeDetail` option set and runs the opt-in passes it names over one open handle: `.hdr10Plus` + scans demuxed video packets for ST 2094-40 carriage, `.atmos` is the bounded E-AC-3 JOC decode + that `probeDetectingAtmos` already ran (which stays, as a spelling of `detecting: .atmos`). Asking + for both costs one connection rather than two. `SourceProbe` gains `carriesHDR10PlusMetadata`, and + a source the container called HDR10 reads `.hdr10Plus` once the payload is seen, so a host can + label a title correctly on first play instead of waiting for the session's own mid-playback + upgrade. The scan opens no decoder: HDR10+ rides an in-band ITU-T T.35 SEI that no demuxer parses + (only `hevcdec` surfaces it, post-decode), and Matroska's `AV_PKT_DATA_DYNAMIC_HDR10_PLUS` side + data is read as the second carriage. Bounded by `HDR10PlusDetectionOptions` (32 packets, 16 MiB, + 2 s) and additive: a cap leaves the base probe's answer exactly where it was, and a negative means + "not seen inside the budget", never "proven absent". `aetherctl probe` gained + `--detect-hdr10plus` and `--detect-atmos`. Suggested by Geordie. ## [7.8.1] - 2026-09-20 diff --git a/Scripts/make-hdr10plus-fixture.py b/Scripts/make-hdr10plus-fixture.py new file mode 100755 index 000000000..9dd4f33da --- /dev/null +++ b/Scripts/make-hdr10plus-fixture.py @@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +"""Build a tiny HEVC/PQ MP4 that carries a real ST 2094-40 (HDR10+) T.35 SEI. + +Layout of the payload follows libavutil/hdr_dynamic_metadata.c's parser exactly, so +`ffprobe -show_frames` reporting "HDR Dynamic Metadata SMPTE2094-40" is the proof the +fixture is genuine rather than a byte pattern that merely looks like one. +""" +import subprocess, sys, base64, os + +class BitWriter: + def __init__(self): + self.bits = [] + def u(self, n, value): + for i in range(n - 1, -1, -1): + self.bits.append((value >> i) & 1) + def bytes(self): + while len(self.bits) % 8: + self.bits.append(0) + out = bytearray() + for i in range(0, len(self.bits), 8): + byte = 0 + for b in self.bits[i:i + 8]: + byte = (byte << 1) | b + out.append(byte) + return bytes(out) + + +def hdr10plus_payload(): + w = BitWriter() + w.u(8, 0) # application_version + w.u(2, 1) # num_windows + w.u(27, 500 * 10000) # targeted_system_display_maximum_luminance (den 10000) + w.u(1, 0) # targeted_system_display_actual_peak_luminance_flag + # window 0 + for maxscl in (17000, 16000, 15000): + w.u(17, maxscl) + w.u(17, 12000) # average_maxrgb + percentiles = [(1, 1000), (5, 2000), (10, 3000), (25, 5000), (50, 8000), + (75, 12000), (90, 16000), (95, 18000), (99, 20000)] + w.u(4, len(percentiles)) + for pct, val in percentiles: + w.u(7, pct) + w.u(17, val) + w.u(10, 100) # fraction_bright_pixels + w.u(1, 0) # mastering_display_actual_peak_luminance_flag + # window 0 tone mapping + w.u(1, 1) # tone_mapping_flag + w.u(12, 1000) # knee_point_x + w.u(12, 1200) # knee_point_y + anchors = [100, 200, 300, 400, 500, 600, 700, 800, 900] + w.u(4, len(anchors)) + for a in anchors: + w.u(10, a) + w.u(1, 0) # color_saturation_mapping_flag + body = w.bytes() + header = bytes([0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04]) + return header + body + + +def emulation_prevent(rbsp): + out = bytearray() + zeros = 0 + for b in rbsp: + if zeros >= 2 and b <= 3: + out.append(0x03) + zeros = 0 + out.append(b) + zeros = zeros + 1 if b == 0 else 0 + return bytes(out) + + +def sei_nal(payload): + rbsp = bytearray() + rbsp.append(0x4E) # nal_unit_type 39 (PREFIX_SEI_NUT) << 1 + rbsp.append(0x01) # nuh_layer_id 0, temporal_id_plus1 1 + rbsp.append(0x04) # payload_type: user_data_registered_itu_t_t35 + size = len(payload) + while size >= 255: + rbsp.append(0xFF) + size -= 255 + rbsp.append(size) + rbsp += payload + rbsp.append(0x80) # rbsp_trailing_bits + return b"\x00\x00\x00\x01" + emulation_prevent(bytes(rbsp)) + + +def split_annexb(data): + starts = [] + i = 0 + while i < len(data) - 3: + if data[i] == 0 and data[i + 1] == 0 and data[i + 2] == 1: + starts.append((i, 3)) + i += 3 + elif (i < len(data) - 4 and data[i] == 0 and data[i + 1] == 0 + and data[i + 2] == 0 and data[i + 3] == 1): + starts.append((i, 4)) + i += 4 + else: + i += 1 + nals = [] + for idx, (pos, sclen) in enumerate(starts): + end = starts[idx + 1][0] if idx + 1 < len(starts) else len(data) + nals.append(data[pos:end]) + return nals + + +def nal_type(nal): + body = nal.lstrip(b"\x00") + body = body[1:] # drop the 0x01 of the start code + return (body[0] >> 1) & 0x3F + + +def main(): + out_dir = sys.argv[1] + plain = os.path.join(out_dir, "plain.mp4") + annexb = os.path.join(out_dir, "plain.hevc") + injected = os.path.join(out_dir, "injected.hevc") + fixture = os.path.join(out_dir, "hdr10plus-hevc.mp4") + + subprocess.run([ + "ffmpeg", "-y", "-v", "error", + "-f", "lavfi", "-i", "color=c=black:s=64x64:r=10:d=0.2", + "-c:v", "libx265", "-pix_fmt", "yuv420p10le", + "-x265-params", "log-level=none:info=0:keyint=1:min-keyint=1:colorprim=9:transfer=16:colormatrix=9", + "-color_primaries", "bt2020", "-color_trc", "smpte2084", "-colorspace", "bt2020nc", + "-frames:v", "2", plain, + ], check=True) + + subprocess.run([ + "ffmpeg", "-y", "-v", "error", "-i", plain, + "-c:v", "copy", "-bsf:v", "hevc_mp4toannexb", "-f", "hevc", annexb, + ], check=True) + + data = open(annexb, "rb").read() + payload = hdr10plus_payload() + sei = sei_nal(payload) + out = bytearray() + for nal in split_annexb(data): + if nal_type(nal) <= 31: # VCL: the SEI must precede the slice in its access unit + out += sei + out += nal + open(injected, "wb").write(bytes(out)) + + subprocess.run([ + "ffmpeg", "-y", "-v", "error", "-f", "hevc", "-i", injected, + "-c:v", "copy", "-tag:v", "hvc1", fixture, + ], check=True) + + probe = subprocess.run([ + "ffprobe", "-v", "error", "-select_streams", "v:0", "-show_frames", + "-show_entries", "frame=side_data_list", fixture, + ], capture_output=True, text=True) + ok = "SMPTE2094-40" in probe.stdout or "HDR10+" in probe.stdout + print(probe.stdout[:600]) + print("SIZE:", os.path.getsize(fixture), "bytes") + print("HDR10+ PARSED BY FFMPEG:", ok) + if ok: + print("BASE64_START") + print(base64.b64encode(open(fixture, "rb").read()).decode()) + print("BASE64_END") + return 0 if ok else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/Sources/AetherEngine/AetherEngine+Probe.swift b/Sources/AetherEngine/AetherEngine+Probe.swift index 2df605b45..0da5060ed 100644 --- a/Sources/AetherEngine/AetherEngine+Probe.swift +++ b/Sources/AetherEngine/AetherEngine+Probe.swift @@ -82,7 +82,7 @@ extension AetherEngine { options: LoadOptions = .init(), atmosDetection: AtmosDetectionOptions = .init() ) throws -> SourceProbe { - try probeDetectingAtmos(source: .url(url), options: options, atmosDetection: atmosDetection) + try probe(source: .url(url), options: options, detecting: .atmos, atmosDetection: atmosDetection) } /// `probeDetectingAtmos(url:)` for a custom byte source. Same reader-ownership contract as `probe(source:)`: @@ -91,6 +91,59 @@ extension AetherEngine { source: MediaSource, options: LoadOptions = .init(), atmosDetection: AtmosDetectionOptions = .init() + ) throws -> SourceProbe { + try probe(source: source, options: options, detecting: .atmos, atmosDetection: atmosDetection) + } + + // MARK: - Bounded, opt-in detail probe + + /// `probe(url:)` plus the bounded, OPT-IN passes named in `detecting`, over ONE open handle to the source. + /// + /// Everything the base probe reports comes from the container. Two things a host wants to put on a + /// details screen are not in there: + /// + /// - **Dolby Atmos**, which for E-AC-3 means the JOC flag in the dependent substream and only exists + /// post-decode (`.atmos`, see `AtmosDetectionOptions`), and + /// - **HDR10+**, whose ST 2094-40 metadata rides an in-band ITU-T T.35 SEI that no demuxer parses + /// (`.hdr10Plus`, see `HDR10PlusDetectionOptions`). + /// + /// Both cost reads past `avformat_find_stream_info`, which is why `probe(url:)` does neither and stays + /// byte-for-byte what it was. Neither is for the playback-start critical path. Asking for both runs both + /// over one open, one connection: the HDR10+ scan first, out of the packets `find_stream_info` already + /// queued, then the queue-flushing seek the Atmos decode pass needs. + /// + /// Both passes are additive and one-directional. They can only ever SET `isAtmos` / + /// `carriesHDR10PlusMetadata`, never clear what the container already declared, and a pass that hits a cap + /// leaves the base probe's answer exactly where it was. + /// + /// - Parameters: + /// - url: Media source, forwarded verbatim to `probe(url:)`. + /// - options: Forwarded verbatim to `probe(url:)` (`httpHeaders` only). + /// - detecting: Which extra passes to run. Empty is exactly `probe(url:)`. + /// - atmosDetection: Bounds + optional track override for the Atmos decode pass. Ignored without `.atmos`. + /// - hdr10PlusDetection: Bounds for the HDR10+ scan. Ignored without `.hdr10Plus`. + /// - Throws: Only what `probe(url:)` throws (demuxer open / probe). Pass-side failures are never thrown: + /// they only mean a detail stays unconfirmed. + public nonisolated static func probe( + url: URL, + options: LoadOptions = .init(), + detecting: ProbeDetail, + atmosDetection: AtmosDetectionOptions = .init(), + hdr10PlusDetection: HDR10PlusDetectionOptions = .init() + ) throws -> SourceProbe { + try probe(source: .url(url), options: options, detecting: detecting, + atmosDetection: atmosDetection, hdr10PlusDetection: hdr10PlusDetection) + } + + /// `probe(url:detecting:)` for a custom byte source (SMB, WebDAV, a disc image, anything behind an + /// `IOReader`). Same reader-ownership contract as `probe(source:)`: the caller retains ownership, the + /// cursor is left at an unspecified position, and `close()` is NOT called. + public nonisolated static func probe( + source: MediaSource, + options: LoadOptions = .init(), + detecting: ProbeDetail, + atmosDetection: AtmosDetectionOptions = .init(), + hdr10PlusDetection: HDR10PlusDetectionOptions = .init() ) throws -> SourceProbe { let demuxer = Demuxer() let displayURL: URL @@ -104,19 +157,37 @@ extension AetherEngine { } defer { demuxer.close() } - let base = makeSourceProbe(demuxer: demuxer, displayURL: displayURL) - // Flush what `avformat_find_stream_info` left queued before the decode pass starts. Those packets - // were read before `detectAtmos` sets AVDISCARD_ALL, so libavformat hands them back regardless of - // the hint: on a source whose audio does not sit at the head, the pass burns its whole foreign-packet - // fuse on that queue and reports "not Atmos" for genuinely Atmos media without ever reading a byte - // of audio. Seeking to the start discards the queue so the discard takes effect from the first read. - // A source that cannot seek is no worse off than before. - demuxer.seekBounded(to: 0, timeout: Self.atmosProbeFlushSeekTimeout) - let targetIndex = Self.atmosDecodeTargetIndex( - options: atmosDetection, defaultAudioStreamIndex: demuxer.audioStreamIndex) - let outcome = Self.detectAtmos(demuxer: demuxer, targetIndex: targetIndex, options: atmosDetection) - guard outcome.confirmedAtmos else { return base } - return Self.enrichAtmos(base: base, confirmedTrackID: Int(targetIndex)) + var probe = makeSourceProbe(demuxer: demuxer, displayURL: displayURL) + + // HDR10+ first, and before any seek: `avformat_find_stream_info` leaves its packets queued and + // `av_read_frame` hands those back first, so at the head of a container the scan gets video packets + // that have already been paid for. Running it after the Atmos pass would mean re-reading them. + if detecting.contains(.hdr10Plus) { + let outcome = Self.detectHDR10Plus( + demuxer: demuxer, videoIndex: demuxer.videoStreamIndex, options: hdr10PlusDetection) + if outcome.carriesHDR10Plus { + probe = Self.enrichHDR10Plus(base: probe) + } + } + + if detecting.contains(.atmos) { + // Flush what is still queued before the decode pass starts. Those packets were read before + // `detectAtmos` sets AVDISCARD_ALL, so libavformat hands them back regardless of the hint: on a + // source whose audio does not sit at the head, the pass burns its whole foreign-packet fuse on + // that queue and reports "not Atmos" for genuinely Atmos media without ever reading a byte of + // audio. Seeking to the start discards the queue so the discard takes effect from the first read. + // A source that cannot seek is no worse off than before. (It also puts a source the HDR10+ scan + // has just walked back at the start.) + demuxer.seekBounded(to: 0, timeout: Self.atmosProbeFlushSeekTimeout) + let targetIndex = Self.atmosDecodeTargetIndex( + options: atmosDetection, defaultAudioStreamIndex: demuxer.audioStreamIndex) + let outcome = Self.detectAtmos(demuxer: demuxer, targetIndex: targetIndex, options: atmosDetection) + if outcome.confirmedAtmos { + probe = Self.enrichAtmos(base: probe, confirmedTrackID: Int(targetIndex)) + } + } + + return probe } /// Flip `isAtmos` to `true` on exactly the one confirmed audio track, leaving everything else identical. @@ -136,6 +207,18 @@ extension AetherEngine { return probe } + /// Record HDR10+ carriage on a probe: set the flag, and move `videoFormat` if it is the one label the + /// payload describes (see `hdr10PlusUpgradedFormat`). + /// + /// Mutates a copy rather than rebuilding the struct field by field: the memberwise init carries defaulted + /// parameters, so a hand-copy silently drops any field added later and the compiler stays quiet about it. + nonisolated static func enrichHDR10Plus(base: SourceProbe) -> SourceProbe { + var probe = base + probe.carriesHDR10PlusMetadata = true + probe.videoFormat = Self.hdr10PlusUpgradedFormat(base.videoFormat) + return probe + } + /// Assemble a `SourceProbe` from an open demuxer. Shared by static probe entry points and `load(source:)`'s internal probe stage so all report identical metadata. nonisolated static func makeSourceProbe( demuxer: Demuxer, diff --git a/Sources/AetherEngine/PlayerState.swift b/Sources/AetherEngine/PlayerState.swift index 9d57eb08f..7fd483ceb 100644 --- a/Sources/AetherEngine/PlayerState.swift +++ b/Sources/AetherEngine/PlayerState.swift @@ -950,7 +950,10 @@ public struct SourceProbe: Sendable { /// 0 for live streams / pipes. public let durationSeconds: Double /// `.sdr` when no HDR signaling or no video track. - public let videoFormat: VideoFormat + /// + /// Settable inside the module so the `.hdr10Plus` upgrade from `probe(url:detecting: .hdr10Plus)` lands + /// here rather than rebuilding the struct field by field. + public internal(set) var videoFormat: VideoFormat /// FFmpeg AVCodecID raw value; 0 (AV_CODEC_ID_NONE) when no video track. public let videoCodecID: Int32 /// Codec name from libavcodec (e.g. "hevc", "h264", "av1"). nil when unavailable. @@ -964,6 +967,15 @@ public struct SourceProbe: Sendable { public let isDolbyVision: Bool /// Dolby Vision profile number (5, 7, 8, 10) read from the dvcC/dvvC configuration record; nil when not DV. public let dvProfile: Int? + /// HDR10+ (ST 2094-40) dynamic metadata was SEEN in this source's video. + /// + /// Always `false` unless the probe was asked for `.hdr10Plus` (the container carries no such declaration, + /// so there is nothing to read without looking at packets). `false` therefore means "not asked, or not + /// seen inside the scan budget", never "proven absent": a positive is evidence, a negative is not. + /// + /// Separate from `videoFormat == .hdr10Plus` because a Dolby Vision source can carry an HDR10+ layer too + /// (Blu-ray Profile 7 and the 8.1 remuxes of it), and that source keeps reading `.dolbyVision`. + public internal(set) var carriesHDR10PlusMetadata: Bool /// Settable inside the module so `probeDetectingAtmos` can enrich one track without rebuilding the struct field by field. public internal(set) var audioTracks: [TrackInfo] /// Includes both text and bitmap (PGS / DVB) variants. @@ -983,6 +995,7 @@ public struct SourceProbe: Sendable { videoFrameRate: Double?, isDolbyVision: Bool, dvProfile: Int? = nil, + carriesHDR10PlusMetadata: Bool = false, audioTracks: [TrackInfo], subtitleTracks: [TrackInfo], metadata: MediaMetadata = MediaMetadata(title: nil, artist: nil, album: nil, artworkData: nil), @@ -998,6 +1011,7 @@ public struct SourceProbe: Sendable { self.videoFrameRate = videoFrameRate self.isDolbyVision = isDolbyVision self.dvProfile = dvProfile + self.carriesHDR10PlusMetadata = carriesHDR10PlusMetadata self.audioTracks = audioTracks self.subtitleTracks = subtitleTracks self.metadata = metadata diff --git a/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift b/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift new file mode 100644 index 000000000..1b4361ff6 --- /dev/null +++ b/Sources/AetherEngine/Video/HDR10PlusDetectionProbe.swift @@ -0,0 +1,200 @@ +import Foundation +import AetherLibavformat +import AetherLibavcodec +import AetherLibavutil + +/// Bounds for the pre-playback HDR10+ (ST 2094-40) carriage scan, `AetherEngine.probe(url:detecting:)` with +/// `.hdr10Plus`. +/// +/// Separate from `LoadOptions` and from the lightweight `probe(url:)` path, exactly like +/// `AtmosDetectionOptions`: a host opts into reading real video packets without the default probe changing +/// behaviour or cost. +/// +/// Unlike the Atmos pass this one opens no decoder. It reads demuxed video packets and looks for the T.35 +/// signature (see `HDR10PlusMetadataScan`), so its cost is I/O and a memory scan, not decode. +public struct HDR10PlusDetectionOptions: Sendable, Equatable { + /// Stop after this many video packets have been scanned. Default 32. + /// + /// HDR10+ metadata is per frame, so a carrying source almost always confirms on the very first video + /// packet; the budget exists for the source that starts with a run of frames without the SEI, and for + /// the adversarial one that never has it. + public var maxPackets: Int + + /// Stop after this many cumulative video-packet bytes. Default 16 MiB. + /// + /// This is the cap that actually binds on the content the feature targets: one UHD HEVC keyframe runs to + /// several MB, so a handful of packets can cross it long before `maxPackets` does. 16 MiB leaves room for + /// a keyframe plus the frames after it on a 4K remux while staying finite on a hostile source. + public var maxBytes: Int64 + + /// Soft wall-clock budget, checked BETWEEN packet reads. NOT preemptive: one blocking `av_read_frame()` + /// on a stalled remote socket can still overrun it, the same AVIO-layer limitation `Demuxer.seekBounded` + /// documents. Default 2 seconds. + public var timeBudget: TimeInterval + + public init( + maxPackets: Int = 32, + maxBytes: Int64 = 16 * 1024 * 1024, + timeBudget: TimeInterval = 2.0 + ) { + self.maxPackets = maxPackets + self.maxBytes = maxBytes + self.timeBudget = timeBudget + } +} + +/// Result of the bounded HDR10+ carriage scan. Internal: hosts read the enriched `SourceProbe` instead. It +/// exists at module visibility so the stop conditions are unit-testable without media. +struct HDR10PlusDetectionOutcome: Sendable, Equatable { + enum StopReason: Sendable, Equatable { + /// No video stream at the resolved index, or the source has no video at all. + case noVideoTrack + /// The T.35 signature (or Matroska's decoded side data) was seen. The only positive answer. + case found + /// `maxPackets` video packets were scanned without a hit. + case packetCap + /// `maxBytes` cumulative video-packet bytes were scanned without a hit. + case byteCap + /// `timeBudget` elapsed (checked between reads) without a hit. + case timeCap + /// The demuxer reached EOF without a hit (a source short enough to scan whole). + case demuxEOF + /// `Demuxer.readPacket()` threw. Tolerated, never rethrown. + case demuxError + } + + let stopReason: StopReason + let packetsRead: Int + let bytesRead: Int64 + + /// A negative is never authoritative and is never published as one: only `.found` sets the flag, and the + /// flag is only ever set, never cleared. Every other reason means "not seen inside this budget", which + /// for a cap is genuinely inconclusive and for EOF is only as conclusive as the source is short. + var carriesHDR10Plus: Bool { stopReason == .found } +} + +/// Which extra, strictly more expensive detail a probe should resolve on top of the container metadata. +/// +/// Each member costs real reads past `avformat_find_stream_info`, which is why the base `probe(url:)` never +/// does any of it. They combine into one pass over one open handle, so a host that badges both Atmos and +/// HDR10+ pays one connection rather than two. +public struct ProbeDetail: OptionSet, Sendable { + public let rawValue: Int + public init(rawValue: Int) { self.rawValue = rawValue } + + /// Authoritative E-AC-3 JOC (Dolby Atmos) via a bounded decode pass. See `AtmosDetectionOptions`. + public static let atmos = ProbeDetail(rawValue: 1 << 0) + + /// HDR10+ (ST 2094-40) carriage via a bounded packet scan. See `HDR10PlusDetectionOptions`. + public static let hdr10Plus = ProbeDetail(rawValue: 1 << 1) +} + +extension AetherEngine { + + /// Pure stop-condition check for the scan loop, in cap priority order (packets, bytes, time). `nil` while + /// inside all three budgets. + nonisolated static func hdr10PlusScanCapReached( + packetsRead: Int, + bytesRead: Int64, + elapsed: TimeInterval, + options: HDR10PlusDetectionOptions + ) -> HDR10PlusDetectionOutcome.StopReason? { + if packetsRead >= options.maxPackets { return .packetCap } + if bytesRead >= options.maxBytes { return .byteCap } + if elapsed >= options.timeBudget { return .timeCap } + return nil + } + + /// Packet ceiling for the AVDISCARD_ALL fuse, saturating rather than trapping: `maxPackets` is public and + /// `Int.max` is a plausible "no limit" value to pass. + nonisolated static func hdr10PlusForeignPacketFuse(maxPackets: Int) -> Int { + let (product, overflowed) = maxPackets.multipliedReportingOverflow(by: foreignPacketFuseMultiplier) + return overflowed ? Int.max : product + } + + /// The label an HDR10+ finding produces, given what the container already said. + /// + /// The same rule the running session applies in `handleHDR10PlusDetected`, in one place so probe and + /// session cannot drift: `.hdr10` is the only format that moves, because the ST 2094-40 payload rides an + /// HDR10 base. A Dolby Vision source keeps its label (Profile 7 and the 8.1 remuxes of it carry an HDR10+ + /// base layer under the RPU), an HLG or SDR one has no HDR10 base for the payload to describe, and + /// `SourceProbe.carriesHDR10PlusMetadata` carries the evidence in all of those cases. + nonisolated static func hdr10PlusUpgradedFormat(_ detected: VideoFormat) -> VideoFormat { + detected == .hdr10 ? .hdr10Plus : detected + } + + /// Bounded scan for HDR10+ carriage on `videoIndex`. Opens no decoder: it reads demuxed packets and asks + /// `HDR10PlusMetadataScan` about each one, stopping at the first hit or the first cap. + /// + /// Deliberately runs BEFORE any queue-flushing seek, unlike `detectAtmos`. `avformat_find_stream_info` + /// leaves the packets it read queued, `av_read_frame` hands those back first, and at the head of a + /// container those are video packets: the common case is answered out of bytes that are already paid for, + /// with no further I/O at all. (The Atmos pass has to throw that queue away precisely because what it + /// needs is audio, which may sit far into the file.) + /// + /// `Demuxer.readPacket()` failures fold into `.demuxError` rather than propagating: an unreadable stream + /// fails to confirm HDR10+, it does not fail the probe. + nonisolated static func detectHDR10Plus( + demuxer: Demuxer, + videoIndex: Int32, + options: HDR10PlusDetectionOptions + ) -> HDR10PlusDetectionOutcome { + guard videoIndex >= 0, let stream = demuxer.stream(at: videoIndex), + let codecpar = stream.pointee.codecpar, + codecpar.pointee.codec_type == AVMEDIA_TYPE_VIDEO else { + return HDR10PlusDetectionOutcome(stopReason: .noVideoTrack, packetsRead: 0, bytesRead: 0) + } + + // Matroska's BlockAdditional carriage is attached by the demuxer to the packet, so the scan needs the + // packets themselves either way; dropping the other streams keeps the byte budget spent on video. + demuxer.discardAllStreamsExcept([videoIndex]) + + let start = DispatchTime.now() + var packetsRead = 0 + var bytesRead: Int64 = 0 + var packetsSeen = 0 + let fuse = Self.hdr10PlusForeignPacketFuse(maxPackets: options.maxPackets) + + while true { + let elapsed = Double(DispatchTime.now().uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000 + if let cap = Self.hdr10PlusScanCapReached( + packetsRead: packetsRead, bytesRead: bytesRead, elapsed: elapsed, options: options + ) { + return HDR10PlusDetectionOutcome(stopReason: cap, packetsRead: packetsRead, bytesRead: bytesRead) + } + + let packet: UnsafeMutablePointer? + do { + packet = try demuxer.readPacket() + } catch { + return HDR10PlusDetectionOutcome( + stopReason: .demuxError, packetsRead: packetsRead, bytesRead: bytesRead) + } + guard let pkt = packet else { + return HDR10PlusDetectionOutcome( + stopReason: .demuxEOF, packetsRead: packetsRead, bytesRead: bytesRead) + } + + var found = false + packetsSeen += 1 + if pkt.pointee.stream_index == videoIndex { + packetsRead += 1 + bytesRead += Int64(pkt.pointee.size) + found = HDR10PlusMetadataScan.packetCarriesHDR10Plus(pkt) + } + av_packet_unref(pkt) + av_packet_free_safe(pkt) + + if found { + return HDR10PlusDetectionOutcome( + stopReason: .found, packetsRead: packetsRead, bytesRead: bytesRead) + } + + // AVDISCARD_ALL is advisory, so a container that keeps handing back foreign packets still ends. + if packetsSeen >= fuse { + return HDR10PlusDetectionOutcome( + stopReason: .packetCap, packetsRead: packetsRead, bytesRead: bytesRead) + } + } + } +} diff --git a/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift b/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift new file mode 100644 index 000000000..4b9c9aa74 --- /dev/null +++ b/Sources/AetherEngine/Video/HDR10PlusMetadataScan.swift @@ -0,0 +1,64 @@ +import Foundation +import AetherLibavcodec +import AetherLibavutil + +/// Where HDR10+ (SMPTE ST 2094-40) can be seen in a demuxed packet, without decoding a frame. +/// +/// One definition for both readers: the segment producer's per-packet scan during playback +/// (`HLSSegmentProducer.finalizeAndWriteVideo`) and the pre-playback pass behind +/// `AetherEngine.probeDetectingHDR10Plus`. They used to be one call site and a literal byte array; a probe +/// that answered differently from the session it precedes would be worse than no probe at all. +/// +/// Two carriages, because FFmpeg produces exactly two (checked against the pinned FFmpeg tree): +/// +/// - **In-band ITU-T T.35 SEI.** This is where HEVC HDR10+ lives, and NO demuxer parses it: `mov`, `mpegts` +/// and `matroska` hand the SEI through inside the video packet, and only `hevc/hevcdec.c` surfaces it, +/// post-decode, as `AV_FRAME_DATA_DYNAMIC_HDR_PLUS`. Hence the byte scan: it is the only way to see HDR10+ +/// without opening a decoder. +/// - **`AV_PKT_DATA_DYNAMIC_HDR10_PLUS` packet side data.** `matroskadec.c` alone attaches this, from a +/// BlockAdditional whose T.35 header it has already stripped (the VP9/AV1-in-Matroska carriage). The bytes +/// are a decoded `AVDynamicHDRPlus` by then, so the signature scan cannot see it and the type has to be +/// asked for separately. +/// +/// The byte scan is a signature match over the whole packet payload, not a NAL walk, and it can in principle +/// match those six bytes inside compressed slice data. That is the trade the producer has always made, and it +/// is kept here deliberately: one false positive mislabels a badge, while a NAL walk that disagrees with the +/// producer's scan would mislabel the session against its own probe. +enum HDR10PlusMetadataScan { + + /// `country_code` 0xB5 (US), `provider_code` 0x003C (Samsung), `provider_oriented_code` 0x0001, + /// `application_identifier` 0x04: the four fields that open every ST 2094-40 T.35 payload, in bitstream + /// order. `matroskadec.c` gates its BlockAdditional conversion on the same four values. + /// + /// All four are needed. Dolby's RPU rides a T.35 payload too, under provider code 0x003B, so a scan for + /// the country code alone would call every Profile 5 / 8 source HDR10+. + static let t35Signature: [UInt8] = [0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04] + + /// Whether the ST 2094-40 T.35 signature appears anywhere in `size` bytes at `data`. + /// + /// `nil` data, a non-positive size, or a payload shorter than the signature answer `false` without a read. + static func bytesCarrySignature(_ data: UnsafePointer?, size: Int) -> Bool { + guard let data, size >= t35Signature.count else { return false } + return t35Signature.withUnsafeBufferPointer { needle -> Bool in + memmem(data, size, needle.baseAddress, needle.count) != nil + } + } + + /// Array convenience for tests and call sites that already hold the bytes. + static func bytesCarrySignature(_ bytes: [UInt8]) -> Bool { + bytes.withUnsafeBufferPointer { bytesCarrySignature($0.baseAddress, size: $0.count) } + } + + /// Whether this packet carries HDR10+ in either of the two carriages. + /// + /// Side data is asked first: it is a type lookup over a short list, while the signature scan walks the + /// whole payload, and on the one container that produces side data the payload no longer holds the T.35 + /// header at all. + static func packetCarriesHDR10Plus(_ packet: UnsafePointer) -> Bool { + var sideDataSize = 0 + if av_packet_get_side_data(packet, AV_PKT_DATA_DYNAMIC_HDR10_PLUS, &sideDataSize) != nil { + return true + } + return bytesCarrySignature(packet.pointee.data, size: Int(packet.pointee.size)) + } +} diff --git a/Sources/AetherEngine/Video/HLSSegmentProducer.swift b/Sources/AetherEngine/Video/HLSSegmentProducer.swift index af2ba52ea..fb836bd5a 100644 --- a/Sources/AetherEngine/Video/HLSSegmentProducer.swift +++ b/Sources/AetherEngine/Video/HLSSegmentProducer.swift @@ -4235,18 +4235,9 @@ final class HLSSegmentProducer: @unchecked Sendable { packet.pointee.stream_index = muxer.videoOutputStreamIndex - if !hdr10PlusDetected, let data = packet.pointee.data { - let size = Int(packet.pointee.size) - if size >= 6 { - let needle: [UInt8] = [0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04] - let found = needle.withUnsafeBufferPointer { n -> Bool in - memmem(data, size, n.baseAddress, n.count) != nil - } - if found { - hdr10PlusDetected = true - onFirstHDR10PlusDetected?() - } - } + if !hdr10PlusDetected, HDR10PlusMetadataScan.packetCarriesHDR10Plus(packet) { + hdr10PlusDetected = true + onFirstHDR10PlusDetected?() } // #131: A53 caption extraction rides the same per-packet spot as the HDR10+ scan: decode diff --git a/Sources/aetherctl/Probe.swift b/Sources/aetherctl/Probe.swift index 39b4cbe02..ab9be45ce 100644 --- a/Sources/aetherctl/Probe.swift +++ b/Sources/aetherctl/Probe.swift @@ -3,13 +3,19 @@ import AetherEngine // MARK: - probe -func runProbe(url: URL) -> Int32 { +func runProbe(url: URL, detecting: ProbeDetail = []) -> Int32 { EngineLog.handler = { print($0) } print("aetherctl probe: \(url.absoluteString)") + if !detecting.isEmpty { + var passes: [String] = [] + if detecting.contains(.hdr10Plus) { passes.append("hdr10plus (packet scan)") } + if detecting.contains(.atmos) { passes.append("atmos (bounded decode)") } + print("detail passes: \(passes.joined(separator: ", "))") + } print("") let probe: SourceProbe do { - probe = try AetherEngine.probe(url: url) + probe = try AetherEngine.probe(url: url, detecting: detecting) } catch { print("ERROR: \(error)") return 1 @@ -26,6 +32,10 @@ func runProbe(url: URL) -> Int32 { if probe.isDolbyVision { print(" HDR/DV: Dolby Vision signaled") } + if detecting.contains(.hdr10Plus) { + // A negative here means "not seen inside the scan budget", never "proven absent". + print(" HDR10+: \(probe.carriesHDR10PlusMetadata ? "ST 2094-40 metadata seen" : "not seen")") + } print("") if probe.audioTracks.isEmpty { diff --git a/Sources/aetherctl/main.swift b/Sources/aetherctl/main.swift index 1d1393c4a..ba17a7779 100644 --- a/Sources/aetherctl/main.swift +++ b/Sources/aetherctl/main.swift @@ -66,7 +66,7 @@ func printUsage() { aetherctl: standalone AetherEngine repro harness Usage: - aetherctl probe + aetherctl probe [--detect-hdr10plus] [--detect-atmos] aetherctl serve [--no-dv] [--force-dv] [--dv-base-layer] [--start-position S] aetherctl validate [--no-dv] [--force-dv] [--dv-base-layer] aetherctl swdecode [--frames N] @@ -891,6 +891,11 @@ if ["probe", "serve", "validate", "swdecode", "extract", "audio", "customio"].co let extractLoops = takeIntFlag("--loops", from: &rest) ?? 1 let extractWidth = takeIntFlag("--width", from: &rest) ?? 320 let snapshotMode = takeFlag("--snapshot", from: &rest) + // The opt-in detail passes of `AetherEngine.probe(url:detecting:)`, so both are observable from the CLI + // instead of only through a host. Each costs reads past find_stream_info; the bare `probe` does neither. + var probeDetail: ProbeDetail = [] + if takeFlag("--detect-hdr10plus", from: &rest) { probeDetail.insert(.hdr10Plus) } + if takeFlag("--detect-atmos", from: &rest) { probeDetail.insert(.atmos) } let inMemory = takeFlag("--memory", from: &rest) let forwardOnly = takeFlag("--forward-only", from: &rest) let customAudioIndex = takeIntFlag("--audio-index", from: &rest).map(Int32.init) @@ -976,7 +981,7 @@ if ["probe", "serve", "validate", "swdecode", "extract", "audio", "customio"].co } switch first { case "probe": - exit(runProbe(url: url)) + exit(runProbe(url: url, detecting: probeDetail)) case "serve": runServe(url: url, dvModeAvailable: dvModeAvailable, forceDVWithoutDisplay: forceDV, dolbyVisionHandling: dvHandling, diff --git a/Tests/AetherEngineTests/HDR10PlusDetectionOptionsTests.swift b/Tests/AetherEngineTests/HDR10PlusDetectionOptionsTests.swift new file mode 100644 index 000000000..882b71b9c --- /dev/null +++ b/Tests/AetherEngineTests/HDR10PlusDetectionOptionsTests.swift @@ -0,0 +1,78 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// Stop-condition and outcome semantics for the bounded HDR10+ carriage scan. Pure: no demuxer, no media. +@Suite("HDR10PlusDetection: bounds and outcome semantics") +struct HDR10PlusDetectionOptionsTests { + + @Test("Defaults are finite and non-zero on all three axes") + func defaultsAreFinite() { + let options = HDR10PlusDetectionOptions() + #expect(options.maxPackets > 0) + #expect(options.maxBytes > 0) + #expect(options.timeBudget > 0) + } + + @Test("Within every budget no cap is reported") + func withinBudgetsNoCap() { + let options = HDR10PlusDetectionOptions(maxPackets: 10, maxBytes: 1000, timeBudget: 1) + #expect(AetherEngine.hdr10PlusScanCapReached( + packetsRead: 9, bytesRead: 999, elapsed: 0.9, options: options) == nil) + } + + @Test("Cap priority is packets, then bytes, then time") + func capPriority() { + let options = HDR10PlusDetectionOptions(maxPackets: 10, maxBytes: 1000, timeBudget: 1) + // All three exceeded at once: the packet cap is the one reported, so a log line names the same + // reason whichever axis a source happens to also cross. + #expect(AetherEngine.hdr10PlusScanCapReached( + packetsRead: 10, bytesRead: 5000, elapsed: 5, options: options) == .packetCap) + #expect(AetherEngine.hdr10PlusScanCapReached( + packetsRead: 1, bytesRead: 1000, elapsed: 5, options: options) == .byteCap) + #expect(AetherEngine.hdr10PlusScanCapReached( + packetsRead: 1, bytesRead: 1, elapsed: 1, options: options) == .timeCap) + } + + @Test("Only .found is a positive answer; every other stop reason is inconclusive") + func onlyFoundIsPositive() { + let reasons: [HDR10PlusDetectionOutcome.StopReason] = [ + .noVideoTrack, .packetCap, .byteCap, .timeCap, .demuxEOF, .demuxError + ] + for reason in reasons { + let outcome = HDR10PlusDetectionOutcome(stopReason: reason, packetsRead: 3, bytesRead: 300) + #expect(!outcome.carriesHDR10Plus, "\(reason) must not confirm HDR10+") + } + let found = HDR10PlusDetectionOutcome(stopReason: .found, packetsRead: 1, bytesRead: 100) + #expect(found.carriesHDR10Plus) + } + + @Test("The foreign-packet fuse saturates instead of trapping on Int.max") + func fuseSaturates() { + #expect(AetherEngine.hdr10PlusForeignPacketFuse(maxPackets: Int.max) == Int.max) + #expect(AetherEngine.hdr10PlusForeignPacketFuse(maxPackets: 32) + == 32 * AetherEngine.foreignPacketFuseMultiplier) + } + + @Test("An HDR10 source upgrades to HDR10+, and only that format does") + func formatUpgradeIsScopedToHDR10() { + // Mirrors the session's own rule (`handleHDR10PlusDetected`): the payload rides an HDR10 base, so + // that is the only label the scan moves. A Dolby Vision source keeps saying Dolby Vision even when + // it carries an HDR10+ base layer (Profile 7, and 8.1 remuxed from it), and the flag carries the + // evidence instead. + #expect(AetherEngine.hdr10PlusUpgradedFormat(.hdr10) == .hdr10Plus) + #expect(AetherEngine.hdr10PlusUpgradedFormat(.hdr10Plus) == .hdr10Plus) + #expect(AetherEngine.hdr10PlusUpgradedFormat(.dolbyVision) == .dolbyVision) + #expect(AetherEngine.hdr10PlusUpgradedFormat(.hlg) == .hlg) + #expect(AetherEngine.hdr10PlusUpgradedFormat(.sdr) == .sdr) + } + + @Test("ProbeDetail combines and tests as a set") + func probeDetailSetSemantics() { + let both: ProbeDetail = [.atmos, .hdr10Plus] + #expect(both.contains(.atmos)) + #expect(both.contains(.hdr10Plus)) + #expect(!ProbeDetail.atmos.contains(.hdr10Plus)) + #expect(ProbeDetail().isEmpty) + } +} diff --git a/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift b/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift new file mode 100644 index 000000000..98e9cb663 --- /dev/null +++ b/Tests/AetherEngineTests/HDR10PlusMetadataScanTests.swift @@ -0,0 +1,136 @@ +import Testing +import Foundation +import AetherLibavcodec +import AetherLibavutil +@testable import AetherEngine + +/// Pure byte-level tests for the HDR10+ (ST 2094-40) carriage scan that `AetherEngine.probeDetectingHDR10Plus` +/// and `HLSSegmentProducer` share. No demuxer, no decoder, no media. +/// +/// The two carriages tested here are the two FFmpeg actually produces, verified against the pinned +/// `~/Dev/FFmpegBuild/build/ffmpeg-src` tree: +/// +/// - in-band ITU-T T.35 SEI in the video bitstream, which is where HEVC HDR10+ lives and which no demuxer +/// parses (`hevc/hevcdec.c` surfaces it only post-decode as `AV_FRAME_DATA_DYNAMIC_HDR_PLUS`), and +/// - `AV_PKT_DATA_DYNAMIC_HDR10_PLUS` packet side data, which `matroskadec.c` alone attaches, from a +/// BlockAdditional carrying the same T.35 payload. +@Suite("HDR10PlusMetadataScan: T.35 signature and packet side data") +struct HDR10PlusMetadataScanTests { + + /// country_code 0xB5 (US), provider_code 0x003C (Samsung), provider_oriented_code 0x0001, + /// application_identifier 0x04. The same four fields `matroskadec.c` gates its BlockAdditional + /// conversion on, in the same order. + static let signature: [UInt8] = [0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04] + + @Test("The signature is found at the head of a packet payload") + func signatureAtHead() { + let payload = Self.signature + [0x01, 0x02, 0x03] + #expect(HDR10PlusMetadataScan.bytesCarrySignature(payload)) + } + + @Test("The signature is found in the middle, where a real SEI NAL puts it") + func signatureInMiddle() { + let payload: [UInt8] = [0x00, 0x00, 0x01, 0x4E, 0x01, 0x04, 0x2F] + Self.signature + [0x00, 0xFF] + #expect(HDR10PlusMetadataScan.bytesCarrySignature(payload)) + } + + @Test("The signature is found at the very end of the payload") + func signatureAtTail() { + let payload: [UInt8] = [0xAA, 0xBB, 0xCC] + Self.signature + #expect(HDR10PlusMetadataScan.bytesCarrySignature(payload)) + } + + @Test("A payload without the signature does not match") + func absentSignature() { + let payload: [UInt8] = Array(repeating: 0xB5, count: 64) + #expect(!HDR10PlusMetadataScan.bytesCarrySignature(payload)) + } + + @Test("A prefix of the signature that runs off the end of the payload does not match") + func truncatedSignature() { + let payload: [UInt8] = [0x00, 0x00] + Self.signature.dropLast() + #expect(!HDR10PlusMetadataScan.bytesCarrySignature(payload)) + } + + @Test("A payload shorter than the signature is rejected without reading past it") + func payloadShorterThanSignature() { + #expect(!HDR10PlusMetadataScan.bytesCarrySignature([])) + #expect(!HDR10PlusMetadataScan.bytesCarrySignature([0xB5, 0x00, 0x3C, 0x00, 0x01])) + } + + @Test("A Dolby Vision T.35 payload (provider 0x003B) is not mistaken for HDR10+") + func dolbyProviderDoesNotMatch() { + // Dolby's provider_code is 0x003B, one below Samsung's. A scan that only checked the country code + // would report every P5/P8 RPU carried in a T.35 SEI as HDR10+. + let payload: [UInt8] = [0xB5, 0x00, 0x3B, 0x00, 0x01, 0x04, 0x11, 0x22] + #expect(!HDR10PlusMetadataScan.bytesCarrySignature(payload)) + } + + @Test("A packet carrying AV_PKT_DATA_DYNAMIC_HDR10_PLUS side data is detected without an in-band SEI") + func packetSideDataIsDetected() { + guard let packet = av_packet_alloc() else { + Issue.record("av_packet_alloc failed") + return + } + defer { + var p: UnsafeMutablePointer? = packet + av_packet_free(&p) + } + // Payload bytes are irrelevant to this carriage: matroskadec strips the T.35 header and stores a + // decoded AVDynamicHDRPlus, so the scan must key off the side-data TYPE, not the bytes. + var size = 0 + guard let hdrplus = av_dynamic_hdr_plus_alloc(&size) else { + Issue.record("av_dynamic_hdr_plus_alloc failed") + return + } + let added = av_packet_add_side_data( + packet, AV_PKT_DATA_DYNAMIC_HDR10_PLUS, + UnsafeMutableRawPointer(hdrplus).assumingMemoryBound(to: UInt8.self), size) + guard added >= 0 else { + av_free(hdrplus) + Issue.record("av_packet_add_side_data failed: \(added)") + return + } + #expect(HDR10PlusMetadataScan.packetCarriesHDR10Plus(packet)) + } + + @Test("A packet with neither carriage is not detected") + func emptyPacketIsNotDetected() { + guard let packet = av_packet_alloc() else { + Issue.record("av_packet_alloc failed") + return + } + defer { + var p: UnsafeMutablePointer? = packet + av_packet_free(&p) + } + var payload: [UInt8] = Array(repeating: 0x00, count: 32) + payload.withUnsafeMutableBufferPointer { buf in + packet.pointee.data = buf.baseAddress + packet.pointee.size = Int32(buf.count) + #expect(!HDR10PlusMetadataScan.packetCarriesHDR10Plus(packet)) + packet.pointee.data = nil + packet.pointee.size = 0 + } + } + + @Test("A packet with the in-band signature is detected") + func packetInBandIsDetected() { + guard let packet = av_packet_alloc() else { + Issue.record("av_packet_alloc failed") + return + } + defer { + var p: UnsafeMutablePointer? = packet + av_packet_free(&p) + } + var payload: [UInt8] = [0x00, 0x00, 0x00, 0x01, 0x4E, 0x01] + Self.signature + [0x2A, 0x80] + payload.withUnsafeMutableBufferPointer { buf in + packet.pointee.data = buf.baseAddress + packet.pointee.size = Int32(buf.count) + #expect(HDR10PlusMetadataScan.packetCarriesHDR10Plus(packet)) + packet.pointee.data = nil + packet.pointee.size = 0 + } + } +} diff --git a/Tests/AetherEngineTests/HDR10PlusProbeIntegrationTests.swift b/Tests/AetherEngineTests/HDR10PlusProbeIntegrationTests.swift new file mode 100644 index 000000000..d310592a4 --- /dev/null +++ b/Tests/AetherEngineTests/HDR10PlusProbeIntegrationTests.swift @@ -0,0 +1,203 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// End-to-end tests for `AetherEngine.probe(url:detecting: .hdr10Plus)` against two fixtures that differ in +/// exactly one thing: the presence of an ITU-T T.35 SEI carrying ST 2094-40 metadata. +/// +/// Both are 64x64 HEVC Main 10, PQ / BT.2020, two frames, generated locally (`Scripts/make-hdr10plus-fixture.py`) +/// and embedded here because they are ~1 KB each. The positive one's payload is a real HDR10+ payload built to +/// `libavutil/hdr_dynamic_metadata.c`'s bit layout, and `ffprobe -show_frames` on it reports +/// "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)": FFmpeg's own parser accepting it is what makes this a +/// fixture of the case rather than a byte pattern that resembles it. +/// +/// The negative fixture is the same encode WITHOUT the SEI. It is the load-bearing half: a scan that reports +/// HDR10+ for everything would pass every positive test in this file. +@Suite("HDR10+ probe: bounded pre-playback detection against real HEVC fixtures") +struct HDR10PlusProbeIntegrationTests { + + /// 64x64 HEVC Main 10, PQ/BT.2020, 2 frames, each access unit preceded by a prefix SEI (NAL type 39) + /// carrying a valid ST 2094-40 T.35 payload. + static let hdr10PlusBase64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAAAAhmcmVlAAAAwG1kYXQAAABFTgEEQLUAPAAB + BABCYloAhNA+gB1MC7gkCA+gKB9AUC7gyE4hkH0CWLuC0PoC+RlDGTiAZE+hLCRkMhLGQfSWK8yD + hACAAAAADigBr3jrrvv//FtlXy08AAAARU4BBEC1ADwAAQQAQmJaAITQPoAdTAu4JAgPoCgfQFAu + 4MhOIZB9Ali7gtD6AvkZQxk4gGRPoSwkZDISxkH0livMg4QAgAAAABAoAa8J4CQEyH//J2Eew0j8 + AAADw21vb3YAAABsbXZoZAAAAAAAAAAAAAAAAAAAA+gAAADIAAEAAAEAAAAAAAAAAAAAAAABAAAA + AAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + AAIAAALtdHJhawAAAFx0a2hkAAAAAwAAAAAAAAAAAAAAAQAAAAAAAADIAAAAAAAAAAAAAAAAAAAA + AAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAQAAAAABAAAAAQAAAAAAAJGVkdHMAAAAc + ZWxzdAAAAAAAAAABAAAAyAAAAAAAAQAAAAACZW1kaWEAAAAgbWRoZAAAAAAAAAAAAAAAAAAST4AA + A6mAVcQAAAAAAC1oZGxyAAAAAAAAAAB2aWRlAAAAAAAAAAAAAAAAVmlkZW9IYW5kbGVyAAAAAhBt + aW5mAAAAFHZtaGQAAAABAAAAAAAAAAAAAAAkZGluZgAAABxkcmVmAAAAAAAAAAEAAAAMdXJsIAAA + AAEAAAHQc3RibAAAAWRzdHNkAAAAAAAAAAEAAAFUaHZjMQAAAAAAAAABAAAAAAAAAAAAAAAAAAAA + AABAAEAASAAAAEgAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABj//wAA + AMdodmNDAQQIAAAAnagAAAAAHvAA/P36+gAADwOgAAIAF0ABDAH//wQIAAADAJ2oAAADAAAeugJA + ABdAAQwB//8ECAAAAwCdqAAAAwAAHroCQKEAAgArQgEBBAgAAAMAnagAAAMAAB6gIIEE2W6kkyvA + WoSIBIIAAAMAAgAAAwAUEAArQgEBBAgAAAMAnagAAAMAAB6gIIEE2W6kkyvAWoSIBIIAAAMAAgAA + AwAUEKIAAgAHRAHBcrAiQAAIRAHBcrAiQAAAAAATY29scm5jbHgACQAQAAkAAAAAEHBhc3AAAAAB + AAAAAQAAABRidHJ0AAAAAAAAHMAAABzAAAAAGHN0dHMAAAAAAAAAAQAAAAIAAdTAAAAAHHN0c2MA + AAAAAAAAAQAAAAEAAAACAAAAAQAAABxzdHN6AAAAAAAAAAAAAAACAAAAWwAAAF0AAAAUc3RjbwAA + AAAAAAABAAAALAAAAGJ1ZHRhAAAAWm1ldGEAAAAAAAAAIWhkbHIAAAAAAAAAAG1kaXJhcHBsAAAA + AAAAAAAAAAAALWlsc3QAAAAlqXRvbwAAAB1kYXRhAAAAAQAAAABMYXZmNjIuMTIuMTAx + """ + + /// The same encode with no SEI injected: HDR10, no dynamic metadata. + static let hdr10Base64 = """ + AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAAAAhmcmVlAAAALm1kYXQAAAAOKAGveOuu+//8 + W2VfLTwAAAAQKAGvCeAkBMh//ydhHsNI/AAAA8Ntb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPo + AAAAyAABAAABAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAA + AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAC7XRyYWsAAABcdGtoZAAAAAMAAAAAAAAAAAAA + AAEAAAAAAAAAyAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAA + AEAAAAAAQAAAAEAAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAAAMgAAAAAAAEAAAAAAmVtZGlh + AAAAIG1kaGQAAAAAAAAAAAAAAAAAEk+AAAOpgFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAA + AAAAAAAAAFZpZGVvSGFuZGxlcgAAAAIQbWluZgAAABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYA + AAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAAB0HN0YmwAAAFkc3RzZAAAAAAAAAABAAABVGh2 + YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAQABAAEgAAABIAAAAAAAAAAEAAAAAAAAAAAAAAAAA + AAAAAAAAAAAAAAAAAAAAAAAAAAAY//8AAADHaHZjQwEECAAAAJ2oAAAAAB7wAPz9+voAAA8DoAAC + ABdAAQwB//8ECAAAAwCdqAAAAwAAHroCQAAXQAEMAf//BAgAAAMAnagAAAMAAB66AkChAAIAK0IB + AQQIAAADAJ2oAAADAAAeoCCBBNlupJMrwFqEiASCAAADAAIAAAMAFBAAK0IBAQQIAAADAJ2oAAAD + AAAeoCCBBNlupJMrwFqEiASCAAADAAIAAAMAFBCiAAIAB0QBwXKwIkAACEQBwXKwIkAAAAAAE2Nv + bHJuY2x4AAkAEAAJAAAAABBwYXNwAAAAAQAAAAEAAAAUYnRydAAAAAAAAAXwAAAF8AAAABhzdHRz + AAAAAAAAAAEAAAACAAHUwAAAABxzdHNjAAAAAAAAAAEAAAABAAAAAgAAAAEAAAAcc3RzegAAAAAA + AAAAAAAAAgAAABIAAAAUAAAAFHN0Y28AAAAAAAAAAQAAACwAAABidWR0YQAAAFptZXRhAAAAAAAA + ACFoZGxyAAAAAAAAAABtZGlyYXBwbAAAAAAAAAAAAAAAAC1pbHN0AAAAJal0b28AAAAdZGF0YQAA + AAEAAAAATGF2ZjYyLjEyLjEwMQ== + """ + + private static func writeFixture(_ base64: String, name: String) throws -> URL { + let cleaned = base64.replacingOccurrences(of: "\n", with: "") + guard let data = Data(base64Encoded: cleaned) else { + throw CocoaError(.fileReadCorruptFile) + } + let url = FileManager.default.temporaryDirectory + .appendingPathComponent("aether-hdr10plus-\(name)-\(UUID().uuidString).mp4") + try data.write(to: url) + return url + } + + @Test("The base probe does not report HDR10+ on a carrying source, and does not read packets to find out") + func baseProbeStaysHDR10() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + let probe = try AetherEngine.probe(url: url) + #expect(probe.videoFormat == .hdr10) + #expect(!probe.carriesHDR10PlusMetadata) + } + + @Test("Asking for .hdr10Plus finds the T.35 payload and upgrades the label before playback") + func detectingUpgradesToHDR10Plus() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + let probe = try AetherEngine.probe(url: url, detecting: .hdr10Plus) + #expect(probe.carriesHDR10PlusMetadata) + #expect(probe.videoFormat == .hdr10Plus) + } + + @Test("A plain HDR10 source is not upgraded") + func plainHDR10IsNotUpgraded() throws { + let url = try Self.writeFixture(Self.hdr10Base64, name: "plain") + defer { try? FileManager.default.removeItem(at: url) } + + let probe = try AetherEngine.probe(url: url, detecting: .hdr10Plus) + #expect(!probe.carriesHDR10PlusMetadata) + #expect(probe.videoFormat == .hdr10) + } + + @Test("Everything the base probe reports is unchanged by the scan") + func baseMetadataSurvivesTheScan() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + let base = try AetherEngine.probe(url: url) + let scanned = try AetherEngine.probe(url: url, detecting: .hdr10Plus) + #expect(scanned.videoCodecName == base.videoCodecName) + #expect(scanned.videoWidth == base.videoWidth) + #expect(scanned.videoHeight == base.videoHeight) + #expect(scanned.durationSeconds == base.durationSeconds) + #expect(scanned.audioTracks.count == base.audioTracks.count) + #expect(scanned.subtitleTracks.count == base.subtitleTracks.count) + #expect(scanned.isDolbyVision == base.isDolbyVision) + } + + @Test("An empty detail set is exactly the base probe") + func emptyDetailSetIsTheBaseProbe() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + let probe = try AetherEngine.probe(url: url, detecting: []) + #expect(!probe.carriesHDR10PlusMetadata) + #expect(probe.videoFormat == .hdr10) + } + + @Test("A zero packet budget cannot confirm, and leaves the base answer alone") + func zeroBudgetCannotConfirm() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + let probe = try AetherEngine.probe( + url: url, detecting: .hdr10Plus, + hdr10PlusDetection: HDR10PlusDetectionOptions(maxPackets: 0)) + #expect(!probe.carriesHDR10PlusMetadata) + #expect(probe.videoFormat == .hdr10) + } + + @Test("The scan confirms on the first video packet") + func confirmsOnFirstPacket() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + let demuxer = Demuxer() + try demuxer.open(url: url) + defer { demuxer.close() } + let outcome = AetherEngine.detectHDR10Plus( + demuxer: demuxer, videoIndex: demuxer.videoStreamIndex, options: HDR10PlusDetectionOptions()) + #expect(outcome.stopReason == .found) + #expect(outcome.packetsRead == 1) + } + + @Test("A source scanned to its end without a hit reports EOF, not a cap") + func exhaustedSourceReportsEOF() throws { + let url = try Self.writeFixture(Self.hdr10Base64, name: "plain") + defer { try? FileManager.default.removeItem(at: url) } + + let demuxer = Demuxer() + try demuxer.open(url: url) + defer { demuxer.close() } + let outcome = AetherEngine.detectHDR10Plus( + demuxer: demuxer, videoIndex: demuxer.videoStreamIndex, options: HDR10PlusDetectionOptions()) + #expect(outcome.stopReason == .demuxEOF) + #expect(!outcome.carriesHDR10Plus) + } + + @Test("A source with no video track degrades to .noVideoTrack rather than throwing") + func noVideoTrackDegrades() throws { + let url = try Self.writeFixture(Self.hdr10Base64, name: "plain") + defer { try? FileManager.default.removeItem(at: url) } + + let demuxer = Demuxer() + try demuxer.open(url: url) + defer { demuxer.close() } + let outcome = AetherEngine.detectHDR10Plus( + demuxer: demuxer, videoIndex: -1, options: HDR10PlusDetectionOptions()) + #expect(outcome.stopReason == .noVideoTrack) + } + + @Test("A custom byte source is scanned like a URL one (the SMB / WebDAV shape)") + func customReaderIsScanned() throws { + let url = try Self.writeFixture(Self.hdr10PlusBase64, name: "plus") + defer { try? FileManager.default.removeItem(at: url) } + + guard let reader = FileIOReader(url: url) else { + Issue.record("FileIOReader could not open the fixture") + return + } + defer { reader.close() } + let probe = try AetherEngine.probe( + source: .custom(reader, formatHint: "mp4"), detecting: .hdr10Plus) + #expect(probe.carriesHDR10PlusMetadata) + #expect(probe.videoFormat == .hdr10Plus) + } +} diff --git a/docs/api.md b/docs/api.md index cbcbce951..673897172 100644 --- a/docs/api.md +++ b/docs/api.md @@ -411,7 +411,10 @@ try await player.reloadAtCurrentPosition() | `prepareForItemReplacement()` | One-shot. Keeps the current native `AVPlayerItem` attached until the next `load()` replaces it atomically, for a host that mounts the engine's own player layer and would otherwise show a black layer across the nil-item gap of a foreground episode or playlist change. Consumed by the next `load()`, cancelled by `stop()`, no effect when the outgoing session is not native. PiP hosts do not need it: an active PiP window already forces the handover (AE#158). | | `stop(resetDisplayCriteria:finalTeardown:)` | Ends the session, `state` becomes `.idle`, `startupProgress` becomes nil. `resetDisplayCriteria: false` keeps the panel in its current mode across an item handoff. | | `AetherEngine.probe(url:options:)` / `probe(source:options:)` | `nonisolated static throws -> SourceProbe`. Demux-only metadata read, no decoders, no session. `options` is read for `httpHeaders` only. For a custom reader the caller keeps ownership, `close()` is not called, and the cursor is left unspecified. | -| `AetherEngine.probeDetectingAtmos(url:options:atmosDetection:)` | `probe` plus a bounded decode pass that authoritatively resolves E-AC-3 JOC for an Atmos badge. Strictly more expensive; never on the playback-start path. Decode-side failures degrade to "not confirmed" rather than throwing. | +| `AetherEngine.probeDetectingAtmos(url:options:atmosDetection:)` | `probe` plus a bounded decode pass that authoritatively resolves E-AC-3 JOC for an Atmos badge. Strictly more expensive; never on the playback-start path. Decode-side failures degrade to "not confirmed" rather than throwing. Same thing as `probe(url:detecting: .atmos)`. | +| `AetherEngine.probe(url:options:detecting:atmosDetection:hdr10PlusDetection:)` / `probe(source:...)` | `probe` plus the opt-in passes named in `ProbeDetail`, over ONE open handle: `.atmos` (the bounded JOC decode above) and `.hdr10Plus` (a bounded packet scan for ST 2094-40 carriage). Asking for both runs both on one connection. Empty set is exactly `probe(url:)`. Both passes only ever SET `isAtmos` / `carriesHDR10PlusMetadata`; a pass that hits a cap leaves the base answer untouched, and pass-side failures are never thrown. | +| `ProbeDetail` | `OptionSet`: `.atmos`, `.hdr10Plus`. | +| `HDR10PlusDetectionOptions` | Bounds for the HDR10+ scan: `maxPackets` (32), `maxBytes` (16 MiB), `timeBudget` (2 s). The byte cap is the one that binds on UHD remuxes, where a single keyframe runs to several MB. | | `AetherEngine.externalSubtitleTrackIDBase` | `100_000`. Synthetic ids of external subtitle tracks start here. | ### Warming a source before it is loaded @@ -869,7 +872,7 @@ All flags default to safe values; the table is the full set. Depth for the media | Type | Carries | | --- | --- | -| `SourceProbe` | `url`, `durationSeconds`, `videoFormat`, `videoCodecID` / `videoCodecName`, `videoWidth` / `videoHeight`, `videoFrameRate`, `isDolbyVision`, `dvProfile`, `audioTracks`, `subtitleTracks`, `metadata`, `isLive`. | +| `SourceProbe` | `url`, `durationSeconds`, `videoFormat`, `videoCodecID` / `videoCodecName`, `videoWidth` / `videoHeight`, `videoFrameRate`, `isDolbyVision`, `dvProfile`, `carriesHDR10PlusMetadata`, `audioTracks`, `subtitleTracks`, `metadata`, `isLive`. `carriesHDR10PlusMetadata` is `false` unless the probe was asked for `.hdr10Plus`, and a `false` means "not asked, or not seen inside the budget", never "proven absent". When it is true and the container said HDR10, `videoFormat` reads `.hdr10Plus`; a Dolby Vision source keeps `.dolbyVision` and carries the flag alongside. | | `TrackInfo` | `id`, `name`, `codec`, `language`, `channels`, `bitrate`, `isDefault`, `isForced`, `isHearingImpaired`, `isCommentary`, `isAtmos`, `assHeader`, `isExternal`, `isNativelyRenderedSubtitle`. The last one marks a subtitle the playback backend draws itself (a remote-HLS rendition AVFoundation renders), so no cue reaches `subtitleCues` and an overlay control (position, delay, styling) has nothing to act on. | | `MediaMetadata` | `title`, `artist`, `album`, `artworkData`, `hasDisplayMetadata`. There is no separate album-artist field: a container's album artist is a fallback the parser folds into `artist`. | | `SubtitleCue` | `id`, `startTime`, `endTime`, `body` (a `SubtitleCue.Body`: `.text`, `.richText`, `.image`), `placement`, plus `text` and `isForced` conveniences. | diff --git a/docs/cli.md b/docs/cli.md index d03514b86..8b1b49ce6 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -33,6 +33,14 @@ Twenty-one subcommands plus the bare-URL `serve` alias. Opens the demuxer, prints the codec / resolution / frame rate of the video track, the audio track list (codec, channels, language, Atmos flag), the subtitle track list, the parsed container metadata (`MediaMetadata`: title / artist / album / albumArtist + embedded cover art presence), then exits. No HLS server is started. +`--detect-hdr10plus` and `--detect-atmos` add the opt-in detail passes of `AetherEngine.probe(url:detecting:)`, and both can be given at once (one open, one connection). HDR10+ is the interesting one to watch: the bare `probe` reads only what the container declares, and ST 2094-40 is declared nowhere, so a carrying source prints `format: hdr10` without the flag and `format: hdr10Plus` plus `HDR10+: ST 2094-40 metadata seen` with it. `not seen` means "not inside the scan budget", not "proven absent". + +```bash +swift run aetherctl probe --detect-hdr10plus /path/to/hdr10plus.mkv +``` + +`Scripts/make-hdr10plus-fixture.py ` builds a ~1 KB HEVC/PQ fixture that carries a real ST 2094-40 T.35 SEI (and prints it base64, which is how the two fixtures embedded in `HDR10PlusProbeIntegrationTests` were made). It verifies itself: it only emits the file when `ffprobe -show_frames` reports `HDR Dynamic Metadata SMPTE2094-40` on it, so the payload is one FFmpeg's own parser accepts rather than a byte pattern that resembles one. + ## serve The original behavior. The CLI prints the loopback URL and parks until Ctrl-C; from another terminal you can: