Skip unselected streams during decode - #7
Closed
ksindi wants to merge 1 commit into
Closed
Conversation
decode_media now marks every stream that was not selected for decoding with AVDISCARD_ALL after stream selection, so the demuxer skips their packets instead of parsing them — and index-based demuxers (mp4/mov) skip reading their chunks entirely on seekable inputs. The packet loop still drops packets without a matching filter context, as defense in depth for demuxers that ignore the discard flag. To let the forward skips become bandwidth savings on gs://s3:// inputs, the cloud reader's forward-seek threshold (previously hardcoded to 64 MiB) is now configurable via AVTENSOR_CLOUD_SEEK_THRESHOLD_BYTES, and when a seek past the threshold spawns a new range request, readers that would keep streaming through the skipped gap are truncated at the fetched frontier (via a shared atomic range end) so they stop instead of downloading bytes FFmpeg skipped over.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Audio-only loads of remote videos (
gs://,s3://, http(s)) currently pay for the video track too:decode_medianever marks unselected streams as discarded, so FFmpeg demuxes every packet of every stream, and the cloud reader's eager streaming downloads all of their bytes. For data-loading pipelines that only want audio out of large A/V masters, that is wasted CPU and bandwidth.What changes
1. Unselected streams are discarded at the demuxer (
src/decoder/mod.rs)After stream selection, every stream whose index was not selected is marked
AVDISCARD_ALL(newdiscard_unselected_streams, via rsmpeg's safestreams_mut()+set_discard()). Index-based demuxers — mp4/mov in particular — then skip unselected streams' samples instead of parsing them, and on seekable inputs seek past their chunks instead of reading them. Multiple selected audio streams are respected; the packet loop's drop-unmatched-packets branch stays as defense in depth for demuxers that ignorediscard.2. Cloud reader turns forward skips into stopped downloads (
src/decoder/io.rs)AVTENSOR_CLOUD_SEEK_THRESHOLD_BYTES, parsed with the same pattern asAVTENSOR_MAX_CLOUD_OBJECT_BYTES. Default unchanged (64 MiB), so behavior is unchanged unless opted in.closest_available_byte + 1) and stop, instead of downloading the skipped bytes to end-of-object.ReadRange.endbecame a sharedArc<AtomicUsize>;cloud_asset_readerchecks it per chunk. The truncation decision is a pure function (reader_truncations) with unit tests.Expected impact (honest version)
gs:///s3://by default: skips smaller than the seek threshold are still streamed through by the eager reader (existing behavior, deliberately kept). LoweringAVTENSOR_CLOUD_SEEK_THRESHOLD_BYTEStrades more range GETs for fewer skipped bytes downloaded; each request has a fixed cost, so the right value depends on chunk sizes.Testing
Ran locally against FFmpeg 7.1.1 (shared) + torch 2.11 on Linux x86_64:
make test— 74 passed, 0 failed, 3 ignored (NVDEC, no GPU)cargo fmt --check,cargo clippy --no-default-features --all-targets -- -D warnings— cleancargo llvm-covline coverage 67.7% (floor 65%)New tests: audio-only decode produces bit-identical samples to the audio stream of a full A/V decode; unselected streams carry
AVDISCARD_ALLafter setup while selected ones do not;reader_truncationsedge cases; seek-threshold env var (default/valid/invalid). Not covered locally: end-to-endgs:///s3://byte-savings measurements (no cloud test infra in this repo — end-to-end cloud tests live in the consuming project); the wheel build and Python-layer checks are left to CI.