PyAV decoding backend no longer works

[datumbox]

The PyAV video decoding backend no longer works. I believe this can be tracked back to commit 99a655b. Though the implementation of the torchvision_decode backend was updated to receive lists for some arguments and return more parameters, the one at pyav_decode wasn't, leading to a runtime error.

A quick and dirty workaround is to apply the following patch, though this causes the entire video to be read hence it's slow:

diff --git a/slowfast/datasets/decoder.py b/slowfast/datasets/decoder.py
index 7ebf713..231c76b 100644
--- a/slowfast/datasets/decoder.py
+++ b/slowfast/datasets/decoder.py
@@ -414,6 +414,7 @@ def pyav_decode(
     fps = float(container.streams.video[0].average_rate)
     frames_length = container.streams.video[0].frames
     duration = container.streams.video[0].duration
+    start_end_delta_time = duration = None
 
     if duration is None:
         # If failed to fetch the decoding information, decode the entire video.
@@ -450,7 +451,7 @@ def pyav_decode(
 
         frames = [frame.to_rgb().to_ndarray() for frame in video_frames]
         frames = torch.as_tensor(np.stack(frames))
-    return frames, fps, decode_all_video
+    return [frames], fps, decode_all_video, start_end_delta_time
 
 
 def decode(
@@ -513,7 +514,7 @@ def decode(
             assert (
                 min_delta == -math.inf and max_delta == math.inf
             ), "delta sampling not supported in pyav"
-            frames_decoded, fps, decode_all_video = pyav_decode(
+            frames_decoded, fps, decode_all_video, start_end_delta_time = pyav_decode(
                 container,
                 sampling_rate,
                 num_frames,

[lyttonhao]

Thanks for summarizing the issue and sharing workaround. We have ongoing PR to solve this PyAV issue here

[datumbox]

@lyttonhao oh sorry. Sounds like this issue is duplicate. Feel free to close if you track it elsewhere!

[lyttonhao]

Close it as it's tracked in #541