Extract motion vectors from MPEG files

[alexarje]

I came across an interesting project called MV-Tractus and is based on extracting motion vectors from h264 encoded videos. The idea is that the MPEG compressor has already done motion analysis, which is actually stored inside of an MPEG file. So if we are able to extract those motion vectors, it may not be necessary to run a new motion analysis on the file (or at least if would be much faster?). Not sure how this works in practice, but it could be cool to test out.

[joachimpoutaraud]

I started to look for some information regarding the extraction of motion vectors from MPEG files with MV-Tractus but it requires to compile FFmpeg inside the toolbox in order to generate the binary files for retrieving motion vector data. This might be feasible but a bit problematic as it will drastically increase the size of the repository and cause some compatibility problems within different OS (i.e. Linux, MacOS, or Windows).

Moreover, I found another repository (mv-extractor) that is also interesting but the pip package is only provided for x86-64 linux at the moment.

So far, I manage to visualize motion vectors using FFmpeg and the codecview filter that can be used to show motion vectors as small arrows for each macroblock. Here is an example:

Now, I am trying to find a way to retrieve the macroblocks info in order to get information regarding the motion vectors positions. This requires to parse NAL unit types and to convert this in a meaninful way for reusing it in Python.