Hi MLVC team,
First of all, thank you very much for open-sourcing MLVC — the codebase, pretrained checkpoints, conversion tooling and benchmark anchors are all extremely valuable to the community. Really appreciate the effort. 🙏
I have a small question about a piece of pretrained weights that I could not locate in the repo/README.
What I'm trying to do
Run the optical-flow computation defined by video/configs/dataset/run_optic_flow.yaml (invoked by run_optic_flow.py), which is also used by calculate_frame_sequence_metrics.py to compute training/validation frame-sequence metrics.
However, I couldn't find a download link or SHA for me_spynet.pth.tar anywhere in the README — the "Prepare auxiliary models" section covers DCVC-RT I-frame, LPIPS, DeQA-Score and face segmentation, but not the SpyNet ME weights.
I also inspected the released mlvc-psnr-v1.ckpt hoping to extract the ME_Spynet weights from it, but its state_dict only contains the following top-level modules:
bit_estimator_z, decoder, encoder,
feature_adaptor_i, feature_adaptor_p, feature_extractor,
hyper_decoder, hyper_encoder,
q_decoder, q_encoder, q_feature, q_recon,
recon_generation_net, temporal_prior_encoder,
y_prior_fusion, y_spatial_prior
i.e. there is no optic_flow weights are not bundled inside the released MLVC checkpoint either. The same appears to be the case for mlvc-perceptual-v1.ckpt / mlvc-s-*.ckpt (I only verified psnr-v1).
Questions
Do you plan to release the pretrained me_spynet.pth.tar used by run_optic_flow.yaml and calculate_frame_sequence_metrics.py? If yes, could you add a download link (with SHA-256) to the "Prepare auxiliary models" section of the README, similar to LPIPS / DeQA-Score?
If the ME weights are borrowed from an existing project (e.g. DCVC / the original SpyNet), could you point us to the exact source checkpoint and the key-mapping you used, so we can reproduce me_spynet.pth.tar locally?
If SpyNet is only used as an offline optical-flow estimator for dataset preparation (and is intentionally not part of the codec checkpoints), it would be great to clarify this in the README to avoid confusion.
Thanks again for the excellent work and for your time!
Hi MLVC team,
First of all, thank you very much for open-sourcing MLVC — the codebase, pretrained checkpoints, conversion tooling and benchmark anchors are all extremely valuable to the community. Really appreciate the effort. 🙏
I have a small question about a piece of pretrained weights that I could not locate in the repo/README.
What I'm trying to do
Run the optical-flow computation defined by video/configs/dataset/run_optic_flow.yaml (invoked by run_optic_flow.py), which is also used by calculate_frame_sequence_metrics.py to compute training/validation frame-sequence metrics.
However, I couldn't find a download link or SHA for me_spynet.pth.tar anywhere in the README — the "Prepare auxiliary models" section covers DCVC-RT I-frame, LPIPS, DeQA-Score and face segmentation, but not the SpyNet ME weights.
I also inspected the released mlvc-psnr-v1.ckpt hoping to extract the ME_Spynet weights from it, but its state_dict only contains the following top-level modules:
bit_estimator_z, decoder, encoder,
feature_adaptor_i, feature_adaptor_p, feature_extractor,
hyper_decoder, hyper_encoder,
q_decoder, q_encoder, q_feature, q_recon,
recon_generation_net, temporal_prior_encoder,
y_prior_fusion, y_spatial_prior
i.e. there is no optic_flow weights are not bundled inside the released MLVC checkpoint either. The same appears to be the case for mlvc-perceptual-v1.ckpt / mlvc-s-*.ckpt (I only verified psnr-v1).
Questions
Do you plan to release the pretrained me_spynet.pth.tar used by run_optic_flow.yaml and calculate_frame_sequence_metrics.py? If yes, could you add a download link (with SHA-256) to the "Prepare auxiliary models" section of the README, similar to LPIPS / DeQA-Score?
If the ME weights are borrowed from an existing project (e.g. DCVC / the original SpyNet), could you point us to the exact source checkpoint and the key-mapping you used, so we can reproduce me_spynet.pth.tar locally?
If SpyNet is only used as an offline optical-flow estimator for dataset preparation (and is intentionally not part of the codec checkpoints), it would be great to clarify this in the README to avoid confusion.
Thanks again for the excellent work and for your time!