Implement real-time variance-based partitioning

[jianj-g]

No description provided.

[leolzhao]

Please provide results for your code change. @jianj-g

[jianj-g]

Please provide results for your code change. @jianj-g

Here's the command line we use now to test RTC mode:

./avmenc --passes=1 --bit-depth=8 --obu --lag-in-frames=0 --kf-min-dist=1000 --kf-max-dist=1000 ~/Videos/inp_webrtc.640_360.yuv -w 640 -h 360 -o out.ivf --fps=15/1 --target-bitrate=400 --auto-alt-ref=0 --enable-tpl-model=0 --enable-keyframe-filtering=0 --enable-deblocking=1 --enable-restoration=0 --enable-intra-edge-filter=0 --enable-flip-idtx=1 --enable-masked-comp=0 --enable-onesided-comp=0 --enable-interintra-comp=0 --enable-smooth-interintra=0 --enable-diff-wtd-comp=0 --enable-interinter-wedge=0 --enable-interintra-wedge=0 --enable-global-motion=0 --enable-warped-motion=0 --enable-smooth-intra=0 --enable-paeth-intra=0 --enable-cfl-intra=0 --force-video-mode=1 --enable-overlay=0 --enable-angle-delta=0 --enable-trellis-quant=0 --enable-qm=0 --use-intra-dct-only=1 --coeff-cost-upd-freq=2 --mode-cost-upd-freq=2 --mv-cost-upd-freq=3 --frame-parallel=0 --aq-mode=0 --deltaq-mode=0 --frame-boost=0 --noise-sensitivity=0 --cdf-update-mode=1 --reduced-reference-set=1 --enable-ref-frame-mvs=0 --erp-pruning-level=0 --use-ml-erp-pruning=0 --enable-ext-partitions=0 --enable-mrls=0 --enable-pc-wiener=0 --enable-wiener-nonsep=0 --enable-tip=0 --enable-bawp=0 --enable-cwp=0 --enable-imp-msk-bld=0 --enable-ist=0 --enable-inter-ist=0 --enable-inter-ddt=0 --enable-cctx=0 --enable-ibp=0 --max-drl-refmvs=0 --max-drl-refbvs=0 --enable-refmvbank=0 --enable-opfl-refine=0 --enable-ccso=0 --enable-lf-sub-pu=0 --enable-adaptive-mvd=0 --enable-flex-mvres=0 --enable-joint-mvd=0 --enable-refinemv=0 --enable-mvd-sign-derive=0 --enable-parity-hiding=0 --enable-warp-delta=0 --enable-warp-extend=0 --tile-columns=0 --gf-min-pyr-height=0 --gf-max-pyr-height=0 --end-usage=q --use-fixed-qp-offsets=0 --psnr --threads=1 --enable-sdp=0 --min-gf-interval=8 --max-gf-interval=8 --explicit-ref-frame-map=1 --enable-bru=0 --enable-ext-seg=0 --enable-six-param-warp-delta=0 --enable-cdef=1 --enable-cdef-on-skip-txfm=0 --limit=1000 --dpb-size=3 --use-inter-dct-only=0 enable-fsc=1 --enable-idtx-intra=1 --enable-palette=1 --enable-intrabc-ext=0 --enable-intrabc=1 --tune-content=0 --rt --cpu-used=6 --min-qp=100 --max-qp=100 --enable-gdf=0 --max-reference-frames=1

Here's the result on the 360p clip:

With variance based partitioning:

Bitrate(kbps)  |  PSNR(Y)  |  PSNR(U)  |  PSNR(V)  |  PSNR(Avg)  |  PSNR(Overall)  |  Encoding time (FPS)
407.401600  |  41.390690  |  44.641992  |  45.486243  |  42.303441    |  42.302828        |    18.0s (16.6 fps)

Without:

Bitrate(kbps)  |  PSNR(Y)  |  PSNR(U)  |  PSNR(V)  |  PSNR(Avg)  |  PSNR(Overall)  |  Encoding time (FPS)
336.636800  |  41.987292  |  45.136939  |  46.031441  |  42.885996    |  42.885658        |  2022.1s (0.1 fps)

[yeqing-wu]

I’ve reviewed the code and have a few comments. Since the code changes are extensive, I’ll read them again tomorrow.

[jianj-g]

I’ve reviewed the code and have a few comments. Since the code changes are extensive, I’ll read them again tomorrow.

Thanks for the careful review! I've addressed the comments.

[yeqing-wu]

@yunqingwang1, since you have a better understanding of the code for the RT use case in libavm, could you please help review this PR as well? The code changes are quite a lot, so let's review them carefully to solve the issues as early as possible.

Thanks
Yeqing

[jianj-g]

The failed integer / undefined sanitizer runs don't seem to be related to this PR...

[urvangjoshi]

The failed integer / undefined sanitizer runs don't seem to be related to this PR...

Yes, the current head of av2-enc seems to have this failure. Filed issue: #5231