Project description
Task: Create a plugin for GStreamer, which implements VMAF video quality metrics.
VMAF is a new full-reference perceptual video quality metric developed by Netflix, which has a high correlation with subjective quality scores. It is open-source and ML-based metric, and it's base model is constantly updating. The developers add new features to each new version: such as 4K support, 95% confidence intervals, versions for measuring quality of displaying on phones and other viewing options.
GStreamer is a framework for building multimedia applications. This framework allows you to create various applications. GStreamer has a plug-in architecture and has a very large set of plug-ins that can solve 99% of the needs of all multimedia developers. But until recently, GStreamer had only one metric for video quality --- DSSIM. The goal of my GSoC project was to create VMAF plugin for GStreamer.
Plugin description
New iqa-vmaf plugin is based on GstVideoAggregator and supports all libvmaf options. It sends frame-by-frame quality scores in the end of video sequence processing to GStreamer pipeline using messages.
Additionally, new plugin added a support of the following metrics:
- PSNR
- SSIM
- MS-SSIM
since libvmaf may calculate them when calculating VMAF.
Thus, the number of video quality metrics in GStreamer increased five times :).
Resources
The code was written in the following repositories
- GStreamer’s gst-build-bad repository, which contains a number of plugins. Old iqa plugin with DSSIM was renamed to iqa-dssim plugin and iqa-vmaf plugin was added. The iqa-vmaf plugin consists of this files:
- GStreamer’s gst-build repository, which contains scripts for assembly. The gst-build is an auxiliary repository for GStreamer compilation. It uses Meson build system and I created a wrapper for VMAF submodule in it.
- Original VMAF repository. To integrate VMAF into gst-build, I wrote a meson configuration file for the VMAF project. I also made a small patch which was necessary for correct work with exceptions from libvmaf.
Merge requests
- Merge Request to gst-plugins-bad: Status (26-08-19) – on review
- Merge Request to gst-build: Status (26-08-19) – on review
- Merge Request to vmaf: Status (26-08-19) – on review