Introduction to Video Codecs Comparison
MSU Graphics & Media Lab (Video Group)
There are a lot of different codec comparisons on the internet - long and short, emotional or not, biased and unbiased. But most of the authors of such comparisons just put two frames from 2 movies nearby, forgetting following important issues:
1. Every codec gives different quality on different frames of same sequence
Various factors play a role here. First, all codecs have a bitrate control mechanism, that produces quality fluctuations even in very good implementation. Second, bitrate control strategy is selected by user, and in CBR case on slow scenes quality will be low, and on fast ones it will degrade. Third, codecs usually have a so-called key frame, whose quality is determined separately and differs from other frames. Next, final quality depends on prefiltrations, that is included on all present-day codecs. That means that on every sufficiently long movie (average movie is about 150.000-200.000 frames) it is possible to select very good and very bad frames, especially if one-pass CBR compression was used, and movie had enough motion. As a result, having two codecs almost identical in terms of quality, you can easily show the superiority of either codec on specific frames. That method is widely used, alike in press-releases and video processing sites.
Diagram brings into accord with metrics PSNR (if higher, then better quality).
If you make a comparison on "peak" frames, the VP result will be better then DivX one.
Other frames show the opposite conclusion.
2. Different codecs are tuned to different film types.
It is known that some time ago simultaneously existed two different codecs - DivX 4.1 LowMotion and DivX 4.1 FastMotion. Playback was the same, but their internal parameters were tuned to low and high motion, and compression produced different results. Note - FORMAT was same in both cases, but compression was different, so equal size movies had different quality. Similar devotion to specific data type exists in all codecs. Some codecs better perform on high bitrates, while others better compress low ones. Some are tuned to high motion, and some better do with low. Codecs can be tuned to better compress animation or real video. Noisy and clear movies require separate tuning also. That means, that if you have almost equal codecs A and B, it is easy to find movies where A is better than B and vise versa. Curiously this could be same movie, but one could be taken from DVD and other re-filmed in cinema.
This is frames quality in conventional colors (black - no difference with original,
blue and green - more difference, red - huge difference). If we will use this frames
for comparison - DivX 3.1 will be better DivX 5.1.
3. Compression quality could seriously depend on coding parameters.
Most people don't think about coding parameters, blaming it on the developers. Most codecs have many settings that can seriously affect quality First of all, these are bitrate strategy parameters, especially selection between "quality-based" and "bitrate-based" modes. After that prefiltration parameters play a role. For example, deinterlacing (because some movies were compressed with MPEG4 in interlaced mode), denoising (noise reduction), deflicking (removal of effect) and so on. There are parameters of key frame frequence, B-frame mask, control of prefiltering dependence from movie. etc. Here it is even easier to manipulate the results of a comparison of two codecs by simply optimizing the parameters of one of them. One can even put both movies on his site so everyone could see the difference (like some companies do) :-). In that case fact that one codec worked 10 times slower than another somehow becomes lost - why the people (and journalist) should to know such uninteresting information? In that case source sequence could not be found ( due to size, for sure), and coding parameters are unknown (and now you know why). Really there are many more delicate ways to receive "advantage" and even real advantage - for example there are no criteria to determine how frame skipping is significant for perception. But these methods are most common. So, if you study "fooling" methods, you could see that sufficiently large testing is required for correct comparison. As a minimum it is important to compare sequences with different characteristic on different bitrates, while using integral (average) quality characteristics throughout the sequence. So, let us wish you successful detailed comparison.
Quality fall in one of the bitrates on this codec, because of the rate control error.
This bitrate is unlucky for this codec.
MSU video codecs comparisons resources:
- Introduction to Video Codecs Comparison
- Lossless Video Codecs Comparison 2004 (October 2004)
- MPEG-4 SP/ASP Video Codecs Comparison (March 2005)
- JPEG 2000 Image Codecs Comparison (September 2005)
- First Annual MPEG-4 AVC/ H.264 Video Codecs Comparison (January 2005)
- Second Annual MPEG-4 AVC/H.264 Video Codec Comparison (December 2005)
- Subjective Comparison of Modern Video Codecs (February 2006)
- MPEG-2 Video Decoders Comparison (May 2006)
- WMP and JPEG2000 Comparison (October 2006)
- Third Annual MPEG-4 AVC/H.264 Comparison (December 2006) (All versions for free!)
- Lossless Video Codecs Comparison 2007 (March 2007)
- Fourth Annual MPEG-4 AVC/H.264 Comparison (December 2007) (All versions for free!)
- Options Analysis of MPEG-4 AVC/H.264 Codec x264 (December 2008)
- Fifth MPEG-4 AVC/H.264 Comparison (May 2009) (All versions for free!)
- Sixth MPEG-4 AVC/H.264 Comparison (May 2010)
- Seventh MPEG-4 AVC/H.264 Comparison (May 2011)
- Eighth MPEG-4 AVC/H.264 Comparison (May 2012)
- Ninth MPEG-4 AVC/H.264 Comparison (Dec 2013)
- Tenth Video Codec Comparison (HEVC) (Oct 2015)
- Eleventh Video Codec Comparison (HEVC) (Aug 2016)
- Twelfth Video Codec Comparison (HEVC) (Aug 2017)
- Thirteen Video Codec Comparison (HEVC) (Aug 2018)
- Fourteen Video Codec Comparison (HEVC) (Sept 2019)
- Cloud Encoding Servoces Comparison 2019 (Dec 2019)
- Fifteen Video Codec Comparison (HEVC) (Dec 2020)
- Codec Analysis for Companies:
Server size: 8069 files, 1215Mb (Server statistics)
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab