Second Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison
For professional users and codec authorsMSU Graphics & Media Lab (Video Group)
Return to second H.264/AVC comparison home page!
Part 4. Additional information and conclusions
Additionally to one-pass mode in Main profile of standard H.264 we tested two-pass mode and compression using possibilities of High profile. Only x264, Elecard, Ateme and Fraunhofer IIS codecs took part in this test. This test was held on "foreman" and "battle" sequences and without speed measuring.
As it is seen on picture below, it is difficult to analyze results using PSNR/Bitrate graphs. That is why we use Delta Y-PSNR graphs instead of absolute Y-PSNR graphs in this part. In all graphs reference is Main Profile one pass results for current codec.
In addition to average metrics' values we measured metrics' values for separate frames. Basing on this data, Rate Control of each codec could be analyzed. As a rule, on such graphs it is clearly seen whether codec uses B-frames or not ("striation" of graph), location of I-frames (abrupt peaks or collapses on graph), scene changing reaction, quality of R-D models, etc.
On the next graphs there are frames on abscissa axis and bit rates on axis of ordinates (low bit rates are below). Color shows metric's value: the more red, the lesser is value. Thus, for Y-PSNR metric red color means bad quality, blue - good quality. All graphs of one sequence have the same colors (every color corresponds to only one value of metric on all graphs).
Analyzing these graphics we should take into account bit rate handling of codecs since these graphs do not consider it.
On this graph compression quality does not significantly depend from bit rate, because DivX codec does not maintain low bit rates well, and works badly on high ones.
Even in spite of appearance of new metrics, that more adequately reflect human perception of video, subjective video tests still do not lose their importance.
We set ourselves a problem to compare video codecs using only existing objective metrics. But in addition to various graphs that impartially evaluate quality of obtained results, below we give several frames from "bbc" and "battle" sequences, compressed with different codecs.
As it is clearly seen on the graphs, codecs show significantly different results on different sequences. Selected sequences have different types in terms of motion and noise, and this allows checking which codecs are tuned to all types of sequences, and which one - to only one. But also it is very interesting to evaluate codec behavior on all test data.
We considered averaging of results between different sequences as incorrect, and used a method of assigning points to codecs for different measurements. Points are united in arbitrary informal rating in the end.
We tried to reflect real situation on test data as much as possible. We are still developing methodology of our informal comparison. Therefore it is better to consider following results as subjective opinion of comparison authors.
Separate comparisons were carried out for "Best Quality" and "Best Speed" presets. Several graphs were chosen for each preset, on which comparison was held. Obtained marks were summed with some predefined weights.
Codecs received points in a following way:
Following graphs were used for "Best Quality" preset with corresponding coefficients:
Following graphs were used for "Best Speed" preset with corresponding coefficients:
Average values were computed as arithmetic average of all successfully encoded sequences (i.e. codecs' errors were not taken into account).
- MSU H.264 Video Codec Comparison 2005 (eng) - PDF (3560 Kb)
- MSU H.264 Video Codec Comparison 2005 (eng) - ZIP (2636 Kb)
See all MSU Video Codecs Comparisons
MSU video codecs comparisons resources:
- Introduction to Video Codecs Comparison
- Lossless Video Codecs Comparison 2004 (October 2004)
- MPEG-4 SP/ASP Video Codecs Comparison (March 2005)
- JPEG 2000 Image Codecs Comparison (September 2005)
- First Annual MPEG-4 AVC/ H.264 Video Codecs Comparison (January 2005)
- Second Annual MPEG-4 AVC/H.264 Video Codec Comparison (December 2005)
- Subjective Comparison of Modern Video Codecs (February 2006)
- MPEG-2 Video Decoders Comparison (May 2006)
- WMP and JPEG2000 Comparison (October 2006)
- Third Annual MPEG-4 AVC/H.264 Comparison (December 2006) (All versions for free!)
- Lossless Video Codecs Comparison 2007 (March 2007)
- Fourth Annual MPEG-4 AVC/H.264 Comparison (December 2007) (All versions for free!)
- Options Analysis of MPEG-4 AVC/H.264 Codec x264 (December 2008)
- Fifth MPEG-4 AVC/H.264 Comparison (May 2009) (All versions for free!)
- Sixth MPEG-4 AVC/H.264 Comparison (May 2010)
- Seventh MPEG-4 AVC/H.264 Comparison (May 2011)
- Eighth MPEG-4 AVC/H.264 Comparison (May 2012)
- Ninth MPEG-4 AVC/H.264 Comparison (Dec 2013)
- Tenth Video Codec Comparison (HEVC) (Oct 2015)
- Eleventh Video Codec Comparison (HEVC) (Aug 2016)
- Twelfth Video Codec Comparison (HEVC) (Aug 2017)
- Thirteen Video Codec Comparison (HEVC) (Aug 2018)
- Fourteen Video Codec Comparison (HEVC) (Sept 2019)
- Cloud Encoding Servoces Comparison 2019 (Dec 2019)
- Fifteen Video Codec Comparison (HEVC) (Dec 2020)
- Codec Analysis for Companies:
Server size: 8069 files, 1215Mb (Server statistics)
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab