MSU Subjective Comparison of Modern Video Codecs

Part 4. Conclusions

Contents

Following graphs present MOS value for each codec after averaging among all sequences and all bitrates.

Picture 21.    Average MOS for all codecs

Picture 22.    Average MOS for all codecs and bitrates

Plots from previous part of report reflect average subjective opinion on codecs. To calculate rank of each codec on the whole test set, we apply following method: count rank of codec for each sequence and then count sum of ranks ("Overall" row). Sorting this array we get rank of codec in comparison (Ref. - uncompressed sequence, was evaluated by experts as well as others).

x264 supremacy is obvious for both 690 kbps and 1024 kbps, it is interesting that its' average mark for 690 kbps is better than the mark of WMV on 1024 kbps. DivX is the second, and WMV and XviD are last. Last place of XviD can be explained by the lack of deblocking during the assessment.

• Open source x264 codec of the new H.264 compression standard was found out to be better than long-developed proprietary solutions.
• Subjective comparison is stable and productive method for video systems' assessment when all testing conditions are precisely adhered to.
• Subjective tests can not always be replaced by objective ones with sufficient precision. SSIM was the best objective metric in our comparison.

1. International Telecommunication Union, "Methodology for the subjective assessment of the quality of television pictures (ITU-R BT.500-11)" (2002)

2. European Broadcasting Union, "SAMVIQ - a New EBU Methodology for Video Quality Evaluations In Multimedia", web site www.ebu.ch

3. Feng Xiao, "DCT-based Video Quality Evaluation", Final Project for EE392J (2000)

4. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity", IEEE Transactions on Image Processing, vol. 13, no. 4 (2004)

5. VQEG, "Final Report from the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment" (2000), web site www.vqeg.org

6. MSU Graphics & Media Lab (Video Group), "MSU Perceptual Video Quality Tool" www.compression.ru/video/quality_measure/perceptual_video_quality_tool_en.html

7. MSU Graphics & Media Lab (Video Group), "MSU Video Quality Measurement Tool" www.compression.ru/video/quality_measure/video_measurement_tool_en.html

8. M.H. Prinson, S. Wolf, "The Impact of Monitor Resolution and Type on Subjective Video Quality Testing", NTIA TM-04-412

9. Avery Lee, "VirtualDub", web site www.virtualdub.org

Contents

Download

• MSU Subjective Comparison of Contemporary Video Codecs - PDF (853 kB)
• MSU Subjective Comparison of Contemporary Video Codecs - ZIP (721 kB)

Contact Information

Additional information

See all MSU Video Codec Comparisons

New comparisons versions:

Other materials

Video resources:

Public MSU video filters Here are available VirtualDub and AviSynth filters. Commonly we develop a whole family of some kind of a filter. Generally there are also versions optimized for PC and hardware implementations (ASIC/FPGA/DSP). These optimized versions can be licensed to companies. Please contact us for details via video(at)graphics.cs.msu_ru. MSU Cartoon Restore (New!) MSU Noise Estimation (New!) MSU Frame Rate Conversion (New!) MSU Image Restoration (New!) MSU Denoising (Top!) MSU Old Cinema MSU Deblocking (Top!) MSU Smart Brightness and Contrast (Top!) MSU Smart Sharpen (Top!) MSU Noise generation MSU Noise estimation MSU Motion Estimation Information MSU Subtitles removal MSU Logo removal (Top!) MSU Deflicker (Top!) MSU Field Shift Fixer AviSynth plug-in MSU StegoVideo MSU Cartoonizer (Top!) MSU SmartDeblocking (Top!) MSU Color Enhancement MSU Old Color Restoration MSU filters FAQ (Read me!) MSU filters statistics (New!)	MSU filters for companies We are working with Intel, Samsung, RealNetworks and other companies on adapting our filters other video processing algorithms for specific video streams, applications and hardware like TV-sets, graphics cards, etc. Some of such projects are non-exclusive. Also we have internal researches. Please let us know via video(at)graphics.cs.msu_ru if you are interested in acquiring a license for such filters or making a custom R&D project on video processing, compression, computer vision. Automatic Objects Segmentation (New!) Semiautomatic Objects Segmentation New Frame Rate Conversion (New!) New Deinterlacer (New!) MSU-Samsung Deinterlacing Project Digital TV Signal Enhancement Old Film Recovery Tuner TV Restore Panorama Video2Photo SuperResolution SuperPrecision MSU-Samsung image and video resampling MSU-Samsung Frame Rate Conversion Motion Phase filter Deshaker (video stabilization) Film Grain/Degrain filter (New!) Deblurring filter Video Content Marking
Codecs comparisons Objective and subjective quality evaluation tests for video and image codecs 4-th MPEG4-AVC/H.264 Comparison (New!) Lossless Video Codecs Comparison 2007 (New!) 3-rd MPEG4-AVC/H.264 Comparison HD Photo and JPEG 2000 Comparison MPEG-2 Decoders Crash Test (Top!) Subjective Compar. of 4 Modern Codecs (Top!) 2-nd MPEG-4 AVC/H.264 Comparison (Top!) FAQ of 2-nd H.264 Comparison 1-st MPEG-4 AVC/H.264 Codecs Comparison FAQ of 1-st H.264 Comparison JPEG 2000 Image Codecs Comparison (Top!) Lossless Video Codecs Comparison 2004 MPEG-4 SP/ASP Codecs Comparison (Top!) Video codecs comparison (old) Ads link: x264 parameters efficiency comparison	Video quality metrics Programs with different objective and subjective video quality metrics implementation MSU Video Quality Metric - objective metrics for codecs and filters comparison (Top!) Implemented metrics short info: PSNR, Delta, MSAD, MSE, SSIM, VQM, MSU Blurring, MSU Blocking MSU Brightness Flicking Metric MSU Brightness Independent PSNR MSU Drop Frame Metric MSU Noise Estimation Metric MSU VQM Metric Tool FAQ MSU Human Perceptual Quality Metric - several metrics for exact visual tests (Top!) Implemented metrics short info: ITU-R BT.500-11: DSIS, DSCQS I & II, SCACJ; EBU: SAMVIQ; MSUCQE MSU PVQ Metric Tool FAQ
Video codecs projects Different research and development projects on video codecs MSU Lossless Video Codec (Top!) MSU Screen Capture Lossless Codec (Top!) MSU MPEG-2 Video Codec x264 Codec Improvement	Other Other information Crazy gallery (filters screams :) License for commercial usage of MSU VideoGroup Public Software (please be careful: some soft like metrics has another license!)

Bookmark this page:   Add to Del.icio.us     Digg It     reddit
 

Last updated: 12-October-2007

---

Search (Russian): Server size: 7467 files, 922Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab