Hot news:

If you find a spelling error, please select an incorrect text and press Ctrl+Enter. Thank you!

Compression project >> Video Area Home

MSU Subjective Comparison of Modern Video Codecs

MSU Graphics & Media Lab (Video Group)

Return to Subjective Comparison of Modern Video Codecs home page!

Part 4. Conclusions

Contents

  • Introduction
  • Results of subjective comparison
  • Evaluation of objective metrics
  • Conclusions
  • Results of the comparison

    Following graphs present MOS value for each codec after averaging among all sequences and all bitrates.

    Picture 21.    Average MOS for all codecs

    Picture 22.    Average MOS for all codecs and bitrates


    Plots from previous part of report reflect average subjective opinion on codecs. To calculate rank of each codec on the whole test set, we apply following method: count rank of codec for each sequence and then count sum of ranks ("Overall" row). Sorting this array we get rank of codec in comparison (Ref. - uncompressed sequence, was evaluated by experts as well as others).

    Battle

    Rancho

    Matrix sc.1

    Matrix sc.2

    Overall

    Place

    Ref.

    1

    1

    2

    1

    5

    -

    x264 1024

    2

    3

    1

    2

    8

    1

    DivX 1024

    3

    4

    5

    3

    15

    2

    XviD 1024

    7

    2

    3

    5

    17

    3

    x264 690

    5

    6

    4

    4

    19

    4

    WMV 1024

    4

    5

    6

    6

    21

    5

    DivX 690

    6

    8

    7

    7

    28

    6

    WMV 690

    8

    7

    9

    9

    33

    7

    XviD 690

    9

    9

    8

    8

    34

    8

    x264 supremacy is obvious for both 690 kbps and 1024 kbps, it is interesting that its' average mark for 690 kbps is better than the mark of WMV on 1024 kbps. DivX is the second, and WMV and XviD are last. Last place of XviD can be explained by the lack of deblocking during the assessment.

    Conclusions

    References

    1. International Telecommunication Union, "Methodology for the subjective assessment of the quality of television pictures (ITU-R BT.500-11)" (2002)

    2. European Broadcasting Union, "SAMVIQ - a New EBU Methodology for Video Quality Evaluations In Multimedia", web site www.ebu.ch

    3. Feng Xiao, "DCT-based Video Quality Evaluation", Final Project for EE392J (2000)

    4. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity", IEEE Transactions on Image Processing, vol. 13, no. 4 (2004)

    5. VQEG, "Final Report from the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment" (2000), web site www.vqeg.org

    6. MSU Graphics & Media Lab (Video Group), "MSU Perceptual Video Quality Tool" www.compression.ru/video/quality_measure/perceptual_video_quality_tool_en.html

    7. MSU Graphics & Media Lab (Video Group), "MSU Video Quality Measurement Tool" www.compression.ru/video/quality_measure/video_measurement_tool_en.html

    8. M.H. Prinson, S. Wolf, "The Impact of Monitor Resolution and Type on Subjective Video Quality Testing", NTIA TM-04-412

    9. Avery Lee, "VirtualDub", web site www.virtualdub.org


    Contents

  • Introduction
  • Gratitude
  • Comparison abstract
  • Overview
  • Rules and goals
  • Sequences
  • Codecs
  • Results of subjective comparison
  • MOS+PSNR/bitrate graphs
  • Z-test resuts
  • MOS+PSNR graphs, grouped by bitrate
  • Evaluation of objective metrics
  • Correlation of objective metrics and subjective scores
  • Conclusions
  • Average MOS for all codecs
  • General conclusions

  • Download



    Contact Information



    Additional information


    Call for HEVC codecs 2019
    See all MSU Video Codecs Comparisons

    MSU video codecs comparisons resources:


    Other materials


    Video resources:

    Bookmark this page:   Add to Del.icio.us Add to Del.icio.us     Digg It Digg It     reddit reddit

     
    Last updated: 09-September-2019

    Search (Russian):
    Server size: 8069 files, 1215Mb (Server statistics)

    Project updated by
    Server Team and MSU Video Group


    Project sponsored by YUVsoft Corp.

    Project supported by MSU Graphics & Media Lab