Compression project >> Video Area Home

Project, ideas: Dr. Dmitriy Vatolin
Implementation: Oleg Petrov

Version 1.0 (DSCQS type II added, bugfixing)
Version 0.9 (DSCQS, DSIS and SAMVIQ added, AviSynth support)

“MSU Perceptual Video Quality tool” is a tool for subjective video quality evaluation.
Download!
Information about subjective methods
FAQ
Order now (for companies)

Main features:

• Six methods of subjective comparison, four are from the ITU-R BT.500 standard and
  one was recently submitted for standardization!
• Three methods of averaging results.
• Supports .avi files and .avs (AviSynth).
• Results are saved in simple CSV format.

How to make subjective video quality comparison

There are a lot of ways to carry out subjective comparison. If you want to make comparison on professional level, you should get a standard that describes every detail of it, like ITU-R BT.500 “Methodology for the subjective assessment of the quality of television pictures”.
It includes detailed description of test environment, tests methodologies and results processing.
Subjective comparison usually consists of following steps:

1. Choose video sequences for testing (they are often named SRC).
2. Choose settings of systems that you want to compare (often named HRC).
3. Choose test methodology (how sequences are presented to experts and how their opinion is collected).
4. Invite sufficient number of experts (not less than 15 are recommended).
5. Calculate average marks for each HRC based on their opinion.

How to use this tool

This tool consists of two parts.
First part is "MSU Perceptual Video Quality - task manager". You can use it to create task file and to collect expert's results - just add video files(.avi, .avs) that you want to compare, set amount of tests that you want expert to go through, give name to task (+some additional options) and save the task file.
If you use .avs file, your AviSynth script should return a value, i.e. have string "return clip" at the end of it.
Second part is "MSU Perceptual Video Quality - player". It can play video files from task and record expert's reaction in a way that is defined by task methodology. Expert can run "MSU PVQ - player" and open any task on his or her choice.
It is possible to group any number of tasks, so expert doesn't need to open them. To do this, create .bat file with following contents:

"MSU perceptual video quality player.exe" "c:\tasks\task1.tsk" "c:\tasks\task2.tsk" ...

In addition, it is possible to conduct test for color blindness, just add -c key to the command line:

"MSU perceptual video quality player.exe" -c "c:\tasks\task1.tsk" "c:\tasks\task2.tsk" ...

Results of this test (standard Ishihara charts are used) will be added to results of all tasks that an expert will go through.

Settings

Task manager settings

1. Name of task - Task name; When expert finishes task, subfolder with name of task is created in folder that contains task file. Results from all experts are stored there. Results for particular expert are stored in file "expert_name".csv.
2. Amount of tests - Total tests amount. When set to minimum, each file is compared with each other only once. You can increase number of tests, it will allow system to measure expert's reliability (measure of reliability is based on average disperse of answers).
3. Add/Remove file - Add/Remove file from test list.This tool supports .avi and .avs files. Add only files that can be opened on experts’ computer! It is assumed that all files are different versions of one sequence (for example, compressed with different codecs).
4. Task type - With this tool you can make six types of subjective comparisons.
   See Subjective video quality methods information
5. Count results - There are three ways of calculating final mark: simple averaging, averaging from ITU-R BT.500 and our own MSU averaging that uses reliability marks for experts. Results from any method that you choose are added to the final file "average_mark.csv".
6. Other task options - Detailed description of all task options you can find in Help file attached to the program.

When you start "MSU Perceptual Video Quality - player", you need to input your name. Your results will be stored in file "expert_name".csv.

1. Playback controls - You can play/pause film, rewind it 3 seconds back, restart current test or whole task (you can disable pause and rewind in task settings). When you cancel test or task your results are discarded.


2. Give your mark - This is an example of a dialog that asks expert about his or her opinion (from ‘Double Stimulus Impairment Scale‘ test method).

Download

• Download ( ZIP, 375 KB).
• Program license for usage in companies
  If you have discount code (e.g. as successful tester) please use this form
• Program license

MSU Video Quality Measurement tools
• MSU Video Quality Measurement Tool - program for objective comparison
  • Features
  • Setting & screenshots
  • FAQ
  • Metrics info
  • PRO-version
  • Download & purchase
  • Plugins & Plugin SDK
    • MSU Brightness Flicking Metric
    • MSU Brightness Independent PSNR
    • MSU Drop Frame Metric
    • MSU Noise Estimation Metric
    • MSU Scene Change Detector
• MSU Human Perceptual Quality Metric - several metrics for exact visual tests
  • Implemented metrics short info:
    ITU-R BT.500-11: DSIS, DSCQS I & II, SCACJ; EBU: SAMVIQ; MSUCQE
  • MSU PVQ Metric Tool FAQ
  • MSU Perceptual Video Quality Tool PDF

Other resources

Video resources:

Public MSU video filters Here are available VirtualDub and AviSynth filters. For a given type of digital video filtration we typically develop a family of different algorithms and implementations. Generally there are also versions optimized for PC and hardware implementations (ASIC/FPGA/DSP). These optimized versions can be licensed to companies. Please contact us for details via video(at)graphics.cs.msu_ru. MSU Cartoon Restore (New!) MSU Noise Estimation MSU Frame Rate Conversion MSU Image Restoration MSU Denoising (Top!) MSU Old Cinema MSU Deblocking (Top!) MSU Smart Brightness and Contrast (Top!) MSU Smart Sharpen (Top!) MSU Noise generation MSU Noise estimation MSU Motion Estimation Information MSU Subtitles removal (New!) MSU Logo removal (Top!) MSU Deflicker (Top!) MSU Field Shift Fixer AviSynth plug-in MSU StegoVideo MSU Cartoonizer (Top!) MSU SmartDeblocking (Top!) MSU Color Enhancement MSU Old Color Restoration MSU TV Commercial Detector (New!) MSU filters FAQ (Read me!) MSU filters statistics	MSU filters for companies We are working with Intel, Samsung, RealNetworks and other companies on adapting our filters other video processing algorithms for specific video streams, applications and hardware like TV-sets, graphics cards, etc. Some of such projects are non-exclusive. Also we have internal researches. Please let us know via video(at)graphics.cs.msu_ru if you are interested in acquiring a license for such filters or making a custom R&D project on video processing, compression, computer vision. Automatic Objects Segmentation Semiautomatic Objects Segmentation New Frame Rate Conversion New Deinterlacer MSU-Samsung Deinterlacing Project Digital TV Signal Enhancement Old Film Recovery Tuner TV Restore Panorama Video2Photo SuperResolution SuperPrecision MSU-Samsung image and video resampling MSU-Samsung Frame Rate Conversion Motion Phase filter Deshaker (video stabilization) Film Grain/Degrain filter Deblurring filter Video Content Search
Codecs comparisons Objective and subjective quality evaluation tests for video and image codecs 5-th MPEG4-AVC/H.264 Comparison (New!) Codec Analysis for Companies: (New!) Research and Development Projects (New!) MSU Video Codecs Scoring System (New!) MSU Video Quality Measurement Tool (New!) Options Analysis of Codec x264 (New!) 4-th MPEG4-AVC/H.264 Comparison (Top!) Lossless Video Codecs Comparison 2007 3-rd MPEG4-AVC/H.264 Comparison HD Photo and JPEG 2000 Comparison MPEG-2 Decoders Crash Test (Top!) Subjective Compar. of 4 Modern Codecs (Top!) 2-nd MPEG-4 AVC/H.264 Comparison (Top!) FAQ of 2-nd H.264 Comparison 1-st MPEG-4 AVC/H.264 Codecs Comparison FAQ of 1-st H.264 Comparison JPEG 2000 Image Codecs Comparison (Top!) Lossless Video Codecs Comparison 2004 MPEG-4 SP/ASP Codecs Comparison (Top!) Video codecs comparison (old) Ext. link: x264 parameters efficiency comparison	MSU Video Quality Measurement tools Programs with different objective and subjective video quality metrics implementation MSU Video Quality Measurement Tool - objective metrics for codecs and filters comparison (New!) (Top!) Implemented metrics short info: PSNR, Delta, MSAD, MSE, SSIM, VQM, MSU Blurring Metric, MSU Blocking Metric MSU Brightness Flicking Metric MSU Brightness Independent PSNR MSU Drop Frame Metric MSU Noise Estimation Metric MSU Scene Change Detector MSU VQMT FAQ MSU Human Perceptual Quality Metric - several metrics for exact visual tests (Top!) Implemented metrics short info: ITU-R BT.500-11: DSIS, DSCQS I & II, SCACJ; EBU: SAMVIQ; MSUCQE MSU PVQ Metric Tool FAQ
Video codecs projects Different research and development projects on video codecs MSU Lossless Video Codec (Top!) MSU Screen Capture Lossless Codec (Top!) MSU MPEG-2 Video Codec x264 Codec Improvement	Other Other information Crazy gallery (filters screams :) License for commercial usage of MSU VideoGroup Public Software (please be careful: some soft like metrics has another license!)

Bookmark this page:   Add to Del.icio.us     Digg It     reddit
 

Last updated: 28-April-2009

---

Search (Russian): Server size: 7898 files, 968Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab