Compression project >> Video Area Home

MSU Quality Measurement Tool: FAQ

MSU Graphics & Media Lab (Video Group)

MSU VQMT Information

Basic Information About VQMT
Setting & Screenshots (advanced files settings, visualization settings)
Metrics Info (PSNR, MSE, MSAD, SSIM, VQM, MSU Blurring/Blocking)
FAQ (Frequently Asked Questions)
Plugins & Plugin SDK[BI-PSNR, DFM, BFM, NE, SCD]
MSU VQMT program performance (New!)
Subjective quality metrics comparison (New!)

PRO Version Info (with command-line interface) [Full Documentation, PDF, 2.98 MB]
Download & Purchase [Brochure, PDF, 475kB]

MSU Video Quality Measurement Tool FAQ

What about command-line interface? Does MSU VQMT support it?
How can I perform job processing with MSU VQMT?
Your program reports that it failed to open input file.
What's about values of the metrics? What values for each metrics means, that quality is better?
Where can I get more information about your own metrics (MSU Blocking/Blurring metrics and others)?
I recieve unequal results for AviSynth scripts. What's wrong?
Is it possible, to open video files, which are not supported by MSU VQMT directly (.mp4, .mpg, .mov etc.)??
Results are unequal (too low) for one ore more codecs?
What about masking? I really need it!
Your masking is not comfortable; I want to draw a mask on a video frame!
Why MSU VQMT has four variants of PSNR calculation?
What about OPSNR? Why MSU VQMT does not have it?
Why PSNR is slightly different from previous version?
Why MSU VQMT has two variants of SSIM calculation?
Why neither of your SSIM calculations does not match AviSynth SSIM plugin?
Why SSIM (fast) visualization seems to be shifted?
Why MSU Noise Estimation Metric plugin does not match to its previous version?
Why MSU Noise Estimation Metric plugin does not match to VirtualDub MSU Noise Estimation filter?
How can I obtain version for Linux?

Q: What about command-line interface? Does MSU VQMT support it?
A: Public version doesn't support command line. We have such feature in PRO version, but it's available only with commercial license.

Q: How can I perform job processing with MSU VQMT?
A: you can't perform job processing with the free version. There is PRO version with command line interface that supports this feature.

Q: Your program reports that it failed to open input file.
A: Our tool supports .avi, .yuv, .avs, .bmp files as input. Other file types (.wmv, .vob, .mpeg, etc.) can be opened through AviSynth. Our tool can generate the necessary AviSynth script automatically. For more information read AviSynth Introduction paragraph.
If you experience a problem with .avi, then it is possible that MSU VQMT is not compatible with this codec. Please send us information about this codec, or part of such avi file (1-2 megabytes) by e-mail.

Q: What's about values of the metrics? What values for each metrics means, that quality is better?
A:

Metric	Full Reference (FR)/No Reference (NR)	Interpretation
PSNR, PSNR (256), APSNR, APSNR (256)	FR	100 for equal frames, higher values are better
SSIM (fast), SSIM (precise)	FR	Higher values are better, 1 for equal frames
VQM	FR	0 for equal frames, lower values are better
MSU Blocking metric	NR	Lower value corresponds to lower blocking.
MSU Blurring metric	NR (but 2 files are necessary in case of measure visualization)	Lower value corresponds to higher blurring
Delta	FR	0 means equal frames, positive and negative values mean deviation, lower absolute values are better
MSAD	FR	0 means equal frames, lower values are better
MSE	FR	Lower values are better, 0 for equal frames
MSU Brightness Flicking Metric	NR	The higher the value, the higher the brightness flicking in comparison with the previous frame
MSU Brightness Independent PSNR	FR	100 for equal frames, higher values are better
MSU Drop Frames Metric	NR	Metric has two values: 0 means that current frame exists, 1 that it is dropped
MSU Noise Estimation Metric	NR	The higher the value, the higher the noise level
MSU Scene Change Detector	NR	Metric has two values: 1 means that current frame is the first frame in a scene, for other frames metric value is 0.

Q: Where can I get more information about your own metrics (MSU Blocking/Blurring metrics and others)?
A: We plan to prepare some paper first and next publish these assessment methods on our webpage.

Q: I recieve unequal results for AviSynth scripts. What's wrong?
A: Try to review resulting video from AviSynth script processing in VirtualDub to find out possible errors in the script.

Q: Is it possible, to open video files, which are not supported by MSU VQMT directly (.mp4, .mpg, .mov etc.)?
A: Our program supports AviSynth, it is possible to open such files using that software. Most of files you can open using scripts auto generation. If it is not working, try to write AviSynth script on your own. Script examples are provided in the installation folder.

Q: Results are unequal (too low) for one ore more codecs?
A: Some codecs (for example DivX 6.0) shift video file one or two frames back, or make two first frames equal. Check for this shift in your video files after encoding. You can open two encoded videos in two different Virtual Dubs and switch to the 10th frame, for example. And then do switching between them to see if 10th frames in two videos correspond to each other or not. If shift really exists, use AviSynth scripts with DeleteFrame() and DuplicateFrame() to fix it.

Q: What about masking? I really need it!
A: Since 2.01 beta MSU VQMT supports masking. A mask file should be a two color file: one color should mark masked area and another color should mark unmasked area. One of the colors should be black.
User can specify mask color. Masked areas are filled with the specified mask color. User has two choices for mask color definition.
- Specify that the black color is a mask - it means that a black color is taken as a mask color
- Specify that not black color is a mask - it means that a non black color is taken as a mask color
NOTES:
- We assume the color with three zero components in RGB and with zero Y-component in YUV as "black"
- If, for example, the mask file is provided in RGB and calculations are performed in YUV, then mask color is converted into YUV using provided conversion settings and the masked areas are filled with the color achieved
- When black color is used as a mask an unmasked area may contain different colors - it is useful when the user wants to mark mask area on an actual video frame

Q: Your masking is not comfortable; I want to draw a mask on a video frame!
A: Actually you can. Specify black color as a mask and draw black mask on frames, everything will be fine.

Q: Why MSU VQMT has four variants of PSNR calculation?
A: PSNR formula is:

"PSNR" and "APSNR" use the correct way of PSNR calculation and take maximum possible absolute value of color difference as MaxErr. But this way of calculation gives an unpleasant effect after color depth conversion. If color depth is simply increased from 8 to 16 bits, the PSNR will change, because MaxErr should change according to maximum possible absolute value of color difference (255 for 8 bit components and 255 + 255/256 for 16 bit components). Thus "PSNR (256)" and "APSNR (256)" are implemented. They would not change because they use upper boundary of color difference as MaxErr. The upper boundary is 256. This approach is less correct but it is used often because it is fast.
The difference between "PSNR" and "APSNR" is the same as between "PSNR (256)" and "APSNR (256)" and is in the way of average PSNR calculation for a sequence. The correct way to calculate average PSNR for a sequence is to calculate average MSE for all frames (average MSE is arithmetic mean of the MSE values for frames) and after that to calculate PSNR using ordinary equation for PSNR:

This way of average PSNR calculation is used in "PSNR" and "PSNR (256)". But sometimes it is needed to take simple average of all the per frame PSNR values. "APSNR" and "APSNR (256)" are implemented for this case and calculate average PSNR by simply averaging per frame PSNR values.
The next table summarizes the differences:

Metric	MaxErr calculation	Average PSNR calculation
PSNR	correct	correct
PSNR (256)	256 (fast, inexact)	correct
APSNR	correct	averaging
APSNR (256)	256 (fast, inexact)	averaging

"PSNR" metric is recommended for PSNR calculation since it is implemented according to the original PSNR definition.

Q: What about OPSNR? Why MSU VQMT does not have it?
A: Actually, MSU VQMT does have OPSNR. OPSNR is a shortening from the "Overall PSNR" which means that average PSNR for the sequence is calculated in the "correct" way. Thus "PSNR" and "PSNR (256)" of the MSU VQMT are two implementations of OPSNR. Please refer to "Why MSU VQMT has four variants of PSNR calculation?" paragraph to find out the difference between these two.

Q: Why PSNR is slightly different from previous version?
A: New version of MSU VQMT uses IPP L2 measure to calculate MSE for PSNR, since IPP does not have direct MSE calculation function. This is a cause of slight difference.

Q: Why MSU VQMT has two variants of SSIM calculation?
A: SSIM (fast) is identical to the previous version of MSU VQMT SSIM. SSIM (precise) is implemented as a more precise and correct SSIM. It uses Gauss blur instead of box filter, as SSIM (fast) does.

Q: Why neither of your SSIM calculations does not match AviSynth SSIM plugin?
A: SSIM (fast) is close to AviSynth SSIM plugin. But MSU VQMT does not use luminance masking, because it is not used in the original paper.

Q: Why SSIM (fast) visualization seems to be shifted?
A: This effect originates from the sum calculation algorithm for the box filter. The sum is calculated over the block to the bottom-left or up-left of the pixel (depending on if the image is bottom-up or top-down).

Q: Why MSU Noise Estimation Metric plugin does not match to its previous version?
A: The previous version of MSU Noise Estimation Metric plugin has a bug: it calculates measure correctly only when MSU VQMT opens file in RGB color space. The bug is fixed now, but color space conversion sequence has changed. The closest result is achieved when conversion algorithm is set to PC.601. But result is not the same because previous conversions were made using integer data, and now the conversion is made using floating point data.

Q: Why MSU Noise Estimation Metric plugin does not match to VirtualDub MSU Noise Estimation filter?
A: The reason is MSU VQMT and VirtualDub sequences of color space conversions are different. For example, if the file is opened in YV12 color space then MSU VQMT plugin get Y component of the image right away. But VirtualDub plugin gets RGB, converted from YV12 by VirtualDub, and extracts Y component from this RGB. The closest result is achieved when conversion algorithm is set to PC.601. But result is not the same because VirtualDub conversions are made using integer data, and MSU VQMT conversions are made using floating point data.