Compression project >> Video Area Home

Call for MPEG4-AVC/H.264 codecs

Sixth H.264 video codec comparison
For real researchers and developers in field of high-end video compression

MSU Graphics & Media Lab (Video Group)

Deadline is extended. New requirements for presets are described.

Important Dates

February, 15	Deadline for receipt of a H.264 codec with required presets
March, 2	Deadline for settling technical problems with codec’s functioning
April, 8	Draft version of report that will be sent to all participants
April, 15	Deadline for reception of comments to the draft
April, 21	Comparison report release

Task of the Comparison

To perform comparative unbiased analysis of the current software implementations of H.264/MPEG-4 AVC video coding standard using objective metrics

Scope of test

Summary report topics:

Objective measurements (SSIM, PSNR, Average Advantage and etc.)

Encoding time

Analysis of averaged objective results

Leaders in different areas

Some special points of this comparison

Codecs options analysis (see example at Options Analysis of MPEG-4 AVC/H.264 Codec x264)

Pareto optimal presets construction for all codecs

Comparison between Pareto optimal presets between different codecs

Options analysis for codecs - analyze the effectiveness of some options (ME methods, rate-control options, pre- and post filtering, etc.)

Additional subjective analysis for video codes

For psychovisual enhancement in codecs analysis

For fade processing analysis

For animation movie compression analysis

Special types of analysis for codecs

Using natural sequences' special modification

Using synthetic sequences

Separate analysis of codecs main subsystems

See previous comparison for details.

New in This Comparison

This comparison will includes some new features, that previous compasrisons do not include

Subjective comparison for the codes

More codecs - we will include codecs publicly available and will choose optimal presets by Option Analysis

Comparison between Pareto optimal presets between different codecs

Effectiveness analysis of some encoder options

New video sequences

New presets requirements (new use-cases)

GPU-used encoding analysis

Comparison Rules

There will be two types of testing sequences and for every type there will be specific set of presets, bitrates, and profiles:

HDTV streams

resolutions: 1920x1080, 1280x720

bitrates 0.7-10 Mbps

3 presets: "High Speed", "High Quality", "Normal"

speed requirements (at 3 Mbps, 1280x720 sequences):

Minimum 100 fps for "High Speed" preset (1 pass, no B-frames, 1 reference frame)

Minimum 50 fps for "Normal" preset (2 passes)

Minimum 20 fps for "High Quality" preset

Movie streams

different types of content (movie, animation)

resolutions: PAL, NTSC, 4CIF

bitrates 500-2000 Kbps

3 presets: "High Speed", "High Quality", "Normal"

speed requirements (at 750 Kbps, 4CIF sequence):

Minimum 120 fps for "High Speed" preset (1 pass, no B-frames, 1 reference frame)

Minimum 80 fps for "Normal" preset (2 passes)

Minimum 40 fps for "High Quality" preset

We are planning to include new codecs that did not participate in previous comparison by choosing presets for them ourself. For that task we will use option analysis. So we will be glad to have a direct contact with codec developers. The main benefit of direct participation for developers is receiving Pro version of comparison free of charge.

The main reason of speed limitation is to compare objective quality without regarding speed/quality trade-off, so if preset provided by developer will work much faster than requirements but with low quality this can lead to low total results.

An important restriction on a preset is encoding time for it. A few iterations of compliance testing and preset optimization are possible to meet the requirements set above. Please pay attention that we will use 4-cores CPU for encoding, so you can use multi-threading.

All speed measurements will be performed using our testing hardware (detailed description see below).

Decoding process will be the following: all encoded sequences are decoding with the JM decoder.

All measurements will be performed using

YUVsoft Video Codec Scoring System and

MSU Video Quality Measurement Tool (Pro Version).

Before results' publishing each developer will receive the results of its codec and competitive free codecs. Developers of each codec can write a comment (one paragraph) about the comparison results. That comment will be included in the report.

We are willing to completely or partially delete information about some codec in the public version of comparison report only in exceptional cases (e.g. critical errors in a codec).

If your company wants to receive results of your codec testing without publication and information disclosure, you should pay for measurements and report preparing before comparison's beginning. You can join to comparison for free if you agree that your codec's results will be published.

Full version of comparison report is available for direct participants for free.

Testing Hardware Characteristics

4-cores processor: Intel Core i7 920, 2.67GHz

OS Name: Microsoft Windows 7 Professional 64-bit

Total Physical Memory: 6 GB

Codec Requirements

Presets for different types of video sequences should be provided by the developers

Codec should allow to set arbitrary bitrate of resulting stream

3 variants of codec interface are possible:

Console codec version (with batch processing support — bitrate and file names must be possible to assign from the command line). This variant is most preferable.

Video for Windows Codec with correct state saving (batch processing support).

Direct Show filter. In this case software for batch processing should be provided by the developers.

Codec should open and save *.yuv or *.avi (YV12 colorspace) files

Encoder should be compatible with JM reference decoder