Brief descriptions of the codecs used in our comparison are given in Table 2.
XviD was used as a good quality MPEG-4 ASP reference codec for comparison purposes.
Detailed descriptions of all codecs used in our comparison
can be found in
Results for Movies
This version contains only Y-SSIM results only for Normal preset.
RD Curves
The Normal preset results for each sequence are presented in next figures. The results do not depend strongly on the metric used.
SSIM metric: The leader is x264; MainConcept placed second, and DivX H.264 placed third. For the "Ice Age" and "Up" sequences, the
MediaSDK and Elecard encoders exhibited similar results.
For the "Indiana Jones" and "State Enemy" sequences, the XviD encoder placed fourth.
Encoding Speed
Absolute speed results are presented in next figures. All the encoders except Microsoft Expression and Theora have a
similar growth rate for encoding time versus increasing bitrate. Elecard is the fastest encoder. The encoding speed
of the Microsoft Expression and Theora encoders exhibits almost no dependency on bitrate, and Microsoft Expression’s
encoding speed decreases at 1,200 kbps for the “Ice Age” sequence. XviD’s encoding speed exhibits little dependency
on bitrate for the “Up” sequence.
Speed/Quality tradeoff
Detailed descriptions of the speed/quality trade-off graphs can be found in
Appendix 7. Sometimes, codec results are not present in the particular graph owing to the
codec’s extremely poor performance. The codec’s RD curve has no intersection with the
reference’s RD curve.
The speed/quality trade-off graphs simultaneously show relative quality and encoding speed
for the encoders tested in this comparison. XviD is the reference codec, for which both quality
and speed are normalized to unity for all of the graphs. The terms “better” and “worse” are
used to compare codecs in the same manner as in previous portions of this comparison.
Please note that the method of averaging among all sequences assumes that all codecs produced results
for each sequence. When this is not the case, only existing results are taken into account.
Next figures show results for the Normal preset. The results differ depending on the chosen metric.
Y-SSIM: The three best codecs (no codec performs faster with higher quality) in terms of speed and
quality are XviD, Elecard and x264 on average, except for the "Ice Age" and "Indiana Jones" sequences.
The x264 encoder is better on average than DivX H.264 and MainConcept.
Almost all encoders except Microsoft Expression exhibited better results on average than did Theora.
Bitrate Handling
Encoders with Normal presets, except the Microsoft Expression encoder, demonstrate good bitrate
handling for all sequences. For the “Up” sequence, XviD showed less than stellar results, especially
at high bitrates. For all sequences, MainConcept, x264, Elecard and DivX H.264 showed the best results
compared with other encoders. DivX H.264’s bitrate decreases at 1,500 kbps.
MediaSDK slightly decreases bitrates, but the results are stable.
Theora has a good bitrate handling mechanism but is not very stable.
Relative Quality Analysis
Next table shows relative bitrates for a fixed-quality output for all codecs.
Note that these tables do not include information about the speed of the encoder.
Note that each number in the tables below corresponds to some range of bitrates (see Appendix 7 for more details). Unfortunately, these ranges can differ significantly because of differences in the quality of compared encoders. This situation can lead to some inadequate results when three or more codecs are compared. This comparison technique will be improved in the future.
The leaders are x264 and MainConcept, depending on the quality metric (the bitrate difference is 18% for a fixed quality).
Table 3. Average bitrate ratio for a fixed quality—usage area “Movies,” Normal preset, Y-SSIM metric
|
DivX H.264 |
Elecard |
Expression |
MediaSDK |
MainConcept |
Theora |
x264 |
XviD |
| DivX H.264 |
100% |
124% |
127% |
118% |
87% |
138% |
76% |
128% |
| Elecard |
81% |
100% |
103% |
95% |
70% |
115% |
61% |
104% |
| Expression |
79% |
97% |
100% |
93% |
69% |
114% |
61% |
101% |
| Media SDK |
84% |
105% |
107% |
100% |
74% |
120% |
64% |
108% |
| MainConcept |
115% |
142% |
144% |
136% |
100% |
157% |
88% |
145% |
| Theora |
72% |
87% |
88% |
84% |
64% |
100% |
57% |
88% |
| x264 |
131% |
163% |
164% |
155% |
114% |
176% |
100% |
164% |
| XviD |
78% |
96% |
99% |
93% |
69% |
113% |
61% |
100% |
Next figure depicts the data from the table above.
Each line in the figures corresponds to one codec.
Values on the vertical axis are the average relative bitrates compared
with the codecs along the horizontal axis. A lower bitrate indicates better relative results.
Contents
Results for HDTV
Speed/Quality tradeoff
Detailed descriptions of the speed/quality trade-off graphs can be found in
Appendix 7. Sometimes, codec results are not present in the particular graph owing to the
codec’s extremely poor performance. The codec’s RD curve has no intersection with the
reference’s RD curve.
The speed/quality trade-off graphs simultaneously show relative quality and encoding speed
for the encoders tested in this comparison. XviD is the reference codec, for which both quality
and speed are normalized to unity for all of the graphs. The terms “better” and “worse” are
used to compare codecs in the same manner as in previous portions of this comparison.
Please note that the method of averaging among all sequences assumes that all codecs produced results
for each sequence. When this is not the case, only existing results are taken into account.
Contents
Conclusions
Movies
The leading encoders in this usage area are MainConcept and x264. The quality of the Theora encoder is rather low.
High Speed Preset
The x264 encoder demonstrates better quality on average, and MainConcept shows slightly lower quality. These codecs’ bitrate handling algorithm is acceptable for this usage area. The MediaSDK codec places third, along with the dicas encoder (MediaSDK has slightly higher quality, but its encoding speed is 25% lower). The top four codecs for this preset are the following:30.04.2010
- x264
- MainConcept
- MediaSDK
Normal Preset
The results for the Normal preset differ from those for the High Speed presets only in third place. The x264 encoder demonstrates better quality on average, and MainConcept shows slightly lower quality. The DivX H.264 encoder holds third place. The top three codecs for this preset are the following:
- x264
- MainConcept
- DivX H.264
High Quality Preset
The results for this preset are similar to those of the Normal preset. The leaders are the x264 and MainConcept codecs. DivX H.264 takes third place. The top three codecs for this preset are the following:
- x264
- MainConcept
- DivX H.264
*The encoding speed of these encoders is not in the requirements range owing to our chosen presets or to a lack of options
**The MediaSDK encoder does not meet the requirements for High Speed presets–specifically, no B-frames and exactly one reference frame
HDTV
The leaders in the HDTV area are x264, DivX H.264 and MainConcept. The XviD encoder trails all other H.264 encoders. Theora demonstrates rather poor results compared with other encoders.
High Speed Preset
The x264 encoder demonstrates better quality on average, and MainConcept shows slightly lower quality. The top three codecs for this preset are the following:
- x264
- DivX H.264 (this preset does not meet speed requirements)
- MainConcept
Normal Preset
The results for the Normal preset differ from those of the High Speed presets. MainConcept shows the best results. The difference between best codecs are 2-4% of bitrate for the same quality. The top three codecs for this preset are the following:
- MainConcept
- x264
- DivX H.264
High Quality Preset
The results for the High Quality preset are very interesting: x264 held first place, MainConcept was in second place, and two codecs (DivX H.264 and Elecard) shared third place (their quality results are very close, but DivX H.264 is 1.2 times faster than Elecard). The top four codecs for this preset are the following
- x264t
- MainConcept
- DivX H.264 and Elecard
*The encoding speed of this encoder is not in the requirements range owing to our chosen presets or to a lack of options
**The DivX H.264 High Speed preset for HDTV does not meet the speed requirements
Overall Conclusions
Overall, the leader in this comparison is x264, followed by MainConcept and DivX H.264. The Theora encoder demonstrates the worst results among all codecs tested.
*The encoding speed of this codec does not fall in the required range owing to our chosen presets or to a lack of options.
The overall ranking of the codecs tested in this comparison is as follows:
- x264
- MainConcept
- DivX H.264
- MediaSDK
- Elecard
- XviD
- Theora
Microsoft Expression Encoder 3 could not be placed in this list because of its much longer encoding time compared with other encoders (except Theora).
Using the standard Theora interface, we could not find any multithreading options; Theora works only in single-thread mode. The leader in this comparison is x264—its quality difference (according to the SSIM metric) could be explained by the special encoding option ("tune-SSIM"). Interestingly, using the PSNR metric for MainConcept yielded results comparable with or better than those of x264. This means that no encoder can achieve the best results for both SSIM and PSNR when using the same parameters.
The difference between the MainConcept and DivX H.264 encoders is not overly significant, so these encoders tied for second in this comparison. The developers of the Elecard encoder do not provide a High Speed preset, so its ranking is based solely on the results for the Normal and High Quality presets. The dicas encoder holds third or fourth place for some sequences.
This rank is based only on the encoders’ quality results. Encoding speed is not considered here.
Contents
The Graphics & Media Lab Video Group would like to express
its gratitude to the following companies for providing
the codecs and settings used in this report:
The Video Group would also like to thank these companies for
their help and technical support during the tests.
