MSU Cloud Benchmark 2020

Video Transcoding Cloud Services Comparison
conducted by Moscow State University Graphics & Media Lab Video Group

Find the best encoding service for your task

Video group head: Dr. Dmitriy Vatolin
Project head: Dr. Dmitriy Kulikov
Measurements, analysis: Dr. Mikhail Erofeev,
Egor Sklyarov,
Anastasia Antsiferova


Contents


About

We perform independent comparison of cloud-based video-encoding services. This kind of encoding tools is quickly gaining popularity and selecting one for your tasks is quite difficult. In our experience, encoding performance may entirely differ from service to service. The results of our previous comparison showed that in the case of cloud services that provide the same quality, the file-size difference can reach 100% and the cost per minute difference is 700%.

Comparison main points

  • Encoding time and objective videos quality are measured
    We perform a series of mesurements to estimate average time needed for transcoding. Subjective comparison and objective algorithms are used for estimating the resulting video quality
  • Two encoding use cases: default encoding options and options provided by service maintainers
    This year we will perform two types of tests: encode test videos using service default encoding settings and using the settings recommended by service maintainers for our scenario. If no settings are provided we will do our best to find and use optimal encoding settings (for example, see "Tunned Qencode" in the previous comparison report). If a service does not have an option to vary encoding settings, it will not be presented in the second test case
  • FullHD videos of different type and content (including UGC, vertical and noisy videos, etc.)
    5-10 FullHD videos will be used for tests. We will perform two types of measurements: the main case involved encoding into FullHD, and the additional case will include transcoding into several resolutions (the results will be published in the report appendix)
  • Cross-checking is performed by industry experts and independent users
    We ask the users of participating services to encode several videos from test set to validate the results
  • Anonymous accounts are used for testing
    For fair comparison, we use accounts which are not connected to us as the comparison organizers. This is done to prevent the services reserving separate servers/rules for comparison test videos processing

Comparison Stages

1. Application
February, 18 - April, 10
Application for participation is open for all cloud encoding services and users!

Use the form to tell us which services you want to see in this comparison



2. Measurements We encode comparison videos in each service and measure encoding performance


3. Verification We conduct independent cross-checking of obtained results. You can become a contributor and verify comparison results. If you have an account in any of cloud encoding services, we will provide test sequences and pay for encoding time which needed for measurements


4. Release PDF and HTML reports are published on this page. There will be two versions (free and enterprise). All contributors will receive full (enterprise) report

List of services to be compared

Cloud Encoding Service Supported standards
1 Alibaba H.264, HEVC
2 Amazon Elastic Transcoder H.264
3 AWS Elemental MediaConvert H.264, HEVC
4 Coconut H.264, HEVC
5 Qencode H.264, HEVC
6 Zencoder H.264, HEVC
7 We are open to add other services to the comparison. Please use the form below if you have any suggestions.
Which cloud encoding services you want to see in the comparison?

We are looking for people who have an account in any cloud encoding services and able to use it for verificating encoding results. If you want to participate in our comparison, contact us using this form or via cloud@compression.ru


Download previous report

Video Transcoding Clouds Comparison 2019
Download free full version
  • 6 cloud encoding services
    Alibaba, Amazon Elastic Transcoder, AWS Elemental MediaConvert, Coconut, Qencode, Zencoder
  • 4 FullHD video sequences
    epson, fountains, hawk and hockey from Vimeo website
  • Three resolutions
    FullHD, HD, SD
  • Three encoding use cases (presets)
    Encoding with similar options, default options of tested services and with tuned options for our test videos
  • HTML and PDF documents
    118 interactive charts and 33 pages

Take part in cloud video-encoding services comparison

We ask users of cloud encoding/transcoding services to take part in this comparison by encoding several videos to compare the results with our mesurements. Approximate timeline of verification measuremets for contributors:

  1. We send you several videos for encoding
  2. You send us compressed video streams
  3. We analyse the results and prepare a report
  4. You receive enterprise version of the report

If you have an account in any cloud encoding service and want contact us usint the form below. We are ready to compensate the cost of encoding.



Contact Information


compression.ru
in cooperation
with
Lomonosov MSU
Graphics & Media Lab
(Video Group)
Dubna State University
Institute for Information
Transmission Problems RAS

Subscribe to report updates


Materials about MSU Codec Comparison


Call for cloud encoding comparison participation 2020
See all MSU Video Codecs Comparisons

MSU video codecs comparisons resources:


Other Materials


Video resources:

Last updated: 03-April-2020


Server size: 8069 files, 1215Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab