[Google VP9] HDR Encoding with ffmpeg

High Dynamic Range (HDR) video is a maturing technology. There are today a number of gradually-harmonizing standards.

For a maintained picture of HDR video evolution see the WikiPedia article High-dynamic-range video.

FFmpeg can be used along with VP9 to both (1) compress HDR sources to Standard Dynamic Range (SDR) outputs or (2) to repackage an HDR source into HDR web-friendly formats.

Input sources

Currently, when encoding HDR with VP9, any high bit depth (10-bit) codecs supported by FFmpeg can be used as input (for example, VP9 Profile 2 and 3, H.264 (10-bit), DNxHR, HEVC, ProRes, et al.). Also see the YouTube support article Upload High Dynamic Range (HDR) videos.

Encoded output reach:

Currently and in general, target devices supported include any device that can decode VP9 Profile 2 and output to an HDR display.

• Chromecast Ultra + HDR-capable TV
• Chrome 64 Canary on Windows 10 Fall Creators Update, with HDR flag turned on. (This is a good debugging tool).
• Any HDR TV with VP9 Profile 2
  • All Samsung 2017 HDR models (flatpanelshd.com list) and 2016 models starting with “KS”
  • All LG 2017 HDR models (also some 2016 LG G6 models)
  • On mobile, devices that support hardware-accelerated VP9 Profile 2 10-bit HDR decode (Meizu Pro 7)
  • UHD Bluray from 2017

FFmpeg command-line settings to enable VP9 Profile 2 and HDR EOTFs

Your FFmpeg will need to have been built with 10-bit (or even 12-bit) support.

In the examples below we use a statically-built ffmpeg with 10-bit support. You may use our build script to make your own, or follow the guidance on the FFmpeg site.

Following is a usable FFmpeg command line for HDR encoding to SMPTE 2084 (PQ EOTF) standards with VP9:

ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 /dev/null && \
   ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 2 \
   -pix_fmt yuv420p10le \
   -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
   -maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 \
   strobe_scientist_18Mbps.webm

Breaking out the key parameters introduced here:

Example encodes

The following examples all use the source file strobe_scientist.mkv

As a general note on bitrates with HDR encoding, a 25-30% higher bitrate than equivalent 8-bit SDR encoding is recommended. The examples typically target 18Mbps. Good HDR results can be achieved with 12Mbps bitrates with optimal tuning.

The encodes were performed on an Ubuntu Linux system with the following specifications:

• Processor: 4x Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
• Memory (RAM): 8060MB (1492MB used)
• Graphics: Intel HD Graphics 530 (Skylake GT2)
• OS: Ubuntu 16.04 LTS

Convert to YUV 4:2:0 10-bit PQ

Example A: 6Mbps 4k 2-pass

ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -speed 4 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 8000000 -minrate 4000000 \
  -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 8000000 -minrate 4000000 \
  -profile:v 2 -vcodec libvpx-vp9 \
  2pass_HDR_strobe_scientist_6Mbps-static.webm

• Output file: 2pass_HDR_strobe_scientist_6Mbps-static.webm
• Encode time: 142m30s
• File size on disc: 51.2MB

Example B1: 18Mbps 4k 2-pass

ffmpeg -y -report -i strobe_scientist.mkv -b:v **18000000** -speed 4 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate **20800000** -minrate **15040000** \
  -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate **20800000** -minrate **15040000** \
  -profile:v 2 -vcodec libvpx-vp9 \
  2pass_HDR_strobe_scientist_18Mbps-static.webm

• Output file: 2pass_HDR_strobe_scientist_18Mbps-static.webm
• Encode time: 213m32s
• File size on disc: 161MB

Example B2: 18Mbps 1080p 2-pass

ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -speed 4 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 20800000 -minrate 15040000 \
  -profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
  -f webm /dev/null && \
  ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 20800000 -minrate 15040000 \
  -profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
  1080_2pass_HDR_strobe_scientist_18Mbps-static.webm

• Output file: 1080_2pass_HDR_strobe_scientist_18Mbps-static.webm
• Encode time: 113m9s
• File size on disc: 160MB

Convert HDR to SDR with a look-up table (LUT)

Converting HDR to SDR requires the transcoder to understand how to map brightness levels and colors to the smaller BT.709 color space and non-HDR range of brightness. For best results, an LUT describing how to perform that mapping — usually specific to the content — can be used. Examples C and D show how to use a LUT. We used bt2020_to_bt709_example.cube

Example C: 18Mbps 4k HDR to SDR with LUT

ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -speed 4 -pass 1 \
  -pix_fmt yuv420p \
  -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 -profile:v 0 \
  -vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
  -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p \
  -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 -profile:v 0 \
  -vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
  -vcodec libvpx-vp9 -f webm SDR_strobe_scientist_18Mbps-static.webm

• Output file: SDR_strobe_scientist_18Mbps-static.webm
• Encode time: 124m23s
• File size on disc: 165MB

Convert HLG to VP9 HLG10 10-bit PQ

Requires an HLG source. We used the input file strobe_scientist_hlg.mkv

Example D: 18Mbps 4k 2-Pass HLG

ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 1 -speed 4 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 \
  -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 \
  -profile:v 2 -vcodec libvpx-vp9 \
  HLG_HDR_strobe_scientist_18Mbps-static.webm

• Output file: HLG_HDR_strobe_scientist_18Mbps-static.webm
• Encode time: 186m10s
• File size on disc: 165MB

Summary

The output files are of varying size. The input was 4.3GB.

Note that 2-pass encoding is recommended for HDR, since 1-pass can vary in certain configurations.

Tools needed to signal HDR in WebM and MP4 output (Matroska command-line tools)

At the time of writing, FFmpeg doesn’t provide a mechanism for specifying SMPTE 2086 static metadata in WebM/MKV, although it will propagate it from an input stream when the stream contains it.

The mkvmerge tool, part of mkvtoolnix, can be used to insert or modify this metadata if your application requires it.

Following is an example of inserting HDR metadata into one of the files created above. It is especially suited to preparing HDR content for upload to YouTube.

mkvmerge \
  -o HDR_strobe_scientist_18Mbps.mkv\
  --colour-matrix 0:9 \
  --colour-range 0:1 \
  --colour-transfer-characteristics 0:16 \
  --colour-primaries 0:9 \
  --max-content-light 0:1000 \
  --max-frame-light 0:300 \
  --max-luminance 0:1000 \
  --min-luminance 0:0.01 \
  --chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
  --white-colour-coordinates 0:0.3127,0.3290 \
  HDR_strobe_scientist_18Mbps.webm

This example produces the file HDR_strobe_scientist_18Mbps.mkv, in 0.6 seconds.