HDR Encoding Guide for Premiere Pro

Note that HDR clips will usually have more saturated colors than their comparable SDR versions, and some movies also have different color grading in HDR as well, but it's possible something could be happening here.

There will likely be some inaccuracies in color, because of the way Premiere handles HDR source files. Premiere converts them to rec709 first, for display purposes. The format they use is okay if you want to use Premiere's own H265 HDR export, but for Voukoder, the gamma and color needs to be converted back to the raw ST2084-BT2020 data using my preset. Unfortunately, because premiere already converted it into a rec709 color space, converting it back is not going to be 100% perfect. While I do use accurate rec709 to rec2020 conversion calculations, you're going to lose a bit of the original wider color gamut, and there may be rounding errors in both conversions that create that effect you're describing. Gradation between colors though should remain just as smooth as the conversions happen in floating point.

Ideally, the best way to encode these videos with Voukoder would be to ensure Premiere doesn't recognize them as HDR to begin with, so it doesn't perform that conversion, and you can then output 1:1 color information perfectly (or as accurate as the encoder itself is anyway). One way I've found that can work with this is if you append the HDR video, to a video file that has the same resolution, codec, frame rate, and bit depth, but is encoded with rec709 instead of rec2020 metadata. Maybe encode a blank video with the same attributes, including audio codecs, and then use ffmpeg to concatenate the two files together. After concatenating, the output file will use the first file's metadata, so it will ignore the HDR metadata from the second file, and premiere will think the whole thing is rec709. Then you won't need the preset to pre-format the color correctly, as it will already be ready to export. If you can successfully get Premiere to ignore the HDR metadata in the file, you'll create a much more accurate encode in voukoder, but my preset should still get you pretty close.

I don't believe premiere tells you whether it thinks the video is SDR or HDR unfortunately. You just have to look at the video. Do the colors look more natural? Are some of the highlights blown out? If it was displaying the raw data, the color would look flat and unnatural, slightly yellow/green tinted, and no highlights would be blown out (unless they're just clipped in the source)

The way you can tell Premiere is going to read a video that way is by opening it in MediaInfo. If the clip mentions having BT2020 color and/or PQ transfer characteristics, rather than rec709, then premiere will read it as HDR. Basically, you want to retain the same encoded video data, without those pieces of metadata in there. So appending it to a non HDR video with the same resolution, frame rate, codec, audio format, and container will trick the metadata into thinking it's SDR, and premiere will be tricked too.

If you want to bypass all this work, it might just be easier to use Premiere's own H265 encoder now with HDR options turned on, but if you want to use Voukoder for better control over the encode, it might be worth trying.

Zitat von dcsimpson

Forgive me but I still don't quite understand. So you're saying if I concatenate a 4k rec 709 SDR file with my actual 4k HDR source file and import that into premiere, premiere will treat it like a rec709 clip and it will look correct to the naked eye without any conversion, presets or anything?

And then are you saying just export from Premiere using Voukoder like normal as a rec709 sdr 4k file?

What I'm saying is, in order to get the most accurate color reproduction out of Voukoder when encoding in HDR, Premiere has to wrongly think the video you imported is SDR. It will display incorrectly, with flat colors and tinting, as shown in the examples in the first post, however, despite displaying incorrectly in the program monitor, voukoder will encode it correctly because the raw ST2084/BT2020 data is being carried over untouched.

If you concat a rec709 4K file (same resolution, codec, frame rate, and audio) with the HDR file, premiere will think you're importing a rec709 file, and therefore not perform any conversion on the colors. Because it doesn't perform any conversion, you won't need my preset to "undo" what Premiere typically does with HDR files normally, and it's already ready for export.

For an example, here is a project I'm working on right now. Here's what it looks like in Premiere:

This is the raw unprocessed ST2084/BT2020 footage. Because of this I can use voukoder without my preset. Premiere (wrongly) thinks it's loading rec709 footage because my footage looks like this in MediaInfo:

and here is how the same frame appears with the HDR tonemapped to SDR correctly:

much more natural

and here's what Premiere WOULD have looked like if it originally recognized my source footage as HDR:

Colors and highlights quite a bit more blown out. While I've used a Lumetri preset to simulate what Premiere does with HDR footage, it's essentially the same idea. They apply a conversion to the original raw data to make it viewable on an SDR screen, but just as a flat 100nit clip instead of tonemapping. Thankfully because of the floating point nature of their video processing, I can reverse this conversion for the most part with my preset, but because that then becomes two colorspace conversions in a row, some loss of color accuracy occurs in the process.

I would just use premiere and voukoder to make a blank rec709 file to concat with.

But for example, assuming your source is 3840x2160 and 23.976 fps, I would open premiere, create a new sequence that matches those aspects, and insert a "black video" (file->new->black video) into the sequence. By default it's 15 seconds but you can trim it down to just a few seconds, and then in voukoder do something like this:

(this is for x265)

My exported file doesn't list rec709 but that's okay. As long as it doesn't list rec2020 and/or PQ, you should be good.

As for the frame rate, if exporting as MP4 gives you variable frame rate, then try exporting as MKV. I believe ffmpeg might still be able to concat from different containers because it's using the original track data. If not, you can remux the MKV to MP4. Oddly enough, the easiest method I've found to remux MKV to MP4 is in the program OBS, which contains a tool specifically for this purpose. But the app AVIDemux would probably work fine too.

Now, I'm not 100% certain what would happen if you try to concat a video only file with video+audio. It might work, but otherwise you might need to ensure the blank video includes a matching audio codec. If your HDR source has something weird for audio, you might need to trim out the original audio somehow, cut to the same length as your blank clip above. AVIDemux might be able to help you with that as well, but that's a more complicated process. If the source is just AC3 audio or something, you can include that codec in your voukoder settings.

Zitat von dcsimpson

This time I embedded rec709 in the file i was going to pair with the source file and that all seemed to work ok and my output from the concat lists rec 709 in mediainfo so that part was good. But when I import into Premiere the hdr portion still looks really bad with highlights clipped. Bummer.

But let's say it did work for me. How do you actually export from premiere at this point? At what stage did you produce that nice tone mapped image of Ellie? Do you export using all those methods that you described initially that include embedding the rec2020 color in the VUI tab and all that?

The tonemapping is another part of my own personal process as I also produce SDR color graded versions of my videos, but as for how to encode it once you have the video looking like premiere isn't processing the HDR-to-SDR conversion itself, that's listed in the first post. If you've already got the MaxCLL and MaxFALL numbers from the source file, you can skip the HDR Metadata section of the guide, as that's what I use to analyze my source footage (I currently have over 2000 frames captured from my current project for that purpose)

So yeah, just follow along where I say what to put into the VUI section for x265 and it should be good. Skip the part after the edit unless you want to use some other codec.

Hmm I'm having trouble with MP4/MKV files as well (my files are HEVC)

I did this one time successfully. It was with my ProRes footage stored in an MOV file. My Atomos Ninja Inferno can record HDR either with the HDR metadata, or without, and one time I had a recording where I accidentally let it record with the metadata, but I had a short rec709 clip with the same frame rate and resolution and codec, so I used concat to combine the two and it worked great. But I don't seem to be having the same results with my HEVC files. Premiere actually glitches trying to read them which is particularly strange.

Perhaps there is something inherently different and incompatible between SDR HEVC-10bit and HDR HEVC-10bit, while perhaps with my ProRes footage the only difference was in the container's metadata. I would have thought the same codec would be able to mux together like that, but perhaps there's something deeper in HEVC preventing that. I even tried creating two MKV files and using mkvtoolnix to append them and that had the same result, glitchy video that wouldn't play. I'm surprised yours is even playing after concat haha

Unfortunately if you cannot successfully remove the metadata from the file before Premiere reads it, then you'll need to use my preset. I'll keep researching this to see if there is any other way to strip that info but I think it's buried deep in the encoded stream so it may not be possible.

If you want to just make really basic cuts like trimming out a part, without any effects or transitions, you might want to try using AVIDemux for this purpose, as it can perform that task without re-encoding at all. Then if you still want to re-encode it, you can use the file it created and encode directly in ffmpeg, although learning the full command line for something like that might be complex.

But yeah, if you're doing full editing work, I think the preset is the best way to handle that type of footage at least until we can figure something else out. Or I suppose you could use Adobe's HEVC encoder instead, although I'm not a fan of the way that thing works myself.

Okay, I think I found out an option that might help you, although it requires a full re-encode to ProRes. ProRes is what I use to capture my footage, and it is nearly a perfect representation of the original pixel data. It is compressed, but it's such a small level of compression that it's not going to impact your final encode in any visible way. ProRes is often used during post production of products because you can potentially re-encode it multiple times and not notice a difference. The main drawbacks then are the time it takes to create this intermediate file, and that the file will be huge. Here is the command line I used:

ffmpeg -i input.mp4 -c:v prores_ks -profile:v 2 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709" -c:a copy output.mov

I've used the ProRes 422 format here. If you don't mind using extra file space, you can use the 422 HQ format which is even less compression by changing to -profile:v 3, although personally I don't think it's necessary unless you're doing several re-encodes. There are also profiles 4 and 5 for even less compression if you're crazy lol

Then you should be able to import that into Premiere, and Premiere will think it's looking at a rec709 file, meaning you can perfectly retain color information while exporting with Voukoder, and not need to use my preset as a workaround.

Here is the mediainfo of the file I created:

Here is the video imported into Premiere:

And here's what the original video looked like imported into premiere. before using this method to strip the HDR metadata:

Here's a tonemapped version using MadVR's default tonemapping:

Zitat von dcsimpson

Ellie AND Aloy !? Truly a man after my own heart. Whatever you're working on sounds incredible. And fascinating find on going ProRes. I was actually going to do that initially when I first started down this rabbit hole, but not for HDR reasons, I had no idea what was awaiting me there, but I usually go ProRes444 on all my videos I shoot and produce outside of this hobby so I thought I would just use that again. But yeah HQ should be fine, you're probably right.

I actually just started an encode of a 2 hour film with your old method right before I checked this board but I'm going to let it run and see what I come up with. My test clips before I committed using your old method still seemed over saturated but I had reached the point where I was willing to live with it as it was still the best method I had seen thus far.

My question now would be when I go to export via voukoder can I simply just export a 10bit UHD file? Or do I need to add any of those parameters that you listed in the VUI section and embed HDR data? How would you export specifically? I don't believe Plex will tonemap unless I embed HDR data back in the file somehow but I could be wrong about that, I will test that tomorrow.

Use the parameters I mention in the first post. For metadata, if you already have that in your source file, you can modify the "Master Display" section to match that. Then go to the section beginning "Now you're ready to export to Voukoder. "

If you don't already have that information, start at the section beginning with "In order to output HDR10 correctly, you will need metadata." I designed a tool for that purpose but it will require exporting several stills from your project. If you want to speed things up, you can focus on areas that are bright scenes overall and areas that have bright highlights. Personally, I like to have a solid number of stills representing every scene. I use 16bit TIFFs for the most accurate measurement.

You can also use NVENC or VP9 or other codecs, but I don't think you will be able to include MaxCLL and MaxFALL in those, although there is a way to add that metadata to an MKV container. I know MPC or at least MadVR in MPC supports that format, but it's not very standard.

As for my projects, I make movie edits of video game stories for my Youtube channel. Working on TLOU2 right now. Been doing HDR since I got my Ninja Inferno and Voukoder started supporting these capabilities last year.

If you don't mind 4444 you can certainly use that if you want. That's profile 4. Profile 5 is 4444 XQ. But if your source comes from a UHD Blu-ray, then those are 4:2:0 meaning 444 is probably unnecessary. I do use that for any post production effects work I include in my projects (I often use VFX to remove HUD elements where possible), as I want those to be as lossless as possible and I also use the alpha channel for that.

Zitat von dcsimpson

Well I can confirm this method works beautifully. My outputs now look very very close to the SDR equivalent and tone map splendidly in Plex. The HDR metadata tool is awesome, thank you for sharing that with us. I would have never figured out how to do this without your help.

There is a slight con however, which in all fairness you did call out. My intermediate ProRes HQ file was enormous. It was over 600 GB. Which in the end resulted in my final output file being double than what is typically created. I'm willing to live with that for my library but surely there is a better way to embed the REC709 info into my intermediate file? Even if it's just compressing the intermediate file with a very low CRF like 10 or something and embedding the 709 parameters. The absolute greatest thing would be to be able to change the color space metadata without the need for re compressing at all. That would be my dream.

Oh yeah, you could use another codec instead of ProRes. I just chose ProRes to maintain quality, and because it's also very easy to work with in Premiere. But you could go low CRF x265 and as long as you include the same setparams filter in the command line it should work fine, so here's an example:

ffmpeg -i input.mp4 -c:v libx265 -crf 10 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709" -c:a copy output.mp4

Note that CRF on x265 is different in HDR than it is with SDR. For example I found that I needed to drop from CRF18 to CRF14 for my HDR to match the quality of my SDR encode of the same content, so you may want to consider that when choosing your CRF for this. If 10 is what you would have chosen for an SDR intermediate, you may want to consider lowering it for HDR content.

Based on our experiences earlier I think it's safe to say that stuff is buried in the HEVC codec itself, so remuxing won't remove it.