HDR Testing

[Pauven]

I'm trying to figure out why HDR doesn't work consistently on my projector setup. Some discs look incredible, some look washed out, some won't display an image at all, and the results vary by player and by player config.

To both catalog and share my results, I'm taking sample pictures with my Pixel 6 XL, during the daytime in a 95% light controlled room - so there is some ambient/uncontrolled light affecting the image a bit (some loss of shadow detail).

I'm using the 2023 Spears & Munsil Ultra HD Benchmark set to begin this evaluation. These first images are from the demo disc, MPLS 0022 (which is what MPC-BE plays by default).

I checked in PowerDVD, and this is the menu item linked to MPLS 0022 - "Dolby Vision 10,000 BT.2020"

0_Menu.jpg (115.94 KiB) Viewed 5285 times

Note that somehow the menu is smart and reflects that I don't have Dolby Vision nor HDR10+ nor SL-HDR2 (whatever that is), and that my playback will use standard HDR10, so I assume that to mean my playback will use HDR10 10,000 BT.2020. To keep my tests consistent between MPC and PowerDVD, I played this same MPLS 0022 in both apps.

Note that my projector, an Optoma UHZ50, only supports HDR10 and HLG. Regarding it's performance, here's an excerpt from All Home Cinema's review:

The Optoma supports HDR10 and HLG. The color range comes in at 77% P3, which is rather on the small side, but the projector makes good use of what it has. It does, however, ignore metadata and perform tone mapping up to over 3,000 nits. The brightest shades are therefore somewhat dark. The projector retains pretty good white detail, and certainly has enough light to show off bright scenes. The contrast was very good in HDR. The images will never have the real impact of HDR, the color range and dynamic range are too small for that, but the result is quite good.

Also note that my projector, as calibrated & configured, probably maxes out at 2700 lumens or less, which equates to about 790 nits or less. HDR10 600 is about the best this projector can achieve.


Windows HDR Off, MPC-BE default Enhanced Renderer

01_Win_HDR_OFF.jpg (130.9 KiB) Viewed 5285 times

While the chameleon looks amazingly detailed, the coloring comes across as almost black and white, with subtle hints of yellow/brown and moss green.

Windows HDR On, MPC-BE default Enhanced Renderer

2_Win_HDR_ON_MPC_Def_Rnd.jpg (125.25 KiB) Viewed 5285 times

With Windows HDR now enabled, the image looks slightly more washed out, the chameleon seems ghostly.

Windows HDR On, MPC-BE MPC Video Renderer set to Passthrough HDR to display

3_Win_HDR_ON_MPC_Vid_Rnd_Passthrough.jpg (141.04 KiB) Viewed 5285 times

Now this looks amazing. Colors are vibrant, tone mapping is great. Camera struggles to do this justice, looks better in person. Very 3-dimensional, looks like it could walk off the screen.

The info overlay is now working with the MPC Video Renderer, so here's a closeup of the text. Note the HDR Processing indicates Passthrough, 10,000 nits, as expected.

3_Info.jpg (181.09 KiB) Viewed 5285 times

Windows HDR On, MPC-BE MPC Video Renderer set to convert HDR to SDR

4_Win_HDR_ON_MPC_Vid_Rnd_HDR2SDR.jpg (1.18 MiB) Viewed 5285 times

While easily the 2nd best MPC image of these tests, this SDR converted image lacks the color accuracy and vibrancy of the true HDR image. In person, the difference is larger than this photo captured. I think if I was quizzed, I could have guessed that these colors were mapped into the SDR range, they look a bit artificial, unnatural while remaining punchy, like the color saturation was cranked too high.

Windows HDR On, MPC-BE MPC Video Renderer set to handle HDR

5_Win_HDR_ON_MPC_Vid_Rnd_Handle HDR.jpg (129.53 KiB) Viewed 5285 times

Very similar to the MPC Default Renderer. The info text now reads that HDR Processing is "Not used"... hmmm.

Windows HDR Off, PowerDVD

6_Win_HDR_OFF_PDVD.jpg (141.8 KiB) Viewed 5285 times

PowerDVD does admirably here, showing a nice image with Windows HDR turned off. It certainly did better than MPC's default Enhanced Renderer in the same situation.

Windows HDR On, PowerDVD

7_Win_HDR_ON_PDVD.jpg (149.95 KiB) Viewed 5285 times

With Windows HDR enable, PowerDVD now looks fantastic. To my eyes, this seems nearly identical to MPC's MPC Video Renderer set to HDR Passthrough. I'm not sure I could tell them apart or pick a winner. I'll post test image 3 again below to make it easier to compare MPC's best side-by-side:

3_Win_HDR_ON_MPC_Vid_Rnd_Passthrough.jpg (141.04 KiB) Viewed 5285 times

Yep, it's amazing that two different players can produce an image so nearly identical. Though this actually matches my expectations that the image should accurately represent what's encoded on the disc, and the player should not be artificially manipulating the results. In this case, a 4K image that is being viewed on a 4K screen, with HDR likely being offloaded to the display for tone mapping, the two players produce the same result as I would expect.

Note that if you do perceive a difference between these 2 photos, it's likely a result of my Pixel 6 XL's camera/processing, and not something you would see in person.


Here's the above test again but using a different sample image:

Windows HDR Off, MPC-BE default Enhanced Renderer

1_Squirrel.jpg (121.49 KiB) Viewed 5285 times

Decent image quality, looks like a standard Blu-ray but with more sharpness. Less saturated and contrasty than with HDR.

Windows HDR On, MPC-BE default Enhanced Renderer

2_Squirrel.jpg (114.2 KiB) Viewed 5285 times

Turning on Windows HDR, MPC's default renderer is very muted and loses contrast.

Windows HDR On, MPC-BE MPC Video Renderer set to Passthrough HDR to display

3_Squirrel.jpg (154.61 KiB) Viewed 5285 times

Just like the chameleon above, this is the best image quality combo.

Windows HDR On, MPC-BE MPC Video Renderer set to convert HDR to SDR

4_Squirrel.jpg (147.2 KiB) Viewed 5285 times

For some reason, my camera struggled to take this picture without flickering, and the flickering made it into the picture as color banding. It doesn't look this way in person. But the colors are definitely off, they have a weird magenta cast to them, saturation is artificially high, and the tone mapping is poor.

Windows HDR On, MPC-BE MPC Video Renderer set to handle HDR

5_Squirrel.jpg (127.84 KiB) Viewed 5285 times

Same color banding photo issue again. Magenta color cast still present, plus colors are more washed out, tone mapping worse, definite lack of contrast.

Windows HDR Off, PowerDVD

6_Squirrel.jpg (144.1 KiB) Viewed 5285 times

Once again, PowerDVD does admirably here, showing a nice image with Windows HDR turned off. Just like before, it certainly did better than MPC's default Enhanced Renderer in the same situation.

Windows HDR On, PowerDVD

7_Squirrel.jpg (162.2 KiB) Viewed 5285 times

Again, a virtual tie with MPC Video Renderer using HDR Passthrough (showing below again for better side-by-side comp).

3_Squirrel.jpg (154.61 KiB) Viewed 5285 times

Extra Test - HDR10 600 nits
As one last test for this round, I played the HDR10 600 BT.2020 and HDR10 10,000 BT.2020 clips. Interestingly, the HDR10 600 nit clip has an asterisk denoting it is derived from the Dolby Vision clip, while the HDR10 10k nit clip is not. Regardless of what clip I played, they all look identical to me, same as the Dolby Vision clip used for all of the tests above. Here's a couple photos of the 600 nit result:

8_PDVD_HDR10_600nits_Clip.jpg (157.45 KiB) Viewed 5285 times

8_Squirrel.jpg (152.89 KiB) Viewed 5285 times

TEST 1 CONCLUSIONS
These results with the Spears & Munsil demo disc do NOT match my issues with Guardians of the Galaxy Vol. 3, which looked so bad it prompted my exploration of this issue. In GotGv3, the disc wouldn't display at all using the MPC Video Renderer, and looked pretty bad with the default Enhanced Renderer. So I switched over to PowerDVD. At times, the image looked good, but often it appeared washed out, similar to some of the washed out examples above.

But in today's results, PowerDVD always looked good. It did a good job of showing HDR material with HDR disabled in Windows, so I assume this is falling back to standard Blu-ray quality but at 4K. PowerDVD certainly faired better here than MPC.

And once HDR was enabled in Windows, MPC's alternate MPC Video Render (which only shows a black screen on GotGv3) looks virtually identical to PowerDVD, and both look amazing. I really couldn't pick a winner between the two, and wouldn't complain about either. Contrast was amazing and colors were fully saturated while remaining natural, not artificially pumped.

This was not the experience I encountered with GotGv3. Again, I wonder why all discs don't perform the same way, playing HDR content effortlessly.

This also makes me think that PowerDVD is doing HDR passthrough, since the results match so closely to MPC in that mode, and also because the HDR10 10,000 nit clip looked the same as the HDR10 600 nit clip. If passthrough is being used, then it makes sense that the projector is simply mapping things down to it's own colorspace/range capabilities. I believe my projector's capabilities are in the <800 nit range, so that would also explain why the 600 nit clip and 10k nit clip look virtually the same, since everything is being tone mapped down to around 600 nits of range.

So if PowerDVD worked so well in these tests, why did it fail in GotGv3? To be fair, I've only ever seen this movie once, and only at home with PowerDVD, so it might be that I'm mistaking the director's intent for what I perceive as a washed out image.

My next round of tests will be with that GotGv3. I'll play both the UHD 4K with HDR and the standard Blu-ray disc. I'll also play it on Disney+ streaming, which I believe will let me do both standard and HDR. So hopefully I will determine what the film is supposed to look like, and see if HDR is really acting up.

[Pauven]

Since I was on a roll, I decided to go ahead and test the Guardians of the Galaxy vol 3 discs and streaming. For the discs, I tested all in PowerDVD.

First, here's the Blu-ray menu:

GotGv3_BR_Menu.jpg (183.98 KiB) Viewed 5281 times

And here's the 4K UHD menu in PowerDVD with HDR enabled:

GotGv3_4K_Menu.jpg (191.65 KiB) Viewed 5281 times

Instantly you can tell somethings not right in HDR land. Gamora is minty green. The shadows under I am Groot's arm are blue instead of black. The red in the background nebula lack saturation and punch, Mantis' leather pants look whitewashed. Yes, it's subtle, but keep in mind the camera fixes some of these issues, so it's hard to capture in photos what looks like a white haze to your naked eye.


I probably didn't pick the best sample image, so I may need to do this all again. Here's a frame from around 2:09:15, when Quill is falling back into the exploding ship.


Disney+ no-HDR

GotGv3_Disney_No_HDR.jpg (143.27 KiB) Viewed 5281 times

Disney+ with HDR Streaming enabled in Windows (but not sure if it's actually doing anything...)

GotGv3_Disney_HDR.jpg (127.04 KiB) Viewed 5281 times

Blu-Ray

GotGv3_BR.jpg (141.06 KiB) Viewed 5281 times

4K HDR

GotGv3_4K_HDR.jpg (147.74 KiB) Viewed 5281 times

4K with Windows HDR Disabled

GotGv3_4K_NO_HDR.jpg (134.22 KiB) Viewed 5281 times

My Thoughts
Of the 5 images, the Disney+ streaming without HDR seems obviously the best. The blacks are actually black, there's plenty of contrast, the lights are bright, and the image has a nice 3-dimensional effect. I won't go so far as to say this is a great image, only that it's the best of this bunch.

The Disney+ streaming with HDR enabled in Windows (including the Streaming option) appears worse, slightly washed out. I don't think this is working right. I'm not sure if the Disney+ app properly enabled HDR, as this looks more like regular content with improper gamma applied on an HDR display. I've never had a good streaming HDR experience, and this is no different, so just consider this an interesting observation but nothing to be deduced from it.

The Blu-ray image seems just generally washed out a bit, like the black level is set too high. But in other scenes I was struck by how much more vibrant it appeared versus the 4K HDR image. So I think I chose a poor sample image here, as generally I was liking the Blu-ray better.

The 4K HDR image here is also atypical of what I normally had issues with. Instead of looking like a sheet of wax paper was laid on top, this image looks overly dark, lacking dynamicism - the lights really aren't that bright, and all-in-all it reads muddy. It seems easy to lose Quill in the backdrop.

Here's a couple crops of Quill, from almost the same frame but perhaps a few frames off. Neither is impressive. Quill's suit is darker, but there's no detail in those shadows, in fact it seems like less detail.

Quill_BR.jpg (44.71 KiB) Viewed 5281 times

Quill_4K_HDR.jpg (39.21 KiB) Viewed 5281 times

Turning off HDR made the 4K image horribly blown out. Unlike with the S&M demo disc, PowerDVD didn't do a good job of reverting it to a standard Blu-ray appearance.

I need to watch the movie again and find better examples of the HDR issues, but the above is the best I can do for the moment.

[Pauven]

I began to wonder if the 4K UHD issue of Guardians of the Galaxy Volume 3 was widely considered lackluster, perhaps it was a poorly mastered release, so I did some searching and found a review by High-Def Digest (Review Link) in which they proclaim:

Guardians of the Galaxy Volume 3 has an exquisite 2160p 4K UHD image with HDR10 enhancements. This is a beautiful-looking movie. Guardians has the best and most colorful palette of all the MCU properties. This third installment allows those fun planets and spaceships to thrive through boasting colors.

That first sequence in Nowhere gives off those rusty amber hues with some fun neon signage in the background that perfectly contacts with Krgalin's pinkish-red arrow. The Gold paint on Warlock is a deep gold that also gives a nice reflective metallic look. On other planets, the blue skies and green plants all look impressive that mix well with the purple outfits, and Gamora's green skin. Inside the living, breathing spaceship, the orange and yellows pop brightly that lead into a perfect white balance of walls and floors where red droplets are flawlessly blended in. The Black levels are inky and rich in those darker corridors and the bleakness of outer space. The climactic fight scene showcases a rainbow of colors through alien bodily fluids, gun blasts, and costumes. This film is such a wonder to view within its rich and colorful palette that everyone should be satisfied.
...
The skin tones are fantastic throughout too. The HDR upgrades the color palette quite a bit in both bright and dark sequences which is noticeable from the 1080p counterpart, particularly during the heaven sequence for Rocket and during the final battle. This is a demo-worthy image presentation with no issues.

The emphasis is mine, illustrating that my main takeaway is that any video quality issues I'm having are on my end.

In that review they gave a few sample images from the film, and this is the image used to represent the excellent HDR quality:

image.png (374.59 KiB) Viewed 5267 times

It is an impressive image, going from "inky blacks" to bright whites, with rich reds and purples and an interesting slate blue tile floor. Chukwudi Iwuji's skin tone is also well exposed and natural. They review didn't provide the timestamp, so I had to spend some time to find the right spot. I found it at around 27:27. Here's a screenshot taken on my office PC from the Disney+ webpage in Firefox:

HDD_Disney_1080p.jpg (572.24 KiB) Viewed 5267 times

My office PC really isn't set up for viewing movies, and I'm not sure what visual enhancements (or restrictions) Disney imposes when their content is viewed through Firefox, and I don't have HDR capabilities on this rig, so it's not surprising that this isn't a stellar screenshot. I believe this is from the IMAX version of the film, evidenced by the taller framing. The dynamic range in my screenshot is absolutely crushed, everything is darker and looks underexposed. Obviously, this is not an HDR image, but it does look unusually muted compared to the HDR sample above.

Ironically, the image quality looked fine while I was viewing the film on my PC, it was only when put the screenshot side-by-side with High-Def Digest's HDR sample that it lost its appeal.


For the following photos I switched to using my Canon DSLR, which allows me to use manual exposure settings and get a consistent record of what my eyes are seeing. I started off with the Blu-ray, and tweaked the camera settings so that the captured image closely matched what I'm seeing first-hand.

HDD_BR.jpg (288.33 KiB) Viewed 5267 times

When I view this photo on my office PC's monitors, I see a strong magenta color cast that is not present in my projector's image. I think it is because my office monitor is not color calibrated. My projector, being well calibrated, has very neutral colors - the tile floor appears very gray, not blue-gray, the silver cabinet doors appear very naturally metallic with no color cast, and the reds look more candy apple vs. burgundy in the photo. Perhaps the image looks correct on your monitor, so take this with a grain of salt.

I think my Canon DSLR is also enhancing the color saturation a bit. I tried cranking up the ISO to 800 to give the image sensor less exposure time, and this helped reduce the saturation some, but the camera is still using a fairly long exposure and capturing more light than my eyes, which makes the purple jacket look incredibly saturated. In real-life, what my eyes see is very close to the 2nd image, the screenshot from Disney's website in the Firefox browser.

The main point here is that this photo may not represent how well my projector is calibrated, and that this photo is really best used to compare and contrast against the following photos taken with the same camera, same lens, and same exposure settings.

My first-hand impression of this Blu-ray image is that it isn't super sharp (obviously 1080p being upscaled in PowerDVD), and I sense a heavy dose of artificial film grain, likely to help meld the CGI environment with the actors. The exposure and colors seem natural if not overwhelming, perhaps a touch on the dark side. The left and right backdrops show very little detail, particularly the right side that merely hints that there's a structure there. For the most part, there's nothing for me to complain about here, it's a nice Blu-ray presentation, and at best I would say the gamma could be raised a notch or two.

I then enabled Windows HDR with Streaming support, and loaded up the same scene in the Disney+ app:

HDD_DisneyApp_HDR.jpg (293.79 KiB) Viewed 5267 times

To me, this is obviously worse than the Blu-ray. The dynamic range seems reduced - the blacks are lighter and the whites are darker, colors are muted, and the resolution seems dramatically reduced. If this is the best that streaming can do, the looming death of physical media is a sad future indeed.

Here's the two spliced together, BR on the left:

HDD_BR&DisneyAppHDR_Overlay.jpg (280.21 KiB) Viewed 5267 times

Honestly, the difference between the two is lost a bit in this spliced image, but it's most apparent in Chukwudi's skin on his face and hand, but look closer and you can discern the purple of his suit is less rich, and the red tiles in the backdrop are hazy.

Next I loaded up the 4K UHD version with Windows HDR enabled. Some time has passed, and more post noon sunlight is sneaking into my theater, which may be partially responsible for the grayer blacks in this next photo. That said, I did not find the HDR image impressive, in fact it seems worse than the Blu-ray to my eyes. The hint of paneling in the back right that I noted in the Blu-ray is nearly absent here. Yes, the darks seem darker (in spite of the sunlight's disastrous effects), the candy apple red richness of the cabinet is now muted, and overall the image seems more like the Disney+ streaming version.

HDD_4K_HDR.jpg (282.49 KiB) Viewed 5267 times

Here is the the BR (left) and 4K (right) images spliced together.

HDD_BR_4KHDR_Overlay.jpg (275.41 KiB) Viewed 5267 times

While I know that my projector is far from capable of displaying pristine HDR, it seems to me that the 4K HDR content should at least look better than the bog standard Blu-ray. To me, the Blu-ray looks better. Here is a side-by-side of the High-Def Digest, Blu-ray, and 4K UHD HDR images:

HDD_BR_4K_TripleSideBySide.jpg (169.42 KiB) Viewed 5267 times

This 4K HDR image on the right looks slightly more muted and hazy compared to the Blu-ray in the middle, and is a far cry from the High-Def Digest sample image on the left.

I know, I know... it's just bad tone mapping, right? But I have a hard time accepting this since the Spears & Munsil HDR content absolutely blows me away. How can the same setup do so well on one disc do so poorly on another?

[Pauven]

As a follow up to my PowerDVD testing, I've installed a trial of JRiver Media Center 31.0.56 (which includes the new auto-close on complete option that will be incorporated into CMC v7).

This first image is bone stock JRMC, using the JRVR renderer.

HDD_JRMC_Stock.jpg (276.82 KiB) Viewed 5267 times

Next I enabled the HDR passthrough option:

HDD_JRMC_HDR_OUT.jpg (285.31 KiB) Viewed 5267 times

Next I switched to the madVR renderer with HDR passthrough:

HDD_JRMC_madVR_stock.jpg (269.91 KiB) Viewed 5267 times

Then I enabled Tone Mapping via Pixel Shaders. I had to play around with the settings for a while to even notice a difference. I discovered that setting the target peak nits to 100 (about as low as it goes) brightened up the image:

HDD_JRMC_madVR_ToneMapPixelShaders.jpg (265.1 KiB) Viewed 5267 times

Turns out that my camera adjusted the exposure back down, as this photo looks the same as the others, but is noticeably brighter in person. I checked the exposure settings, and this exposure was 0.5 seconds, while the previous photo was 0.6 seconds. Basically, this last tone mapped image was about 18% under exposed by my camera, making it look the same as the others.

Which I guess means that my test methodology has failed. I thought I had my camera on full manual, but it's been adjusting exposure the entire time, which is masking some of my results.

I will say that this tone mapped madVR image has caught my attention. It's far from perfect, but it's also the first improvement I've seen in the HDR image.