OK, I must confess I was one of the first to try the AI upscaling (read: Gigapixel) for one of my projects; at the beginning it was amazing - a lot of small details came out of seemingly nowhere!
Then, slowly, I understood that, apart the obvious details that are literally invented - hence taking the image away from the content creator intent - there were other side effects; it cleaned a lot (if not all) of the grain, and let people appear made of plastic - even worst than "waxy" effect made by too much DNR.
Now, I don't want to detract its merits - used with moderated settings, result may be fair or even good, depending on the source of course... but frankly using it with setup to 11 to upscale a whole movie from HD to UHD is not only useless, but also harmful: the resultant file is 4x bigger, grain is gone, details even if clearer are too clear so that they seem unreal... without mentioning the fact someone may believe it is the "real deal" - take a look around for example at Aliens or AvatarUHD, just to mention two from the first letter of the alphabet, and you will see.
I firmly believe that modern TVs with their own AI upscalers could do the same job, if not better, leaving the source at its original resolution. So, if you really want to upscale video for whatever reason, it's much better - albeit a pain in the... neck - to find out several sources, upscale each with classic methods, and then average the upscaled versions; result will be surely less detailed than the real higher resolution but, if the latter does not exist or is not available, will look much better than any AI upscale - just use that AI eventually for the few frames or shot needed to fill the gap, when needed, on a project.
Just my 2c. 
Note: a research discovered that, when people watched native UHD 4K material on an UHD 4K TV, most of them cannot see the difference with the upscaled version on an UHD 8K TV, and some even prefer the former!
I agree. Most AI upscales are awful, with details smeared away indiscriminately. Then you have a spend a lot more effort to recreate or restore the details. Otherwise you are left with this:
![[Image: 2120.jpg?width=880&quality=85&fit=max&s=...c309f253b7]](https://i.guim.co.uk/img/media/7e3cf6395f32273f249019910f841aeb05721407/0_141_2120_1272/master/2120.jpg?width=880&quality=85&fit=max&s=7c04ef5b36f39692fc6155c309f253b7)
I tried restoring some Blade Runner outtakes a couple years ago and the upscaling was easy. But even spliced into the 2007 VC-1 copy of the movie, the upscaled scenes looked very out of place. I never figured out how to make them look quite right.
I personally have a mostly good experience with AI upscaling using Video Enchance AI from Topaz Labs, but it was far from a simple one pass process. I aimed for a more detailed look and the end result looks oversharpened instead of the typical waxy look of AI upscales. Pick your poison I guess. Adding artificial grain also helped masking some of the issues.
I first started by obtaining four different 1080p encodes of the same movie, one MPEG-2 from the only existing official bluray, one H264 from Disney+, one HEVC from Amazon and one VP9 from Youtube.
After discarding the VP9 version entirely for being so bad, I then upscaled all three remaining sources to 4K using the same settings inside Topaz Labs and exported the results into a sequence of 16bit RGB tiff files. A total of around 72 rendering hours and 12Tb of files.
The use of multiple sources with different encoding issues was enough to give a considerably different result on all three sources.
I then imported all three into Davinci Resolve as a Multi-Cam source, cut it all down shots by shots and chose the best looking version for all shots one by one. In some case I had to take a note of some awful looking shots and went back to Video Enhance AI to upscale these with their own settings.
I also went the extra mile and made my own HDR10+ grading that I then converted into Dolby Vision using an unofficial tool. This was a completely awful process as I do not have the proper equipment or knowledge to do this and the end result even tough I think looks good is not accurate to the original creative intent by any means.
I can share more about the process and possibly screenshots if anyone is interested.
(This post was last modified: 2022-09-10, 01:29 AM by babouin.)
Interesting insight, thanks! I'm very curious to know more about this; please start a new dedicated thread!
So you end up with three viable source, right? Did you try to upscale each with classic upscaling method (lanczos, spline etc.) or a less aggressive AI (NNEDI3, AIUpsale) and then average? If not, I'd give this a try - using a short clip, say 30s, full of small details, so to give you an idea of the possible final result.
I might create a dedicated thread later.
In the meantime, here are four comparisons I made early on before starting actual work on this project. AI upscaling was done using my own preset in Topaz Labs.
https://imgsli.com/MTI1Mjg0
https://imgsli.com/MTI1Mjg1
https://imgsli.com/MTI1Mjg2
https://imgsli.com/MTI1Mjg3
I was never satisfied with the official Blu-ray, many years ago I had done a 1080p version using a simple avisynth sharpen plugin and fake grain. A few months ago I got curious to see if any of the streaming versions were better and was disappointed to see that they're not better and in ways even worse due to the lower bitrate.
I then tried to make my own version using more modern tools like AI upscaling, I don't remember precisely which software and methods I tried but out of the few I did try, Topaz Labs was the only one that was configurable enough to find a proper balance of being worth the upgrade without messing up everything.
I think it looks better in motion without staring at all the imperfections, I could provide a video sample of the HDR10/DolbyVision output to anyone interested. It would have been easier to compare the upscaling alone with a SDR version but I didn't think to create one back then and I no longer have the intermediate files before HDR grading so I can't go back and make one without re-rendering everything.
Well, this is (luckily) an AI upscale example using gentle settings, plus grain plate that, personally, I think boosts a bit the perceived increased resolution.
But there are around quite some bad examples of over the top AI upscaling with all grain scrubbed out, and details over sharpened - where the originals when upscaled using simple old methods are so much better - that frankly I don't see the point of doing that: why someone should get an upscaled version of a movie with four times a given bitrate, when you can get a better result using the original resolution? Because, maybe, people think that 4X mpbs > 1X mbps?!?
Bitrate is not an exact science, at the contrary... but this topic should deserve a thread on its own!
I've made a post about using the PaNup technique with HD sources, and compared result with GigaPixel AI: https://forum.fanres.com/showthread.php?tid=4733
Frankly I like the result obtained with MergeHdUp more than GigaPixel AI - but it's my personal opinion. What do you think?
(2022-09-19, 10:16 PM)spoRv Wrote: I've made a post about using the PaNup technique with HD sources, and compared result with GigaPixel AI: https://forum.fanres.com/showthread.php?tid=4733
Frankly I like the result obtained with MergeHdUp more than GigaPixel AI - but it's my personal opinion. What do you think?
I agree that MergeHdUp gave a better looking result in that comparison. It does indeed looks like MergeHdUp is squeezing details that's there but lost in compression and using multiple sources brings some of it back.
Personally, I think AI upscaling has potential if you're looking to "enchance" the source and creating details where there may never have been. It's not exactly a restoration anymore but more of an upgrade if done properly, which is likely pretty difficult in many cases.
Taking my use case (the movie Dinosaur from Disney) as an example, I believe that the original master (that all HD sources so far are derived from) has issues, it's overall soft but has signs of oversharpening in places. There are also some shots that have signs of being poorly scanned.
Some AI upscaling Video Enhance AI from Topaz Labs are not just upscaling the image but also processes it and try to fix multiples issues. Presets and AI models from GigaPixel specifically might not be trained and tuned to fix issues specific to videos while Video Enhance AI has specific features like fixing common video compression artifacts and reducing the effects of previous sharpening.
What possibly made my experience with AI more positive is that the movie does not have any humans. Upscaling human faces while still looking natural is likely a much more difficult process than upscaling a dinosaur scales. Knowing there were no humans is the only reason I gave AI upscaling a chance on this project. In my opinion AI should probably be used as last resort when no better source exists.
(This post was last modified: 2022-09-22, 03:01 AM by babouin.)
Agree that AI upscale works better without people.
But I was surprised that, using MergeHdUp with three sources, result was dramatically better (IMHO): https://screenshotcomparison.com/comparison/27866
With five sources it should be spectacular! 
I know this is a late reply, but I had a similar experience with gigapixel. Was great at the start, but eventually you do realize its faults. The best thing I did with it was an upscale of the Halloween 2 TV cut from the dvd, that actually looked pretty solid. Other than that, I think it usually looks really meh. I would usually prefer watching the lower quality version than the upscaled one unless maybe it's done professionally.
|
