[In progress] Felidae 35mmized AI upscale

[TomArrow]

Some may know this movie is one of my favorites of all time.

After years of searching and waiting, I managed to get hold of a 35mm trailer scan of a Felidae trailer.

I then took the same trailer from the DVD (frame-perfect match), cropped the 35mm scan to fit the DVD trailer and set up these two sources for AI training. This is a trick I learned from someone else some time ago, thanks to that person!

In the end, I want to use the resulting AI model to upscale the entire movie. Now, the trailer is actually letterboxed where the DVD itself is anamorphic, but I figured I could give it a try anyway, and so far the results seem promising.

This is what I got after something around ~24 hours of training (good results can take weeks though!):


DVD on the left, AI upscale on the right. The low contrast of the output has to do with how I set up the training, pay no attention to that.

Here's a direct comparison between Bicubic upscaling and the AI upscaling with slightly adjusted contrast:
http://www.framecompare.com/image-compar...n/7GGWWNNX

Already I think it's very promising. You can see it seems to have the most issues with emulating the 35mm grain, the imitated grain comes out way too uniform and synthetic looking. However this was actually much worse at around 5 hours of training and it's been improving ever since, so I'm hopeful that it will eventually improve to the point of being fully acceptable.

If that doesn't happen, maybe because the DVD just doesn't have enough information to properly reconstruct any reasonable semblance of grain, I will try adjusting the training set by temporally degraining the 35mm scan for the upscaling. That should almost perfectly preserve the image content (as opposed to spatial degraining) while getting rid of the grain, meaning the AI will no longer have to emulate the grain and could just deliver a clean picture. I could then try and add the grain with some other method later.

Anyway, I'll keep this training going for at least a few more days and see how it improves.

[spoRv]

Sadly the pupils of the cat on the right are almost lost in the AI upscale...
Would you mind to try with Gigapixel, to see the difference?

[TomArrow]

Sadly the pupils of the cat on the right are almost lost in the AI upscale...
Would you mind to try with Gigapixel, to see the difference?

It's a very small area of the image, I don't think it will be noticeable.

I've tried Gigapixel and Video Enhance and Waifu2x, Photoshop Preserve Detail 2.0 and others. It was all more or less trash sadly, at least in my opinion. It just wasn't optimized/trained for this specific kind of animation. That acquaintance who taught me this even tried with models he did for some other cartoon upscales, but they were mostly for digitally animated stuff, so they didn't work properly on classical animation and as a result also produced trash.

Topaz Video Enhance was probably by far the best out of those I've tried, but still not exactly satisfying.

Feel free to try yourself: https://mega.nz/file/LswkSKBS#vPaoFbX5lY...T3gQj8Naso

[spoRv]

Still think that the old approach of clean&sharp is still good with animation:
https://i.postimg.cc/9VzzMTHy/seq-000017...pscale.png

[TomArrow]

Still think that the old approach of clean&sharp is still good with animation:
https://i.postimg.cc/9VzzMTHy/seq-000017...pscale.png

Nah, doesn't look good to me.   I love this movie. It's gotta look good.

Side note: After a few more hours of training, and judging on my past experience, I've decided to train against a degrained 35mm scan and then to regrain with something afterwards. The current approach likely won't cut it, since it probably won't be able to generate really pretty grain for some reason. Also, totally dark areas will be left scrubbed of grain, which looks unnatural.

[spoRv]

I've decided to train against a degrained 35mm scan and then to regrain with something afterwards.

I guess it's a good decision!

[TomArrow]

Was a good call:


Much better now.

Suppose a 35mm trailer scan had not been not available.  Could you 1) downscale the DVD and then 2) use the downscaled DVD versus the full-res DVD for training?

[TomArrow]

Suppose a 35mm trailer scan had not been not available.  Could you 1) downscale the DVD and then 2) use the downscaled DVD versus the full-res DVD for training?

I doubt it but you could try I guess.

[TomArrow]

Good news, around 80% of the frames have been processed/upscaled and I have also found a way to replicate the original grain rather accurately. Stay tuned.

Edit: All frames upscaled.