Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
ColorMatch 3D (new colormatching tool)
#1
Just something I was working on:

https://github.com/TomArrow/ColorMatch3D/releases

[Image: 3j0PMJPF_t.png]

Give it an input and output image. Hit the red button, wait, then click the export button.

Benefits over DrDre's colormatch:
- Blazing fast. Depending on image size and your CPU, it can be less than a second
- Should be able to deal with more complex changes in theory due to a completely different algorithm

Downsides over DrDre:
- Doesn't generalize well. It will only match the colors in the image and no others, so applying the LUT to any other image will result in garbage. It might not even generalize well across an entire scene, so its use currently is to match single frames and maybe very similar frames.

Todo for the future:
- Try to integrate some interpolation option to make the 3D LUTS generalize better.
- Create an AVISynth plugin

The algorithm is childishly simple: Create an RGB cube in the size of the output LUT (32x32x32 atm). Take each color in the source image and create a vector to the color in the output image. Average all these vectors and place them in the RGB cube. Export RGB cube as LUT. Profit.

As you can see, the algorithm explains both the downsides and upsides. If I manage to do some interpolation (or maybe extrapolation is a better word), it would probably slow it down quite a bit, but maybe it could be made optional and controlled somehow, aka you could choose how much interpolation is to be done. We'll see.

Important note: Most of the options in the tool don't actually do anything, so don't bother with them. They are stuff I wanted to implement or might implement some day. The only option that really does anything currently is the "Aggregation Variable" being set to Absolute or Vector. Absolute is the old algorithm I had for months but it wasn't precise enough. Instead of averaging vectors, it was averaging absolute values, which gave decent results but in some situations the results were disappointing. The Vector algorithm is default now and solves that issue.

Edit: Since this is open source, feel free to contribute if you want to ofc.
Reply
#2
Nice work! This could be very useful for inserting replacement frames to fill gaps and so forth.
Reply
Thanks given by: TomArrow
#3
Did a little test involving a low-quality 70mm cell frames scan(?) of The Abyss and thought to post the results from Dr. Dre and TomArrow's Color Matchers with the same inputs at 250.

70mm & Cinemax Inputs
[Image: Abyss70-Comp-A.png] [Image: Abyss70-Comp-B.png]
Cinemax
[Image: vlcsnap-2020-11-15-02h06m44s230.png] [Image: vlcsnap-2020-11-15-02h07m04s394.png]
Regrade (Dr. Dre)
[Image: 70-Test-1-7-1.png] [Image: 70-Test-1-8-1.png]
Regrade (Tom Arrow)
[Image: Abyss70-Tom-Test-1-13-1.png] [Image: Abyss70-Tom-Test-1-14-1.png]
Although TomArrow's tool seems to get more appealing / accurate results in these two frames than Dr. Dre's, I notice banding in some areas (namely the helmets).
Don't have anything else to add really other than it's a nice starting point for something bigger @TomArrow, and I hope o see this tool built upon in the feature (whether it be by you or the community).
[Image: ivwz24G.jpg]
Reply
Thanks given by: TomArrow
#4
@LucasGodzilla Ah yes indeed. Those artifacts I encountered before. I think that to get rid of them, I might have to do the interpolation I was talking about. Currently the LUT has a lot of "black spaces" in it at the edges of the used colors, those artifacts are probably when the tool you are using tries to interpolate using those "not set" values that it just sees as black (which technically they are).

Thanks for doing a test!

Edit: About the banding ... might have to do with those artifacts too or might just be a limit of the source? Worst case could add some noise before application of the LUT.

Edit 2: Did you actually do the matching with those little images? That might be the reason it didn't work properly since the downscaling ditches some of the colors necessary to form the LUT. I wonder what results you would get if you do it at full resolution. Should be fast enough to do that.
Reply
Thanks given by: LucasGodzilla
#5
So, I might be an idiot, but I can't get the program to do anything at all. It calculates a histogram, but the images doesn't change at all. Am I doing something wrong?
There are the images I've used:
https://mega.nz/file/4NlTXAyC#V1eB2g_zR-...egi2y6Ak9o
https://mega.nz/file/RElD2CQD#_6JPPYtnPv..._sbZ2APeR0
Reply
Thanks given by: TomArrow
#6
(2020-11-20, 06:46 PM)Cactus Wrote: So, I might be an idiot, but I can't get the program to do anything at all. It calculates a histogram, but the images doesn't change at all. Am I doing something wrong?
There are the images I've used:
https://mega.nz/file/4NlTXAyC#V1eB2g_zR-...egi2y6Ak9o
https://mega.nz/file/RElD2CQD#_6JPPYtnPv..._sbZ2APeR0

Pardon for not being clearer. You need to export the 3D LUT and apply it in a suitable program. It doesn't show a preview.
Reply
Thanks given by:
#7
@Cactus

I tried your images. First off, the way you made them would likely not work because they weren't properly aligned.

Here's the aligned images:
[Image: hHE0ZM40_t.png] [Image: Y4DFacxp_t.png]

First, here's the result I get: Not very good. Looks weird as hell:

[Image: cXZSLmfr_t.png]

My explanation for why the match is this bad is that the second image has different areas graded differently, aka some areas are darker/brighter compared to the first image. Which means that there cannot be a perfect 3D LUT. In cases like this DrDre might give better results I think. However I did a little experiment and tried to equalize the brightness of the first image to match the second:

[Image: HD9w6ywv_t.png]

While the result is still far from good (I'm not sure something like this could even be done perfectly), it looks a lot more natural now, tho still not as good as you'd hope:

 [Image: hEweqwR8_t.png]

In short, this method isn't really very suited to matching grades that have had power windows used in them, as it will end up averaging things that don't belong together and such, since it takes everything very "literal".
Reply
Thanks given by: pipefan413 , Cactus
#8
Thank you, TomArrow. I've been working on an extended version of Danny the Dog/Unleashed but came to a standstill a while ago. Not only because of some vastly different grading, but with the American version also having some artificial camera movement added. I shall continue my quest. Smile
Reply
Thanks given by: TomArrow
#9
New version out: https://github.com/TomArrow/ColorMatch3D...s/tag/v0.2

Added a simple interpolation algorithm. It's listed as two separate ones, but it's really just one with slight differences. Single Linear and Dual Linear. Single Linear will give acceptable results with the Vector mode. Dual Linear is pretty useless so far, I suppose the idea behind it wasn't that great.

It probably won't make a big difference when it comes to a single image, but it definitely generalizes a bit better now. In the past you would get big black patches where the image had colors that weren't in the image used for matching. Now you get at least ... something. The further away the color is from the image used for matching, the worse the result gets. But I think it might just be good enough for matching a whole scene now, tho I'm sure there are exceptions to that.

As for speed... in Debug mode it was pretty slow (about 6 seconds per image) but in the Release version (the one uploaded) it actually only takes about 2 seconds per image (depends on size of the image too), so I'd say it's okay. And it's optional, so you can decide whether to use the interpolation. There's definitely a lot of room for optimization I think but until the algorithm performs really well, I figure it's best to postpone that.

Edit: Secret tip, the Single Linear + Vector mode, combined with LUT smoothing in 3D lut creator can give pretty pretty okay results already. Maybe I'll add some kind of smoothing too to the interpolated parts.
Reply
Thanks given by: LucasGodzilla , alleycat
#10
New version.

https://github.com/TomArrow/ColorMatch3D/releases/

Skipped the one in between. Last version added the option to work in CIELab instead of the sRGB of the images that are to be matched, the new function is a brightness equalization similar to what I did in the post above, but now you can just click on the option and have it done, whilst setting a blur radius for the equalization. It gives rather smooth results, but a downside is that the contrast of the reference isn't necessarily respected too well. I'll have to come up with some clever idea to further improve the results.

Smoothing the result is not implemented yet. Sadly I am right now fighting with constant BSOD issues on my PC whenever I work with Visual Studio, so it might take a while until I feel like touching it again. So far, enjoy.

Result I get with 15px equalization radius of the image above:
[Image: t2jTUbQd_t.jpg]

As you can see, the colors are alright, but the contrast is more or less ignored. Will need to come up with something clever to fix that.
Reply
Thanks given by:


Possibly Related Threads…
Thread Author Replies Views Last Post
  [Help] Tool: Taking an Image and Searching Every Frame within a Video for that Image? alexp120 4 384 2021-06-01, 12:40 PM
Last Post: alexp120
  DrDre Colormatching Problems PDB 1 502 2020-05-10, 06:00 PM
Last Post: PDB
  Searching for a tool CaveDoctor 3 1,324 2019-01-16, 02:04 AM
Last Post: spoRv
  Tool to undo improper image stabilization? trondmm 6 3,095 2018-03-10, 10:02 AM
Last Post: TomArrow

Forum Jump:


Users browsing this thread: 1 Guest(s)