A new era for logo removal and Open Matte?

[TomArrow]

Check this out: https://www.youtube.com/watch?v=86QU7_SF16Q

[spoRv]

.
.
.

WOW!!!

.
.

[Hitcher]

Maybe it's the skeptic in me but some of that seems too good to be true.

How does it know what's on the right of this shot before it appears like that wooden frame thing?

You can do this right now, just not with the same degree of AI interpretation. After Effects borrowed content aware fill from photoshop a year or more ago. I’ve already used it on two open matte projects to remove jump in/jump out logos that lasted for 3 or 4 minutes. The key is to add a lot of key frames and if it doesn’t work the first time, try again. Every time AE’s content aware fills takes a slightly different approach. Not the same thing as this but easy to do now.

https://youtu.be/25ltIoHtiO4

[spoRv]

Maybe it's the skeptic in me but some of that seems too good to be true.

How does it know what's on the right of this shot before it appears like that wooden frame thing?

I just guess:
it can't predict beforehand, so it must analyze all the shot first; then would be "trivial" to put content on left and right - even if it's not possible to do it perfectly at the beginning of a shot, as there is nothing to "steal" from; for top and bottom without pans is not possible as well.

Still, I guess that with an UAR version with four empty corners should be relatively easy to fill them. I don't know how much time will take, though (time=money, as IIUC it is a cloud solution).

[Hitcher]

Maybe it's the skeptic in me but some of that seems too good to be true.

How does it know what's on the right of this shot before it appears like that wooden frame thing?

I just guess:
it can't predict beforehand, so it must analyze all the shot first; then would be "trivial" to put content on left and right - even if it's not possible to do it perfectly at the beginning of a shot, as there is nothing to "steal" from; for top and bottom without pans is not possible as well.

Still, I guess that with an UAR version with four empty corners should be relatively easy to fill them. I don't know how much time will take, though (time=money, as IIUC it is a cloud solution).

Yeah, that makes sense. Thanks.

[TomArrow]

My guess is it works by analyzing first the video to see what can be gotten from that, and to then use inpainting for the rest, with the added difficulty of temporal coherence, which is the big impressive thing about this new one. In the video he compares it to older methods which produce a lot of artifacts. But AI these days is so damn smart it really can create not only something like Content-Aware fill but even logically continue images, it's pretty incredible. That same Youtube-channel made a video about that a few weeks ago.

[zoidberg]

This AI would probably work well with 3D conversion WRT in-painting, might even be able to generate decent depth maps