Future of selections in Photoshop with Adobe Sneaks

Hi everyone, this is a weird video, where we watch a video together. We're going to sit down side by side and watch YouTube. Why am I forcing you to do this? You don't have to, skip along, but what I wanted to show you is kind of the future of Photoshop. Just so that you're in the know, I love being in the know. And it's all about masking out things out of images. It's something called 'Deep Fill', which is something Photoshop is developing in the background. It's not in the product yet but will be in the future. Just gives you kind of a secret back stage pass of what they're doing. They announced that at last year's Adobe MAX conference, which is their big conference they have every year, at something called the Sneaks, which is in the last days. It's my most favorite part of the whole conference. They just kind of show you stuff that's going on in the background. Not available yet but will be, and it's pretty amazing. So let's watch it together now, you can skip ahead to the next video, or later on just go and check out Project 'Deep Fill', otherwise sit back, relax, I've skipped the intro. Let's just jump in.

Please welcome Jiahui Yu. Thank you. Hi, I'm Jiahui, and today I'll introduce some image hole filling technologies. As you all know that Photoshop already have, a very powerful image hole filling tool called Content Aware Fill, which can be used to remove distracting objects or undesired regions in an image to make it nicer. It works well in most cases but in my case here is quite complicated. Here, I have a saved file So I want to remove this thing here because it's annoying me. So I'm masking it in here. And let's see what Content Aware Fill can give us. Well! Now I have one, two, three, four, I've got four eyes now. I'm just going to talk over this, and annoying everyone. 

The mere reason for this failure is because the Content Aware Fill does not try to understand the image. And it's only relying on copying the surrounding areas, the surrounding pixels into the hole. We believe a good image hole filling system should be able to understand the face, and fill the nose with the nose, not the eyes, or mouth, or something else. So to bridge this gap and solve this very challenging image hole filling problems we introduce Project 'Deep Fill'. We leverage the power of Adobe Sensei, deep learning and develop that 'Deep Fill' that can understand the image. Here, let's see how it performs. 

I press that 'Deep Fill' button, and yes it can. Cool, huh? What if we master the entire eyebrow? Can 'Deep Fill' return us a new one? Well, let's try it. Here I masked the eyebrow, well, let's see Content Aware Fill first. Well, this time it copies the mouth into the location of the nose. And let's see the 'Deep Fill'. Well, it can successfully hallucinate a new eyebrow for you. We know that 'Deep Fill' works on faces, but in most cases, people travel around the world, and take photos, and find some people you want to remove. For that project, in my case-- here, I take a photo in Perth Canyon National Park. Well, it's wonderful, beautiful weather, but I find two people on the top. So what I'm going to do is, I take the photo-- what he's trying to explain in the background, is that Adobe Sensei is their kind of machine learning artificial intelligence. Watch what happens. Watch it disappear, was this the right one? This image is part of high resolution. It's a bad version.

 
What Adobe Sensei is doing is, it's looking at other people's images that they've found online. That's crazy, it goes out and says, you've taken this photo of a popular spot. I'm going to go off and see if I can find it-- Adobe Sensei is going to go off and see if they can find it online, and grab data from there and put it into your image without you asking. Crazy. Wait for it. This one looks visually realistic, but it's not semantically correct because it's not an arc anymore. So with our 'Deep Fill', by the way, we can mask multiple regions, and hallucinate it in one shot. Let's see it. And yes, this is the Deep Fill. Cool, huh! One more thing, given that Deep Fill users have no control over, what 'Deep Fill' will feel in this master region. Of course, we can provide multiple solutions for users to choose. 

Another thing to mention is that, if you are like me, kind of cross over into the video world, they're looking to do this exact same thing but in live action. So, like people's videos that have gone online can be used, to mask out live actions of people walking in front of your video of the Eiffel Tower, can be masked down, because there are enough videos online. Enough data to kind of re-represent that. It's the best I can take into now. So, using 'Deep Fill', let's see how it forms. And yes. This is the AI-powered user guided image whole filling technology. Project 'Deep Fill'. Sense. That is awesome. 

They just have a bit of a chat afterwards. So yes, it's just looking at data that exist online, and tries to use your image, Photoshop dives into the internet, and says, well here's some images that are the same, or we think are the same, and starts using that to put it together, rather than what's just in the image. They call it Adobe Sensei as their kind of like, background robot learning, machine learning Artificial Intelligence stuff. You'll see more and more of this into Adobe. Content Aware Fill is pretty amazing by itself, but when we get into this kind of, like reaching out, out of Photoshop and out of the image, things are going to get pretty cool. 

Also know, this is Adobe MAX, this is in Vegas last year. It was my first time, it was so good. If you are going to this year's one in LA, drop me a line. What we'll do is, if any students are going, and we want to hang out, we'll just grab a beer one night and we'll see if we can get a few students together. And we'll just have a little chat, doesn't have to be anything too special, but if you are going to Adobe MAX, let me know. That is enough YouTube watching together. Let's jump into the next group of videos. haere rā.