UI vs UX

Photoshop Generative Fill

Daniel Scott

@dan

I’m sure you’ve heard that Artificial Intelligence (AI) is here to stay and that you can’t stop the tide with your fingers or the wind with wishful thinking. And that’s a good thing. AI has been here for decades, but it just reached a new level.

Personally, I believe that AI – in this context, Generative AI – will help us move forward, from basic rights to Human Rights, to finding different jobs and endless new opportunities. I don’t think we should fear AI, as many didn’t fear previous industrial and technological revolutions. We will adapt and add its strengths to our own talent. I think it will help us work smarter and faster, and connect us to new values and new personal and global priorities.

Fancy speeches aside, this post is all about how we can use AI technology in Photoshop. Come along and follow me into (not entirely) uncharted territory. By the way, all the images in this post were generated with the assistance of Photoshop’s AI. Cool, huh?

Ready to have a look at how these amazing features work? Let’s dive into 5 ways you can use generative AI in Photoshop and start learning all the cool stuff the future is bringing!

Pro tip: Using Generative AI doesn’t mean that having strong skills in Photoshop gets thrown away. Not at all. First, because there are still considerable limitations in AI and the way it reads and meets your prompts. It struggles a lot with animal and human anatomies, a lot of solutions it presents are many times distorted or have some random elements thrown in that don’t make any sense.

A man brings a woman a bouquet of flowers

Bonus challenge! See if you can count all the weird things that AI delivered to this basic prompt: “man brings girlfriend some flowers”.

Generative AI in Photoshop – Hands-on tips

From a basic point of view (I may be writing about advanced techniques in the future), realize that Generative AI in Photoshop stands on two pillars: selection and prompts. So, plan your project, sketch it first if necessary and define your selection in context with what you want to remove, replace, or generate brand new – it will impact what Photoshop’s AI delivers. There’s a lot to say about prompts and prompt engineering, but on a starting level I recommend two things:

Forget verbs. You do not have to say “add” something, “remove” something or “paint” something blue. Type the outcome you need, not the work the AI will have. Select a wall and type “window with curtains” and not “add window over the wall and add some curtains to it”.

Be specific, don’t use too many words. If anything is off you can fine tune it afterwards. As you’ll see, typing only one word, like “bridge”, can be enough to start your edit.

Let’s have a look at some cool Generative AI effects!

Edit 1 - Changing Landscape.

Let’s start with a fictional riverside city image and modify it to make it more spectacular! Remember, this is not about accurate composition or realism, we’re exploring the Generative AI’s potentials.

A photo of an AI generated river in adobe photoshop

An AI-generated image of a riverside city.

I think this river needs a bridge. Let’s make our selection. I’ll use the Marquee Tool, press “M” on my keyboard and click and drag to draw a rectangle at the bottom half of the image.

A photo of a river by a city in adobe photoshop

Generative AI responds to human designed prompts.

I’m telling Photoshop that these are the limits for the bridge. Now all I must do is type “bridge” on the generate field in the Contextual Bar below the image and click on “Generate”.

A photo of a river with a bridge in adobe photoshop

A bridge was generated over the river and there are available variations to choose from.

AI generated a bridge over my image. Let’s have a look at what else we have. As you can see in the Layers Panel, the bridge is a smart object, identified as a Generative Layer with a “stars” icon in the corner of the thumbnail, and has its own layer mask. In the Properties Panel, you can adjust your prompt and find three variations for our bridge. If we press the Generate button without changing the prompt, Photoshop will create three more options for you to choose from.

A photo of a river with a stone bridge in adobe photoshop

A different variant is applied from three new generated ones.

We now have six variants, you can browse through them until you find the perfect one, keep the others or delete them, or generate new alternative bridges. Remember that the more variations you have, the more of your computer resources will be necessary, so keep this as clean as possible. Let’s keep the first bridge and add some more cool things.

I want this city to have a fun appeal, so I’ll ask the City Hall for an amusement park. Using the Lasso tool, we create a selection in the background, once again in context with the element’s expected size and proportions.

A photo of a river with a bridge and an amusement park in adobe photoshop

Proper selection is key for a more accurate result.

All we need to do now is move to the Contextual Bar, type “ferris wheel” and click on “Generate”.

A photo of a river with a ferris wheel in adobe photoshop

A ferris wheel has been generated in the background’s context and has its own layer.

We now have a ferris wheel inviting us to a fun amusement park downtown! A new Layer was created over the one with the bridge and you can check out ferris wheel variations and generate new ones, if needed.

Looking carefully at our composition, I feel we need some life generated into the image. Using the Lasso tool, we’ll draw a small shape under the bridge and tell Photoshop’s AI that we need some ducks swimming there. It’s easy, type “ducks swimming” and hit that “Generate” button.

A photo of a river with a bridge and two ducks in adobe photoshop

Living elements in your composition add emotion and realism to your composition.

Happy ducks swimming in our river, so cool! Same thing, our ducks have their own layer in our Layers panel and we can check out variations in the Properties panel or ask for more ducks.

Let’s go crazy. With the Lasso tool selected, we’ll draw a shape in the sky, type “UFO” and click on Generate. I’m using the Lasso tool for these selections, but you can use whatever selection tool you feel more comfortable with.

A photo of a river with a bridge and a UFO in the sky in adobe photoshop

You can create unexpected elements for extra impact on your viewer. Let your imagination run free.

And we now have a UFO hovering over our city! We could make many more changes to this file, including selecting the houses on the left bank and generating new ones or adding clouds to the sky, you can go ahead and try some more magic. I’ll just add a train over the bridge before we move to the next example.

A photo of a river with a bridge and train in adobe photoshop

Through Generative AI, life and motion were added to a rather static representation of a riverside city.

Timeout #1

Is AI a true revolution that will transform the way we work in UX design or a threat to our jobs and careers? Explore this article to understand how Artificial Intelligence will power up all of the Design Thinking stages and open doors to new perspectives and better outcomes.

Edit 2 - Cast reflections.

We’ve seen we can transform images by adding new elements to them. Let’s do some of that but with a twist.

This is an image of a desert with a pyramid on the left. Let’s have some fun and see how Photoshop’s Generative AI works with what surrounds the generated elements.

A photo of a pyramid in adobe photoshop

AI-generated image of a partially revealed pyramid over a desert landscape.

Now let’s add some water to this arid scenery. I’ll draw a selection close to the pyramid and ask for a refreshing lake. The twist: it’s water, it reflects things.

A photo of a pyramid in adobe photoshop

Define your placement area with the assistance of Photoshop’s selection tools.

So, I’ll type “reflecting lake” as a prompt and click on “Generate” – as a shortcut, you can also press Enter after typing your prompt.

A photo of a pyramid with a body of water in adobe photoshop

Generative AI creates a water surface that reflects the surrounding context.

Done! Photoshop’s Generative AI observed the element’s surroundings and placed them as a reflection in the surface of the lake. It’s an amazing effect that adds extra realism to the outcome. Love this feature, has enormous potential, and it’s also great when you are expanding some images, but we will get to that a few paragraphs down.

Edit 3 – Remove and replace things.

For this example, we’re having a look at an ordinary street from anyone’s hometown and we want to eliminate some distractions and add a surprising effect. Let’s see how we can do it.

A photo of a two lane road in adobe photoshop

AI-generated street view.

Let’s remove that waste bin and that bench from the picture. We want the Remove Tool selected, an AI innovation in Photoshop. We can adjust the brush size in the toolbar on top or with the shortcut Alt + right mouse button + drag left or right, and all we must do is paint the object we want to remove.

A photo of a two lane road with a trash can circled in adobe photoshop

The Remove tool helps you to “paint” out undesired elements from a composition.

Next, we remove the bench...

A photo of a two lane road with a bench circled in adobe photoshop

Paint each element to remove it from the image.

...and that annoying lamp post up the...

A photo of a two lane road with a lamp post circled in adobe photoshop

Photoshop reads the background and removes the selected element.

It’s so easy! I think we can remove these white lines as well.

A photo of a two lane road with a white line circled in adobe photoshop

Accurate selections will assist the generated outcome.

We have eliminated all the distractions and now we are free to summon some AI superpowers to generate awesome stuff like this!

A photo of a flowing river on a road in adobe photoshop

The street view is now transformed and the viewer’s attention is now fully focused on the action.

Timeout #2

Thinking about AI, and how it will change societies, must include a wide debate on ethics and a conscious observation of all the opportunities and risks that will rise. The United Nations have been promoting these discussions and trying to influence governments to act responsibly and protect their citizens.

Edit 4 - Replace model’s clothes.

Let’s do some fashion work. You’ll find the model a bit quirky, but I kept it that way to underline what I’ve explained before – AI still struggles with human and animal anatomies, but the goal in this example is to show you how to replace things in an image.

A photo of a man in a white t-shirt in adobe photoshop

Artificial Intelligence still struggles with human anatomy and expressions.

Let’s change his t-shirt! That white color feels boring! We can use the Quick Selection tool, click and drag over the t-shirt and when our selection is complete type “striped t-shirt” and hit Enter! Bang!

A photo of a man and his t-shirt is circled in adobe photoshop

Generative AI has transformed the model’s t-shirt in seconds.

A brand-new shirt, instantly generated, no effort at all! Let’s take it to the next level and change the whole outfit! We do another quick selection, making sure all the clothes are selected.

A photo of a man in a striped t-shirt in adobe photoshop

Combine Selection Tools and prompts to instruct Photoshop’s AI.

Now let’s type in our Contextual Bar “leather jacket” and hit “Generate” or the Enter button.

A photo of a man in a black leather jacket in adobe photoshop

New outfit generated by Generative AI, following our selection and prompt instructions.

We can also add that extra sexy touch with a cool “leather hat” and...

A photo of a man in a black leather jacket and black hat  in adobe photoshop

Though with some visible imperfections, it is clear to see Generative AI’s potential.

Yes, it looks weird, doesn’t it…? Fun, though! Moving on!

Edit 5 - Start from Zero.

This one is special because we are starting with a clean canvas, we’ll expand its limits and add some fun stuff. Let’s go for one last lap!

Let’s open a new artboard with nothing on it, just a white background layer, use the Marquee tool to select the whole artboard and type “beach landscape” as our prompt and gently smash that Enter key.

After Photoshop generates our new beach – don’t forget to check those variations – we are ready to expand its limits.

A photo of a beach in adobe photoshop

AI-generated beach view that needs to be expanded to fit required dimensions.

Let’s try out the Generative Expand feature and break this image’s horizontal limits. To do that, we must pick the Crop tool from the left toolbar or hit “C” on our keyboard. Then we look up at the Properties bar and on “Fill” we click the option “Generative Expand”.

A photo of a beach with grid lines over it in adobe photoshop

The Crop tool’s Generative Expand option will create extra landscape for your design.

Next, we expand our image to the left and to the right and Photoshop will fill in the blank spaces with some extra landscape. As we’ve seen with the reflecting lake, Photoshop reads the original image, considers its context, and limits and does the amazing! You don’t even need to type any further instructions. Hit Enter and wait.

A photo of a beach in adobe photoshop

Landscape has been expanded with new features.

Presto! This is so great! Yes, I know that Content-Aware Fill used to do something like this, but trust me, Generative Fill opens a whole new dimension of possibilities. This is Multiverse material!

I’ll just use what we have learned in these examples to add some finishing touches to our beach project, such as removing part of the sand’s vegetation, adding a cruise ship close to the horizon and a beach bar by the sea to welcome the tourists, some chairs and… well, you get the picture, have a great vacation!

A photo of a beach with lounge chairs and a cruise ship in the background

Go from desert island to tourism heaven with just a few clicks and prompts.

Who owns AI Generated Content?

There are huge questions about copyrights when it comes to AI Machine Learning and AI Generated Contents. Countries all over the world, with the help of legal and tech teams, are trying to legislate on a phenomena that “exploded” too fast and in too many directions at the same time.

Today, as I write this post, there are debates, even lawsuits, around author rights, from the contents used to train AI models, to the ones “translated” into a brand new artworks, and down to the final art itself.

You can read more on how this is being studied and legislated in the US and in Europe. My advice to you, when you are working with AI generated content, make sure to always check each tool’s commercial use terms and conditions - they are being updated all the time and it is safer to always make sure you are acting according to the law and respect for copyrights.

What’s Next?

Jump now to Bring Your Own Laptop and become an active member in our natural intelligence community! Learn more about the skills you need to tackle the new AI challenges and expand your own limits with our 30+ courses and dedicated support for just $7 a month with an annual plan. Of course, cancel anytime (and come back anytime, too)

See you in class! - Dan