Regional Prompt Control in Krita
I've seen the excitement about MOSS's regional prompting and so I'm here to point out that Krita also does this (and also allows you to use whatever model you want). Here is a guide to how I typically use it locally (it can also be used via cloud-based servers but I'm not sure how):
Follow the guide on their website to set up Krita AI if you haven't already. It's 100% free, it just requires downloading a few models that you might not actually want to use. It also requires an instance of ComfyUI to be running in the background.
Set up your style from the gear icon in the AI docker under the styles tab. This lets you create and save sets of model, LoRAs, generic positive and negative prompts, sampler, CFG, and steps. Also under the "interface" tab ensure that "Apply behavior" is set to "New Layer on Top" and "Layer Group". These settings are optional, but your results will differ from this guide if they aren't set. Feel free to mess with them later.
Open an image. I will be showing an inpainting process, but you can also use it to generate new images, just turn the denoise up to 100%.
The first step is to mask out areas that you want to specify prompts for. You can use any method, but I find the following to be easiest. First, create a new paint layer with the '+' icon or Ins key:
Select the brush tool and paint over the area. You can use any color, the mask is opacity-based.
Continue creating layers until you have everything you want covered. I like to fill the whole image when possible because why not, but you don't need to. Anything you don't mask will be generated as a separate background layer. Here I have turned down the opacity on the layers to check overlap between regions. This does not affect the opacity of the mask, only the actual brush settings will do that. Regions are allowed to overlap, just keep in mind that the stacking order will determine what you can see.
If you're not inpainting, skip this step. Right-click each layer and choose "Select Opaque (Replace)". Copy the region from the original image and paste it as its own layer, then delete all the mask layers.
Assign prompts to each layer. With the layer selected, click the "T+" icon on the AI docker (if you don't see this for some reason, it can be turned back on from Settings > Dockers > AI Image Generation)
A new prompt box will pop up. You can see that it's linked to the layer by the blue chain icon. Here I have the full face layer selected. Since I'm putting the eyes in a separate layer I am not adding anything regarding the eyes here.
Take a moment to ensure your layer order is correct. I noticed here that I have the eyes and the jewel below the face, so they will be invisible!
As you're probably aware if you've done much inpainting before, using minimalist prompts splits the model's attention less, which often leads to a better result. Since Krita will generate each layer in sequence, using this method allows the model to really lock in on each region.Add a general prompt in the bottom prompt box. This will apply to every layer including any areas that you didn't mask. Also add the negative prompt here (this can't be unique for each layer unfortunately). Make sure you're not repeating the prompts that are in your style description already.
Verify your generation settings. I'm inpainting using Euler A, so I'm going for 30% denoise. If you're generating from scratch, set to 100%. Also the secret sauce: if your rig can handle it crank up the resolution to 1.5x. This will generate each layer at 1.5x the size before scaling them back down. It makes a huge difference in detail, and it's fairly manageable performance-wise if you keep the layers reasonably-sized.
When you hit generate, Krita will create a new layer group for each prompt region. Here I tried generating at 20% and 30% denoise, so I have 3 images in each group:If you ever get annoyed by having so many layers, you can right click in this frame and select "New layer from visible" to merge them all into one. I2I generations will take whatever is visible as input, but keep in mind that new generations will still get put in their bins. You can change how new images are delivered in the settings.
Here is the result. On the left is a regular I2I generation at 30% and 1.5x. On the right is the generation following the method shown above:
Just for illustration purposes, I tried generating an image from scratch as well. I prompted "An outdoor scene on a sunny day with some grass" along with two regions "A rock" and "A cloud". The bad news is I got them backwards. The good news is that it proves the regions worked!
One thing I like to do with this is to mix and match:The top is using the 30% denoise eyes region, the bottom is 20%. You will notice a seam on the top image. When Krita generates using regions, it keeps the entire image in context and dynamically blends the regions together. So when I slap the 30% region on top of the rest of the image at 20%, it isn't continuous.
One way to fix this is to right-click the layer, select opaque, and then generate it again at low denoise. When you generate with a selection active, Krita will only generate the regions that border and/or overlap it. The effect of this is that the bordering regions will each contain a little "blending strip" that removes the seam.I like to get a little silly with the details. So here I've scaled up the image 2x. This is also a function you can access from the paintbrush icon at the top of the AI docker. I usually use a "bland" sampler like dpm++ 2m (changed in the style settings) for this. Also be sure to activate the "Use Prompt" radio button, which will keep your prompt regions active during the tiled upscale to prevent any shenanigans from the model.
You may have noticed that the jewel region that I originally had went missing partway through. It turns out that there is a minimum size for regions and it just ignored that region when I generated. But now it's much bigger! So I will use the selection feature I mentioned above. Notice that this is not a prompt region that I defined, but each prompt region will generate the part that overlaps it. Also, I'm still generating at 1.5x resolution! I have to be careful how big my selections get now, but I'm still able to generate just this jewel at around 600x600 pixels.
I like the result here, but the generation has burned the colors too much for Krita's color matching to compensate (notice the border where my selection was). Lucky for me, I'm in a full-featured image editor. All I have to do is slide this layer under the original, erase the part I want to replace, and merge the layers!
I mentioned in my article a year or so ago that I still prefer ComfyUI over Krita's generator for its power and versatility, but at this point I mainly generate in Krita and only use Comfy for very customized functionality. This only scratches the surface of the possibilities with this tool.