First i recommend reading the Part 1. It explains the extension and settings. Remember to leave some ⭐(～￣▽￣)～
I have planned to expand more on multidiffusion tutorials:
Workflow on multidiffusion + controlnet tiling
Maybe inpaint workflow tutorial
If you have something you would like to see tutorial on let me know in the discussion.
Please leave feedback and images you manage to create with the tutorial :)
⬇Updates and information below ⬇
Updates:
05/07
V1.0 of Region prompt control
Tutorial focused on Region prompt control.
05/01
V1.0 of Multidiffusion IMG2IMG Workflow
Tutorial more focused on scaling in IMG2IMG
04/29
V1.2 of Multidiffusion upscaler how to use + workflow
Clarified few things in the tutorial
04/25
V1.1 of Multidiffusion upscaler how to use + workflow
Fixed some typos, uncompressed images, wording
04/24
V1.0 of Multidiffusion upscaler how to use + workflow
multidiffusion upscaler for automatic1111
Disclaimer
This is information i have gathered experimenting with the extension. I might get something wrong and if you spot something wrong with guide, please leave comment. Any feedback is welcome.
I am not native English speaker and write such text. I can't do anything about that. :)
I am not the creator of this extension and i am not in any way related to them. They can be found from Gitgub . Please show some love for them if you have time :).
Description
V 1.2 Clarified few things in the tutorial
V. 1.1 Fixed some typos, removed image compression, clarified few things
V. 1.0 New better tutorial
FAQ
Comments (112)
Thank you very much, very interesting tutorial ! Is it actually an equivalent of the ultimate image upscaler that was before available as a side script? Or is it more advanced?
I am not aware of ultimate image upscaler, so i cannot say for sure. But unless it was tiling images i would say this is more advanced. And this does much more than upscale. Will continue updating the guide so please follow for more if you are interested :)
Yes, it is based on tiling too: see here https://github.com/Coyote-A/ultimate-upscale-for-automatic1111 in case you are interested in comparing with multidiffusion
@michoko Ah i see, thanks for the information. I'll see if i find time to fiddle around with it :)
@michoko Was checking the example images and it looks like it enhances and sharpens the images pretty well, but does not add details to the image. This is from generation i did for the preview images:
text2image with tiled diffusion enabled: https://i.imgur.com/OVNR2MQ.png and after that:
image2image scaling with tiled diffusion: https://i.imgur.com/guUf7Ip.jpg
text2image without tiled diffusion: https://i.imgur.com/iti67bu.png
It gives totally different image with different details compared if you use tiled diffusion or not.
I recommend checking this out if you liked ultimate image upscaler :)
@potatovision842 the img2img upscale seems to go overboard with details, especially with those veins in her neck. The colours are more punchy but that’s quite a big jump in contrast, was there anything that caused the added contrast?
My initial impression was Ultimate upscales via tiling, and Tiled Diffusion adds more details by generating in the tiles. But Ultimate seems to do the same too with the right settings. Tiled Diffusion has the region prompt control feature. Any idea how it works?
@jenxi I am not 100% sure what causes the jump in contrast. But my guess would be some prompts might have something to do with it, VAE could be one of the reasons too. I read somewhere that using exactly same seed causes this too. Should test out and figure what is exactly causing this.
With the region control you can specify area in your image you want the prompt to affect. It is nice and effective tool. Idea is somewhat similar to https://github.com/opparco/stable-diffusion-webui-two-shot
It is pretty easy to use. Press "create txt2img canvas" after that enable some regions and it gives GUI to move and resize the regions. Planning on doing tutorial on this one too when i got time. You can find some general information from https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111#regional-prompt-control about this.
Thanks for comment.
@jenxi Hey, updated the tutorial with region prompt control :)
@potatovision842 Thanks!
Very informative, good job.
Although, I am having a lot of trouble with creating an image that is larger than 768 x 1152..
And it seems like I can't go beyond image larger than 1280 x 1280 ish, even with "Overwrite image size" option on.
(SD simply refuse to start generating due to lack of VRAM)
So at this point, I think I will be stuck with classic Upscalers.. =/
try using tiled VAE on default settings, and set image size with the hires fix settings.
If your GPU has atleast 8gb vram then it sounds like you dont have xformers installed, xformers drastically reduces your vram usage in generating and upscaling
@JBro How much VRAM you have? . Like @AkatsukiSakiya said if your GPU supports xformers, it is really good idea to use it. It speeds up renders and reduces the vram usage. IF you have Pascal, Turing, Ampere, Lovelace or Hopper card, you should be able to enable it just by writing --xformers after the "set commandline_args=" in the webui-user.bat.
Also if you use hires. fix with the tiled diffusion, it could be the cause also.
Make sure you have the tiled VAE enabled. And if you still get the error with tiled vae enabled, reduce the tile size for the VAE.
I just tested 1792 x 1792 generate and i was using 5.6gb of vram during generate and when tiled VAE hit it went to 11.7gb.
By reducing the tiled VAE settings:
Encoder Tile Size: 320
Decoder Tile Size: 80
It used 8.4gb.
Thank you for comment. Let me know if you need more help.
may i ask why that's always show the "[Tiled Diffusion] ignore tiling when there's only 1 tile :)"? how to fix it? thx!
I had the same issue. I went through script and it looks like for some reason when it checks if image can be divided into tiles it divides image size by opt_f variable which is set to 8. As workaround i replaced opt_f in files with "1" and it seems to be working now.
@crabbLJ Reducing Latent tile width or Latent tile height will fix this issue. Basically it means you have too few tiles for the image size.
@nimloth639 Interesting. Gotta check this out. I wonder what is the reason to have limitation with how few tiles you can use for the generate. Thanks.
Thanks for comments!
@potatovision842 in my case image was 512x512 and latent tile was 96x96. It was still saying it's just 1 tile.
As far as I was able to check it it's because this in utils.py:
def splitable(w:int, h:int, tile_w:int, tile_h:int, overlap:int=16) -> bool:
w, h = w // opt_f, h // opt_f
Automatic1111 has opt_f set to 8, so even if the real size was 512x512 this divided size by 8 and assumed image is 64x64.
i have another question, what's meaning about the "[Tiled VAE]:input size is tiny and unnecessary to tile"?
It means the image size you are generating is small and the VAE tiling is not necessary. It usually kicks in with the upscaling or when the generate is large resolution.
Thanks for the comment!
it works great on img2img but I have a lot of troubles using it on txt2img.. even using the vanilla 96x96/32 I get a mutants. I tried using diff resolutions: 512x512, 768x768 etc and variations 512x768, 768x1024 etc.. and I get mixed results, not really sure how to adjust the tile wid/hei ratio accordingly, thank you for sharing it, I will be experimenting
to follow up on this.. I think it has to do with the checkpoint. Basically, a well trained checkpoint is flexible enough to handle this script flawlessly, but a weird merge will have issues..
Hey,
Usually i go with along the lines of width and height divided by 8.5~. Using different numbers for width and height sometimes help. Then little over half of what i get from there for tile overlap
For example
824 x 824 image i could go with something like width 96, height 112 and overlap of 52.
Thanks for comment, let me know if you have more problems with it :)
thank you for the tutorial,but i can't see the Tiled Noise Inversion in my webui,how can i fix it?
Hey, i think i answered you in the github page but here it is again :P
It is found under "Tiled Diffusion" and only in "img2img".
Thanks for the comment! :)
@potatovision842 thank you!!!
@Zieten Please leave review if you liked the tutorial 😎
Thanks for the tutorial. For some reason, how to use it and what it does clicked in my head much better reading your tutorial than the original documentation for the extension.
Hey, Tried to make the tutorial in mind about what i needed when i started with it :) Was a lot of trial and error to get it work every time. Glad you liked!
Thanks for the comment and please post some pictures if you have time for that 😎
@potatovision842 I got the setup working, but I still have to fiddle the settings so my 8 year old graphics card doesn't die halfway through. I'll try to have another go today or tomorrow and if I get something decent I'll post it here for you.
@lordbaktor Biggest impact for VRAM should come from the tiled VAE, enable it and lower the tile sizes :) Disable fast encoder as it changes colors.
If your GPU supports xformers enable it too. It reduces VRAM usage.
@potatovision842 I have no idea if my GPU supports xformers, but I have the flag in the launch file just in case. I'll try reducing the tiles on the VAE section and see. Thanks for being helpful.
@lordbaktor No problem, let me know if you run into some issues :) Does it say in the bottom of the UI "xformers: 0.0.17.dev464" or something similar, if it's not enabled it should not say version number. I think the xformers is supported by Pascal, Turing, Ampere, Lovelace or Hopper cards. Read this information somewhere, can't say for sure how accurate it is.
@Potatovision Had to stay away from SD for a couple weeks, but I got the whole Multidiffusion thing working now. I've posted an image that I made following your tutorial.
@lordbaktor Nice :) Glad you got it working.
This is getting longer than i expected 🤔
Thank you very much! I managed to improve an img. I uploaded the comparison on imgsli with the key MTY5MDQ4. (civitai won't let me put the link). https://imgsli.com/MTY5MDQ4
(edit: it seems I can put the link)
Very nice 😎 The difference is huge! Really nice picture. Please post the result below and leave review if you have time for it 🙂
very neat, multi diffusion region control is kind of like Latent Couple, and just as janky! It does work, but it seems like a struggle lol.
I managed to get two different characters in it and rendering properly, but I had to clean it up in inpaint manually since I can't figure out how to let it make a large high quality image out the gate without screeching at me
That said, Latent Coupling also barely, barely, barely works so this is like, at least 12% upgrade over that
Thank you very much for the guide, it's super helpful! I can definitely see how useful multi diffusion can be as a tool
Yeah, it is definitely one of my favorite extensions for sure.
Agree on the janky part. It needs some tuning to get really nice results. I think where it shines is when you use it to add props to your main prompt.
Latent couple is really fun and strong extension too. Have you tried using the paint fork? It's pretty nice upgrade and a bit easier to manage areas :) Don't install it on top of the other latent couple. Create it as an another extension and disable the other if you decide to try it.
Thanks for the comment, remember to leave some shiny stars :)
@potatovision842 I actually tried leaving a review below twice with an image I created following your guide, but it doesn't seem to let me post it, weird
@PettankoPaizuri Sometimes the tag thing seems to bug and need to re entry it.
I rewrote some stuff to take off some of my opinions that might give wrong information. And now there is actually somewhat good PDF instead of that crude pile of land waste 😏
Thank you so much for this awesome Guide, it helped me a lot improving Image Quality!
Glad to hear :)
Thanks for the excellent guide. I always have the problem that more figures appear to me.. I've tried everything, any suggestions on the parameters?
Hey,
There are several things that can cause this.
1. It is usually issue with too small Latent tile width, Latent tile height or/and latent tile overlap. I usually use thumb rule of dividing the width/height with 8.5-10 for the tiles. And around half of width/height tile for overlap. This works for me most of the time.
example: 512 x 712, i would use 64 width, 80 height, 36/40 overlap. Usually rising overlap fixes small issues like many hands etc..
2. Your prompt might be also adding them, if you have for example "girl" several times in the prompt, it wants to add it several times. for example "cute girl, girl with hoodie, girl with jeans" -> "cute girl, hoodie, jeans"
3. Adding (solo:1.2) or stronger can help.
Thanks for the comment, please leave review if you have time :)
is your tutorial only for Nvidia cards? Didn't work for my AMD gpu
Hey. Sorry i personally have Nvidia card and can't say how well things work for AMD. A lot of stuff seems to rely on CUDA.
Thanks for the tutorial. I have some questions come with this upscaler. When I use different character lora at 2 regions, it usually comes out weird and blurry pictures although I set a very high resolution, how to fix it? Another question is that if I use this upscaler, does it means I can draw higher resolution image than usual? Any difference between generating image of 2048*2048 directly and image of 1024*1024 with a 2 times highfix?
1. If you mean the region prompt control. It does not use tiling when it is active, and it makes it more blurry than what it would be without region prompt control. If you want to use region generating with tiling i recommend checking extension called: https://github.com/opparco/stable-diffusion-webui-two-shot . Seems to work ok with tiled diffusion.
2. As it tiles the images, you can make larger scale images. Remember to have VAE tiling on, or this will run you out of VRAM when you make large images. Usually i scale images in img2img with tiled diffusion. I do not know how big you can go with Hires. fix. Tiled diffusion does not use upscaler in text2image, so you should either upscale it with hires. fix or in img2img or both ;)
Where in A1111 I can find regional prompts?
You need the extension for it. After you have installed it you can find it under "tiled diffusion"
Attempted to add a couple images to the gallery here, but CivitAI's resource-detector-thinger basically prevented it from showing up here. Very frustrating.
https://civitai.com/posts/155271
Anyhow, looked into the Regional Control thing, which is frankly nifty as heck, although it definitely suffers from "Works when it wants to" syndrome. Gotta be very finicky with feathering, tile size/overlap, region size, etc.
Seems to also work best at low CFG values.
I think my biggest Sad is that there is little, if any, ability for the subjects in different regions to interact with each other, which is what I was REALLY hoping for. However, this is still giving me MUCH better results for distant shots, especially outdoors (that second image, with the dude in the bedroom, fought like hell to not make a lick of sense. The uploaded image was the first one that even attempted sanity). That said, I'd presumably get much better results if I made each individual component of the room a separate region, rather than just saying "bedroom, GO!"
Still, informative tutorial for a less than intuitive setup (just due to the sheer number of OPTIONS available - the userfriendliness is an unfortunate sacrifice for power, and I think I'd rather have the power).
Thanks for the writeup!
Hey, thank you for the comment.
Currently it seems to be impossible to crosspost without model attached to it. I have reported it as a bug. :)
Personally i am not big fan of the region control as it is extremely hard to get results that are satisfying. I like to use two-shot fork that lets you paint the areas and it seems to work a lot better https://github.com/ashen-sensored/stable-diffusion-webui-two-shot have been planning on adding this to the tutorial also . It feels like the multidiffusion one does not use tiling for the regions which results in not so great images.
I have been experimenting more hi-res fix + multidiffusion and it has given pretty good results. Planning on adding more info on the tutorial about the findings, hit a follow/like for the update to pop up in your feed :) Going to update when i find time for it.
The extension has really nice potential and after you get over the finding right settings it will show really nice results. If you have any questions let me know :)
Cheers
@elahrai Hey, seems like the crosspost function is working now, also updated the tutorial if you have interest in reading it :)
@Potatovision Awesome, thank you! Actually already halfway through the new edition :D I'll give the xpost thing a try afterwards.
@elahrai Very nice 😎 What were your thoughs on the new tutorial if i may ask?
@Potatovision Looked pretty good! Was a good combination of brief and informative, with lots of informative screenshots showing what you mean. Good writeup!
Were you looking for suggestions for any future volumes, my vote would be expanding the regional prompts section. What's there isn't bad, but it could afford more details and/or more examples just due to how bloody confusing that feature can be (or, at least, it was confusing for me lol).
@elahrai I was thinking it is probably good idea to make separate tutorials on different things. Instead of just one big lump of tutorial. The main point was to be as short, simple and easy to follow as possible. Glad it succeeded to some extend at least :)
I could make deep dive on the regional prompts next. Thanks for the suggestion and comments!
my issue i keep get the same face on all the figures in the image. how to get different face for each? what prompt or whatever will work?
Hey, it could be issue with the model, you could try to change hair, eyes color, try emotion like smile etc.. If you are talking about inpainting, then the issue could be too low Denoising strength.
@Potatovision i mean this...u can see the settings i use and model and prompts in meta data. it doenst matter the model i still get look a like faces
images2.imgbox.com/f1/71/BZdQeOUu_o.jpg
@emotionaldreams4 Hey, can't see any settings from the image, maybe the site you uploaded it in removes metadata. You could try to specify each girl. Like "Girl in the left has x, girl in the middle has x, girl in the right has x. I do not know if this actually works. Can you post your prompt on this post, so i can check.
@Potatovision (3girls, multiple girls:1.3) a photo of skaijackson wearing white evening gown standing together with [(ariana) wearing red evening gown and (zendaya) wearing yellow mini skirt], (laughing), (facing viewer), movie stars, modelshoot style, (extremely detailed CG unity 8k wallpaper), Intricate, High Detail, Sharp focus, dramatic, photorealistic painting art by midjourney and greg rutkowski , (waist chain), ((movie premiere gala)), ((standing on the red carpet)), ((paparazzi in the background)), (looking at viewer), (detailed pupils:1.2), dimly lit
Negative prompt: (((kids))), (((child))), (((children))), (((loli))), canvas frame, cartoon, 3d, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)),((close up)),((b&w)), blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render, (knees), (full body), (((nsfw)))
Steps: 35, Sampler: Euler a, CFG scale: 7, Seed: 2471623824, Face restoration: CodeFormer, Size: 512x768, Model hash: 61b359f3fe, Model: NSFW-Style_consistentFactor_v30, ENSD: 31337
@emotionaldreams4 Hey, it could be face restoration. And i would recommend using textual inversion negatives instead of long list of negative. Those can be causing issues too if they are strong. Are those "skaijackson" "ariana" "zendaya" LORA's or special commands in the model? Could be that it takes the first and inserts the face from it to all of them.
Did fast generate, and to my eye the faces seem different https://i.imgur.com/VJklzJU.jpg
(3girls, multiple girls:1.3) a photo of skaijackson wearing white evening gown standing together with [(ariana) wearing red evening gown and (zendaya) wearing yellow mini skirt], (laughing), (facing viewer), movie stars, modelshoot style, (extremely detailed CG unity 8k wallpaper), Intricate, High Detail, Sharp focus, dramatic, photorealistic painting art by midjourney and greg rutkowski , (waist chain), ((movie premiere gala)), ((standing on the red carpet)), ((paparazzi in the background)), (looking at viewer), (detailed pupils:1.2), dimly lit
Negative prompt: (low quality, worst quality:1.5), [Unspeakable-Horrors-16v:Unspeakable-Horrors-24v:0.5],
Steps: 50, Sampler: Euler a, CFG scale: 13, Seed: 774200380, Size: 512x712, Model hash: e2f6e0733a, Model: coffeemix_v20, Denoising strength: 0.45, Clip skip: 2, Tiled Diffusion: "{'Method': 'MultiDiffusion', 'Latent tile width': 64, 'Latent tile height': 80, 'Overlap': 56, 'Tile batch size': 8}", Hires upscale: 2, Hires upscaler: 4x-UltraSharp, Dynamic thresholding enabled: True, Mimic scale: 7, Threshold percentile: 100, Mimic mode: Half Cosine Up, Mimic scale minimum: 4, CFG mode: Half Cosine Up, CFG scale minimum: 4
Saved: 00425--774200380-coffeemix_v20-Euler a.jpg
@Potatovision "skaijackson" "ariana" "zendaya" are texture inversions
@emotionaldreams4 Those could be causing the issue, try without them and see if the problem still persists
tried ur suggestion. but still same faces..ugh
@Potatovision ..what u mean try without them? i wanted to those 3 in the same image(or something similar). they work fine by themselves though
@emotionaldreams4 Using many Loras ,hypernetworks or embeddings don't always work well with eachother. For example you have 2 persons and use 2 different character loras. Sometimes it only takes character from one of the loras and adds them to both persons.
You ser, are a scholar and a fine gentleman!
Thank you for kind words your excellency 😊
You are using multidiffusion incorrectly. It's suppsoed to be used to make large upscales and add detail. You only use 1.1 scale on a high res image, which defeats the purpose. You can skip the high res step and use img2img on the first image and use scale 2 or more.
Hey, thank you for your input. It does say in the tutorial along the lines of "higher scale give more detail". Yes maybe i should have used higher scale on the img2img to make it more clear. In my opinion tiling is the most amazing thing about this extension, it does add detail alongside with the upscaling. In my opinion this extension is more than just a upscaler.
There is different ways to use the upscaling, for example t2t -> img2img 1.3x ->img2img 1.3x upscaling can give more details with right settings than just simply doing 2x upscale.
The main idea of this tutorial was to give general idea of what things do.
Thanks for feedback, i will clarify this in the tutorial when i have more time.
@adry Hey, i did update on the tutorial and clarified it more clearly that hires. fix is not necessary and you can just jump in to img2img and do 2x upscale for example ;)
Thanks again.
hopefully someone will do video tutorial :/ too much for my brain T_T
:P
Hey thanks for the tutorial. i have been trying some of your prompts but i cant find the Tiled Diffusion upscaler and Tiled Diffusion scale factor settings in the Tiled Diffusion section. I installed the extension from: ( https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111 ) that i think is the right repo but these settings just wont show up. Can you guide me what i am doing wrong? and how can i fix it?
Hey, upscaler and scaling is only available in img2img. There is button in text2image under the generated image called "send to img2img", with that button you can send the image to img2img and scale it there :)
Let me know if you need more help!
@Potatovision ah I see. Can you guide me to how i can get detailed/clear eyes in a full body shot image? No matter what I try I always get blurry eyes. I tried the upscaling methods from various youtube videos but they never work. What would be the best workflow for it? I see people upload full body images with detailed eyes but I just cant figure it out.
Thanks for the quick reply mate <3
@gtama979728 a bit busy at the moment, i will give reply in 10-15 min
@Potatovision Sure no problem. I am testing the inpainting method you mentioned in the guide right now. Hoping the it works.
@gtama979728 The eyes can be a bit tricky sometimes, and also depends a lot on model you are using. Some can do faces from far and some are bad at it. FP32 models are better at it.
Adding things like (detailed eyes:1.1), (extremely detailed blue iris:1.2) helps. Detailed, intricate detail, highly detailed, extremely detailed, realistic. Pretty much any quality tag is going to enhance them in some way. This depends on model how it reacts.
Higher the resolution more detail, this is pretty obvious, and multidiffusion helps with this a lot, as you can bring the resolution up with tiling.
Upscale, using upscaler like 4x-Ultrasharp that i mentioned in the tutorial is pretty good at bringing detail.
inpaint is the best if you want something really detailed, but can be really time consuming.
These are some tips that i can think of now, let me know if you need more help.
@gtama979728 This was with (detailed eyes:1.1), (extremely detailed blue iris:1.2) and upscaled to 1328x2016 , no inpainting in this. https://civitai.com/images/629594?period=AllTime&periodMode=published&sort=Most+Reactions&view=categories&modelVersionId=54518&modelId=34726&postId=175950
@gtama979728 forgot to mention sampling steps, some sampling methods like euler a needs more to get detail.
@Potatovision Perfect i will try it
@gtama979728 I hope it helps :)
@Potatovision So i tried it in img2img with your guide but the rendering time jumped to 1.5 hours. Everything including the image size was same as in the guide. 12 GB VRAM by the way. Any idea why this happening?
@gtama979728 Do you have xformers enabled, that speeds up the generation time and vram usage too. Did you put latent tile overlap too high? What GPU you have? I have rtx 3060 and i think it took like 7-11 minutes to generate with the img2img setting on. Try disabling it if you put it on.
Did you let it generate a little time, it shows longer time at start as it upscales it first.
@gtama979728 Oh yeah, and did you remember to put Latent tile batch size to 8. I think i forgot to write about that in the guide (っ °Д °;)っ That affects time a lot. In the image it is at least 8.
@Potatovision I tried it in txt2img and got some fine details in with hires enabled. Normally, even with hires it doesn't give that much details. But img2img is not working. Yes, Xformers enabled, Latent tile batch size to 8, and RTX 2060 12 gb vram.
@gtama979728 Hmm.. You have tiled VAE enabled and the VRAM usage does not go to max? If it goes to max it can slow it down insanely. What does the CMD line say about the tile batches: x, and how many steps it is trying to do?
@Potatovision it showed 1200 tiles i guess
i have retry it to se the vram usage
@gtama979728 Hmm.. Should only show 30-100 🤔 With the settings i have in there.
@Potatovision so, i have Fast Encoder and Fast Encoder Color Fix enabled. do you think it is because of these settings?
@gtama979728 No i don't think that is the issue. There must be something wrong with the tiling settings if it is showing 1500 on CMD.
You have exactly these settings in there https://i.imgur.com/2nnPZ7C.png ?
@Potatovision yes except for the upscaler i am using 4x-ultasharp
Ok so now it worked
@gtama979728 I tested with the exact settings, and it is giving me 2 minutes for generate time. Are you sure it is using your cuda cores? When you have generate on, go to task manager -> performance -> GPU and click 3D and change it to CUDA. It should be using 95-100% of the GPU cuda when generating.
@gtama979728 Oh good :) What was the issue?
@Potatovision hmm i have no idea. Maybe i use Hires in txt2img and sent a 1024x1552 sized image in img2img. Because i am doing without hires and i get 4 mintues for rendering.
@gtama979728 Haha, yeah that definitely sounds like something that would cause it (～￣▽￣)～
I have planned to expand more on multidiffusion tutorials:
+ Workflow on region prompt control
+ Different workflow text2img -> img2img, more focused on scaling in img2img
+ Maybe inpaint workflow tutorial
If you have something you would like to see tutorial on let me know, and i'll see what i can do.
so, let me tell you.. the img2img part of this extension is actually very IMPRESSIVE... but I dont get the txt2img part on your guide... so whats the point of using tiled diffusion if you gonna use hire fix anyways? isnt redundant? and it might actually be bad for your result (maybe some extra noise or "ghosts"), I learnt a lot, thank you for the guide!
Hey, it is one of my workflows i was testing when i wrote the guide and now looking back i should have started with different route, i will be making another workflow soonish that is more focused on img2img.
The main idea of the guide was to show how things work, and as most people seem to like to use hires. fix it seemed like good idea back then ;)
The img2img part in the tutorial does add more detail, but i should have used higher scaling. I clarified this more in v1.2 version of the guide, that more scaling adds more detail.
Also the tiling adds a lot of detail to the picture, and adds details to hires. fix too.
Yeah the img2img is very impressive, i have been lately testing a lot of loopback img2img and have gotten some interestingly detailed results. Fun thing to experiment with. Have not seen too much ghosts or extra noise, higher tile size usually fixes these. One thing to note, if your images start getting darker it is most likely because you are using model that has baked VAE + using another VAE.
Thank you for the comment, and please leave some images you manage to make :)
@Potatovision ok posted one of my results, if I do t2i from the beginning .. I get ghosts. I think if I lower the denoise on img2img I could fix it.. and I didnt know there was an updated guide. Ill check it out tomorrow for sure, ty for your work
@romoe Hey, I uploaded new tutorial more focused on the img2img workflow
MultiDiffusion hooked into DPM++ 2M Karras sampler. Tile size: 64x80, Tile batches: 1, Batch size: 8 [Tiled VAE]: input_size: torch.Size([1, 3, 563, 563]), tile_size: 480, padding: 32 [Tiled VAE]: split to 2x2 = 4 tiles. Optimal tile size 256x256, original tile size 480x480 | 0/20 [00:00<?, ?it/s] Error completing request Arguments: ('task(o7769ays1rpef3p)', 0, 'portrait of milad,in cyberpunk city, cinematic, hyper resolution, very realistic, very detailed, cyberpunk art, 4k hd wallpaper, trending in artstation, octane render, by greg rutkowski, centered', 'easynegative', [], <PIL.Image.Image image mode=RGBA size=512x512 at 0x121545CBB20>, None, None, None, None, None, None, 20, 15, 4, 0, 1, False, False, 1, 1, 7, 1.5, 0.75, -1.0, -1.0, 0, 0, 0, False, 1200, 1200, 0, 0, 32, 0, '', '', '', [], 0, False, 'prompt keyword', 'keyword1, keyword2', 'None', 'textual inversion first', 'None', '0.7', 'None', True, 'MultiDiffusion', False, 10, 1, 1, 64, False, True, 1024, 1024, 64, 80, 56, 8, 'ESRGAN_4x', 1.1, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, True, False, True, False, False, 480, 64, False, '', 0, 'Send to Canvas Editor', True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, False, True, False, False, 0, 'Gustavosta/MagicPrompt-Stable-Diffusion', '', False, 7, 100, 'Constant', 0, 'Constant', 0, 4, False, 'x264', 'mci', 10, 5, False, True, True, True, 'intermediate', 'animation', <controlnet.py.UiControlNetUnit object at 0x00000120000754B0>, <controlnet.py.UiControlNetUnit object at 0x00000121545CB910>, False, False, 'Denoised', 5.0, 0.0, 0.0, 'Standard operation', 'mp4', 'h264', 2.0, 0.0, 0.0, False, 0.0, True, True, False, False, False, 1, False, False, False, False, '1:1,1:2,1:2', '0:0,0:0,0:1', '0.2,0.8,0.8', 20, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, None, None, '', '', '', '', 'Auto rename', {'label': 'Upload avatars config'}, 'Open outputs directory', 'Export to WebUI style', True, {'label': 'Presets'}, {'label': 'QC preview'}, '', [], 'Select', 'QC scan', 'Show pics', None, False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 'mizuni-concept-v1', 1.0, 1, '', 0, '', 0, '', True, False, False, False, 0, 'Blur First V1', 0.25, 10, 10, 10, 10, 1, False, '', '', 0.5, 1, False, 'Not set', True, True, '', '', '', '', '', 1.3, 'Not set', 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', 1.3, 'Not set', False, 'None', <PIL.Image.Image image mode=RGB size=512x512 at 0x121545C9DE0>, 1, 256, 256, 127, 81, False, 20, 50, 256, 'D:\\SD\\stable-diffusion-webui/outputs/animation.gif', None, True, None, None, False, '', False, 'Normal', 1, True, 1, 1, 'None', False, False, False, 'YuNet', 512, 1024, 0.5, 1.5, False, 'face close up,', 0.5, 0.5, False, True, [], True, True, True, 0.0, 0, None, 'gif', None, False, None, False, 50, 0, 0, 512, 512, False, False, True, True, True, False, True, 1, False, False, 2.5, 4, 0, False, 0, 1, False, False, False, 'Euler a', 0.95, 0.75, '0.75:0.95:5', '0.2:0.8:5', 'zero', 'pos', 'linear', 0.2, 0.0, 0.75, None, 'Lanczos', 1, 0, 0, '<p style="margin-bottom:0.75em">Will upscale the image depending on the selected target size type</p>', 512, 0, 8, 32, 64, 0.35, 32, 0, True, 0, False, 8, 0, 0, 2048, 2048, 2) {} Traceback (most recent call last): File "Z:\stable-diffusion-webui\modules\call_queue.py", line 56, in f res = list(func(*args, **kwargs)) File "Z:\stable-diffusion-webui\modules\call_queue.py", line 37, in f res = func(*args, **kwargs) File "Z:\stable-diffusion-webui\modules\img2img.py", line 172, in img2img processed = process_images(p) File "Z:\stable-diffusion-webui\modules\processing.py", line 503, in process_images res = process_images_inner(p) File "Z:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs) File "Z:\stable-diffusion-webui\modules\processing.py", line 594, in process_images_inner p.init(p.all_prompts, p.all_seeds, p.all_subseeds) File "Z:\stable-diffusion-webui\modules\processing.py", line 1056, in init self.init_latent = self.sd_model.get_first_stage_encoding(self.sd_model.encode_first_stage(image)) File "Z:\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in <lambda> setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs)) File "Z:\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in __call__ return self.__orig_func(*args, **kwargs) File "Z:\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "Z:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 830, in encode_first_stage return self.first_stage_model.encode(x) File "Z:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\autoencoder.py", line 83, in encode h = self.encoder(x) File "Z:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl return forward_call(*input, **kwargs) File "Z:\stable-diffusion-webui\extensions\multidiffusion-upscaler-for-automatic1111\scripts\vae_optimize.py", line 377, in __call__ return self.vae_tile_forward(x) File "Z:\stable-diffusion-webui\extensions\multidiffusion-upscaler-for-automatic1111\scripts\vae_optimize.py", line 264, in wrapper ret = fn(*args, **kwargs) File "Z:\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "Z:\stable-diffusion-webui\extensions\multidiffusion-upscaler-for-automatic1111\scripts\vae_optimize.py", line 618, in vae_tile_forward result[:, :, out_bboxes[i][2]:out_bboxes[i][3], out_bboxes[i][0]:out_bboxes[i][1]] = crop_valid_region(tile, in_bboxes[i], out_bboxes[i], is_decoder) RuntimeError: The expanded size of the tensor (3) must match the existing size (8) at non-singleton dimension 1. Target sizes: [1, 3, 36, 36]. Tensor sizes: [8, 36, 36] [Tiled VAE]: Executing Encoder Task Queue: 98%|███████████████████████████████████ | 355/364 [00:00<00:00, 463.66it/s]
keep getting this error
@hoodady Hey, newest version seems to have caused some errors. There was update 5 hours ago, try to see if it helps. https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111/issues/155