PonyFlow
Works with Pony, Illustrious, and SDXL!
Please post images generated to this workflow page!
I would love to see your creations! Unfortunately I can't automatically tag this workflow as a resource. The checkpoints and loras you used will be automatically tagged, and show up in the model's images gallery automatically!
NOTE: READ THE INSTALL INSTRUCTIONS ABOUT THE HIGHRESFIX NODE. IT WILL NOT WORK THE FIRST TIME YOU LOAD THIS WORKFLOW.
This workflow is focused on creating high-quality, unique images for Pony and SDXL.
This is my personal workflow and wildcard files I have built up over a year of working with StableDiffusion!
The showcase images are some of my favorite generations over the past year using some variation of this workflow.
Included is a
README.md full of install instructions, where to download the models, and usage tips.
ponyflow_wildcards.yaml a wildcards
ponyflow_v1_workflow.json - The workflow to load into ComfyUI
If you have any questions, let me know in the comments below!
V3 Changelog:
I've simplified this workflow by removing the HiRes Fix node, and instead added an img2img Upscaler that gives similar/ better results.
V2 Changelog:
Added Prompting/ Wildcard/ CFG/ Usage notes inside the workflow to answer commonly asked questions
You can now disable HighResFix/ Upscaling entirely to iterate on styles quickly
No more sample mismatch
Hopefully fixed HighResFix breaking the first time 馃
Copy of README:
## Installation
Install ComfyUI https://github.com/comfyanonymous/ComfyUI
Install ComfyUI Manager https://github.com/ltdrdata/ComfyUI-Manager
- Open ComfyUI
- Load the workflow
- Manager > Install Missing Custom Nodes
聽 聽 - Install all of the missing nodes (I use nightly versions for everything)
- Restart ComfyUI
- Install all of the models listed below to their proper places
- Copy the ponyflow_wildcards.yaml wildcard file into the ComfyUI\custom_nodes\comfyui-impact-pack\wildcards directory.
- Restart ComfyUI
- Trying to run this initially, you may run into an error with the HighRes-Fix Script node. To remedy this, create a new version of the node, make sure the paramters are the same, then reconnect the new one to the KSampler node and delete the old one. Make sure to disabling "use_same_seed" as well. No clue why this happens, but it happens consistently the first time...
## Model Downloads
### Checkpoint
Install to ComfyUI\models\checkpoints
Any Pony model will work. CyberRealistic Pony is one of the most popular models that works out of the box.
https://civarchive.com/models/443821/cyberrealistic-pony?modelVersionId=1346181
### Controlnet
Install to ComfyUI\models\controlnet (I renamed mine to controlnet_union_promax.safetensors)
### Depth Anything
The Depth Anything V2 - Relative node will automatically download and install the needed model.
### Upscale models
Install to ComfyUI\models\upscale_models
https://civarchive.com/models/147759/remacri
### Loras
Install to ComfyUI\models\loras
https://huggingface.co/tianweiy/DMD2/blob/main/dmd2_sdxl_4step_lora.safetensors
https://huggingface.co/tianweiy/DMD2/blob/main/dmd2_sdxl_4step_lora_fp16.safetensors
### CLIP-ViT-H-14-laion2B-s32B-b79K
Install to ComfyUI\models\clip_vision
https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K/blob/main/model.safetensors
### IPAdapter ip-adapter-plus_sdxl_vit-h
Install to ComfyUI\models\ipadapter (You must manually create this folder)
https://huggingface.co/h94/IP-Adapter/blob/main/sdxl_models/ip-adapter-plus_sdxl_vit-h.safetensors
## Usage
This workflow is focused on creating high-quality, unique images for Pony and SDXL.
It contains different features, all controlled by the Fast Groups Bypasser (rgthree) node. The names of the features correspond to the named groups in the workflow. Turning one off will bypass the entire group.
For general text2img, I use:
- Enable Initial Gen
- Enable Highres
- Enable Film Grain
You can toggle on and off previews if you want to mess with scheduler/ upscaling settings to see differences.
### Skimmed CFG
https://github.com/Extraltodeus/Skimmed_CFG
Many people ask how I make my images. They're hard to replicate solely because of this node. Skimmed CFG makes allows your to turn up CFG values and prompt weights (photorealistic:1.5). Try out 5/12/24/48 CFG while setting Skimmed CFG to 3/5/7!
It allows you to weight your prompts more heavily and develop unique styles. For example, try adding (abstract minimal art:1.9) to your next prompt!
For a baseline, I recommend Skimming CFG at 5.0 and CFG at 12. Please test these values out per-model though because they change the output significantly.
## ImpactWildcardEncode
The positive and negative prompt inputs are wildcard encoders. These process the included yaml file into new unique prompts! This lets you change up pieces of the scene, person, or the entire prompt! The default example shows my normal workflow.
Pick a random person, give them a random body size, and put them in a random scene. You can specify more or less detail, or not use wildcards at all!
There are many techniques to using wildcards that I give examples of in ponyflow_wildcards.yaml.
For more information on wildcards, refer to the guide here: https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/ImpactWildcard.md
When making changes to wildcards, refresh them in the ComfyUI menu under Edit > Impact: refresh Wildcard
I probably need to write an entire article on lessons learned using wildcards...
## Film Grain
I like using the post processing film grain node included in the workflow for realistic images. Try out diffferent values for scale, strenght, and saturation as you like.
## Controlnet Depth
Upload an image to the Load Controlnet Image node to use it as a template for your image. This will take the "shape" of an image, and mold your output to the shape of the input image. Enable the "Depth Sample" node to see what the image's influence is.
## Style Transfer
Upload an image to the "Load Style Image" node to copy its color scheme, and artistic qualities.
Adjust "weight_style" and "end_at" parameters to adjust the strength of the style transferred. As a default I like them both at 0.5, but each style image requires tuning.
## Extra VAE
I've included an extra "Load VAE" node in case the model you use does not have its own VAE. You can use the standard SDXL vae, or one of the many avilable on CivitAI
Description
V1!
FAQ
Comments (78)
I have tried a few times to recreate the Hi Res Fix script node. I wonder what I should do to fix it.
I found that if you copy and paste the node, it still errors out.
However, if you doubleclick in ComfyUI, search the HighRes Fix Script node, then set all the parameters manually then is works. I really wish there were a better way, but I think its just a bug in the node definition itself.
Let me know if that works for you!
@prompt_bit_sorcerer聽Worked for me, but for some reason, Remacri isn't showing up in that specific list of upscalers, even though it shows up in one of the other nodes.
@haasetomoko203聽Thats odd. I haven't seen that happen before. Are you sure you installed it to the right folder and refreshed (press "r") your models list?
@prompt_bit_sorcerer聽Well it shows up in "CR Upscale Image". Maybe i just need to restart CUI.
@haasetomoko203聽Yeah I would try a restart in that case.
@prompt_bit_sorcerer聽I got caught up in some other stuff, thans for the message I will try it now and report back
@haasetomoko203聽I got remacri to work by running a python script that the comfyui GPT assistant gave me. I created a venv and installed safetensors and some other stuff required for converting a pickle file or whatever you got, proabably a zip folder with .pt, then you probably tried renaming it to .pth, I did that and it still didnt work, but then i ran the safetensors conversion script and I ended up using remacri in another workflow as i couldnt get the high res thingy to work in this one! But i will now try another fix for the high res node and report.
@haasetomoko203聽try running a safetensors conversion script in a venv (for good measure) that takes the remacri zip folder you probably have and turns it into a useable model that will show up in upscalers. I got that thing to work before the res node.
@prompt_bit_sorcerer聽I don't know, maybe i will just reinstall the node or something. I tried a few times loading the node (not cloning/duplicating the existing node) no success my friend. I appreciate your work and your replies.
@xobythepoet776聽 It worked out when i restarted.
Alright here's the deal. Since I manage packages in Docker + Venv, with systematic pipdeptree inspections I tend to trouble shoot the long and hard way. I rarely use the comfyui node manager except for very new nodes and... for downgrading to older nodes. Once i rolled back the high res script node to its earliest version i was able to get it to work! Nice job on the work flow.
Key Takeaways:
1. remacri.pth <--- rename was insufficient, i ran a venv with its own safetensors + other packages needed to run a python script for "unpickling" , again this could be because i have a containerized set up with docker.
2. High res script node <--- remake, delete and create new was not sufficient, what worked was rolling back to the earliest version of that node in comyfui node manager.
Good luck friends, this is a great workflow once it runs.
@xobythepoet776聽Sorry for the issues with the Highres node. I think there鈥檚 some bug in the node definition itself because it breaks consistently the first time you load the workflow but then it鈥檚 fine every time after. Thanks for posting your troubleshooting steps, I鈥檓 sure they鈥檒l help someone.
Cant wait to see what you create!
I've been looking forward to your workflow for a while now. Glad you're finally releasing it! It's a really nice style.
Thanks! I'm excited to see what you can make with it!
Um..how do i make it make adults instead of 16yo?
Depending on the model sometimes cute face or kawaii girl can slant the faces too young. Comment those out in the wildcard file, or add child/ young to the negative prompt with a higher weight
@prompt_bit_sorcerer聽Thank you
Got any idea why it's looking for the depth models in such a weird directory?
" No such file or directory: 'E:\\ComfyUI_windows_portable_nvidia_cu118_or_cpu\\ComfyUI_windows_portable\\ComfyUI\\custom_nodes\\comfyui_controlnet_aux\\ckpts\\depth-anything\\Depth-Anything-V2-Large\\.cache\\huggingface\\download\\depth_anything_v2_vitl.pth.a7ea19fa0ed99244e67b624c72b8580b7e9553043245905be58796a608eb9345.incomplete' "
Hmm I haven鈥檛 seen that before. I don鈥檛 use the portable version of Comfyui so there may be some differences to my install.
@prompt_bit_sorcerer聽It seems the node hasnt downloaded the files, so it's a node/CUI problem rather than workflow problem i suppose.
For me the "HighRes-Script" node gives me an error so I have to disable it to make it work
Yeah - check the install instructions for how to make it work.
@prompt_bit_sorcerer聽My bad to not finish the sentece... sorry!
We've posted a fix - there is use same seed. Turn to false, change -1 to 1, then back to true if you like.
It doesn't give me errors, but simply doesn't generate at all, the prompts nodes glow in green and just stop generate. Why?
Yes, same here
Got it: check the highres workaround the autor mentioned before. This is the solution.
@SD_Agamemnon聽I understood that it works just if I disable it... than why is there?
@Kanginak聽Were you able to follow the steps to fix the HighResFix node?
@prompt_bit_sorcerer聽As I said I just managed to disable that node so now it works, so I don't really know why some of the nodes are there.
@prompt_bit_sorcerer聽I did remove and readd the node (I assume its the same) with the next prompt being red again. Not sure its a bunch of models (etc) missing, but I'm loading up whatever i'm missing. The script is the one that just doesnt work.
@jsc244482聽Did the same - deleted and re-added and it is still broken for me as well. It works disabled but it's got me curious how well it works with the node
@hotgirlcharacterart聽I fiddle with AI prompting for the error. There is a seed at -1, under use the same seed. by design or not, toggle that off and then change over to 1, and toggle use same seed again. Granted I had a lot of missing py libs and I doubt thats related. but loads of things for a newb to get it running
@jsc244482聽Hmm yeah I would try to set "use_same_seed" to false, then click randomize and see if that helps. Unfortunately the node is not written very well. There are quite a few issues on their Github about it.
My friend excellent workflow. However I am having trouble getting the high res sample output to pass into the film grain, which yesterday was automatic, and so far I can't get the image save node to work. I used two different paths to no success. Ill check the readme again.
thanks so much!
Nevermind I am such a noob i didn't know where comfyui typicaly saves images, I literally just copy them out of the workfow... anyways thanks again good stuff.
Love this workflow, its really great! Is it possible to make a workflow with an optional Facedetailer or Adetailer?
Thanks! I originally had Adetailer nodes in this workflow, but they changed the image too much in a way I didn't like. If you want an example - you can download this image which has an older version of this workflow built in. You should be able to copy/ paste the "Fix Face"/ "Fix Eyes" section into the new flow pretty easily.
https://civitai.com/images/60517437
@prompt_bit_sorcerer聽thank you!
Got the HighRes-Fix Script node to work by recreating the node as suggested AND by disabling "use_same_seed".
I just replied to you about the fix too! Glad you found it and posted it!
Thanks for posting this! I didn't run into this error specifically, but it seems like others are.
thanks!
HighRes-Fix Script Fix Context
So I ran into the same issue as other users, a quick look into the log/console showed, that the issue was the renaming of the file - simply rename into the recommended "controlnet_union_promax.safetensors". Fixed it for me.
How does It work to run loras in this model? If I put in the loras paste will run with all of them at the same time?? Im new to this
In the "Positive WildcardEncode" node, there's a "Select to add LORA" section which will list all of your loras.
Select one, and it will append it to your prompt like: <lora:loraname>. You can then change the weights of the lora by adjusting the strength like "<lora:loraname:0.5>"
OMG tysm, I can almost use everything only the style transfer, which has: "IPAdapterStyleComposition
Error(s) in loading state_dict for ImageProjModel: size mismatch for proj.weight: copying a param with shape torch.Size([8192, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 1024])."
Idk if I downloaded the wrong model, and any tips for making the same face on this workflow? Or I do need to add more nodes? Love the work man!
@TropicalCat聽Hmm. I just doublechecked the links in the README and they seem correct. Models should be available here:
https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K/blob/main/model.safetensors
https://huggingface.co/h94/IP-Adapter/blob/main/sdxl_models/ip-adapter-plus_sdxl_vit-h.safetensors
Tysm I downloaded from the comfyUI menager, what exactly Style does? I though It was for faceswap, What u recommend to implement if I need to use faceswap? (I want to make the same models), ty for the support in advance
@prompt_bit_sorcerer聽ah, it was that simple all along.
Hey, good evening. This workflow is amazing, and thanks for sharing it with the community. This might be a noob question, but can I create photos with consistent people using this workflow?
Thanks man.
You can use character LORAs with this workflow. I know people have faceswap adapters, but I have personally never used them. You can likely find one on CivitAI though and copy the relevant nodes into this workflow.
@prompt_bit_sorcerer聽 Thank you again. I appreciate it.
There is now autocompletion for the wildcards ? not sure how to use this.
What I like the most in your workflow is the trick for the upscale with dmd2, never found a use for this (results were meh) but in this case it enables quick upscale, way faster than ultimate SD Upscale I usually use.
I added wildcard instructions in the README. The most usage instructions are here: https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/ImpactWildcard.md
You don't have to use wildcards if you don't want to. You can just delete them and prompt like normal.
@prompt_bit_sorcerer聽yeah, of course, I did that. But was curious about how it worked exactly, I'll read the github link you posted, thanks.
How do we add this workflow to images we post? I can't seem to find it in any of the categories (resource/tool/technique)
On this page, scroll down to the gallery section and click "Add Post". That will automatically associate it!
@prompt_bit_sorcerer聽
Got it!
@prompt_bit_sorcerer聽When I do that, it just asks me to upload a picture form from my computer. (NVM I'm dumb.)
This is awesome! I've been using A1111 for a long time, and have been trying to make the jump to ComfyUI, but it's very overwhelming. Do you have any recommended guides or tutorials to learn how all this stuff works? Following your directions, I can make this workflow work brilliantly, but I'd like to learn more about each part, and how it all works together. Thanks!
I started using it early on - so I've kind of just learned from downloading other workflows and learning by trial and error. Looks like there's a pretty good guide here: https://blenderneko.github.io/ComfyUI-docs/
@prompt_bit_sorcerer聽Thanks! I'll check it out.
Thank you for sharing, this is very helpful for someone who's starting out.
One thing is that the %time token comes out written as "False" in the file names. Any idea why?
Not sure. It used to work - but a recent update seems to have broken it.
@prompt_bit_sorcerer聽I figured it out. The Imager Saver node has a "time_format" input towards the bottom. For some reason it was set as "false" for me when I opened the workflow initially. I created a new node and copied the tokens from it (%H%M%S).
@xdice聽Aha! That must be a new addition. The time formatting used to just work and I guess I never saw that option. Thanks! I'll make sure to include that in v2!
hey, thanks for all the content!
i was looking on recomendations on hardware.
could you share your setup or what is a good enough setup to generate images comfortably and high quality?
not sure what to look for
generating videos would be nice as well
I can't speak to minimum thresholds for generations.
I currently have an i9/ nvidia 4090/ 64GB of RAM which is pretty comfortable for most image generations. You basically want to shoot for an Nvidia card with the highest VRAM possible.
For video, even the 4090 can be a bit lacking. I would probably recommend just running these on some cloud service. I think runpod is popular - but I have no experience there.
@prompt_bit_sorcerer聽thanks for the fast reply, and wish you al the best
@prompt_bit_sorcerer聽What do image generation times look like for you? It is currently 90 seconds on a 4090 with 16GB VRAM.
@jayebaek2198聽With Highres + Upscale enabled usually ~ 45 seconds
Is the "Image Saver" node not working for anyone else?
"Failed to validate prompt for output 85:
* (prompt):
- Required input is missing: images
* PreviewImage 85:
- Required input is missing: images
Output will be ignored
Failed to validate prompt for output 196:
* (prompt):
- Value not in list: strip_a1111_params: 'False' not in ['nothing', 'positive prompt', 'negative prompt', 'positive and negative prompt']
* Image Saver 196:
- Value not in list: strip_a1111_params: 'False' not in ['nothing', 'positive prompt', 'negative prompt', 'positive and negative prompt']
Output will be ignored"
Sorry I can't say that I've hit that error before.
Is your HighRes section enabled? That one sets the "generated image" variable. If it is disabled then the image won't be set for the ImageSaver Node.
To work around this, you might try dragging the "Image" output of the "KSampler (Efficient)" node down into the "images" input of the Image Saver node. I'm working on a version 2 of this workflow that allows you to toggle it off.
@prompt_bit_sorcerer聽Thanks, no i've mostly been generating small images until i find something worth upscaling as it takes several minutes to upscale.
I tried making a "set_Generated Image" node from the Initial sample, but i couldn't make it work.
Hi, someone has this worflow adaptation for inpaint image and style transfer ?
Thanks