PonyFlow
Works with Pony, Illustrious, and SDXL!
Please post images generated to this workflow page!
I would love to see your creations! Unfortunately I can't automatically tag this workflow as a resource. The checkpoints and loras you used will be automatically tagged, and show up in the model's images gallery automatically!
NOTE: READ THE INSTALL INSTRUCTIONS ABOUT THE HIGHRESFIX NODE. IT WILL NOT WORK THE FIRST TIME YOU LOAD THIS WORKFLOW.
This workflow is focused on creating high-quality, unique images for Pony and SDXL.
This is my personal workflow and wildcard files I have built up over a year of working with StableDiffusion!
The showcase images are some of my favorite generations over the past year using some variation of this workflow.
Included is a
README.md full of install instructions, where to download the models, and usage tips.
ponyflow_wildcards.yaml a wildcards
ponyflow_v1_workflow.json - The workflow to load into ComfyUI
If you have any questions, let me know in the comments below!
V3 Changelog:
I've simplified this workflow by removing the HiRes Fix node, and instead added an img2img Upscaler that gives similar/ better results.
V2 Changelog:
Added Prompting/ Wildcard/ CFG/ Usage notes inside the workflow to answer commonly asked questions
You can now disable HighResFix/ Upscaling entirely to iterate on styles quickly
No more sample mismatch
Hopefully fixed HighResFix breaking the first time 🤞
Copy of README:
## Installation
Install ComfyUI https://github.com/comfyanonymous/ComfyUI
Install ComfyUI Manager https://github.com/ltdrdata/ComfyUI-Manager
- Open ComfyUI
- Load the workflow
- Manager > Install Missing Custom Nodes
  - Install all of the missing nodes (I use nightly versions for everything)
- Restart ComfyUI
- Install all of the models listed below to their proper places
- Copy the ponyflow_wildcards.yaml wildcard file into the ComfyUI\custom_nodes\comfyui-impact-pack\wildcards directory.
- Restart ComfyUI
- Trying to run this initially, you may run into an error with the HighRes-Fix Script node. To remedy this, create a new version of the node, make sure the paramters are the same, then reconnect the new one to the KSampler node and delete the old one. Make sure to disabling "use_same_seed" as well. No clue why this happens, but it happens consistently the first time...
## Model Downloads
### Checkpoint
Install to ComfyUI\models\checkpoints
Any Pony model will work. CyberRealistic Pony is one of the most popular models that works out of the box.
https://civarchive.com/models/443821/cyberrealistic-pony?modelVersionId=1346181
### Controlnet
Install to ComfyUI\models\controlnet (I renamed mine to controlnet_union_promax.safetensors)
### Depth Anything
The Depth Anything V2 - Relative node will automatically download and install the needed model.
### Upscale models
Install to ComfyUI\models\upscale_models
https://civarchive.com/models/147759/remacri
### Loras
Install to ComfyUI\models\loras
https://huggingface.co/tianweiy/DMD2/blob/main/dmd2_sdxl_4step_lora.safetensors
https://huggingface.co/tianweiy/DMD2/blob/main/dmd2_sdxl_4step_lora_fp16.safetensors
### CLIP-ViT-H-14-laion2B-s32B-b79K
Install to ComfyUI\models\clip_vision
https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K/blob/main/model.safetensors
### IPAdapter ip-adapter-plus_sdxl_vit-h
Install to ComfyUI\models\ipadapter (You must manually create this folder)
https://huggingface.co/h94/IP-Adapter/blob/main/sdxl_models/ip-adapter-plus_sdxl_vit-h.safetensors
## Usage
This workflow is focused on creating high-quality, unique images for Pony and SDXL.
It contains different features, all controlled by the Fast Groups Bypasser (rgthree) node. The names of the features correspond to the named groups in the workflow. Turning one off will bypass the entire group.
For general text2img, I use:
- Enable Initial Gen
- Enable Highres
- Enable Film Grain
You can toggle on and off previews if you want to mess with scheduler/ upscaling settings to see differences.
### Skimmed CFG
https://github.com/Extraltodeus/Skimmed_CFG
Many people ask how I make my images. They're hard to replicate solely because of this node. Skimmed CFG makes allows your to turn up CFG values and prompt weights (photorealistic:1.5). Try out 5/12/24/48 CFG while setting Skimmed CFG to 3/5/7!
It allows you to weight your prompts more heavily and develop unique styles. For example, try adding (abstract minimal art:1.9) to your next prompt!
For a baseline, I recommend Skimming CFG at 5.0 and CFG at 12. Please test these values out per-model though because they change the output significantly.
## ImpactWildcardEncode
The positive and negative prompt inputs are wildcard encoders. These process the included yaml file into new unique prompts! This lets you change up pieces of the scene, person, or the entire prompt! The default example shows my normal workflow.
Pick a random person, give them a random body size, and put them in a random scene. You can specify more or less detail, or not use wildcards at all!
There are many techniques to using wildcards that I give examples of in ponyflow_wildcards.yaml.
For more information on wildcards, refer to the guide here: https://github.com/ltdrdata/ComfyUI-extension-tutorials/blob/Main/ComfyUI-Impact-Pack/tutorial/ImpactWildcard.md
When making changes to wildcards, refresh them in the ComfyUI menu under Edit > Impact: refresh Wildcard
I probably need to write an entire article on lessons learned using wildcards...
## Film Grain
I like using the post processing film grain node included in the workflow for realistic images. Try out diffferent values for scale, strenght, and saturation as you like.
## Controlnet Depth
Upload an image to the Load Controlnet Image node to use it as a template for your image. This will take the "shape" of an image, and mold your output to the shape of the input image. Enable the "Depth Sample" node to see what the image's influence is.
## Style Transfer
Upload an image to the "Load Style Image" node to copy its color scheme, and artistic qualities.
Adjust "weight_style" and "end_at" parameters to adjust the strength of the style transferred. As a default I like them both at 0.5, but each style image requires tuning.
## Extra VAE
I've included an extra "Load VAE" node in case the model you use does not have its own VAE. You can use the standard SDXL vae, or one of the many avilable on CivitAI
Description
Added Prompting/ Wildcard/ CFG/ Usage notes inside the workflow to answer commonly asked questions
You can now disable HighResFix/ Upscaling entirely to iterate on styles quickly
No more sample mismatch
Hopefully fixed HighResFix breaking the first time 🤞
FAQ
Comments (37)
The workflow performed excellent right out of the box. (so to speak). I had been having issues with v1.0. Thanks!!
Glad to hear! Hope you enjoy and I look forward to seeing what you make!
What approach are you using for avoiding "Pony face" on Cyberrealistic?
I don't do anything special. Just (cute face:1.1) or (beautiful face:1.2) or some weight. Sometimes I use the igbaddie lora at a fixed weight.
@prompt_bit_sorcerer Was asking because many of your images have a more unique and believable face with Cyber And Yogi's. I wonder if me using DMD2 lora for 8 step gens is making this worse somehow.
I get what you're saying.I think this guy →prompt_bit_sorcerer and some of them is using some kind of face or character Loras without telling us.
@634777656lwr374Â I'm not. I use the workflow posted here exactly with the loras linked in my images.
Please don't project your lack of imagination and talent onto others.
Hi mate, do you know of an accurate way to use IPAdapter/FaceID with Pony? The setup I use for RealVisXL works great, but doesn't work on Pony at all. After some research I read that it doesn't work well with Pony because the Pony model is too different to SDXL, even though it's finetuned from it.
I noticed the same thing happens in this workflow, the IPAdapter doesn't recreate the intended face.
Eventually I want to train my own SDXL/Pony LoRA for my character, but in the meantime I'm just using IPAdapter to generate consistent images for my dataset.
Worst case is having to do a IPAdapter afterwards as IMG2IMG with SDXL to get my desired face on the Pony image. Ideally I want to just generate a Pony image with consistent faces either using IPAdapter or LoRA of my character.
Any advice?
I don't have any advice there - it's not something I've ever done.
I recognize there is face swap kind of thing in ipadpter,just go to their gihub page to find or youtube
@634777656lwr374Â I am using FaceID Plus v2 and Plus Face with IPAdapter. It doesn't work with Pony models though from what I've tried/read.
@xdice try instantID
Amazing workflow! Thank you. I'm still frustrated by the lack of hand consistency with Pony and SDXL models. CyberRealistic_Negative_PONY embedding helps a little. Have you experimented with MeshGraphormer at all, and if so what controlnet model did you use for that? Any other ideas on how to augment this for more consistent hands? I'm tired of fixing in Photoshop, lol
I don't have any experience there. I just use text2image mostly and only select images with good hands.
@prompt_bit_sorcerer Cool, thanks for the response. Any idea which ControlNet model would make the most sense to use if I were to try the Meshgraphormer node?
why the highres-fix script is red? Its not linked to something?
if a node is red, that means that the node is missing and the node needs to be installed.
Does someone know how to connect xyplot to the ImpactWildcardEncode?
What are you trying to accomplish? Most likely an XY Input Prompt S/R node is the closest
Hi, when I try to run this I get the following error: "Cannot execute because a node is missing the class_type property.: Node ID '#227'" anyone know why that might happen? Thanks in advance!
It's probably coming from the node before set sampler or scheulder. That does a lookup based on node id to grab values that are passed to ImageSaver. If you change any of the nodes around you might run into an issue.
Might be helpful to turn on the node ID badges in the UI. In the settings window, go to 'Lite Graph' on the left, scroll to the 'Node' section on the right, then set 'Node ID badge mode' to 'Show all'.
Then update the failing node's ID property to the correct one's ID. Probably the initial KSampler.
these images look great.
It is taking 50 seconds to generate without upscale. With upscale it crashes (Allocation on device; OOM). I have a RX 7900 XTX. Do I just have a bad card for this?
Amazing work, nonetheless. Thanks!
I'm not sure. Can't speak to AMD cards.
l love you
I'm on the latest portable build for rtx5000 cards. Everything works, but there is no preview images. How do I get preview images to show up? I tried setting it to auto in manager and that didn't fix it.
I've had this happen a couple of times to me. Usually it was something wrong with ComfyUI itself. Refreshing the page usually fixes it for me.
I dont know what happened. All it does is spits back what it selected from the wildcard adn does not generate any image. what am i missing?
is the only way to add more loras to include them in the positive wildcardencode? they dont seem to be making a difference.
I had issues with the HighRes fix node not running. Turns out is because of the control net model, even if you are not using control net.
The solution is to enable control net in the HighRes Fix node (set to TRUE) , select it manually (controlnet_union_promax.safetensors or something like that) and then disable it again.
Exactly. When you first load the workflow and don't have the file "controlnet_union_promax.safetensors" in your system, the workflow throws an error message because of that missing file even though controlnet is disabled. I solved it by making sure that there is a Controlnet file of that exact name but CNJ's solution is of course even better.
CyberRealistic v8 used to work plug and play for me with this workflow, but now when I try to run it I get weird distorted faces and washed out/desaturated imagery when I use the upscaler, even on a freshly downloaded version of the workflow. Any tips on how to fix that?
This could work in Forge too.
Vit-H is not a big difference.
Maybe a tutorial?
I'm sorry I'm not familiar with Forge