DEPRECATED - HERE'S MY SUGGESTION:
This workflow, while it has some neat ideas in comfyui, is not being supported, and I don't suggest anyone use it unless they're comfortable enough troubleshooting/fixing workflows themselves. I don't mind questions, but I haven't loaded up this workflow or used flux in a long time. Personally, I suggest moving on to other models than flux for upscaling. z turbo looks promising, as do many others, including many illustrious/noobai models used in multi-stage workflows.
https://civarchive.com/models/2151914/verisimilis-photoreal
Try out my realistic illustrious merge with a simple two stage (hi-res fix) workflow, then use a tiling upscaling step after that if you like the image.
SDXL 2 FLUX Upscaling Workflow (Comfyui) With Custom Flux Lora AND Pony Lora
v 1.1 - added FLUX IPADAPTER, and changed some problematic nodes, made compatible with current version of comfyui (as of 11-24-24)
(Custom Loras linked below, flux Lora is the critical piece, you can load images or use your own txt2img methods for stage 1)
This workflow and flux lora combo applies flux's increased realism and detail to your nsfw images generated via sdxl/pony (or really any image you want to load).
It is designed to take a NEARLY realistic image (i.e. approaching as close as sdxl/pony can get) and push it over the edge to get the 'flux' level of realism and detail.
I put a LOT of time into getting this working, but it WORKS. You'll need to have comfyui installed and know how to use the comfyui manager to install missing nodes (google 'comfyui manager missing nodes' and if you are lost, maybe skip this one).
The workflow should be self-explanatory for those with some basic familiarity with comfyui.
Two loras I custom trained for this workflow on the same dataset:
Trained on pony realism v2.2: https://civarchive.com/models/938116?modelVersionId=1050165
Trained on flux dev: https://civarchive.com/models/771009?modelVersionId=1084068
Models Used:
Pony realism v2.2: https://civarchive.com/models/372465/pony-realism
Flux Dev: https://civarchive.com/models/618692?modelVersionId=691639
Upscaler model (save to models/upscale_models in comfy)
https://openmodeldb.info/models/1x-SkinContrast-High-SuperUltraCompact
Comfui Nodes that may require manual installation of node or it's requirements:
Brightness/contrast/saturation node: https://github.com/mingsky-ai/ComfyUI-MingNodes
IPAdapter Flux: https://github.com/Shakker-Labs/ComfyUI-IPAdapter-Flux
Comfyui Inspire Pack: https://github.com/ltdrdata/ComfyUI-Inspire-Pack
CXH JoyCaption: https://github.com/StartHua/Comfyui_CXH_joy_caption
Face Restore: https://github.com/mav-rik/facerestore_cf
Description
FAQ
Comments (27)
Hey there, Caution! The Workflow is looking beautiful! Although I'm getting errors when running it. As soon as the SDXL model has generated an image and we get to the ClipTextEncodeFlux Stage 3 Node, the error
"CLIPTextEncodeFlux 'NoneType' object has no attribute 'tokenize' "
pops up and stops the Workflow dead in it's tracks. Do you have a possible solution?
Thanks! Sounds like a vae might be set wrong. Reload the workflow and get a screenshot of the default model selections to compare
@CautionMergeAhead That seem to have been resolved after setting the model you used in the original workflow, thanx! I am getting another error when running Step 3. Flux Upscale Tiled; " UltimateSDUpscale
Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_mm) " Is this actually an OOM error? I'm running on a 12GB Nvidia GPU.
@heresy I've seen that, if completely closing and reloading comfy doesn't fix, try running only that portion of the workflow with everything else disabled. Load the image output by the previous step in the load image group
Hi there..can u explain me please, the difference between v 1 and v1.1? thanks
I disabled/swapped a couple of problematic nodes and added the new ipadapter.
@CautionMergeAhead Ah, yes, sorry, I hadn't realized that it was in the explanations. Well, I've been reading a bit, I've downloaded the project. I have two questions, one is that I can't install the art-venture node and it's not new, I haven't been able to do it for a while, I don't know if you know anything about it, and then... how much space do you think this project takes up once everything necessary is downloaded? I'm a bit short on space. Thanks for answering.
@tupu That section is for randomly generating captions and not really the main focus of the workflow.
I'd suggest deleting the plugin folder, updating comfy, reinstalling if you want to get it to work, or just use llm party or another better one.
Honestly, the flux upscaling is pretty slow, and it's not likely you'll want to use this workflow to generate a bunch of randomized images.
I'd suggest finding a workflow that generates random sdxl/pony images separately, generating a bunch, then upscaling the ones you like
@CautionMergeAhead Thanks
Could I use this lora in diffusers :)
If you're talking about the custom trained lora(s), I don't see why not. I have used my safetensor loras trained the same way for image generation with the diffusers back end in my own custom python scripts that allow me to use the full 16 bit flux/sdxl/sd3.5 model sizes. You have to import some other dependencies and find some code examples but for the most part the diffusers documentation is enough to get you going.
if you mean the comfyui workflow - I'm unaware of any comfyui nodes that leverage the diffusers back end.
Ironically, I did start to write one that had basic functionality but lots of issues to sort out to be properly functional for a release.
Hi Caution - I have the following two missing node types:
LayerColor: Brightness Contrast
OverrideMODELDevice
Can't figure out how to install them or get past this error. Any help would be much appreciated. Thanks!
OverrideModelDevice is a script you have to copy in yourself Delete it unless you use multiple gpus.
The layercolor brightness contrast - I had issues with that one, try v1.1 of the workflow or another brighness/contrast (or nothing, it's not essential)
Cannot execute because a node is missing the class_type property.: Node ID '#136', and additionally, theres a bunch of nodes that even the latest version of CUI manager cant install. im not going to become a computer science engineer to figure out if this workflow can figure out female anatomy though, so im out already.
This is a complex workflow and if you're not comfortable with installing comfyui custom nodes and troubleshooting some python dependency issues, it's probably best to skip.
Honestly, this is now outdated. I've created better upscaling from nvidia sora stock recently.
@CautionMergeAhead It appears "Image Multi-Switch" and "String Multi-Switch", both by RvTools, are offline now (two nodes Comfy Manager says are missing for this workflow). The creator's Github no longer exists it appears. The node#136 error is from the custom node missing and it can't be installed by ComfyUI Manager due to the Github disappearing I think. I'm new to ComfyUI, I'm not offering a solution or complaining or asking for a solution, just sharing my experience/insight for future users who might run into this issue. 👍
@spamzorz9293 Thanks for the feedback. At this point I'd just suggest people use the workflow as a starting point and rip out the complexity. I don't have the time to keep it updated unfortunately.
@CautionMergeAhead could you share what you are doing with nvidia sora?
@CautionMergeAhead looking to do n,s.f/w for flux, whats the latest workflow, and workaround for censor? I have had a hard time finding general web talk about it, but I got here, downloaded your workflow, installed some missing nodes, some are still missing, but your saying its outdated. Im missing the clip g and clip I nodes in your workflow.. you got a link to where they are in hugging? Or can you offer some better advice/workflow? Please and thanks!
Hey Caution, I am currently struggling with flux IPadapter node. I got the IPAdapter file from InstantX on huggingface. According to some YT videos it is supposed to be in a different folder than in regular IPAdapter folder. Something like models/xlabs/ipadapter. But even though the node could not recognize the file. Not sure if I should do a path in pth file after that also. My skillissue.. Seems like evertime I need to do something with ipadapter I am cornered.
I followed the instructions on the GitHub page for the node. You should be able to get good results without using the ipadapter node. Honestly, the nvidia sora model has given me better up scaling results faster and without any complex workflow. Sorry. Can't give further info, been a while since I messed with it and have been very busy
@CautionMergeAhead Thank you for your response, I made it work after some time :))
Hello, i'm stuck with this problem:
When loading the graph, the following node types were not found. This may also happen if your installed version is lower and that node type can’t be found.
Image Multi-Switch // RvTools
String Multi-Switch // RvTools
ControlNet Set Union Types (Flux) // RvTools
Can't seem to find them, any idea ?
I posted that workflow almost a year ago, so it was definitely an older version.
https://github.com/whitmell/ComfyUI-RvTools
Look up how to manually install a custom node (just download and unzip the project into the custom nodes folder)
If it doesn't work,
The switches can be replaced with any switch or just bypassed. They were convenient auto switches that reduced setting up multi - function workflow routing but did not contribute to anything beyond that.
The control net nodes should be replaceable by any similar. The union nodes ran the multifunction control net model, but you can replace it with a single function control net model node group if needed.
@CautionMergeAhead thanks
Hello, I have a problem.
A lot of nodes in the scale to megapixels area and aroud wd14 are not connected and I don't know how to do it. also I don't where to put my prompt. I already have all the images so I'll be using upscale refining joy caption and flux upscale tiled.
The workflow is very old and left up for reference. I suggest using something other than flux for upscaling, tbh. Check out the workflow on some of the images posted under my realistic illustrious model, or use one of the other illustrious realistic models, or qwen.