video TBA
Update: v35 txt2img + Lora & Canny ControlNet
Update: v82-Cascade Anyone
The Checkpoint update has arrived !
New Checkpoint Method was released. All Workflows were refactored.
https://huggingface.co/stabilityai/stable-cascade/tree/main/comfyui_checkpoints
put both inside /models/checkpoints/
v30-txt2img
- updated workflow for new checkpoint method.
- Text 2 Image.
links at top
v32-txt2img-lora
- updated workflow for new checkpoint method.
- lora loader
- Text 2 Image.
links at top
v35-txt2img-canny
- updated workflow for new checkpoint method.
- lora loader
- ControlNet Canny
- Text 2 Image.
links at top
v40-img2img
- updated workflow for new checkpoint method.
- Image to Image with prompting, Image Variation by empty prompt.
links at top
v42-img2img-lora
- updated workflow for new checkpoint method.
- added Lora Loader for testing new trained Lora's
- Image to Image with prompting, Image Variation by empty prompt.
links at top
v45-img2img-canny
- updated workflow for new checkpoint method.
- Lora Loader
- canny support
- Image to Image with prompting, Image Variation by empty prompt.
links at top
v50-img2vision
- updated workflow for new checkpoint method.
- Image to CLIP Vision + Text Prompt.
links at top
v54-img2vision-lora
- updated workflow for new checkpoint method.
- added Lora Loader for testing new trained Lora's
- Image to CLIP Vision + Text Prompt.
links at top
v55-img2vision-canny
- updated workflow for new checkpoint method.
- Image to CLIP Vision + Text Prompt.
- adds canny support
links at top
v60-img2remix
- updated workflow for new checkpoint method.
- Multi-Image to CLIP Vision + Text Prompt.
links at top
v65-img2remix-canny
- updated workflow for new checkpoint method.
- Multi-Image to CLIP Vision + Text Prompt.
- adds canny support
links at top
v66-img2remix-lora
- updated workflow for new checkpoint method.
- added Lora Loader for testing new trained Lora's
- Multi-Image to CLIP Vision + Text Prompt.
links at top
v70-img2remix-faceswap
- updated workflow for new checkpoint method.
- Multi-Image to CLIP Vision + Text Prompt.
- Use an HD Face image with Reactor.
links at top
v75-img2faceswap-canny
- updated workflow for new checkpoint method.
- Multi-Image to CLIP Vision + Text Prompt.
- canny support added
- Use an HD Face image with Reactor.
links at top
v82-Cascade-Anyone
- Add high quality Face image with 4 character reference images using prompts.
- built from v70 to estimate custom characters without training or Cnet
links at top
v85-Anyone-canny
- Add high quality Face image with 4 character reference images using prompts.
- built from v70 to estimate custom characters without training or Cnet
- canny support added
links at top
v95-img2vision-canny
- Add 3 high quality reference images for Vision
- img2img with canny using the same image
- built from v85 to do complex remix variations
- canny control net and lora support added
links at top
UPDATE: removed Photomaker version, because it actually had no effect.
I want to stress that you MUST update your comfyUI to the latest version, you should also update ALL your custom nodes because there is no way to know which ones might have affect the UNET, CLIP and VAE spaces which cascade is now using to generate our images.
In addition, i have disabled a lot of custom nodes i did not need on that run. it's easy, just add ".disabled" to the folder name. This is what the button does in the manager. it's very easy to "switch off" some custom nodes in this way.
~
Everything Below applied to the early Method for loadings all the models here: official repo: https://huggingface.co/stabilityai/stable-cascade
~ i will leave the Early method here, for anyone wishing to use it :)
UltraBasic Stable Cascade Workflows for ComfyUI:
Article here: https://civarchive.com/articles/4161
IMG2IMG UPDATE:
these older workflows were deprecated on day 4 by a new method, however still work fine.
v10 = txt2img Stable Cascade here: https://civarchive.com/models/310409?modelVersionId=348385
v12 = v10 txt2img without custom nodes: https://civarchive.com/models/310409?modelVersionId=351470
v16 = img2img (stage C) Stable Cascade Workflow here: https://civarchive.com/models/310409?modelVersionId=351400
v17 = v16 img2img without custom nodes for scaling: https://civarchive.com/models/310409?modelVersionId=351464
v18 = v16 img2img (stage B and C ) now supported by new default node: https://civarchive.com/models/310409?modelVersionId=351658
You can squeeze it only any GPU if you use the correct combination.
These notes are in the Workflow also ;)
Cascade Combos:
stage_b + stage_c ~ 22GB
stage_b_bf16 + stage_c_bf16 ~ 12GB
stage_b_lite + stage_c_lite ~ 8GB
stage_b_lite_bf16 + stage_c_lite_bf16 ~ 5GB
I put together to paths you need to put all the models in case you had to manually DL each of them, due to a poor connection or whatever :)
Huggingface has the models we need, follow the chart below to find where they go
https://huggingface.co/stabilityai/stable-cascade
Text Encoder
ComfyUI Path: models\clip\Stable-Cascade\
HF Filename: /text_encoder/model.safetensors
text encoder CLIP = 1.39GB
Stage C
ComfyUI Path: models\unet\Stable-Cascade\
HF Filename: stage_c.safetensors
stage_c = 14.4GB
stage_c_bf16 = 7.18GB
stage_c_lite = 4.12GB
stage_c_lite_bf16 = 2.06GB
Stage B
ComfyUI Path: models\unet\Stable-Cascade\
HF Filename: stage_b.safetensors
stage_b = 6.25GB
stage_b_bf16 = 3.13GB
stage_b_lite = 2.8GB
stage_b_lite_bf16 = 1.4GB
Stage A
ComfyUI Path: models\vae\Stable-Cascade\
HF Filename: stage_a.safetensors
stage_a = 73.7mb
Effnet Encoder
ComfyUI Path: models\vae\Stable-Cascade\
HF Filename: effnet_encoder.safetensors
img2img VAE encoder = 81.5mb
Description
image 2 image and Image Variation support.
you will need to place the effnet_encoder.safetensors inside your /models/vae/ folder
and update comfyUI to the latest version, this was only added today ;)
FAQ
Comments (10)
Runs fine on my RTX 2060 too!
Do you have any tips (or is there some good article) about settings for the two KSamplers? I am wondering how to set steps, CFG, and samplers now that there are two nodes instead of older models' one.
the best tip i can offer is that the first ksampler seems to be the one you must use denoise with. there is a lot of posts around on the best settings. there is a rush right now to find them. The settings i have in the workflows are my best defaults, if i find better settings, future versions will have those baked in.
If you find nice settings recipes, please post them in the comments :D
`Error occurred when executing CLIPVisionEncode: 'NoneType' object has no attribute 'encode_image' File "D:\work\ai\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "D:\work\ai\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "D:\work\ai\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "D:\work\ai\ComfyUI\nodes.py", line 904, in encode output = clip_vision.encode_image(image)`
I'll take a look and see if i can replicated this now....
EDIT: i updated comfyUI just now and ran the V50-img2vision workflow
It is working as it should be. try to download the workflow again after updating your ComfyUI
I get error when i try the combo you suggested for 8gb vram, error is as follows:
Error occurred when executing CheckpointLoaderSimple: 'model.diffusion_model.input_blocks.0.0.weight' File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI_windows_portable\ComfyUI\nodes.py", line 540, in load_checkpoint out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 500, in load_checkpoint_guess_config model_config = model_detection.model_config_from_unet(sd, "model.diffusion_model.") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI_windows_portable\ComfyUI\comfy\model_detection.py", line 191, in model_config_from_unet unet_config = detect_unet_config(state_dict, unet_key_prefix) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI_windows_portable\ComfyUI\comfy\model_detection.py", line 77, in detect_unet_config model_channels = state_dict['{}input_blocks.0.0.weight'.format(key_prefix)].shape[0] ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
if you are having trouble with the more complicated method of loading all these UNET and CLIP models, consider trying the new updated method which is just two checkpoints that go into /models/checkpoints/
I put a new video up explaining how to get this setup. The UNET method did not use the Checkpoint loader yet, so i reckon if you grab those new models and the new workflows (i had to rewrite them all everyday since launch haha) it should work for you.
I had people with 8GB or less report it's working, so let me know
@driftjohnson Nevermind i got it working with this workflow, almost same but dont know how it worked :D
https://comfyworkflows.com/workflows/15b50c1e-f6f7-447b-b46d-f233c4848cbc, The generated images are very bad though(like stable diffusion 1). Most probably because i am using smaller models.
@Bolly_Diffusion thanks for letting me know :D i'm using vision with reactor in my latest workflow, so be sure to take a look and see if you can use that. It's all built in with the new checkpoints.
@driftjohnson new models just give out of memory from the start while old method works on 8gb (amd)
@patientx737 Thanks for confirming this man, I'll be sure to make a big song and dance about this in the next video. it's great news as all the work that was done on making the UNET/CLIP method workflows is still REALLY useful for anyone on low VRAM. Thanks again - I cannot test this without swapping my GPU out ! Saved me some hassle :)