🚀 JoyAI Image Edit: GGUF Setup Guide
Optimize your ComfyUI workflow with the high-performance GGUF versions of JoyAI Image Edit. Perfect for balancing quality and VRAM efficiency.
🛠️ Required Custom Nodes
To use these models, you must install the following node set:
Node:
ComfyUI_JoyAI_Imageby smthemexHow to install: Search for "JoyAI" in the ComfyUI Custom Nodes Manager and click install.
📂 Critical Folder Installation
[!IMPORTANT] Both the Transformer and the Text Encoder must be placed in the gguf folder. Do not place them in unet or text_encoders.
If you do not have a gguf folder, you must create one manually. Your path should look exactly like this: ComfyUI_windows_portable\ComfyUI\models\gguf\
📥 Download Links
1. Transformers (Place in models/gguf)
Choose the quantization level that fits your hardware:
Download Q6_K (High Quality / Balanced)
Download Q8_0 (Maximum Precision)
2. Und-Merger Text Encoder (Place in models/gguf)
3. VAE (Place in models/vae)
📋 Quick Setup Checklist
[ ] Install ComfyUI_JoyAI_Image custom nodes.
[ ] Create the
models/ggufdirectory.[ ] Drop the Transformer and Text Encoder into
models/gguf.[ ] Drop the VAE into
models/vae.[ ] Refresh ComfyUI and load the JoyAI workflow!
Description
base model with files
FAQ
Comments (8)
Hi! Thanks for doing this and for the heads-up on Reddit. I followed the steps and downloaded the Q6 for both the transformer and the text encoder. I also placed them in the GGUF folder (the drop downs from the nodes see the files and let me select them, so no problem there).
However, I am getting this error:
Matching keys count: 706 Extra keys in state_dict (not in meta_model): 44 - model.language_model.layers.34.self_attn.k_norm.weight - model.language_model.layers.32.mlp.down_proj.weight - model.language_model.layers.33.input_layernorm.weight - model.language_model.layers.32.self_attn.o_proj.weight - model.language_model.layers.32.input_layernorm.weight - model.language_model.layers.34.self_attn.v_proj.weight - model.language_model.layers.35.input_layernorm.weight - model.language_model.layers.33.self_attn.k_norm.weight - model.language_model.layers.34.mlp.gate_proj.weight - model.language_model.layers.35.mlp.up_proj.weight - model.language_model.layers.33.self_attn.q_proj.weight - model.language_model.layers.35.post_attention_layernorm.weight - model.language_model.layers.34.self_attn.o_proj.weight - model.language_model.layers.32.self_attn.v_proj.weight - model.language_model.layers.35.self_attn.o_proj.weight - model.language_model.layers.35.self_attn.k_proj.weight - model.language_model.layers.34.self_attn.q_proj.weight - model.language_model.layers.32.self_attn.k_proj.weight - model.language_model.layers.35.self_attn.v_proj.weight - model.language_model.layers.34.mlp.up_proj.weight Sample matching keys: ['model.visual.patch_embed.proj.weight', 'model.language_model.layers.2.self_attn.q_norm.weight', 'model.language_model.layers.13.self_attn.k_proj.weight', 'model.visual.blocks.12.norm2.bias', 'model.visual.blocks.4.attn.qkv.weight'] !!! Exception during processing !!! model.language_model.layers.0.mlp.down_proj.weight has an expected quantized shape of: (4096, 12288), but received shape: torch.Size([4096, 10080])
Any ideas? Thanks!
Im not sure what the error is tbh, is it loading the encoder onto CUDA or cpu? It HAS to be on cuda, also if its still genning but showing that error its okay, it may be the ggufs compression.
If its NOT genning, update to the latest comfy and ensure your on the NIGHTLY version of the smthemex nodes
@realrebelai Ok, thank you, I will try that and let you know.
If it still gives you issues, when i get home from worl ill take a look at it and debug for you if i can figure out the issue
@realrebelai Thank you for all the help, really appreciated!
Claude says this:
Quick things to check for the intermediate_size mismatch:
Pull smthemex's latest nodes. He recently pushed a fix that got the GGUFs working properly. If you cloned or downloaded before that fix, your nodes are stale. Re-download them fresh.
Make sure your JoyAI-Image-Und/ folder matches what smthemex's nodes expect. His nodes ship with a specific config or expect it at a specific path. If you brought that folder over from the jdopensource repo or from an older JoyAI install, the config's intermediate_size may not match what his GGUFs were quantized against. Check his HF page or node README for where he says to source that folder from, and get it from there.
Don't have multiple JoyAI-Image-Und/ folders on your machine. If older JoyAI installs left leftover folders around, the wrong one might get loaded.
Full ComfyUI restart after updating anything. Close the whole process, reopen.
Sanity check: open your JoyAI-Image-Und/config.json in a text editor and look at the intermediate_size field. If it says 12288, your config is the mismatch — step 1 or 2 fixes it. If it says 10080 already, something else is going on and he should post his full error to smthemex's HF discussion page.
@realrebelai I got it to work: I had to downgrade transformers to be within the requirements, and that fixed the issue. However, it is taking forever to generate, despite using Q6 (lowest quants currently available that I could find). 14 minutes for 1 image! We tried, but this model is not meant for my hardware (3080 10 GB) :D
What is the minimum amount of VRAM needed for this?