V2 - Advanced weighting and various improvements
Highly recommend just using the new workflow (included .zip or in example workflows on the github)
Installation:
You need DiffSynth-Studio FIRST:
CLICK THIS TO GO TO MY GITHUB README FOR INSTALLATION INSTRUCTIONS FOR YOUR SETUP
(But I already installed):
Please go to the link above anyway and confirm THAT is how you installed.
(Optional) Before Starting:
You can use the built in modelscope download to grab the files for you automatically though you can get them much faster manually with huggingface if you know what you are doing (and where to put them) Otherwise, just be patient, it only needs them once.
https://github.com/by-ae/ae-in-workflow
Description
Per-Image Weighting: Added lora_weights parameter for individual image contribution control
Smart Size Reduction: Implemented size reduction option with weight-similarity sorting for intelligent LoRA compression without losing too much quality (experimental) for tons of images in an i2L lora.
Strength Normalization: Added normalized_strength parameter for target LoRA intensity control
Size Compression: Added reduce_size_factor for configurable LoRA size reduction through batching
Custom Decoder: Subclassed ZImageUnit_Image2LoRADecode and added advanced merging logic
Slight Memory Optimization: Replaced tensor stacking with iterative accumulation to reduce VRAM usage
FAQ
Comments (6)
where models and clip is downloaded?
It's the default in the example for DiffSynth but essentially it creates a 'models' folder in the same folder that ComfyUI is executing from.
This is your normal models folder for all but 'Portable' ComfyUI, which is usually 1 level up.
```
models
└─── DiffSynth-Studio
├── General-Image-Encoders
│ ├── DINOv3-7B
│ │ └── model.safetensors
│ └── SigLIP2-G384
│ └── model.safetensors
└── Z-Image-i2L
└── model.safetensors
```
As for the safetensor models to download, its interesting that one of them is 4.3G:
Downloading [SigLIP2-G384/model.safetensors]: 11%|███▌ | 505M/4.34G (SLOWER)
But it's crashing with signature error/different SHA.
When I try to find a site to download it directly, I found many in hugging face, like 'google siglip2-base-patch16-384', which the model.safetensors file is only 1.5G . I tried to download and put in the "DiffSynth-Studio\General-Image-Encoders\._____temp\SigLIP2-G384" folder, but then COMFYUI console insists to download up to 4.34G.
Downloading [SigLIP2-G384/model.safetensors]: 38%|███████████▉ | 1.67G/4.34G
Then it finishes up and got SHA error again. I am now restarting download from zero without trying to get from huggingface.
Manual Download Links
These are direct links to download the specific Safetensors files from Hugging Face. You can copy the full URLs below and use them with a download manager like wget/curl (e.g., wget https://example.com/file).
SigLIP2-G384/model.safetensors (from DiffSynth-Studio/General-Image-Encoders):
https://huggingface.co/DiffSynth-Studio/General-Image-Encoders/resolve/main/SigLIP2-G384/model.safetensors
DINOv3-7B/model.safetensors (from DiffSynth-Studio/General-Image-Encoders):
https://huggingface.co/DiffSynth-Studio/General-Image-Encoders/resolve/main/DINOv3-7B/model.safetensors
model.safetensors (from DiffSynth-Studio/Z-Image-i2L):
https://huggingface.co/DiffSynth-Studio/Z-Image-i2L/resolve/main/model.safetensors
Download them and put them in this structure:
```
models
└─── DiffSynth-Studio
├── General-Image-Encoders
│ ├── DINOv3-7B
│ │ └── model.safetensors
│ └── SigLIP2-G384
│ └── model.safetensors
└── Z-Image-i2L
└── model.safetensors
```
IF YOU ARE RUNNING PORTABLE COMFYUI:
Keep in mind this is will be one folder up from where the models folder usually is (just look where it was putting the files already)
^e.g. if it tries to download them despite you putting them in the right place, read this last bit and make a models folder one folder up
After selecting the images for dataset, the workflow ran for a while, downloaded 2 model.safetensor files then just paused. I've tried restarting comfyui portable but still the terminal says the comfyui is paused. Refer the screenshot in this link -> https://ibb.co/fzFVdBxb.
And to be clear, I have a models folder (One level above the original models folder) with all 3 model.safetensors files inside their respective sub-folders.