[Note: Unzip the download to get the GGUF. Civit doesn't support it natively, hence this workaround]
Flux1.D merged in Flux1.S. It can generate good-quality images (better than Schnell) with just 4 steps, and the quality further improves with more steps, while consuming a very low amount of VRAM. Q_4_0 can produce 1024x1024 images in 45 seconds on my 11GB 1080ti, while using around 6.5 Gigs of VRAM.
It can be used in ComfyUI with this custom node or with Forge UI. See https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/1050 to learn more about Forge UI GGUF support and also where to download the VAE, clip_l and t5xxl models.
Which model should I download?
[Current situation: Using the updated Forge UI and Comfy UI (GGUF node) I can run Q8_0 on my 11GB 1080ti.]
Download the one that fits in your VRAM. The additional inference cost is quite small if the model fits in the GPU. Size order is Q4_0 < Q4_1 < Q5_0 < Q5_1 < Q8_0.
Q4_0 and Q4_1 should fit in 8 GB VRAM
Q5_0 and Q5_1 should fit in 11 GB VRAM
Q8_0 if you have more!
Note: With CPU offloading, you will be able to run a model even if doesn't fit in your VRAM.
LoRA usage tips
The model seems to work pretty well with LoRAs (tested in Comfy). But you might need to increase the number of steps a little (8-10).
Updates
V2: I created the original (v1) from an fp8 checkpoint. Due to double quantization, it accumulated more errors. So I found that v1 couldn't produce sharp images. For v2 I manually merged the bf16 Dev and Schnell checkpoints and then made the GGUF. This version can produce more details and much crisper results.
All the license terms associated with Flux.1 Dev and Flux.1 Schnell apply.
PS: Credit goes to jice and comfy.org for the merge recipe. I used a slightly modified version of https://github.com/city96/ComfyUI-GGUF/blob/main/tools/convert.py to create this.
Description
Q4_0: Memory consumption is similar to NF4 quants.
FAQ
Comments (13)
So you have a compromise between S and D, but you're still stuck to the license of D?
That is their (BlackForestLabs) policy - use Dev in any meaningful way you end up with Dev license
@nakif0968 I read it, any derivative, sad.
My GPU thanks you, this is the future, you are legend.
I have a feeling we're still barely scratching the surface, excited about what the future of Flux holds.
Is it possible for you to make an NF4 version with the clip and TE built in?
I don't know how to make NF4. BTW, Forge UI supports GGUF now, in case you didn't know. Also, you can load T5, CLIP, and VAE separately (see the model description for links).
@nakif0968 Thanks, I just tried it now. Unfortunately with 8gb vram and 16gb ram it takes super long to load everything vs NF4 which has everything.
@low_channel_1503 You're probably using the fp16 version of T5xxl. Try the fp8 version of T5
Thanks mate! This is deffo better quality than the Q4 0/1 GGUF and it doesn't take any longer to render. Almost finished downloading the Q5 and Q6 models to pit them against this monster here :-)
Is the Weight split 50/50?
No. I think all the double blocks except the first one were taken from Dev that directly replaced that of Schnell's, i.e., it's all or nothing, not a weighted average. This was done by jice, according to a tip from comfy.org. See the model description for more info.
Why civitai ain't made no gguf tab gguf is the future!! People make more gguf models! I'm gonna start converting sdxl models!! XD
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.