😉 FLUX.1-dev-ControlNet-Union-Pro-2.0-fp8

Hugging Face Model Card

A Good Reference for Parameters

• Canny: controlnet_conditioning_scale=0.7, control_guidance_end=0.8

• Depth: use depth-anything, controlnet_conditioning_scale=0.8, control_guidance_end=0.8

• Pose: use DWPose, controlnet_conditioning_scale=0.9, control_guidance_end=0.65

• Gray: use Color, controlnet_conditioning_scale=0.9, control_guidance_end=0.8

Folder Structure

Organize your models as follows for FLUX dev and ControlNet workflows:

📂 ComfyUI/
├── 📂 models/
│   ├── 📂 diffusion_models/
│   │   └── 📄 flux-dev.safetensores         # (or gguf)
│   ├── 📂 text_encoders/
│   │   ├── 📄 clip_l.safetensors
│   │   └── 📄 t5xxl_fp8_e4m3fn.safetensors # (or t5xxl_fp16 or t5xxl_fp8_e4m3fn_scaled)
│   ├── 📂 vae/
│   │   └── 📄 ae.safetensors
│   ├── 📂 controlnet/
│   │   └── 📄 FLUX.1-dev-ControlNet-Union-Pro-2.0-fp8.safetensors

Note: Only one T5XXL text encoder is needed—choose based on your hardware and quality/speed needs.

My FP8 Quantization Solution

With modest coding experience, I researched quantization and implemented FP8 compression for the model. The quantized version works perfectly for my needs, enabling all ControlNet workflows with much lower memory requirements and no noticeable quality loss.

Using The Quantized Model

• Supports all original control types: pose, depth, canny edge, etc.

• Drop any reference image, select control type, and generate results with lower memory usage.

Enhanced Prompting with OllamaGemini

I use my customOllamaGemini node for ComfyUIto generate optimal prompts. This, combined with the quantized model, creates a powerful, memory-efficient pipeline for creative image manipulation.

Alternatives for High-End Hardware

If you have a powerful GPU, the original unquantized model from Shakker-Labs offers higher fidelity at the cost of increased memory usage.

Looking Forward

I welcome community feedback! If you find these workflows helpful, please show your support with a 👍 on the project. I'm open to opportunities and appreciate encouragement as I develop these resources.

Feel free to experiment with the model for your creative projects—whether using the memory-efficient quantized version or the original full-precision implementation!

👨‍💻 Developer Information

This guide was created by Abdallah Al-Swaiti:

1. Hugging Face

2. GitHub

3. LinkedIn

4. ComfyUI-OllamaGemini

For additional tools and updates, check out my other repositories.