Original Project found here: <a target="_blank" rel="ugc" href="https://huggingface.co/Djrango/Qwen2vl-Flux">https://huggingface.co/Djrango/Qwen2vl-Flux</a>Qwen2vl-Flux is a state-of-the-art multimodal image generation model that enhances FLUX with Qwen2VL's vision-language understanding capabilities. This model excels at generating high-quality images based on both text prompts and visual references, offering superior multimodal understanding and control.<ul><li>ComfyUI currently doesn't support and there is no available nodes to load the CLIP+LLM portion into it</li><li>This is just for reviewing/testing the finetuned trained part of the Flux model</li><li>CFG set to 1 on KSampler</li><li>Rendered an image in 150s using 8GB GPU @ 512px / 10 steps using the bf16 model</li><li>This model comes will be available in 3 formats named after the folder it should be in<ul><li>diffusion_models - This one is in diffusers format, it is just the merged safetensors file from HuggingFace page</li><li>checkpoints - This one has been converted to Flux Transformers format and prefix for stable_diffusion compatibility, does not include CLIP and VAE</li><li>unet - I will provide the q4_0 and q8 variants, make a comment if you'd like to see any other quants</li></ul></li></ul>

Qwen 2vl Flux

<ul><li>This version goes in the unet folder</li><li>Requires the UNET Loader GGUF node in ComfyUI</li><li>Has been quantsized from bf16 to q4_0</li></ul>

Files with SHA256: 7314b61aed32ddfbf9232d51234a9f03e33924de342b73e76926f39dafe3c289