<h1 id="refcontrol-flux.2-klein-9b-reference-canny-lora">RefControl FLUX.2 Klein 9B – Reference Canny LoRA</h1><h2 id="short-description">📝 Short description</h2>A LoRA for FLUX.2 Klein 9B Base that fuses a reference image (identity) with a canny edge map (structure / contours). It preserves identity and style from the reference while following the shape and composition from the canny control map.Trigger word: <code>refcontrol</code><hr /><h2 id="examples">📊 Examples</h2>Each preview is a single combined image from ComfyUI: Canny → Reference → Result (left to right).Canny → Reference → Result<img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora/resolve/main/images/a%20(1).png" width="960" /><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora/resolve/main/images/a%20(2).png" width="960" /><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora/resolve/main/images/a%20(3).png" width="960" /><span style="background-color:rgb(255, 255, 255);color:oklch(0.446 0.03 256.802);font-family:&quot;Source Sans Pro&quot;, ui-sans-serif, system-ui, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, &quot;Segoe UI Symbol&quot;, &quot;Noto Color Emoji&quot;;font-size:16.8px">---<h2 id="extended-description">📖 Extended description</h2>This LoRA was primarily trained on humans, but it also works with stylized characters and some objects. Its main purpose is to preserve identity — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the structure and contours defined by the canny edge map.FLUX.2 Klein 9B Base already handles reference + edge-guided transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and better preserves character identity and edge fidelity than the base model without LoRA.Part of the RefControl family: reference + control fusion for consistent, controllable generation on FLUX.2 Klein 9B Base.<hr /><h2 id="how-to-use">⚙️ How to use</h2><ol><li>Use the first image as the canny edge map (structure / contours).</li><li>Use the second image as the reference (character, person, or object).</li><li>Add the trigger word <code>refcontrol</code> in your prompt.</li><li>Adjust LoRA weight (recommended 0.8–1.0) depending on how strongly you want to preserve identity.</li></ol><h2 id="comfyui-requirenments">ComfyUI requirenments</h2>Canny extraction in the included workflow uses CannyEdgePreprocessor (via <a target="_blank" rel="ugc" href="https://github.com/Fannovel16/comfyui_controlnet_aux">comfyui_controlnet_aux</a>):<a target="_blank" rel="ugc" href="https://github.com/Fannovel16/comfyui_controlnet_aux">https://github.com/Fannovel16/comfyui_controlnet_aux</a>You can disable the built-in canny preprocessor if your input is already a canny/edge image.<h3 id="base-model">Base model</h3>Trained on and recommended with <code>black-forest-labs/FLUX.2-klein-base-9B</code>. The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, <code>guidance_scale</code> ~4.0).The LoRA also works with the 4-step distilled <code>black-forest-labs/FLUX.2-klein-9B</code> for faster inference (~4 steps, <code>guidance_scale</code> ~1.0), but quality may be slightly lower — especially for identity and edge fidelity — compared to the Base model.<h3 id="example-prompt">✅ Example prompt</h3><pre><code>refcontrol
</code></pre><hr /><h2 id="what-it-does">🎯 What it does</h2><ul><li>Preserves character or object identity across generations.</li><li>Adapts the subject to a new structure or composition defined by the canny map.</li><li>Works best when the edge map has similar proportions and scale to the reference.</li></ul><hr /><h2 id="tips">⚡ Tips</h2><ul><li>Best results when the canny map is not drastically different in body scale or framing from the reference.</li><li>Combine with text prompts to refine background, lighting, or mood.</li><li>Can be combined with other RefControl LoRAs (depth, pose, lineart) on compatible pipelines for multi-constraint generation.</li><li>Canny edge maps on a black background work well as control input.</li></ul><hr /><h2 id="use-cases">📌 Use cases</h2><ul><li>Character restyling while keeping identity and edge-defined structure.</li><li>Consistent character design across different compositions.</li><li>Illustration and storyboard generation with edge-guided layout.</li><li>Object transformations with contour-guided placement.</li></ul><hr /><h2 id="files">📦 Files</h2><ul><li>Weights: <code>flux2_klein_9b_refcontrol_canny.safetensors</code></li><li>ComfyUI workflow: <code>refcontrol_canny_flux_klein_9b.json</code></li><li>Repo: <a target="_blank" rel="ugc" href="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora">thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora</a></li></ul>

Files with SHA256: c816dcf61ee26c55a1abfb655ce424cf5db95afae995d6692b1090d37a979870