<h1 id="minimax-h3-pruned-turbo-lora-(4-step)for-fl2va-model">MiniMax H3 Pruned Turbo LoRA (4-Step)<br />for FL2VA model</h1><p>Special thanks to larryvrh for the original MiniMax H3 Turbo LoRA:</p><p><a target="_blank" rel="ugc" href="https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora">https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora</a></p><p>This adapted release is based on the original acceleration checkpoint:</p><p><a target="_blank" rel="ugc" href="https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora/blob/main/minimax_h3_turbo_4step_ema_ckpt850.safetensors">https://huggingface.co/larryvrh/MiniMax-H3-Turbo-Lora/blob/main/minimax_h3_turbo_4step_ema_ckpt850.safetensors</a></p><p>Please pull<br /><a target="_blank" rel="ugc" href="https://github.com/Comfy-Org/ComfyUI/pull/15353">Add MiniMax H3 pruned complete LoRA support to official LoraLoader by xiaolibai-sys · Pull Request #15353 · Comfy-Org/ComfyUI</a></p><p>or<br /><a target="_blank" rel="ugc" href="https://github.com/xiaolibai-sys/ComfyUI-MiniMaxH3">https://github.com/xiaolibai-sys/ComfyUI-MiniMaxH3</a><br />first</p><h1 id="announcement"><strong>Announcement</strong></h1><h2 id="after-several-days-of-experimentation-i-have-identified-a-serious-issue:-certain-dit-blocks-(from-the-20th-to-the-40th)-in-minimax-h3-are-extremely-sensitive-to-input-perturbationseven-when-the-adaln-modulation-error-is-as-small-as-1e-5-to-1e-4.-after-propagating-through-50-layers-this-can-result-in-a-10percent-difference-in-the-hidden-l2-norm-(compared-to-the-bf16-model-+-lora).-therefore-this-is-not-a-good-approach-and-i-have-decided-to-abandon-it.-perhaps-a-better-alternative-is-to-follow-larryvrh's-method-of-precomputing-t_embedding-and-computing-online.-in-other-words-the-offline-approximated-adaln_t_table-with-low-rank-structure-cannot-adequately-represent-the-rich-dynamics-of-lorait-is-lossy.-however-for-users-with-extremely-limited-vram-this-remains-the-only-way-to-bake-weights-offline-into-the-model-(at-least-for-now).-i-will-provide-a-simple-baking-script-at-https:github.comxiaolibai-sysminimax-h3-lora-bake-where-you-can-bake-the-converted-lora-weights-to-reduce-runtime-lora-memory-usage.-finally-thank-you-all-for-your-support."><span style="background-color:rgb(21, 21, 23);color:rgb(249, 250, 251);font-family:quote-cjk-patch, Inter, system-ui, -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Oxygen, Ubuntu, Cantarell, &quot;Open Sans&quot;, &quot;Helvetica Neue&quot;, sans-serif;font-size:16px">After several days of experimentation, I have identified a serious issue: certain DiT blocks (from the 20th to the 40th) in Minimax H3 are extremely sensitive to input perturbations—even when the AdaLN modulation error is as small as 1e-5 to 1e-4. After propagating through 50 layers, this can result in a 10% difference in the hidden L2 norm (compared to the BF16 model + LoRA). Therefore, this is not a good approach, and I have decided to abandon it. Perhaps a better alternative is to follow Larryvrh's method of precomputing t_embedding and computing online. In other words, the offline approximated AdaLN_t_table with low-rank structure cannot adequately represent the rich dynamics of LoRA—it is lossy. However, for users with extremely limited VRAM, this remains the only way to bake weights offline into the model (at least for now). I will provide a simple baking script at </span><a target="_blank" rel="ugc" href="https://github.com/xiaolibai-sys/MiniMax-H3-Lora-Bake">https://github.com/xiaolibai-sys/MiniMax-H3-Lora-Bake</a><span style="background-color:rgb(21, 21, 23);color:rgb(249, 250, 251);font-family:quote-cjk-patch, Inter, system-ui, -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Oxygen, Ubuntu, Cantarell, &quot;Open Sans&quot;, &quot;Helvetica Neue&quot;, sans-serif;font-size:16px">, where you can bake the converted LoRA weights to reduce runtime LoRA memory usage. Finally, thank you all for your support.</span></h2><h1 id="what-this-is">What This Is</h1><p>This is a complete pruned MiniMax H3 LoRA for 4-step video and audio generation. The original Turbo LoRA targets the full 2688-dimensional AdaLN branch. Pruned models instead use an 8-dimensional adaln_t_table plus per-block projections, so the original AdaLN matrices cannot be applied directly.</p><p>This adapted LoRA bakes the AdaLN adaptation into the pruned model format. It contains the backbone LoRA weights, adaln_t_table, per-block AdaLN projection weights and biases, and final layer AdaLN weights and biases. No h3_silu_temb_grid file is required at runtime.</p><h1 id="available-versions">Available Versions</h1><p>There are two versions of the adapted LoRA.</p><p>The first version uses ALS optimization. The shared table and every per-block projection are jointly refined over several iterations. This version gives the best accuracy for a single complete LoRA. Measured against the exact BF16 full model plus LoRA modulation, block max relative error is around 1.5e-5 and final layer relative error is around 4.5e-5.</p><p>The second version uses a fixed shared table. The table is initialized from builtin:fl2va or builtin:ref2va and is not updated. Only projection and bias are optimized. Block max relative error is around 8.8e-5 and final layer relative error is around 1.27e-4.</p><p>The ALS version is recommended when you only need one adapter and want the closest possible approximation. The fixed-table version is recommended when multiple LoRA adapters must be combined. Because all fixed-table outputs share the same T, projections can be combined linearly:</p><p>P_combined = P_base + sum(strength_j <em>(P_j - P_base))<br />b_combined = b_base + sum(strength_j </em>(b_j - b_base))<br />T_combined = T_common</p><p>Both versions are complete pruned LoRAs and can be placed directly into ComfyUI/models/loras.</p><h1 id="how-to-use">How to Use</h1><p>Place the LoRA into:</p><p>ComfyUI/models/loras</p><p>Then load it with the official LoraLoader, LoraLoaderModelOnly, or the MiniMax H3 LoRA Loader node from ComfyUI-MiniMaxH3.</p><p>Support for the complete pruned LoRA format in ComfyUI is proposed here(DOWNLOAD IT FIRST):</p><p><a target="_blank" rel="ugc" href="https://github.com/Comfy-Org/ComfyUI/pull/15353">https://github.com/Comfy-Org/ComfyUI/pull/15353</a></p><p>Support is also included in ComfyUI-MiniMaxH3 version 1.2.0 and newer(with audio shift fixed):</p><p><a target="_blank" rel="ugc" href="https://github.com/xiaolibai-sys/ComfyUI-MiniMaxH3">https://github.com/xiaolibai-sys/ComfyUI-MiniMaxH3</a></p><p>The table builder repository is available here:</p><p><a target="_blank" rel="ugc" href="https://github.com/xiaolibai-sys/MiniMax-H3-Pruned-Lora-Adapter">https://github.com/xiaolibai-sys/MiniMax-H3-Pruned-Lora-Adapter</a></p><h1 id="credits">Credits</h1><p>Original acceleration LoRA by larryvrh.</p><p>Official ComfyUI integration PR:</p><p><a target="_blank" rel="ugc" href="https://github.com/Comfy-Org/ComfyUI/pull/15353">https://github.com/Comfy-Org/ComfyUI/pull/15353</a></p><p>ComfyUI node package:</p><p><a target="_blank" rel="ugc" href="https://github.com/xiaolibai-sys/ComfyUI-MiniMaxH3">https://github.com/xiaolibai-sys/ComfyUI-MiniMaxH3</a></p>

Files with SHA256: c7ddd5089a89c495e0d9f3fd6d969032ee4b7e0b263afde3f360127cb7cae3bb