Qwen for MinimaxH3 (QP)
NOTE: If you downloaded the day one version I had quantized the wrong files resulting in a lobotimized LLM
Requirements:
Update CUDA to 13.4.2
Update Pytorch to 13.2
Update at minimum comfy-aimdo, comfy-kitchen
Basic workflow shows how to use both first frame text guided and first frame last frame with comfy kitchen backend node to speed up generation by 60% (This prevents pyattention fallback which is slow)
pytorch version: 2.13.0+cu132
xformers version: 0.0.35
Using xformers attention
ComfyUI version: 0.35.0
comfy-aimdo version: 0.5.5
comfy-kitchen version: 0.2.34
Consider --disable-dynamic-vram if you are having OOM issues after a few generation or crash when trying to use comfy kitchen vs pyattention
QP (Quantization Prediction)
QP is theoretically improving 40-50% of the blocks on the trailing 16 Matnitsa bits.
For the other 50-60% that it does not improve it did not degrade them more then what they would have been rounded to in the first place.
This was tested on Full FP32 trainings such as T5.
Description
FAQ
Comments (11)
could you give an ELI5 explanation of what these improve/fix from the non-QP versions? thanks
Well comfy did not provide a MXFP8, so their is that.
QP is theoretically improving 40-50% of the blocks, on the other 50-60% that it does not improve it doesn't degrade them more then what they would have been rounded to in the first place.
Just so I understand, the first two are text encoders meant for people with $4000 GPUs?
Currently use a nvfp4 one at the moment on a 3080 which seems fine, I assume these would OOM.
So the INT8 and MXFP8 will both run and autocast to BF16 far more quickly then a full FP16 or BF16 model, the issue is not so much VRAM for them but rather total RAM of your PC which would need to be 32GB to run smoothly
following up on @Felldude 's reply, if dynamic vram is enabled in comfyui (it should be by default), it depends on how much system ram you have. you can have the diffuser model loaded in your vram and this text encoder will sit in your system ram with basically zero affect on speed.
nvfp4 is a severe downgrade to mxfp8 or int8, and it matters a lot for the text encoder here. They don't have to run at the same time as the model generation so you can plausibly run them without 32gb of vram. It's going to depend on your system, but you said you have a 3080 so get int8. My advice is just try it and see what happens.
I use --disable-dynamic-vram this is do to forcing comfy kitchen attention, which can speed up IT's per second by 60% if your machine is defaulting to pyattention. I found this to be far more stable as comfy shards the memory up when it is pinned
I have a 3080 with 10GB VRAM and 64 GBs of ram but the speed of the ram isn't great. I have a Minimax H3 workflow on Comfy UI, ran through Stability Matrix with stuff like CK Attention, I'm not even sure if dynamic vram is disabled or not.
I use the hybrid ref2va diffusion model and stuff like the nvfp4 encoder. I'm doing 5/8/10 second tests with specific prompts against a node from H3 Optimization pack, called the H3 Sparse Attention, which sacrifices quality for speed, and seeing how fast they can get and how the quality drops with the same seed (with and without it).
I'll probably download these and see what happens, ultimately. The worse thing is really their filesize, my minimax package is already at 40GBs.
Thank you so much for your work
thanks