I've made an open source tool that makes it easy to make your own GGUF's with very low memory requirements, here: https://github.com/qskousen/ggufy I just tested it with this model, found a bug, and fixed it. It's working great now.

@ferretduck Thank you!

@ferretduck I'm not very good at this, but do you plan to add support for NVFP4 converting?

@tima2431 I added NVFP4 support 2 days ago, if you try it out please let me know how it went

this is awesome! I just couldn't find any information about this on github before. I'll definitely try it in the next few days and let you know! do you think I'll be able to do this on an RTX 5060 or 5070? sry if my english bad I used a translator to wrote this

@ferretduck holy shit, just wanted to say it worked perfectly iam absolutely blown away by this tool. Please never abandon this project, it is 100% going to blow up soon

i successfully converted the BF16 miracleIn 1.0 model to NVFP4 in under 2 minutes. My specs are an RTX 5060 + i9 14900kf. The converted model runs flawlessly in SD WebUI Forge Neo without any issues

i will keep testing it on other models and will definitely report back if I find any bugs. You are an absolute lifesaver to the low-VRAM community, thank you so much for your work!!!! <3 <3

@tima2431 thank you! and yeah it doesn't use the video card at all, you don't even need a video card to run it

i tried to do conversion of 1.0_fp8 to gguf q4_k , but am getting error , gguf version is old or not compatible.. please help me how to fix it..

@ajayhirur it looks like there's a bug converting from fp8 perhaps, at least for this model. i'm looking into it. in the meantime you should be able to convert the bf16, i've done that one successfully before

@ajayhirur Found the issue and made a fix, but haven't made a release for it yet.

@ferretduck hey i download 2.0_fp8 and converted to q4 gguf and its working fine on my rtx 3060 with 16gb ddr4 ram.. can you add indian and african races colors .. do i still have to download 2.0_bf16 and convert it to gguf or will it be same as 2.0 fp8 ?thanks again.

@6tZ am trying normal prompt and also use chatgpt, is there a better way to prompt easier that takes nsfw.. i wanted to try qwen 3.5 4b for prompt output, may be its just for sfw..thanks again.. the natural hairy bush.. still needs some working.. coz its still trimmed i feel..i tried in prompt large natural and natural both.. tried to replace with big natural.. nothing works to make it natural hairy bush..Please let me know if i did something wrong.. i tried dark black textured areolas nipples.. and dark black colored pussy , to get colors right..still its giving me dark brown color though..

@ferretduck hi!! ive noticed an issue related to image quality that seems to stem from the conversion process, at least with the z_image_turbo model family (though it might affect others as well)

i ran multiple conversions of these models from FP16 -> NVFP4 and FP8 -> NVFP4. I then compared the outputs with the original FP8 model and a NVFP4 version downloaded directly from civitai. I noticed that the models converted locally through your script lose sharpness and become visually "rough" or artifacted

its hard to describe the exact visual degradation, but it looks like a heavily compressed photo from 2009, or similar to what happens in older sdxl versions when you set the step count way too high. the version nvfp4 from Civitai do not have this issue. if needed, i can send you the comparison test images so you can take a look. Let me know :) !

@tima2431 can you direct me to the nvfp4 one you downloaded from civit? that may give me a clue as to the difference

@ferretduck https://civitai.red/models/2583032/zimageturbonvfp4 i used this version for comparison. its not perfect but the comparison showed that it doesnt produce the artifacts i mentioned. if its unclear, i can provide you with a comparison archive with photo

@tima2431 i personally don't see much of a difference but maybe my eye is bad haha. looking at the tensors in that nvfp4 file and one from ggufy, ggufy does convert many more tensors into nvfp4 than are converted in that file. it results in a smaller file but understandable that would come with tradeoffs.

you could export a template of that nvfp4 and use it when converting, and then ggufy will copy the same tensor quantizations as that file.

@ajayhirur the fix for the issue you had is in the latest release https://github.com/qskousen/ggufy/releases/tag/v0.6.4

@ferretduck I encountered a problem: when I used the tool you provided to quantize the BF16 checkpoint for NVFP4, the results were all 4.82GB of checkpoint files, while the files published by authors who perform NVFP4 quantization themselves are usually 5.23GB.

Additionally, I quantized nvfp4 on both this author's Miraclein NSFW and KleiNova NSFW and got two exact identical hash values, which is quite odd.

By the way, my English isn't good. This passage was translated using software. If my words don't get the message, please allow me to apologize.

@6tZ previously downloaded the 3.0 BF16 version of two checkpoints. After using your recommended tool for NVFP4 quantization, the hash values displayed in ForgeNEO were the same. I'll download it again.

However, I still don't understand why the file I quantized with the tool is about 500M smaller than the file you uploaded ;)

@sar090909 ggufy's quantization settings are more aggressive than what the author used, that is all. Not all the tensors are converted to NVFP4 typically, and ggufy converted more of them by default. If you want to match the quantization settings of the original, you should be able to load the original in ggufy, export a template, and use that template to convert.

I've tried both qwen3_4b and qwen3_8b, but I'm not having any luck either...

@6tZ @managorath337 I am running Forge-Neo without issues using qwen3_8b qwen_3_8b_fp8mixed.safetensors and for VAE FLUX.2-small-decoder

@6tZ @managorath337 Also make sure you're on the correct preset in Forge-Neo, that goes without saying. It makes huge difference.

@JFTC are you using fp8 or nvfp4? I need to know which ones for nvfp4 because I think fp8 is too big for my computer to handle

@managorath337 Hey mate, tested both fp8 and nvfp4, work well with the text and VAE I linked

@managorath337 Use this guide and bookmark it. Wish devs would just include it with each model release. https://github.com/Haoming02/sd-webui-forge-classic/wiki/Download-Models

@6tZ Depends on your goal as a creator. Seems shortsighted that you wouldn't want to make it as easy as possible for anyone to use the model and therefore gain wider adoption. But I'm sure that's not every creator's goal. Sometimes is just a fun hobby for the creator and then who cares who is actually adopting it.

@6tZ Sorry, one more question: what VAE am I supposed to use for the nvfp4 version?

@managorath337 I am running Forge-Neo and I am using black-forest-labs/FLUX.2-small-decoder

Please update ComfyUI and all nodes to their latest versions.

@hoyjooh170 He's running Forge-Neo.

@JFTC thank you! And what text encoder are you using? And by the way, these are for the nvfp4 version right?

Doesnt work in Invoke with the specific VAE and text encoder

Mac's MPS backend still doesn't support FP8 so you can't use anything, meaning any FP8 model, not just this one.