Hunyuan Video

I suggest you give it a go. No one is going to do your personal testing for you

@harmonicdiffusion I think you misunderstand. I am not asking anyone to do personal testing for me. I am asking for information from the one posting it to explain what it is designed for. If there are other users who have a similar build, they can say what their results were from their own experiences inferencing for themselves.

Engineers post heights of bridges and tunnels before trucks go through. Every ride indicates what height you must be to ride. It's not unfathomable that a model creator give a quick note of what works with what settings.

@Phraxas It's really hard to say because it also depends on if you have sageattention or bitsandbytes installed for example and there is stuff like offloading to the CPU. I don't know how it works but when I load the CLIP model without bitsandbytes quantization all my regular RAM gets taken up and if I run it with, it's only taking around 10GB of my 32GB RAM.

I'd say go for the FP16 and watch your resources carefully. Things get slow if all your dedicated VRAM is used and it starts using your shared VRAM. The trick is to stay at around 90/95% of VRAM usage. FP8 takes up less VRAM than FP16.

I'm using a 3070Ti 8GB laptop GPU and 64GB of RAM, running it with native nodes in ComfyUI. I start ComfyUI with --lowvram and --disable-smart-memory arguments to help offloading on RAM.

I can load the full models (bf16 diffusion model, fp16 llava-llama-3, fp32 VAE), my RAM is about 75% during sampling. I just had to tweak the tile_size and temporal_size in the Vae Decode (tiled) node to avoid OOM.

Since I've installed sage attention 2.0.1 in my python environment (I'm using a custom node to apply it only in the sampler, but you can use --use-sage-attention arg to enable it globally), it takes about 10-12 minutes for 20 steps (~34s/it) at 640x480 121 frames (~5s video). Without sage attention, it's about 20 minutes.

You can use the workflow from the ComfyUI example page: https://comfyanonymous.github.io/ComfyUI_examples/hunyuan_video/

@LG89 Ok, so it sounds like FP16 models should be entirely feasible for 16gb VRAM.. but no matter what it seems like it'll be leaking over into the RAM which means slowish times. 10-12 minutes isn't so bad. I'm sure I'll look back on that statement a few years from now and laugh but it works for now.

@Phraxas 16GB of VRAM is still pretty low, especially with models like Hunyuan; the bf16 model is 25GB, and if you add 15.7GB for llava-llama-3 fp16 and ~1GB for vae fp32 we are talking about ~40GB of memory, it's normal for ComfyUI to offload on RAM (I guess even if you had a 4090).
At least with your VRAM you could reach +10sec video at 640x480 :)

@Felldude Ah, yeah, good point. This is precisely the kind of thing I could have easily not set and not known why it wasn't working.

@Felldude That's why bitsandbytes is so good imo. It reduces RAM usage of the CLIP model by a huge amount for me.

so any luck? I also have 4080 and i can only get it to work with very low res videos like 340x240, it takes about 30-40minutes to generate 1s clip in 720p.

Cant figure how im suppose to get it to work. Even the "fast" models takes ages, in fact fast models take 2-3h for 1s clip.

Is something wrong with my setup or i just dont understand how to configure it?

btw. illustrious, flux etc work normally so hardware is not faulty.

@bitzupa I'm busy with another task that is taking time, but once I'm done with that I will be rewarding myself with play time with Hunyuan.

@Felldude yeah, did clean reinstall everything when i was switching from a1111 to reforge few days ago.

Read about it a little and many people suggested using basic default Hunyuan workflow and magically it works, even base Hunyuan works with no issues in decen res.

When i try to use custom workflows and other models it all works fine until i go above 640x480, whats more funny setting 480x640 pretty much doubles generation speed god knows why :/ but if i try 720p it takes forever or just hangs at some point. I guess its bleeding too much to ram but then again i use 12GB work flow on 16GB gpu and still can't get it to fit.

Testing GGUF now to see if i can fit it all in vram and confirm if its simply due to running out of vram.

EDIT:

yeah so it was overflowing to ram even on 12GB workflows. Dont understand how its supposed to work on 12GB while it just barely fits into 16GB and even that seems to be random, sometimes i can get it to fit along with lora, sometimes not at all awhile using same model/lora. I dont understand it. Anyway as long as it loads model completely into vram it all works great.

@bitzupa any update ? what's best model for 16gb? thanks!!!

@landlord123  well I use hunyuan_video_FastVideo_720_fp8_e4m3fn and basic workflow for fast, as long as it fits in vram it works great. GGU models are not worth it, they do take less space in vram but quality is terrible.

Same. Wasn't it due to the distillation?

@Felldude Beg pardon, I don't follow - how so? You mean structure wise or in output?

@firemanbrakeneck The way promoting works in plain language, compared to chunks of small descriptions between commas.

How, link guide you used please? Xx

@Felldude Hmm, I think I see it, down to some of the layer counts.

Do you think that if the sequence length were turned down to 1, hunyuan might be cross compatible with flux loras (at least not throw errors if not in terms of content)? Or even if not?

At the very least, it does seem to function okay as an image generator with that setting.

@Felldude At least renaming is dead simple, comfy does it on the fly, I wrote a small script to convert a few loras and they seemed to work correctly; diffusers' qkv separation for instance is more of a hassle to handle, still not really sure how to do that.

If that's all it takes, and it actually works, even for just some of the existing loras, it'd be really something.

@Felldude That's clever. Shall it work out of the box by setting model type to flux, or are there other incompatible parts besides the video blocks? Or is that the sort of thing requiring more careful examination?

I disagree wholeheartedly with this, Getting Hunyuan to work has been terrible, and the documentation from people is genuinely all over the fucking place. For flux once i figured the whole Clip_L and T5 shit, everything else fell in line.

@finalfight agreed

@Felldude I guess somebody gave it a shot.

https://civitai.com/models/1258103/hunyuan-flux-sim-schnell-unchained