Hunyuan Video
Kijai marked files only for use with Kijai Nodes You do not need them for Comfy Native
Full Guide to picking the correct file above
Workflow for 8GB Card users
Uncensored llama will work with COMFY Native
Using the Kajai marked models on COMFY native will cause rainbow or black output.
I do not recommend the the FP8 VAE unless you are trying to fit all models into GPU, see the guide for 4090 full GPU launch commands.
Technical details regarding "Uncensored"
The model used for Hunyuan was based on llava-llama-3 8 billion parameter LLM. The Intel vision tuned model was used to refine the tokenized model restoring over 5 million values.
Description
FAQ
Comments (50)
I see this page updated sometimes and i am unclear on what i should be using based on my hardware. Any recommendations as to which model/clip, etc. is ideal ones to use on a 4090 24gb vram? Unclear to me and then what nodes i should look for in workflows to ensure compatibility
I'm on a 3090. Personally, I tend to use FP8 everything (hunyuan 720 distill e4m3fn, llama here), except for clip (some unknown old file I had lying around, just 240mb), and vae at bf16 (tiled) - even with teacache and sage attention, you're still looking at minimum 16gb to near saturation for a decent (say 600x800) 5 second video, so I'd rather not waste that on a few extra bits. I haven't really had much luck in the past with lower quantisations and gguf, but I might give those a shot too should the opportunity present itself.
There are some workflows which lower vram further, but they're virtually all based on kjnodes which I dislike. The vanilla nodes don't require much fiddling - I use rgthree's lora loader, welltop's teacache, bleh's sage attention, and the rest is just standard extras from the sample workflow I copied (video manipulation, pythongosss etc).
Comfy Native Hunyuan Fast coming?
Very valuable Hunyuan tips in this article:
https://civitai.com/articles/9584/tips-hunyuan-the-bomb-you-are-sleeping-on-rn
Also here's the prompt formating Hunyuan is expecting:
1. Short Description: Capturing the main content of the scene.
2. Dense Description: Detailing the scene’s content, which notably includes scene transitions and camera movements that are integrated with the visual content, such as camera follows some subject.
3. Background: Describing the environment in which the subject is situated.
4. Style: Characterizing the style of the video, such as documentary, cinematic, realistic, or sci-fi.
5. Shot Type: Identifying the type of video shot that highlights or emphasizes specific visual content, such as aerial shot, close-up shot, medium shot, or long shot.
6. Lighting: Describing the lighting conditions of the video.
7. Atmosphere: Conveying the atmosphere of the video, such as cozy, tense, or mysterious.
Source line code:
https://github.com/comfyanonymous/ComfyUI/blob/master/comfy/text_encoders/hunyuan_video.py#L41
Is it possible to run this on rtx3080?
I always stuck at 0 steps with ?it\s, no matter how long i wait
I also have a 3080 and had the same issue as you. Try following the steps here - https://blog.comfy.org/p/running-hunyuan-with-8gb-vram-and Worked for me!
I use a 2080ti and it works for me, so yes. BUT, only if you have the right settings. You're not going to get high resolutions unless you're willing to wait an hour for a 3second video unfortunately. Try these tweeks to the default workflow.
1. Use the 720p model or the FP8 model with the "Fast Video" lora.
2. Change the step-count to 12-15 from 20 (I've noticed too much noise if I go lower but I've read you can)
3. Change the video length to around 50.
4. Make sure you're using the FP8 model, clip and vae.
5. Change the resolution to 304x304. (Biggest one.)
Follow these 5 steps and I guarantee you, you'll get a video. Even though the resolution sounds horrible, the videos actually come out very well. It's like a mini HD video. Make sure you put in quality-of-life prompts at the end. Much like you would for images. I've just used 3 at the end of my video prompts and have had great results. They are: [Highest resolution, highest quality, sharp focus]
These settings for me on 2080ti make short videos that aren't bad. They take about 3mins each. This at least allows me to experiment and learn the model until I'm able to upgrade my GPU. You'll most likely get better results with a 3080 over my 2080ti but don't expect much more than I've listed here as both of our cards only have 12GB VRAM and that's by far the biggest bottleneck when making these videos.
Based off your results, you can always experiment with higher resolutions, longer lengths, more steps, or the FP16 models/clips/vae.
(Pro tip, sometimes cmd prompt will get stuck if you're trying something beyond your systems capabilities or have multiple programs running while waiting for the video to render. Open your CMD that's running comfyi and press enter.)
Hope this helps and gl with your videos!
can it use for 8gb vram? rtx 3060
For 2-3 second videos that are low res 360x px
How to avoid slow motion,leneth=73,video output =24fps,the 3sec video will occasionally appear in slow motion. I am not sure if it is due to Lora.
please could i get a workflow for this?
future is looking bright
Hey everyone. I was trying to help someone with lower specs. If this is you, than maybe this is worth reading. I posted what's written below as a reply to someone but I'll put it here in case it helps more people. 8GB vram users may have to lower your settings even more as I have no experience with using this model with such low vram. Adjust accordingly based off your systems specs. Instead of rewriting everything, I'm just going to post the original reply here and take from it what you will based off how it correlates with your system's specs. With that said, here you go and GL!
(This was originally a response to someone asking if they can use it with their 3080 as they were having no luck running it)
[My Reply To Them]
I use a 2080ti and it works for me, so yes. BUT, only if you have the right settings. You're not going to get high resolutions unless you're willing to wait an hour for a 3second video unfortunately. Try these tweeks to the default workflow.
-1. Use the 720p model or the FP8 model with the "Fast Video" lora.
-2. Change the step-count to 12-15 from 20 (I've noticed too much noise if I go lower but I've read you can)
-3. Change the video length to around 50.
-4. Make sure you're using the FP8 model, clip and vae.
-5. Change the resolution to 304x304. (Biggest one.)
Follow these 5 steps and I guarantee you, you'll get a video. Even though the resolution sounds horrible, the videos actually come out very well. It's like a mini HD video. Make sure you put in quality-of-life prompts at the end. Much like you would for images. I've just used 3 at the end of my video prompts and have had great results. They are: [Highest resolution, highest quality, sharp focus]
These settings for me on 2080ti make short videos that aren't bad. They take about 3mins each. This at least allows me to experiment and learn the model until I'm able to upgrade my GPU. You'll most likely get better results with a 3080 over my 2080ti but don't expect much more than I've listed here as both of our cards only have 12GB VRAM and that's by far the biggest bottleneck when making these videos.
Based off your results, you can always experiment with higher resolutions, longer lengths, more steps, or the FP16 models/clips/vae.
(Pro tip, sometimes cmd prompt will get stuck if you're trying something beyond your systems capabilities or have multiple programs running while waiting for the video to render. Open your CMD that's running comfyi and press enter.)
Hope this helps and gl with your videos!
16 gb vram cards and 32 gb ram - what model to use? thank you so much!
I'm wondering the same thing but for the best speeds I think you should use FP8 and if you are unsure about which model then look at which one is being used the most in the gallery. That should give you an idea about which models are preffered.
im gonna try this on my 1050 4gb, with some time and some adjustment maybe gonna work, wish me luck
worked, waited 26 hours for a 6s video
workflow?
workflow?
try the Wan2.1 model. Its a little faster than hunyuan
@yofoton174609 didnt know, thank you, anyways, is a laptop1050so im expecting massive heat and at least 4 hours per generation xD
@goodstrk got one form reddit, how I can upload comfyui workflows here? sorry im new to this, still stuck with forge and automatic1111 (those dont destroy my poor laptop as comfyui does when I try to do something)
@yofoton174609 work on anything with 4gb of vram? im interested as even flux take 2 hours per image + upscale
@dingapriya23 I don't know if comfy will partial load 3GB to leave enough room for calculations, with an 8GB card it partial loads 6.7GB and tiled VAE has to be used to not cause an OOM
@dingapriya23 i found one, but my setup cant handle it lol. thank you though!
@yofoton174609 What's the best currently out for img2video? Cuz I've tried the LTX workflow for img2video, and honestly, it seemed to completely ignore the image, and the output was like a bad 90's shockwave flash animation made by a child.
@Lazman I don't think LTX is so good
@yofoton174609 Yea, lol. AI moves fast, and it's been two months since I wrote that. I found out since that Hunyuan works much better, and I'm hearing good things about WAN, but I haven't tried it myself yet.
how do you use Hunyuan? I don't see much videos on how to use this.
This video will help you out :)
https://youtu.be/0jdFf74WfCQ
I want to run Hunyuan on a runpod with a A40, 48gb VRAM and 50GB RAM. Which model and workflow would you recommend?
I would think still BF16 unless you can convert the llama model to Tensor Float
runpod didnt match my expectations. The pods comfyui are outdated and not updateable or there is no access through http worst case not any connection option turns into ready mode but even then the pod(running)time is charged. Would like to know ur experiences
@sikasolutionsworldwide709 IYou can access it through a browser, wdym?... I personally like it, I don't use it too often since my 3060 TI is good enough for SDXL, so I am mainly going to use it for training, Hunyuan or sometimes even heavy image process. Maybe be only $0.44 an hour but still adds up for long training or video. A40 is jussstt over 3x faster than my 3060 TI.
I've had problems with my pod, like it wouldn't start or suddenly I don't have a GPU, so I found my solution. What I did was just create a network volume(persistent/permanent and 0.07cent/GB per month) to mount to the deployed pod. First Run the Ultimate Comfyui invoke ai template, let it do its work, now you have a permanent comfyui installation on your network volume, now all you need to run is the Run Pytorch 2.4 template. But you'll need to make a script to get the requirements installed again on the venv and activate it, then after that you only need a script to just activate venv and run main.py with --listen. Now you update the Comfyui get all your models etc. and you can deploy and terminate pods when you want (you will since you can't "stop" a pod with network volume mounted, but Run Pytorch starts fast, so you can terminate when you are not using it, and you have everything you need on the network volume, just be sure to add the 8188 port when deploying or edit it later.
@Felldude BF16 it is. What workflow would you recommend?
@rgoobty69 I don't have the PC to run it but Iterative up-scaling was thought to be a good one, start a low res then video to video upscale it to 960 or the highest you can, then a final pass with a 2x or 4x upscaler
@rgoobty69 thx for going so deep into details. Runpod was introduced to me via a small tutorial. In this tutorial the established pod has had the option to run comfyui through a browser this option was available by clicking the connect button other options for example were Jupiter notebook and terminal access. The option U presented seems to be the best but I need time to follow the steps as I don't have the know how yet. So for Hunyuan I am still sticking with my 4070. I am still waiting for runpod support reply.
我出现了提示 HyVideoModelLoader
'img_in.proj.weight 我该怎么做
放到c盘试试
I have the same errors
Will it work on 8GB VRAM?
Can this reliably do img2video?
不知道下载到什么地方怎么办
How to add this to my existing Hunyuan workflow