Hunyuan Video
Kijai marked files only for use with Kijai Nodes You do not need them for Comfy Native
Full Guide to picking the correct file above
Workflow for 8GB Card users
Uncensored llama will work with COMFY Native
Using the Kajai marked models on COMFY native will cause rainbow or black output.
I do not recommend the the FP8 VAE unless you are trying to fit all models into GPU, see the guide for 4090 full GPU launch commands.
Technical details regarding "Uncensored"
The model used for Hunyuan was based on llava-llama-3 8 billion parameter LLM. The Intel vision tuned model was used to refine the tokenized model restoring over 5 million values.
Description
FAQ
Comments (37)
So are the diffusion models linked here 'uncensored' or the same as the default ones from tencent and kijai? Which I thought were already uncensored... So why link them in this if it's just an uncensored clip model? I'm confused.
The TE is a LLM that guides the model, Hunyuan used a unique version of Llava LLama only available in FP16, this model used training from intel that was not directly compatible as a drop in to train the FP32 - The uncensored tag via the LLM was it seemed to guide better results for my prompting style
@Felldude ok.. so there's no need to change the standard difussion models to the ones you're linking here? just the text encoder?
@frosty639 The diffusion model has no change other then conversion to Safetensors
So how can I convert Uncensored_Llama_TE_BF16 to fp8_scaled available to comfyui?
Well I would use the FP32 to quantize to FP8
@Felldude Thanks, comfy.org will convert to fp8 e4m3 with “multiply the weight be 448 and then store the value (1/448) for unscaling it during inference in the checkpoint”, since I don't know python well, the conversion on my own is always unsuccessful and vram is not sufficient ......
Hey everyone! There is a lot of confusion around this checkpoint so I thought I'd make a quick guide to help everyone NOT have the experience I had yesterday when trying to learn this model for the first time. To start off with Uncensored_Llama_TE is NOT a checkpoint. I know. It's confusing... Here's a quick guide to help you know what to download assuming you're already familiar with the workflow.
Your workflow is going to have:
Load Diffusion Model.
Dual Clip Loader.
Vae.
All these files are here.
The Uncensored_Llama versions are actually one of the 2 Clips you need to load!!!! Hope this clarifies any confusion.
Use the models at the top in this order based on your system.
-Load Diffusion Model: Comfy 16 or 8 (most likely 8)
- Dual Clip Loader:
1.One of the Vision_Clip-L models based on your system
2. Uncensored_Llama_TE (I know! This is where it goes. The dev recommends 32 if your system can handle it. Read his Full Guide in comments)
- VAE: We all know VAE by now. Pick one.
The Kijai models on the far right are for non-native, requiring special nodes.
Hope this helps you better understand the list up top and what to pick! Clarifying any confusion for new comers! Even though they all show up under "checkpoints", they're not all checkpoint files. Only the Comfy files are the checkpoints. Have fun!
wow.. that got out of hand fast!. Brick killed a guy.. TY!
Is there a way to convert this to huggingface differs format so it can be used in framepack? The original text-ncoder model isn't in this single safetensors format and I haven't gotten this to load at all.
@henk717g I have not used framepack but if it has Llava Llama config defined it should be able to use the safetensors files. If it is relying on the config from hugging face you could modify the script to point to the config on huggingface but load the safetensors file localy
@Felldude My one has an offline copy of hunyuanvideo-community/HunyuanVideo at main
I have tried naming it model.safetensors but for some reason I can't get it to load if I place it in the text_encoder folder instead of the files already there.
Update: Figured it out. I had to use this conversion script:
```
from safetensors.torch import load_file
from transformers import LlamaModel, LlamaConfig
import torch
# Load the safetensors file
file_path = "hunyuanVideoSafetensors_uncensoredLlamaTE.safetensors"
state_dict = load_file(file_path)
# Initialize the LlamaModel configuration
config = LlamaConfig.from_pretrained("text_encoder_")
# Create the model and load the weights
model = LlamaModel(config)
model.load_state_dict(state_dict, strict=False) # Allow partial loading if necessary
model.to(dtype=torch.float16)
# Save the model in Hugging Face format
save_path = "text_encoder"
model.save_pretrained(save_path)
print(f"Model exported successfully to {save_path}")
```
is it even worth it ? it seems as if this is alot of work for nothing
yea made it work but i dont see any difference, is it better for more complex prompts? there is no real example of what it really accomplishes.
Do you know if this works with image to video?
Really good stuff
if you get this error:
Missing Node Type
HunyuanVideoLoaderLora
type this in manager to find the custom node:
ComfyUI-HunyuanVideoMultiLora
wasn't showing up in the Missing folder for me
Can this be used with FramePack? e.g., FramePack Studio (one of the UIs for it)
If it allows for local loading of the LLM
NO
@Felldude FRAMEPACK OR FRAMEPACK STUDIO, does not allow any custom models it fetches everything with custom json structures
@caipes1981135 This person is just spreading bad information!
Guide give 404
Does it say civitai.com or .green - Green will give a 404
^^ normal version work thx
Does this work with Image to Video?
yo tambien lo estoy buscado el image to video :(
Has anybody figured out where these trained checkpoint files go in the wan2GP file structure?
I have the same question, I just finished installing Wan2GP locally and not even on their md files I see anything for this. All I could find was only for ComfyUI which I don't want to use
You got two directories for Lora with Wan2GP :
loras_hunyuan/ - Hunyuan Video text2video loras
loras_hunyuan_i2v/ - Hunyuan Video image2video loras
Please provide more than 1 lora method. in this workflow please
right click clone lora loader
Hello it is possible to set up this model in frampack studio f1 which works with hunyuan video thanks for your help ;)
Since it's not a framepack model unfortunately it's not possible.
@colinuOK I take note thank you for your answer
Wrong. the newest FPS will convert and run ANY Hunyuan Video lora!
HI , does this work for like motion Control reference nsfw ? so i upload an image near to my nsfw Motion control , and run this with that to then generate a nsfw video ? based on the motion control?