This is the Wan version, same dataset. You prompt the same. It works best at 0.5-1.0 strength. It's not that great at unnatural hair colors, and it occasionally shows a watermark so when I redo this I'll remove all the watermarks from the dataset first.
This uses the 14b t2v, and I have a 14b i2v version I'm testing for extending these videos from the last frame.
Any input on ways to improve it or things I should add would be appreciated.
Description
Version 1
FAQ
Comments (24)
Looks like a great Lora but would be nice to have an example prompt.
I kept the metadata on the videos so I assumed that civitai would automatically put the prompts on the videos. Later when I get home from work I'll add a couple.
@asdrabael Thanks bro!
pose is good, the motion is hit or miss tho
so many bouncy loras :D
Yeah, it was a first try. I think if I removed the images and used longer video clips I can get the motion better for v2.
Is model mode biased for african men? All examples are african guys
A little bit. My training videos came from realbooru and I grabbed the first 10 that fit the poses and when I was prepping the files I realized 6 of the 10 had African men. You can get other colors but without prompting the mans race it leans towards that.
@asdrabael can you please add more variety of man and women, having biased model by default is not cool imo
@Rating_Agent it's literally my first try. I know it has issues it needs work on like the motion is spotty too. I'm going to update it when I get the time. I just wasn't paying attention to races since it's not something I really look at and I was more focused on the correct pose and motion. Just prompt it with "white man" or whatever you wantin the meantime.
@asdrabael It's a great first try. I find that when I'm training describing everything but the motion is the way to go unless there is some sort of variety to the motion. People tend to forget to describe faces and ethnicities, which can lead to situations like this and same face syndrome.
@playnproto266 I corrected it with v2 I think. I mixed it up more and the motion came out better. It still needs a little work but I'll try not describing the motion
@asdrabael We're all learning as we go! Thank you for your contribution to the community!
Your lora is great! , can I have a footjob with lora, thank you very much!
Is there an example workflow to run this? I tried to run it on Comfyui and it says "lora key not loaded: lora_unet_up_blocks_3_attentions_2_transformer_blocks_0_ff_net_2.lora_up.weight", the generations look nothing like the lora is supposed to do.
I have no idea why it's doing that. Are you using the hunyuan wrapper nodes?
Same issue. Using Hunyuan FastVideo 720 fp8. Using comfyui native hunyuan nodes.
What clip models do you load in the DualClipLoader?
same...on fast hunyuan gguf...no idea how to solve it..
@lost_moon I used clip_l because thats what Musubi Tuner species in the repo. I used the other one and it used a lot more vram.
Solution for this issue is the following, with musubi. This lora needs to be converted to work with comfyui properly. After doing so I get no error anymore and the lora is working:
python convert_lora.py --input /pathto/lora.safetensors --output /pathto/lora_output/converted_lora.safetensors --target other
@lost_moon I'm using the python_embedded within ComfyUI. When i run the command you mentioned i get this error:
Traceback (most recent call last):
File "D:\ComfyUI_windows_portable\musubi-tuner\convert_lora.py", line 6, in <module>
from utils import model_utils
ImportError: cannot import name 'model_utils' from 'utils' (D:\ComfyUI_windows_portable\ComfyUI\utils\__init__.py)
Any idea what's wrong?
@Melty1989 i don't know what python_embedded in comfyui is. You got to download and install requirements of musubi. Then I open a cmd window, (maybe optional but I did then: active the venv), then run the two lines at once. Double check your paths and backslashes.
@lost_moon So that's basically the bundled python version that comes in with it. I did install pip install all of the requirements.
You need to convert lora to adapt to comfyui before you can use it.https://civitai.com/articles/10335