Version 5.0 works with latest Comfy Handy Nodes 1.24
manually update if you dont see it in Manager by searching for "Handy Nodes"
cd <your custom nodes folder>
git clone https://github.com/trashkollector/TKNodes
Up to 4 prompts/images.
***** Works for Singing or Talking *****
How this works, supply an audio file, size does not matter as the workflow will chunk the audio.
It will then use alternating photos to keep video dynamic.
Audio now is chunked when there is silence in the audio for better transtion.
Please get Handy Nodes 1.1.5 from Manager.
Anyone with low VRAM change this. this tells the workflow to try use 12 second chunks.
Audio driven video using alternating speakers. See Podcast demo.
Please make sure to get the newest TK Handy Nodes version 1.1.0
Description
3.1
Had to fix a bug (oops). Now splitting audio when there is silence for better transition. You will need the latest TKNodes
go to custom_nodes folder
FAQ
Comments (8)
This worked brilliantly, feels so natural and movie like, amazing job. Only two things i don't like, if you use a close up shot and a wide shot, and the camera zooms out with the close up shot, it completely reinvents the clothing. Secondly, i'm not conviced about the fade out effect, that is barely used in movie dialogues, feels rather out of place. However the tech itself is very impressive, i hope you can interate with this for maybe a third shot / two character dialogues etc. , maybe add the recent id-lora that clones voices inside the LTX 2,3 workflow? can't wait to see what you come up with, great job!
ID lora is a VRAM hog.. waiting for Kijai to maybe provide some solutions and then will try to implement.
if the video is zooming too much try this: the subject is talking. the camera is locked. If you add too many details it will start to zoom out to fill in details.
@trashkollector175 Interesting, i'll try that thanks, i'll check the new version later today and see how it works, thanks for keep on supporting this.
I switched the GGUF model for the regular diffusion one, and I keep gettting VAE errors, saying "no vae", even though all the proper Vaes are set in the proper nodes... does this not work with the regular ltx2.3 dev diffusion model?
can you send me the exact error message. Also, you are going to need a lot of VRAM for regular diffusion model
@trashkollector175 I see, I wonder if that could be, even though I have a 5090, i rarely see OOM errors.. I'll test more and will come back if I keep getting errors.
HI, thank's for your workflow!
Can i use fp8 model (ltx-2.3-22b-dev_transformer_only_fp8_scaled.safetensors) with this workflow or it's only for low vram ? (my config : rtx 4080 16gb, ram64gb, ssd)