This workflow is my take on organizing and building an I2V model running against a "smaller" graphics card (specifically 12 GB). I use Q5 GGUF models, though I have also tested against smaller Q4 and Q3 - each having a slight decrease to prompt adherence and quality, but still usable. This includes using the lightspeed lora and notes on where all models / loras can be downloaded from.
I suggest if you have limited RAM or VRAM, to run ComfyUI with the parameter: --cache-none
While this will mean multiple batches against the same video will be slower, you get a much more consistent overall generation speed of your videos (3-4 minutes for 5-6 seconds on moderate home PC configurations).
This also includes using Florence2 (LLM) for image detection and auto-prompt assistance. You only need to add your action to the manual prompt (if desired).
There are a lot of nodes I have seen in various workflows ... but in Wan 2.2 I2V, at least, they tend to have no effect and only increase overhead.
I run my videos typically in 480p @ 480 x 832, and this workflow then upscales by 2x to 960 x 1664.
Custom Nodes Used:
ComfyUI-GGUF (https://github.com/city96/ComfyUI-GGUF)
rgthree-comfy (https://github.com/rgthree/rgthree-comfy)
ComfyUI-KJNodes (https://github.com/kijai/ComfyUI-KJNodes)
ComfyUI-Florence2 (https://github.com/kijai/ComfyUI-Florence2)
ComfyUI-VideoHelperSuite (https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite)
WAS Node Suite (https://github.com/WASasquatch/was-node-suite-comfyui)
Description
Update to latest ComfyUI and dependencies
FAQ
Comments (12)
This is a really solid workflow.
I’m honestly impressed by how consistent the facial features remain across generations.
I’m not entirely sure what the key factor is here, but whatever you’re doing clearly works very well.
Great job, and thank you for sharing this workflow!
I added other loras in the power loader but they are not working unfortunately. Can someone help me on this?
Loras require being for the right model (I2V, WAN2.0 in the case of this workflow) and you need both the high and low lora to have any notable impact - using just high or low can cause artifacts or bad rendering, and using invalid loras will give you completely weird results. With all of this in mind, in order to help more than a generic request, you will have to give more details (what lora, how are you loading the high & low, and what weights are you using).
I was having trouble with getting 2.2 Loras working reliably and downloaded a toolkit called IAMCCS- NODES.
the Lora Loader stack in that toolkit solved some issues for me and is the most reliable lora stack for 2.2 workflows i've found.
to install it:
google it to find the Github page for it online - right click the URL at the top of the page and choose "copy as clean URL", then open custom nodes manager in comfy and hit the "install Via Git URL" button at the bottom and add that link. it should install from there, just restart comfy and you're good to go.
that node will now be in your nodes menu.
an easy way to find it, is to drag a thread off your main wan model node. then right click and select "search" this will filter all your nodes for only ones that plug into the checkpoint/GGUF loader node. then type IAMCC or Lora stack into that search and there it will be.
WARNING: it also conflicted with a few of my other nodes and I had to spend 10-15 mins updating a few of my other nodes to make it all work again. so be aware of that, use at your own risk.
but if you really are having trouble getting 2.2 loras to Load and play nice with your 2.2 checkpoints, thats an option that worked for me.
Hi ! Sorry i'm a newbie with AI generation. Can you help me ? I fail to use your workflow and don't know why... I was pretty sure that all the custom nodes are installed but when I try to run your workflow, 3 nodes became red. You can see a screenshot here :
What problem could it be ?
You have to download the wan and LLM files (URL should be in the notes for each). Once downloaded, make sure to select the right files (I use Q5 in mine for wan gguf, but choose whichever you can use for your hardware) - the LLM is in the sub-group, go in there to select it.
The folders to download & put the files into are also in the notes for each.
@logos011 Thank you !
For me just generating the same original picture with focus an defocus video... 4080 super + 32GB RAM DDR4. Any clue how to sort this out??
I'm not sure I've had that problem before - if you're using the models in the workflow, check that you have the high speed lora enabled and appropriately put into each section (high for high, low for low), and disable any other loras. Even if I run it "base" without any descriptions and let the LLM describe the picture for me, I always get wind or hair moving or some other small effects.
This really isn't a huge divergence from the built-in templates for WAN in ComfyUI - just made to be a bit easier to use & manage by doing things like resizing pictures, grouping loras, and auto-adding descriptions with the LLM. So you could also try the "base" WAN workflows to see if that one produces the same issue for you without any of my adjustments on it.
for some reason having trouble getting the florence node to work
I don't have an option to load loras... How do you load loras in the power lora node? see me screen shot. https://imgur.com/SpEucmv
that is definitely not normal! try adding the base ComfyUI Lora - sadly it only does one at a time, but you can chain as many as you want to make it work.