This project is a node-based implementation for video generation using the Wan2.1 model, with a focus on start and end frame guidance. The source code is a modification of Kijai's nodes code, so for model download and installation instructions, please refer to ComfyUI-WanVideoWrapper. This project specifically adds the functionality of start and end frame guided video generation.
The nodes support Wan2.1 models in both 720P and 480P versions. It is recommended to generate videos with a frame count of 25 or higher, as a lower frame count may affect the consistency of character identity.
Currently, the start and end frame video generation approach is in its early stages. It primarily implements the start and end frame video generation functionality at the code level and does not yet involve model or LoRA fine-tuning, which is planned for future work. Additionally, incorporating end frame guidance in Image-to-Video (I2V) seems to degrade video generation quality, which is another area for future improvement.
Github: raindrop313/ComfyUI-WanVideoStartEndFrames: Start and end frames video generation nodes based on the modified Kijai version Wan2.1 nodes
START FRAME:
END FREAME:
Description
Added start and end frame weight controls for video transitions
Compatible with SLG functionality in KJ's nodes
FAQ
Comments (9)
How much Vram do we need for this? I have 16GB vram and it say Cuda out of memory
It seems that increasing the blockswap and using the fp8 quantization model might solve your problem.
@872409853830 Im currently using Kijai model. Is it different from the normal wan model? Also by fp8 quantization, you mean the Wan2_1-I2V-14B-480P_fp8_e4m3fn model right since it doesnt make a different if I turn the the fp8 setting on. Also block swap seem to still run out of memory but vram_management seem to work although with very bad quality.
After a bit of tinkering, i fixed it. To anyone having this troublem, I fixed it by setting blockswap to 25 and in the load model switch it to main device, fp8 quantized. (Apparently offload device dont use your Vram at all)
Hey!
First of all: Thanks for providing the code and workflow! Really appreciated.
Just a short question: Your Loader is the first I see that gives the choice to use spargeattn.
From your testing: Is there any gain regarding your workflow?
Loader is provided by KJ's nodes. Thanks again to KJ. The new workflow supports the SLG functionality in KJ's nodes, improving the stability of video generation.
For some reason, the generated frames are much darker than the first and last frame.
please i got thi msg
WanVideoSESampler
Empty image embeds must be provided for T2V (Text to Video
Prompt outputs failed validation: WanVideoSESampler: - Return type mismatch between linked nodes: teacache_args, received_type(CACHEARGS) mismatch input_type(TEACACHEARGS)