UPDATED Wan-2.2-HighNoise
ArcShot-Wan2.2-I2V-A14B. It is trained using film material, but I found both anime and comic can used it to generate a good result by adding prompts like anime style or comic style. And it's trained with direction caption , e.g. camera moving from left to right/right to left around the subject. However, it seems not that work on inferencing. You can try. The result is quite good during my testing. You can use MediaInfo to check the generation info in the video.If you like my job , you can donate me here: https://ko-fi.com/ghostshell. Just paying the electricity bill costs me 1,000 yuan a month...This LoRA is not for commercial use. Please lease dm me, If you have any business cooperation inquiries.This LoRA is trained for Wan2.2-high noise. Please make sure add it to the high noise Wan2.2 model.
Suggest Strength : 0.8
Prompts: arc shot, camera moving from left to right around the subject,
Wan-2.1
ArcShot-Wan2.1-I2V-720P. It is trained using film material, but I found both anime and comic can used it to generate a good result by adding prompts like anime style or comic style. And the background will switch to forest usually, you can add some prompts like indoor background to keep the background not changed. Suggest Strength : 0.9-1.0
I uploaded an article of my experience of training Wan2.1 LoRA (Both I2V & T2V): My Personally Training Experience of WAN:Starting with Data.
Description
FAQ
Comments (16)
How to control camera to rotate left or right?
I prompt the direction when I am training, for example, Arc Shot from right to left. However, it works not well when inferencing, you can try, and also try different random seeds, I think you will get the right result
I tried everything, the camera remains static. Do you have prompt samples?
The prompt is: Arc Shot. This is I2V not T2V, the samples in the showcase are all generated by inputing the image and a simple prompt:Arc Shot
Cant get it move, lightx2v just completely cancels it out, even at a strength of 10 on both high and low. Sucks. Bcs lightx2v kills all camera motion in general trained into the model. Uggh, cant wait for the lightx2v lads to fix it, if they even do.
Try PASeer's Accell. I'm using it with other LoRA like this. And I'm getting movement I2Ving weird sci fi landscapes I genned in Flux. Stuff that at first I couldn't get to move at all.
Here's a clue- if 20+ iterations were not needed by the original model, the creators would not specify this in the first place. Gofaster LoRAs are a massive compromise, and mathematically MUST brain-damage the underlying model. That they work as well as they do is a miracle. But A-B testing will show which aspects of the original model they clearly damage.
Anyway the answer is simply, especially with the MoE Wan2.2. In this case just use LightX on low noise. It is the high noise iterations that lay down the camera motion!
@blobby99 The three ksamplers method it's the way to go for best compromise between motion quality/speed of generation.
@sirdrak whats the 3 ksamplers method and does it work with wrapper nodes?
@parallelepipedonlink?
@girlsthatdontexist It's a workflow that divides the High Noise model phase in two advanced ksamplers, using the lightning high Noise lora only in the second ksampler. In the first ksampler, then, the CFG > 1. So this way you have 3 advanced ksamplers, 2 for High Noise model and one for the Low Noise model. In my case, i use 6-7 total steps, 2 for each ksampler or 3 for the last one.
@sirdrak any chance you could share a WF with this setup? sounds interesting for sure
I've tested extremely complex camera movements and hard-to-describe actions. Even a 6-step run without acceleration isn't guaranteed to complete successfully. However, with a 10-step run, the adherence to camera instructions and the overall effect are greatly enhanced. Different samplers also have a significant impact on the results. For text-to-video, I recommend using the 'euler_ancestral_sde' (er_sde) sampler. Among all the parameters I've tested, this one has demonstrated outstanding camera movement effects in text-to-video generation.
@girlsthatdontexist Setup and workflows mentioned here https://huggingface.co/lightx2v/Wan2.2-Lightning/discussions/20#68b10cc0a459d1e003fae631
Works fine for me. I have lightx2v at 2.75 on the high model, using euler with 6 steps total. There must be a problem somewhere else in your workflow. ¯_(ツ)_/¯
Finally, the last two methods i'm using now, fixed all the motion problems... First, the new lightning loras for Wan 2.2 works really well, and you just have to use them the normal way (lora strength 1 for high noise and low noise, 4-6 total steps, CFG 1, Euler/simple) and with shift 5.
And the second method, that gave me a really good results, seen in reddit some time ago:
- In High Noise, use the old wan 2.2 lightning lora at 0.75 strength AND the lightx2v i2v Rank64 Wan 2.1 lora (even in t2v) at 2.20 strength
- In Low Noise, use the old wan 2.2 lightning lora at 1 AND the same lightx2v i2v Wan 2.1 lora at 0.68
- In both Advanced Ksamplers, use 4-6 total steps, sampler Euler and scheduler Beta57 (you need the RES4LYF custom nodes installed in ComfyUI for this to appear), CFG 1
- Use shift 15.87 (the shift to use with the Beta57 scheduler recommended by its authors)
Both methods are slightly different visually but I like the results with both equally, and both gives good motion without issues, so I use them interchangeably. With this it is no longer necessary to use the three-ksamplers method.