V2 is here!

Differences:

• It is a bit weaker,

• follows the prompt better,

• Less bleeding to the other characters

Inference

It seems, the model likes long descriptive prompts. Look at the attached videos for prompt examples. Based on my tests if you use short prompts, the lora effect is weak.

For inference I recommend:

• ComfyUI native nodes

• Kijai's ComfyUI wrapper nodes

Trigger words: csetiarcane, Nfj1nx, blue hair

(If it isn't enough so you don't get the style/character, I recommend adding "animation style" to you prompt. It can help providing the style but in some cases the result will be too cartoony)

Strength: 0.9-1.0

Trainig details (v1):

I tried a method where I first trained the LoRA on images only and then after a certain number of steps continued training only on videos.

• LR: 2e-5

• Optimizer: adamw_optimi

• epochs: 15 (image) + 7 (video)

• steps: 2550 (image) + 2520 (video)

• dataset: 33 images and 35 videos

• repeats: 10

• rank: 128

• batch size: 2

• gradient accumulation steps: 1

For training I used the Diffusion-Pipe repo.

Acknowledgement

Special thanks to Kijai for the tireless and excelent work done for the community to enable us to use these solutions as soon as possible, as well, as to Comfyanonymous for implementing them so quickly with such a good quality natively into ComfyUI, and also to TDRussel for making LoRA training available to the community so quickly.