Claymation Transformation - Wan2.2 14b I2V/Wan2.1 14b T2V

Where objects melt, stretch, and squish into something entirely new!

This LoRA replicates the surreal, handcrafted look of plasticine/claymation animations, where objects morph into others with stop-motion fluidity. Perfect for whimsical transformations.

Wan2.2 I2V Info

While this doesn't generate videos from scratch, it leverages the power of Wan.2.2 I2V 14b to create really stunning transformations. Just input the first or the last frame.

If you have no idea where to source them, you can use the version for Wan2.1 text2video (read below) in the "text2image" mode when you set the input latent frame count to 1 and then save this 1 resulting frame as a picture.

Prompting info:

The trigger words are 'Claymation transformation' and 'plasticine-transforms'. The best prompts are:

Claymation transformation. The video begins with an [object] standing before a [describe background]. [optional object description]. Then the [object] plasticine-transforms into an [another object, object description].

Simplified:

Claymation transformation. The video begins with an [object]. Then the [object] plasticine-transforms into [another object].

Bonus. You can specify multiple transformations in the same video!

Claymation transformation. The video begins with an [subject] sitting on a table. Then the [subject] [does triggering action]. The [subject] squishes, shrinks and the [subject] then plasticine-transforms into an [object_1], which then plasticine-melts into a [object_2].

This LoRA should be quite beginner-friendly.

Wan2.1 T2V Info

Works great both standalone and with VACE for first-last-frame selection and image to video!

The examples illustrate each mode: 3 x pure text2video, 2 x image2video (set start, set end), 1 x flf2video (interpolation).

The generation resolution is advised to be 480p or close (e.g. 512x512). The trained durations are from 1 to 5 seconds, with the most tested one of 3 seconds. (48 frames)

The t2v training was made using diffusion-pipe for 135 epochs and flow shift of 4.0 on the dataset of metamorphoses from the legendary plasticine animation "Last Year's Snow Was Falling".

Due to the specifics of the train material, you may sometimes encounter a hut-like interior with four-sectional windows in the default mode, so it's encouraged to write the background appearance in the text2video mode manually. Though less often, some parts of the main character's appearance can happen (like the villager hat), in this case add 'lys-man' to the negative prompt.

For text2video I tested with cfg 6.0, shift 5.0. For image2video/flf2video recommended to use with the kijai VACE workflows, standard 1.0 lora weight, 6.0 cfg, 16.0 shift + cfg_zero_star. (see videos meta)