Introduction

This ComfyUI workflow is designed for video style transfers, particularly to turn live-action videos into anime. It is a clean and easy-to-use workflow that allows you to render all sorts of stuff.

Tutorial Video:

Video's made with this workflow:

Installation

To install the workflow, you will need the ComfyUI Manager. Once you have the manager installed, open it and click on "Update All" and then "Install Missing Nodes".

Also add the following ControlNets:

Add them to \ComfyUI\models\controlnet

Restart ComfyUI and you are done!

Usage

Import/Export

To use the workflow, you will need to input an input and output folder, as well as the resolution of your video. You can import image sequences with the blue "Import Image Sequence" node. Make sure the import folder ONLY has your PNG-Sequence inside.

Resolution

The "Resolution" node can be used to set the resolution of your output video.

Prompting

The green "Prompt" nodes are used to describe your original video. You can also use the "Auto Prompter" node to describe the first frame of your video automatically.

The purple "Efficient Loader" node is used to choose a checkpoint. The brown and red nodes are used to choose lora's.

Settings

Here are the best settings for an NVIDIA RTX 4070:

Resolution: 1000x417 (12:5 aspect ratio) or 960x540 (16:9 aspect ratio)
Image Description: "Make it an (anime style) [person] with a [object], [background]."
CR OP Face: 1.20
CR OP Body: 1.20
CR Animeline: 0.50
CR depth: 0.90 (& end percent 0.350)
CR Tile: 0.90 (& end percent 0.250)
CR ip2p: 0.25
Ksampler CFG: 13.5 (anywhere between 10 and 15)
Ksampler denoise: 0.50 (turn it up or down for more or less creative freedom by the AI)
Style Lora: Depends on the lora itself and your style preference
Character Lora: Depends on how much your character is in frame

Conclusion & Tips

Once you have set up your workflow, you can start rendering by clicking on the "Queue Prompt!" button.

Shot Selection

The workflow is excellent at close-up shots. The further a character is from the camera, the less detailed it will be. The render sometimes even falls apart completely if the character isn't close enough.

Consistency

I found that the consistency really depends on the Denoise. The lower it is, the more it will follow the original footage. The higher it is, the more it will deviate from the original video. Try a render with a denoise of 0.20 and then 0.70 as a A/B test. You'll see the difference.

Enjoy!

Credits

Created by itskevea.

I do 1 on 1 coaching sessions, build workflows, do renders, etc.!

Interested? Message me on Discord itskevea OR email [email protected]