There's a major post on Reddit across a ton of different hardware PROVING that your video model should live in RAM, not VRAM, and this has no impact on iteration speed. This false information suggesting you need small models to make low VRAM work should end.

What matters is protecting VRAM for the latent frame information, and keeping model data out of VRAM. This can be achieved by launching Comfy with the right arguments. The one potential downside is clip (LLM) being processed by the CPU, which will add a constant overhead to render time, but will not impact iteration speed.

Works. Could be better with lora etc.