You know what sucks about generating short videos? That's right, everything. They're unwatchable. VACE is kind of awesome. It's great for bridging the gaps. Pingponging can only get you so far. Not that anyone even bothers to use it, when appropriate.
Choose a start frame. Choose an end frame. Hit the button.
You can use images for the frames or you can flip the switch and grab the start/end frames of existing videos. There's a Florence auto prompting group that will let you generate a prompt from either of the endpoints, or an interpolated midpoint. The latter will be blurry mess if the images differ by too much but is useful if they're close. You can add prompting about motion and speed in a separate box that is concatenated with the Florence output if you choose to use it, as it obviously won't supply you with that. If you prompt manually, it can all go in the regular box.
There are upscaling, interpolation, and face restoration groups.
I've written detailed notes for each group explaining how to use it.
If a section is important, there is a note somewhere nearby. I promise.
Unless you want to mix video and image inputs, you shouldn't have to yank any node connections. The only adjustments you really should have to make, as far as the intended output is concerned, is the weight of the LoRa (model/CLIP weights can be tuned independently) and the VACE strength. Pretty nice, compared to the usual nightmare of permutation purgatory. Oh btw this is insanely fast. Sage Attention helps, (you may want to disable that if you're scared of it) but it's still crazy fast without it as well. Takes me maybe 6 minutes for a nice 61 frame transition @ 704x1280, which comes out really nice with a 14B 480p model. You can halve that and upscale it, but it's not nearly as good. 720 model quality doesn't seem to be worth the extra time. Yet.
Still a lot to implement on the automation side but any more boolean switches and it will be too confusing for other people to use it so I'll post it like this. I would like to try an auto 3-6 frame cross dissolve, that will help the color issues that still occur, though not as bad now. And of course there is chaining, which I've seen in other WFs, but they all suck horribly and I never got anywhere with them. We shall see.
This workflow is based on the basic Intro to VACE that can be found in comfyui templates.
The major models are linked, you can figure out the rest.
*Original posted WF had an error in the reActor group that could cause premature halting. I corrected this. I also added a slider to the frame selector so you don't have to do any mental mathematizing. It's set to increment appropriately for this use case. The second image set is always n-2, this is still done automatically underneath.
Description
I removed a bunch of automation stuff that I put in there so you would not get confused by the flow. I realize now that it's probably confusing. Which doesn't exactly help. Bypassing looked like it could cause issues, so I removed a bunch of bypassable groups which were redundant anyway. For example, if you choose Image, you'll get the image loader, regardless of the state of the video loaders. It might actually cause a null value in the boolean to halt it anyway. Face restore can be bypassed with no problems now. Also, I moved the restore group to before the upscaler, since it's set to generate at full resolution anyway and the scaler is just smoothing for the most part. See version 1 for the other way around. Florence is bypassed by default. It's tricky; sometimes it's exactly what you need, other times it's like ape pliers in the gearbox. Sometimes the frames are so different that you'd think it would never work, but it does, other times it'll just jump. That's why you can give Flo start/end frames or an interpolation of them, it can help. But mainly focus on the weights in the causvid LoRa loader. Sometimes it's just a lucky confluence of a magic seed and one stupid little word in your prompt. Sometimes enabling another LoRa or two can make the difference. Play with it, it's really awesome. Check out the original in the comfyui templates menu if you need to patch in the smaller models.
It should go without saying that this is NOT a substitute for editing. It just makes certain edits possible that were not before. I'm still working on the color correction part of the transition. Those nodes need forloops, just like masking, which is why neither are in there yet. Also, there is definitely the possibility of chaining generations, either in stages, or loading batches one by one from a directory, like I've used in my other WFs. Would be nice to automate the color-corrected crossfade between clips. How do to that while keeping track of the originals for compositing corrections... I don't know. Tricky. Well, convoluted and annoying more like. Still, everything that I try in comfy I seems to work to one degree or another. It's pretty great. And there are a billion different ways to do everything. So I'll try at some point.
Anyway, try this out. Take a few of your generations and make some tweens. And post the composites. Please, for the love of god, stop posting two second videos. They suck. Constructive criticism. You're really great, I swear. Imagine what you could do if you applied yourself. I only want the best for you.
FAQ
Looks like we don't have an active mirror for this file right now.
CivArchive is a community-maintained index — we catalog mirrors that volunteers upload to HuggingFace, torrents, and other public hosts. Looks like no one has uploaded a copy of this file yet.
Some files do get recovered over time through contributions. If you're looking for this one, feel free to ask in Discord, or help preserve it if you have a copy.