V2 Beta - full control sparse (advanced)

reupload to fix ip adapter bg remove switch

WARNING - this is not an easy flow to follow atm, i made this to see how far i could take sparse controls and to test new ideas for my main videos workflow. There are many options that are not fully explained and as such i recommend you use v1.1 until i can simplify many of this. This beta is fully functional as far as i know, but can go wrong in many places if you are not familiar with all the things i am doing in it. I have added many notes but i do not feel it will be enough to explain properly. To use most of the advanced masking i would recommend 12gb vram, but basic use should be usable with 6 if not masking.

Use at own risk :p i am working on simplfying it for a release. I will be honest i still dont understand why sparse masking works in ip adapter...

- Changed loader to load from any folder

- added saving to save to a folder inside output folder called sparsevids/yourvideoname

- Added 3 optional subject mask groups

- added save/load masks to reload previous (this can get confusing with sparse and skip everys)

- most previews can now save to predetermined folder.

- added 2 optional object mask tint/recolor groups

- added cropping/padding/fliping original video

- added Change FPS/Frames options

- added replace IP ref background

- added presize and after sizing for masking

- added Loader for upscale to skip the first step and load in videos saved from the base previously.

- added so many little things

- added Regional Conditioning for multiple subjects

-- Ip ref image for any mask

-- OPTIONAL prompt for each mask.

included png with 2 masked subject included files.

V1.1

- Changed the way tinting works, now cuts/pastes to properly tint objects.

- Added model options

- added ability to increase FPS of the generation with the same sparse controls. This allows you to use source videos at 4-10 fps and generate the output at double/triple that.

- changed upscale masking to use the upscaled video, it changed nothing but it confused people.

Tips for use

You dont need masks... at ALL.. They are only there to use if you want. Most videos should work strait out with no masks. Subject masks are needed for background removal, and tinting and static RGB can use the object masks. None of that is needed to generate.

Reborn Sparse to video

This is a remake of the sparse video flow. Nothing taken from the old, as I could not get it working properly. Brand new.

Features

• IP adapter (optional)

• LCM (optional)

• Clothing masking and colour tints.

• background replacement

• Sam 2 masking with explanation how to mask things.

• 2 example videos included with workflow PNGs

  1. flashing girl.

  2. girl get out of pool.

Many notes in a step by step explanation, steps 1 through 13 labelled and explained.

Enjoy :)

Reborn - New flow - reuploaded to fix 2 small errors

fixed - scribble grabbing lineart properly, and set second keyframe bypass fixed

Features, 2 image interpolation

background replace

auto prompt scheduling

ip adapter

animate diff

LCM

This was made because the old flow no longer worked, This is ONLY for 2 images interpolation. Video to video will come soon.

Notes in the flow, questions, ask :)

Beta 3 - extended

replaced the color of the padding to black to stop coco from detecting nothing.

Added cropping to end

added load video group to just before the upscaling to allow the reloading of previous videos or anything this skips

more minor fixes

Beta 3

Fixed a lot with the first masking and proper background removal skip, it really frustrated me as nothing I rendered was nearly as good as when I first started. Now it's back to proper clarity.

Masking combined for upscales

few other small fixes.

Beta 2 - needs more testing but throwing it out as i think its better atm than the first and it fixes a few bugs.

Added

-3 mask options, coco is great but slow. Can use any of the three mask options at all steps they are needed. Not sure if they help with memory.  Bypass masking groups you are not needing.

If you're smart and know a better masking solution, you can insert your own group to mask off things

-added anime/cartoon mask creation option

-added face numbered preview to the reactor area to find your needed images

-Added lora groups to all model areas

-Added second save option

-Added Cutoff nodes.  - allows better color control stability in general.

-Many more notes - too many notes... I talk too much.

Edit - Updated to include reference videos used for samples in the main download with the workflow

This is a beta of my sparse control workflow for video to video.

Features

Sparse RGB or Scribble (any line art works) control nets

Full subject replacement

Full background replacement (both animated and static)

Face/body enhancements

Full upscale option

Cropping/padding allowing subject placements of subjects within the video.

Notes in the flow

Adding a few of the ref videos i used to the assets

Will add the requirements when I can, too tired tonight. :P