a distillation lora for anima, trained on ~100 images at 1024x generated by the model itself using 40 steps cfg = 4 er_sde from diverse captions
baked-in negative prompt: worst quality, low quality, score_1, score_2, score_3, blurry, jpeg artifacts, sepia
may or may not be underbaked/overbaked
works best with cfg = 1 and 8 steps for heun/2s samplers, 16 steps for 1s or multistep samplers
the training code is a modification of diffusion-pipe for RL/distillation + custom comfyui nodes for data generation
TODO: get a larger more diverse RL dataset and filter it for artifacts
Description
epoch 50 on 100 images, continued from epoch 15 but with grad accum 2
FAQ
Comments (5)
really amazing work. so far er_sde and simple/normal have worked best with 16 steps and 1 CFG.
it also works with different styles (as in it doesn't have a style lock) but it is slightly more prone to deformations (not hating, just in case you are working on a V2)
uploaded v2, this time i filtered the training set for deformations so it should work better
doing the god's work. its very much a improvement over the v1.
its also a very good improvement on character identities. v1 mixed them up a lot.
amazing work. one thing i found that it sometimes does "x-ray" shots without prompting for it. (which isn't particularly a bad thing, some might call it a improvement, and its like 1 in 4 shots so not that annoying either)
as for the anatomy, its much better than v1, but still struggles when there are 2 or more people, altough for single person images, anatomy issues are pretty much gone in v2. (again, not hating, just "constructive criticism")
@Hycros ep100 does sometimes give me overfit vibes so i may upload ep80 of v2. As for v3 I'll expand the dataset maybe around 2x (if i don't go insane while filtering images) and rank up to 64.
Additionaly I would like to implement ArcFlow training and sampling in comfyui but it's pretty difficult 🫠
@levzzz another thing i would like to point out is that in most images, details feels washed (slightly). kinda like someone adding negative weights on a detailer lora.
i have been doing CFG 1.5 and 8-10 steps with res_multistep+simple. this has mostly fixed everything and now i get almost same quality (80% of the gens) as the base model in 1/3rd of the gen time.
i just wanna say thnak you for this amazing work. may both side of your pillow be forever cold😜. looking forward to what you do with it in future.