v1.0: I think it's time to release v1.0. I've migrated to the krea2 model (never imagined that krea2 would be my next choice...). Fine-tuning the anima model is really fun.
I also uploaded int8 version for ComfyUI hardware int8 (labeled as fp8).
About 8/16 step distillation:
8-step has higher stability. Easy to use and prompt.
16-step (same as previous old versions) has higher diversity and more dynamic details, less sloppy but less stable, more small errors. (I still prefer this one because anime/art model needs a little bit creative and chaotic.)
Comparison images are uploaded as 16-step version showcase images. You can feel the "complexity" of 16-step is noticeable higher than 8-step.
FYI: More distillation = higher stability = lower steps = lower diversity/complexity = more slop-ish. For reference, official anima turbo is 4-step distillation, means highest stability and lowest diversity.
Interesting old version:
b1 v0.39.b: compared to v1.0 16-step, more creative, more dynamic details, but more logical errors. Good for creative, abstract effect, etc.
b1 v0.37.1: 2-stage finetuned, highest aesthetic among all versions, but due to small dataset it has bias. Also has several built-in quality/style trigger words, see update log "b1 v0.32".
EDIT: Why Krea2?
It knows everything. Default quality is off the roof already. (Of course, it's a large 12B commercial model.)
It knows anime/art design, even knows popular characters. (Means it has been trained on a much more creative art dataset. Previous open source models didn't, they mainly for photorealistic)
Official turbo model, 8-step. (FYI: Turbo model is as fast as non-turbo 20-step Anima.)
If you want anime art. All you need is a style/character LoRA.
Life is much easier...
RDBT [Anima]
This is a finetuned model with 10k high aesthetic images paired with natural language captions from LLM. Then distilled to further improve quality and stability. Dataset does not contain any shiny plastic glossy AI image.
It's not overfitted and doesn't have a default style. I use it as a clean starting point to stack more style LoRAs. I can stack whatever I want and get exactly what I stacked.
See this page for update log.
For advanced users: The RDBT model is trained as LoRA natively. See this page for original LoRA.
This model is based on:
prefix with ym: AnimaYume (hf link) (civitai link).
prefix with b,p: Anima pretrained (hf link)
Sharing merges using this model is not allowed. This "restriction" won't affect anyone. It's only aimed at those who steal others' models to sell.
If someone is selling this model as their own, I'm happy to list them here so everyone knows.
Known model thieves: NukeA.I (selling this model behind paywall on tensorart).
I wrote a story about it. Also contains a guide for trainers about "how to bake special trigger word into your model".
Usage:
Settings:
CFG scale: 1~3. This model has been distilled. You can disable CFG (CFG 1) and run the model 2x faster. Cover images are without CFG for demonstration. "RenormCFG" node is recommended
Steps: 16+
Prompt:
Always specify style, or use a style LoRA. Otherwise, you will get random/mixed style. This is a feature, not a bug. This model does not provide overfitted default style.
Quality tags:
It's recommended to omit ALL quality tags. The fine-tuning dataset has higher quality than "masterpiece". Thus they don't have noticeable effects. Omitting those redundant tokens allows LLM to pay more attention on other words
Training settings
All captions are NL from Google Gemini.
Optimizer: adamw, constant lr 0.00002, weight decay 0.1, batch size 16.
LoRA rank/alpha 24.
Timesteps shift 3.
Block 0-2 and adaln linear layers are skipped.
Description
FAQ
Comments (13)
this poster claims to have made the nag work with anima using turbo and 1 cfg but with your model at least , it changes the output too much. Can you take a look at it , maybe we need specific settings for it to work with your model ? https://www.reddit.com/r/StableDiffusion/comments/1sto22j/i_implemented_nag_normalized_attention_guidance/?
I feel like the v27 model is losing its anime style
The further away from V0.24, the less anime it is and the more 3D and 2.5D pollution in data there is :(
Your the stability king for a reason. You have experience on all preview models so I trust yours the most. The official turbo Lora is decent But it isn't as detailed and doesn't follow the prompt exactly unlike yours. Also this is the official NAG that's finally got anima support However, my experience with it has been a little bit mixed:
https://github.com/BigStationW/ComfyUI-NAG-Extended/tree/main
The official Turbo LoRa has the superior style preservation. RDBT the superior compositions. And something went horribly wrong with the NAG implementation. Hope it gets fixed.
@deitychaser I noticed that with I noticed that with NAG. To be honest, I don't think it's really great on anima I think it's more designed for distilled models like Flux Klein 9b/4b and z Imege turbo not really the base model with a distilled LORA.
@AnimaXx Maybe, not sure. It works better on non-distilled Chroma as a negative conditioning guidance than CFG though, in my experience. Simply because you can go up the NAG guidance to 6.0 or something which you can't do with CFG without frying your image, thus in my tests with NAG i could get more control.
Hi boss, is there any possibility that lora version will be released again? Being able to control the strength of lora/distilled model at any time is really useful.
For example, I also use the lore for distillation and it seems to be better, because the built-in lore in the model, it seems to me, has less power overall.
Probably, the reason I decided to release a ckpt because I think mixing distilled models is not a good idea. Distilled lora is not style lora. Distilled lora changes how model works.
@reakaakasky there seems to be many people thinking that RDBT makes @artist less effective, and actually it seems that you don't cite the artists in your work, and that's how your distill different from the official version, what do you think about this issue?
@reakaakasky so putting a distilled lora in a non-distilled base model or vice-versa, either is a bad idea?
@m4rbleye mixing different distilled models, e.g. 0.5 rdbt + 0.5 turbo
@jimzlf if I have 100x compute, I might consider taking care all 20k+ @ tags, but I don't.