Undertale LORAs for SDXL/Flux
I've noticed a lack of Undertale LORAs for SDXL, so I've trained one on Animagine XL v3.x. It can work on some other models as well. I also trained a PonyXL and Base model version s and right now Flux LoRA in process.
Training Data for SDXL:
Used around 800 images in total, with some later additions
Can influence style quite significantly (mostly 3.0 models)
Flux LoRA:
Currently being trained using split mode and fp8 version of dev model
Quality uncertain at this time
Uses smaller dataset of 558 images and less repeats, Florence2/JoyCaption captioning + corrections by me
Possibly would retrain at some point
Characters: Asriel, Chara, Frisk, Muffet, Sans, Papyrus, Undyne, Gaster, Toriel, Mettaton, Grillby, Alphys, Flowey, Asgore, Napstablook
Note: Quality varies between characters. In version 1.0 of Animagine, avoid generating Asgore or Napstablook due to overfitting. Version 1.1 and onwards work better, but caution is still advised.
Parameters:
Trigger words: character_(undertale)
Some characters work well with trigger words alone
Others require additional prompting
Flux: Natural language prompting
SDXL: Booru tagging
The _(undertale) part isn't strictly necessary
Model-Specific Notes:
Flux:
Use guidance of 3.5 or higher for better character likeness
Higher guidance may increase stylistic differences between images
Animagine 3.1:
Model 1.2 surpasses previous versions in flexibility and capabilities
Stylistic differences between versions are subjective
Personal preference: Model 1.2.5 for style, but has biases due to initial overfitting
Base XL:
Second model trained on base model for broader compatibility, as a substitute when other models don't work
Intended for use with any model, but struggles with anime models
Examples not limited to Animagine v3
Currently not fully accurate, performance varies by model
PonyXL:
Trained on civitai
Can produce good outputs
Missed some character features
Note: There may be stylistic inconsistencies, so it's better to specify your preferences when generating images.
Description
This is a mix between 1.2.5 and A31 versions. A31, while flexible, often gets something wrong about characters or less detailed, so the mix makes it to be something in-between. Influences the style closer to what 1.2.5 does. It's actually better with multiple characters than A31.
FAQ
Comments (4)
Most people post generations of models based on PonyXL here, considering the prompts, which are not really good examples - PonyXL can generate some Undertale characters by itself and is also not compatible with Animagine (not much difference with or without LORA).
i mean no offense by this but that's because pony's prompt standard is way more consistent and usable. i actually wanted to suggest making a pony version of this because while you're right that it's fairly capable, it still gets subtle things wrong (like confusing the skeletons and boss monsters, also "boss monster" is not a useful tag) and those subtleties can be hard to steer.
@baph0met_balloon Pony prompting is the same as score tags (and some other) + danbooru tags, difference is minimal from other anime models. Pony is far less consistent and biased outside of tags it knows, while something like Animagine has no problems.
I am not sure if I will be able to see training benefits for Undertale, but I did tried to train Pony model for 2 other models. I don't really know why, but it just won't get their likeness fully, even after hundred of epochs. I tried to increase learning rate, but it breaks easily. Even though I used more or less same settings as other people.
But I will try again, maybe learning rate must be even higher.
@munchkin pony's score tag system is unique to its family and replaces the various "quality tag" incantations that people need to use with other model families to get something to look halfway decent, that might change between models. ultimately, it's preference, but pony's broad style range and steerability is the reason it's got so popular (although it definitely has its own set of issues, hence the de-facto standard being merges like AutismMix)
it's good to hear that you're looking into it, though. the only thing I could suggest with my minimal training experience is to try increasing the dimensionality and/or separating training images into folders, if you're not already doing that; this helped Tigerpillow, the inflation LoRA creator, a lot with separating out the many distinct concepts that project covers.
your loss is gonna be pretty small, since as you said, pony does already have the basics of these characters down. so it might just be down to dimensionality + time.
best of luck! i appreciate this project either way