Please read through the entire description (might need to be expanded) and the version change notes as they cover a lot of information about basic use cases and limitations. The description is always focused on the latest version. Thank you!
This is an SD 1.5 LoRA for the character Lillie / Lilie from Pokemon. It works for models based on SD 1.x only, it will not work for models based on SD 2.x or SDXL!
Example images were picked from base resolution txt2img results + the FreeU extension (https://github.com/ljleb/sd-webui-freeu) enabled with default settings to further improve results (see last example images for a comparison of FreeU disabled vs enabled). They were then re-created at 2x resolutions using txt2img with moderate Hi-res Fix settings (upscaler Lanczos, denoising strength 0.4). This improves faces and other details that are almost impossible to get correctly and consistently at base resolutions (limitation of the technology) while still giving a realistic impression of what it looks like. Base resolution results will have more distorted faces and less detail.
Please see the version change notes for the training and example image generation models as well as the used weights as they might change between versions. Remember that you might need to adjust weights to best suit your use case!
Remember to add the tag lillie \(pokemon\) (with backslashes intact) to your positive prompt.
The training set contained Lillie's two primary signature looks, the "braids look" from the start of the game and the "ponytail look" from the rest of the game. If you want to focus on one of these looks, you will need to put combinations of the following tags in your positive or negative prompts:
Braids look:
dress
duffel bag
hat
kneehighs
slippers
twin braids
Ponytail look:
ankle socks
backpack
mary janes
ponytail
shirt
skirt
You may also be able to mix these looks but I have not tried it. I might look into separating these into different trigger phrases in the future, not sure yet.
As the training set contained a few images with floating hair, you might need to add floating hair to the negative prompts if not desired.
Known Limitations / Problems:
The shoes, socks and bags were not present in a lot of images and not a focus for the training so they sadly will not be consistent at all.
Eyes and hair had different colors (blue and green) in the training data, better to specify explicitly. And maybe try adding multicolored eyes in the negative prompts. Still might come out a bit weird, I need to improve the tagging.
The blue and white parts of the clothing, especially the dress, might get mixed up. No idea how to fix this yet. It might help to put e.g. white dress in the positive prompts.
The hat really loves to appear everywhere, so put hat in the negative prompts, maybe even with additional weight.
Portraits are currently a bit of a weak point, with limited expressions and a fair bit of color bleed. For certain portrait prompts, you will sadly have to get very lucky, especially if they also try to remove the hat. I might be able to improve on this in the future by including a better selection of portrait training images.
Description
There are detailed changes below the next paragraph, you might need to expand this version changes box!
I learned a few new things about creating LoRAs and wanted to see how far I can push my existing LoRAs without changes to their datasets (well... not quite for this one, see below). So there's still no support for alternate outfits, colors etc. for now, sorry! There are other LoRAs if you want those things. Not sure if I'll improve the LoRAs more than that in the future or make new ones but who knows.
LoRA Changes:
Now uses the standard "lillie \(pokemon\)" instead of a custom trigger word to make things easier for everyone (please note though that the LoRA itself is still named "character_pokemon_lillie_vX" as before!)
Switch to 32 dims (now trained directly instead of resizing from 128)
Switch to target weight 1 (increased from 0.7)
Changed a lot of training settings (too many to list, check LoRA metadata if you are interested) and updated to much newer scripts version (~3 months worth of changes)
I had to remove one image from the dataset and slightly change the tags of 3 further images as they confused the new training too much. As a result, the red backpack in particular might sadly become a bit worse in general and especially for close-ups. Hopefully not much else gets worse.
Example image changes:
Use CLIP skip 2 instead of 1 as I accidentally did before
Change settings for hir-res fix: Switch upscaler from None to Lanczos and increase denoise from 0.1 to 0.4 to get better results while still not changing TOO much
Use a negative embedding for the majority of negative prompting
Recommended weight: 1
Training model: Anything V3
Example image generation model: AbyssOrangeMix2 - Hardcore