Illustrious Version Disclaimer: Ok first of all, for the Illustrious version you can use pretty much the same trigger words and tags listed below and a weight of 1. Did this port because it was requested and tried baking it the best I could however this was my first attempt at Illustrious training, if I consider I can improve this model further in the future then I will update it. (Also remember Illustrious DOES NOT use a scoring system in prompts like Pony does, check pictures for prompt examples).
Weight 0.9 seems to be the sweet spot with the Pony version.
Note: Personally I would say her skin tone seems to be quite inconsistent even in her official artworks, she is sometimes depicted with either pale white or a more normal skin tone. You can still use "colored skin", "white skin" and "pale skin" tags as helpers with increased weight (1.4 at most), I noticed it works best in her kunoichi outfit (because the pictures for that outfit dataset mostly had her in with pale skin). I could have just edited the pictures in the dataset for her other outfits before training but I rather used them as is directly from oficial artworks and in-game CGI, if it's an issue for you, it can be fixed by user preference by increasing skin color tag weights.
Her red swimsuit outfit only has 4 pictures I could track, Had to do some sorcery to get it working to a decent state, expect only a few small inconsistencies though.
Swimsuit Outfit Appearance: fubukib3ach, black hair, long hair, braided ponytail, black eyes
Swimsuit Outfit Helpers: colored skin, pale skin, white skin
Swimsuit Outfit: red bikini, thighlet, hair flower, lily \(flower\), camisole
Default Outfit Appearance: fubukidefault, black hair, very long hair, two side up, hair flower, lily \(flower\), black eyes
Default Outfit Helpers: colored skin, white skin
Default Outfit: multicolored dress, black dress, cleavage, pelvic curtain, side slit, long dress, frills, fishnets, detached sleeves, fishnet socks, white footwear, boots
Kunoichi Outfit Appearance: fubukikunoichi, black hair, very long hair, two side up, hair flower, lily \(flower\), black eyes
Kunoichi Outfit Helpers: colored skin, white skin
Kunoichi Outfit: black jacket, cleavage, midriff, miniskirt, pencil skirt, black skirt, brown pantyhose, black shoes, partially unzipped, fingerless gloves, kunai, thigh strap
Description
FAQ
Comments (6)
Please teach me what kind of sorcery that you did to make the Swimsuit one works with just 4 dataset.
Depending on the complexity due to image scarcity I usually duplicate the pictures and flip them, however when the characters are not symmetrical I edit the pictures to avoid confusing the AI on which element is on which side of the character (this includes changing hairpins locations, editing the hair bangs direction, correcting clothing elements after flipping the image, etc) I used a total of 5 source pictures and worked with that to make it up to 11, added another naked picture to stabilize the learning process as it was trained as an independent separated context inside the LoRA.
Although it was not the case here, I sometimes go to the extend of self-commission the art, I'm an artist actually, a graphic designer, before AI became a common thing I was already working with anime characters (I just never published it online, specially with original characters and only shared with closed friends, I'm too overprotective with my stuff) so it is not a problem for me to do some hard proofing picture curation for datasets by directly editing some pictures to prevent feeding garbage into training.
@Hayanaga Hi, thanks for your input and sorry for bumping! Lately I'm trying to train my own character Lora, but struggling when it comes to a character with multiple official outfits. Assuming I have only 10 dataset of that outfit, do you set custom repeat on each of those dataset or not?
@Andrach I did with an old recipe I used for Pony but not anymore, but that also depends a lot on which training settings and optimizer you're using plus how many epochs you're doing.
Right now for Pony I'm using Prodigy... only 1 repeat per concept and 50 epochs (10 up to 30 pictures at maximum).
For Illustrious I also keep the datasets between 10 to 30 pictures (I notice around 25 is fine) and do 6 repeats and 11 epochs on AdamW8bit, you can check the LoRA metadata if you want to know more about the learning rates used for each version because I use very different settings for each although I lately favor more Illustrious over Pony, but just a personal preference, I found Illustrious to be more easy to work with and it delivers what I want.
@Hayanaga Thank you so much! Yeah I'm training on Illustrious too. Just curious though, is it just me or Illustrious is more prone to undertraining/overtraining? Because I got 28 images dataset (8 of them is her official outfit) but, with 2 repeats and 7 epochs, it failed to reproduce her outfit and if I cranked the repeats a bit higher, I got an overtrained lora instead where the details of her outfit "bleeds" into the images even though no related prompts are included.
@Andrach 2 Repeats for a 28 pictures dataset its too low for 7 epochs, even with my very old deprecated recipe I was doing around 9 repeats with 6 epochs for 30 pictures datasets but that was for Pony.
For illustrious try 6 repeats adn 11 epochs (and set it to save each epoch so you can test lower epochs if your LoRA turns a little overtrained) use AdamW8bit, Text Encoder learnin rate 6e-05, Unet learning rate 0.0003 and Learning Rate 0.0003, learning rates are very important, also try Network Dim 8 vs Network Alpha 4, I used to train with higher values in Pony but these seems to be working well for me even with mildly complex outfits.
Extra tip: Make sure you don't use common tags between outfits, for example if both outfits use the tag "dress" it will confuse the AI during training, instead you could do something like "white dress" for one and "black dress" for other.