V10 Disney Muses

TL;DR: All five Muses from Disney's Hercules in one LoRA. Characters were cropped out and trained in isolation, so results are better when prompting one at a time. The LoRA struggled to generalize their outfits, so it's recommended to avoid their default outfits.

Suggested Weight: 0.6-1.1

All images were rendered with AutismMix with PatchModelAddDownscale in ComfyUI, and the two person (or fountain) images used regional prompting to control output.

Tags

The outfit tags and screenshot tag are extremely powerful. If the character appears too realistic or generic, try adding something like screenshot to the positive prompt.

Suggested Negative Tags

bad eyes, bad lips, orange skin

Character Tags

calliope, brown eyes, black hair, white headband

terpsichore, brown eyes, black hair

clio, brown eyes, black hair, ponytail over shoulder

melpomene, brown eyes, black hair

thalia, brown eyes, black hair, short hair, high ponytail, white hair tie

Outfit + Character Tags

calliope, brown eyes, black hair, white headband, white chiton dress, side slit, white arm drapes, white high heels, dark red lipstick, dark red eyeshadow

terpsichore, brown eyes, black hair, white frilled crop top, white skirt with train, white arm drapes, white high heels, dark red lipstick, dark red eyeshadow

clio, brown eyes, black hair, ponytail over shoulder, white capelet, white chiton dress, side slit, white high heels, dark red lipstick, dark red eyeshadow

melpomene, brown eyes, black hair, white chiton dress, single shoulder strap, white single arm drape, side slit, white high heels, dark red lipstick, dark red eyeshadow

thalia, brown eyes, black hair, short hair, high ponytail, white hair tie, white chiton dress, white shoulder drape, dark red lipstick, dark red eyeshadow

Training Information

Training images were highly processed (SUPIR, ESRGANx4plus_anime, manual edits).

Batch Size = 4

Thalia

- screenshot: 15 (x3 repeats)

- fan art: 8 (x3 repeats)

Clio

- screenshot: 14 (x3 repeats)

- fan art: 7 (x3 repeats)

Calliope

- screenshot: 36

- fan art: 6 (x3 repeats)

Melpomene

- screenshot: 16 (x3 repeats)

- fan art: 8 (x3 repeats)

Terscichore

- screenshot: 18 (x3 repeats)

- fan art: 6 (x2 repeats)

Notes

- Eye colors, (most) base character hair styles and red hair highlights/outlines were intentionally left unlabeled.

- Dark skin color was not explicitly tagged and can be difficult to control. There seems to be a strong bias towards orange skin.

- The tag "arm drapes" and "skirt with train" (should be "train (clothing)"?) were not fully learned. The images should have either been altered to remove the details or supplemental datasets for the concepts should have been created and added.

- Props were not explicitly training (e.g. tragedy mask, scroll, feather fan, vase).

- No five-person group shots were trained, which negatively impacts relative heights in group shots.

- Thalia's body is not quite as stylized as in the movie.

- Remove the "screenshot" tag? It is nice as a way to control the style, but it may have absorbed some of the skin color and character detail (e.g. hair swirl details). The base outfit was also over-trained, as it appears in most training images.

- Remove excessive tags, especially related to makeup and clothes. Lipstick and eyeshadow tags were too excessive that the bottom lip could disappear.

- Better data could have been collected from the movie, rather than screenshots from the web.

- Detailed hair style labels so it is clear? (e.g. afro, spiked hair and hair around ears, hair puff and cone hair bun or hair horns, long wavy hair, high ponytail)

TODO

- Fix skin color (how?)

- Fix arm drapes

- make cloned set with all drapes and trains removed, and train on both

- Secondary Datasets

- arm drapes (make sure elbow band is included)

- train (clothing)

- tragedy mask on stick

- feather fan

- scroll

- Remove excessive lipstick and eyeshadow tags (only if large enough in image)

- More close ups of Melpomene's face?

- More repeats on Thalia data?

- Group images and less tight cropping (especially for Thalia to help learn her shorter height.)

- Either remove or get more (but correctly depicted and ideally high quality) fan art so the dataset is better balanced. Ideally with different outfits to act as regularization.

- Fix teeth and eyes? (i.e. double teeth, bad eyes, bad hands, etc.)

- Remove screenshot term? It removes ugly upscaled animation, but may also absorbed some of the character design.