Zlabya is the rabbi's daughter from the Eisner award winning comic Rabbi's Cat. She's a feisty, educated lady exploring her place in the Algerian society and sports an absolutely iconic mix of tan, red and gold. I created the LoRA based on the somewhat more consistently drawn 2011 animated film version.
This time, I didn't remove the backgrounds from the training data, so it's hard to get photorealistic. Then again, if someone wants this this character, they probably also want the Tumblr blob style.
PROMPTING GUIDE:
Generics: 1girl, solo, zlabya, bare midriff, wavy dark hair, dark skin
Outfit: strapless red top with gold lining, puffy red pants with orange belt, bare feet, full body, golden hoop earrings, gold coin necklace, bare feet, golden wristlet,
Feel free to mix as you wish and look at my example pics for inspiration, I always put a lot of effort into those.
Description
First version, no whited out backgrounds so might be glitchy.
Comments (7)
I don't think lack of background removal is what makes it hard for a LoRA to adopt to other styles (including realistic). A LoRA getting an "inherent style" that resists bending moreso comes from a lack of artstyle diversity within the dataset, and training not being able to isolate the style to a particular tag (not really doable here, but tagging some styles that are proper and recognized Danbooru tags like "3D", "1980s (style)", or "lineart" is an effective way to isolate artstyle to those tags, and is particularly useful for certain artstyles you absolutely do not want influencing the final LoRA without being called like "3D"). If all the data is from a single source, a LoRA learns the patterns of that style alongside and as part of the character. This isn't a bad thing for cartoon characters though.
If you really want to avoid inherent style, it's most effective to include some art of a different style in a dataset when possible. Even a small helping of alternate artstyles makes a big change, and the change doesn't have to be that big: Different artists within a "genre" ("anime" fanart by different artists+screencaps of anime) does it fine.
I appreciate you taking the time to comment. I might've phrased myself badly. The main issue during production was that the backgrounds kept being really terrible and cluttered, which makes sense with how this movie was drawn, but isn't very fun when generating, as it even messes with perspectives. During tagging, there's so much stuff that I can see but the training process can't, like random backwards picture frames, patterned wall cloths, carpets and such (these were Escherically confused for each other).
Removing backgrounds from roughly half of the images genuinely helped. However, I don't want to remove backgrounds from every image, because then the character is less grounded and can float somewhat.
Also, I did use QIE to generate the character in anime style and put five such images in the ~170 image dataset because I figured something similar. If I cared enough about this character, I might also generate a "canonical" photorealistic look.
@ChocoGirlTummy OK. I wouldn't recommend synthetic data, even as a minority. The errors are still patterns that will be learned.
@NanashiAnon True, synthetic data can have some strange artifacts that are hard to see initially but accumulate to the end generations and become very distracting. I learned this the hard way with my Orinjade model, where I used DigitalFrames to upscale the poor resolution scans to get a non-pixelated character. In the new version, I cleaned the upscales in GIMP and everything works pretty fine. Funnily enough, this model is less stable because of the sheer amount of off-modeling in it, as these award winning cartoons are known to do.
I would always keep synthetic as a minority and/or reserved for cases where some crucial variety doesn't exist in the source material.
@ChocoGirlTummy 170 images for a single outfit character is totally overkill, even if you do want a strong inherient style (especially if training on site: The default settings will lower your repeats if you have too many images and cost more to increase them). I find ~30 is when it's better to just start purging the low quality images instead.
@NanashiAnon Interesting. I usually have ~40-50 images, but here I felt it was necessary to get more because the style varies so much and I wouldn't want the model to generalize the one image where the head looks a bit off in one way or another. A more on-model character works with significantly fewer images, like Scourge of the Desert which has 35 images but works rather well.
I guess I could train a version of Zlabya with 30 images, but it would be rather hard to select the best ones. Do you have a specific pose/angle breakdown you use?
@ChocoGirlTummy I think characters that need a strong inherent style to be themselves can use some more, but if you're finding inherent style is too strong over 50 is going to be way too many.
I don't think poses are a concern as long as they're not consistent. As for angles, I don't have a firm ratio, but I do think it's important to include at least from side, from behind, and enough data from front the head isn't always at a "perfect" angle, and some full body pics. I can't give a firm ideal lower limit (I tend to make models where I don't at all have the luxury of "perfect" data) but I think 5 of each would be more than enough. Maybe 2 or 3 of each at different angles so it's not just learning patterns for "from behind" (etc.) off one image? Likely depends on how complex (and distinct from front) the rear is.