I can guide you with the best practices, tips, and tricks to achieve your goal in your Stable Diffusion training. Take a look at my Patreon, subscriptions for just $5. Exclusive content. Patreon. :)
About:
This motherfucker LoRA has the purpose to (try) generate Almost Every Shit You Could Fuck Want.
It was trained in 768x768, using captions and 277 high-resolution images, with the most diverse artistic styles from various digital artists.
Since I mixed everything up, they won't be able to curse me out, because it won't look like any specific style. 😂
The demo images aren't edited, but they are cherry-picked and generated in A1111 WebUI, using hires-fix with 0.25 ~ 0.4 denoising and 0.2 extra noise multiplier.
Suggested Settings for Inference:
Model: Probably any model you could fuck want 😂
Positive: 1girl, close up to a skinny, busty, graphite, sweat, ass, short hair, curly hair, green eyes, green hair, indoors, masterpiece, highly detailed shot, epic perspective <lora:ALESYO_alpha_sd15_by_IsnAI:1:0.6)
Negative: verybadimagenegative_v1.3, bad-hands-5, polydactyl
Sampler: DPM++ 2M Karras
Steps: 28
CFG: 5
Resolution: 768x768, 512x768 or 768x512
LoRA Weights: 1:0.6
Note¹: As you can see, the LoRA tag has two weights: <lora:ALESYO_alpha_sd15_by_IsnAI:1:0.6)
This is a feature of the A1111 (which many people aren't aware of), but you can set separate weights for the UNet and the Text Encoder. The TE is always a pain in the ass to get the training right, so it's usually better to reduce its weight in the inferences rather than having it undertrained.
Note²: You will see big variations in style and images depending on the base model used and minor adjustments in resolution.
Note³: If your images are getting weird and bizarre, you probably didn't reduce the TE weight. If you aren't using A1111 WebUI, you can do the common weight reduction: <lora:ALESYO_alpha_sd15_by_IsnAI:0.6), but you will need to test which weight will suit better.
Triggers:
Tags Used on Training: 1girl, blush, 1boy, penis, open mouth, pussy, bangs, long hair, looking at viewer, nude, sweat, short hair, veiny penis, ass, sex, testicles, solo, smile, anal, looking back, panties, brown hair, medium breasts, large breasts, anus, shiny, nipples, lips, blue eyes, thighhighs, gloves, from behind, black hair, long sleeves, saliva, teeth, sidelocks, collarbone, bare shoulders, shirt, green eyes, pussy juice, small breasts, tongue out, spread legs, green hair, pov, thick thighs, curly hair, dress, indoors, tongue, cum, lying, navel, blonde hair, upper teeth only, brown eyes, ass grab, legs up, hair ornament, closed eyes, clothes lift, upper body, dark skin, on back, motion lines, rolling eyes, purple eyes, closed mouth, doggystyle, on bed, glasses, skirt, sitting, breasts out, bent over, clenched teeth, half-closed eyes, pink hair, no bra, purple hair, ahegao, girl on top, monochrome, puffy nipples, clothes pull, medium hair, yellow eyes, straddling, cowgirl position, one eye closed, folded, socks, huge breasts, fellatio, looking up, grin, spread ass, red eyes, pants, nose, vaginal, short sleeves, from side, 2girls, bodysuit, bed sheet, from above, graphite, covered nipples, black eyes, looking at penis, tears, grey hair, aside, choker, fingernails, facial, colored skin, jacket, surprised, barefoot, huge ass, dark-skinned male, nail polish, high heels, wide-eyed, feet, outdoors, back, belt, twintails, trembling, off shoulder, from below, sleeveless, naughty face, kneeling, penis grab, looking at another, toes, blue hair, breasts apart, deep skin, horns, ponytail, bikini, full body, red hair, sideboob, hairband, lipstick, multiple boys, hand up, background, profile, tank top, clothes, superhero, skin tight, necktie, orange hair, skindentation, white hair, breast press, hanging breasts, cameltoe, school uniform, arm support, legs together, hair bun, licking penis, wide hips, cleavage, green skin, heart, pantyhose, very long hair, sweater, leotard, footwear, bob cut, hat, saliva trail, day, penis on face, multicolored hair, kneepits, grabbing another's hair, streaked hair, arm grab, steam, orange eyes, looking down, muscular, dark-skinned female, sports bra, lip biting, leg up, frown, blue sky, messy hair, turtleneck, bdsm, covered navel, 3girls, handjob, fishnets, fake animal ears, animal ears, bouncing breasts, tattoo, torso grab, groping, crop top, wavy mouth, fur trim, high ponytail, flipped hair, curvy, embarrassed, cheerleader, mask, garter belt, pink eyes, constricted pupils, precum, forehead protector, ahoge, garter straps, abs, dark penis, grabbing another's ass, very dark skin, demon tail, demon girl, suit, head grab, hand on another's face, happy, necklace, smartphone, bound, aqua eyes, seiza, armpits, groin, breasts, on side, spiked hair, aqua hair, double penetration, strap-on, whisker markings, fake tail, shorts, twin braids, holding another's wrist, maid, maid headdress, gym, striped, squatting, leg grab, mascara, eyewear, tail, licking, bondage, restrained, female short hair, crossed legs, mouth hold, bowtie, arms up, hair pulled back, orgy, cleft of venus, soles, ok sign, goggles on head, anal fingering, dildo, uzumaki naruto \(mtf\), spoken heart, drooling, furrowed brow, grey skin, against wall, drill hair, waist apron, tan, grabbing own ass, hair down, dark nipples, standing on one leg, female spread legs, pale skin, otoko no ko, gradient hair, hoodie, barbell piercing, sleeves rolled up, looking at phone, white coat, wing collar, foreskin, anger vein, dominatrix, white border, whip, floating hair, spread anus, white scarf, scarf, grey eyes, black bow, own hands together, arm up, fingering, masturbation, hug, headband, low twintails, tiara, selfie, christmas, santa hat, fangs, female nipples, veins, white belt, uniform, colored sclera, thigh grab, sheet grab, baseball cap, lips under eye, testicles under eye, female skirt, female testicles, twitching penis, female tongue out, colored tips, braided ponytail, gaping, sagging breasts, angry, wavy hair, gangbang, multiple penises, yellow ribbon, female from behind, forehead mark, deepthroat, 4girls, tanlines, classroom, paizuri, cheek bulge, kissing penis, female solo, female half-closed eyes, female green hair, large areolae, santa costume, gothic, thigh gap, shoes, side ponytail, sneakers, wet, hair bobbles, spread pussy, hand on own ass, covering crotch, looking to the side, suspenders, bare arms, purple bow, bedroom, sitting on face, female precum, arched back, femdom, unthighhighs, bottomless, tatsumaki, completely nude, unflipped hair, white background, ununderwear, black panties, uncowgirl position, underwear, female bottomless, sex from behind, unvaginal, pillow, all fours, tiptoes
Description
FAQ
Comments (16)
Oh man, can you tell me where to find out more about the unet/encoder double weights and how/when/why to set them differentially??
Hey!
Maybe there's more info on the A1111 wiki, but I haven't looked at that feature there yet.
I learned about this recently, a tip somewhere that this is useful to see how each one is performing in training.
It's pretty straightforward, you just add another weight in the tag. I don't know if it works on CivitAI, but it's worth a try. They have a WebUI extension to detect features, so maybe they are using A1111 as the backend for inferences.
If there is only one number after the lora name and colon, then that weight applies to both Unet and TE. But if you add another colon and another value, then the weights are separated, with the first one being for Unet and the second one for TE:
<lora:ALESYO_alpha_sd15_by_IsnAI:Unet:TE>
😊
Ok i'm understanding that part and I've read that it's basically using different weights of the lora when it breaks it down and then reconstructs the image....any chance you could you give a rough idea of: when you would use that, what makes you think to use it I.E what problem does it solve, and what your baseline settings would be to start like <genericLora :1:?>
we all encounter Lora's that are ok, but overdone or flawed in someway and I'm wondering if this is the key to making them shine.. hoping you can give us a crash on where to start if we wanted to experiment.
TY tip sent for your time.
@d0000d,
Ah! Got it now.
So, I'm not sure how much you know about training LoRA, so sorry if I'm repeating something you already know.
First, I'll give you some context, and then I'll explain about the weights.
When we train a LoRA, we have the option to set a learning rate for the Unet and for the Text Encoder. These parameters are very sensitive, and the TE is much more sensitive than the Unet.
Depending on the combination of parameters, a very high rate and/or too long training, or other factors, it might end up "overtraining" the neural networks (Unet and TE).
Now imagine that game of skipping stones on a lake or river to try making it bounce on the surface. Let's say the LoRA is the stone and the weights of the Unet and the TE are the force you apply to throw the stone.
If you apply too little force, the stone will simply sink, meaning it won't work. If you apply too much force, the stone might go far, maybe bounce once, but it won't be efficient.
However, if you apply the right amount of force, the stone will skip many times on the water.
Then, if you apply too much weight to the LoRA, it will destabilize the main neural network's weights, which is the checkpoint, making the image come out bad, with defects, etc.
If you calibrate it better, reducing the weight of the Unet and/or the TE, you will be creating a better synergy between it and the checkpoint, where despite being weaker, it will work better to influence the checkpoint's weights and create an image that's more pleasing to you.
But of course, there are some LoRAs where the training parameters got so unbalanced, or that were trained with a very limited dataset and/or with bad images, that even adjusting the weight in the image generation won't solve it.
However, I think I could risk saying that many LoRAs that seem bad, can work very well if you calibrate the weights when generating the image.
Btw, thanks for the tip!
😊
@IsnAI Very helpful!, I have zero expernce training Loras, so I appreciate that insight as an end user.
so say I have a flawed lora that does a person pretty well, seems to me it had a limited data set, and is a bit overcooked. if i set it to full strength it's getting the details right, body shape come out right, face comes through recognizable after aDetailer.....but it's producing fused feet, wacky arms that sort of defect, and it overpowers creativity.
I can turn down the lora strength to allow the checkpoint to restore some coherency and creativity, but I start losing the details of the lora... is this the sort of thing you might use, to try to squeeze a little better performance/ balance out and what's your personal rule of thumb as to how you choose those 2 numbers?
out of pure curiosity to maybe draw out your insight and leave it here for future people because I can't be the only curious one about this feature
Thanks again
@IsnAI So it's pretty much a guessing game then with turning knobs and seeing what works and what doesn't, huh? I've been trying my hand at making a concept work but it's so finnicky I wish there was a more concrete way to guess it right (Like maybe some kind of equation used for the number of images and the learning rates and epoch/steps). I HAVE noticed though that some of the loras I make don't work unless I set it to like 1.4 strength, in which I'm wondering if there's a way to know if knowing that would that mean you need more epochs or more learning?...or I guess that too is a guess?
@d0000d unfortunately, when it comes to hands and feet, this is an issue with SD1.5 itself. Even SD XL struggles with hands and feet. The "best" solution for this is to use ControlNet, which recently released a 'hand refiner' preprocessor that helps a bit.
@Light7799, exactly, training neural networks is a complete trial and error game (unfortunately much more error).
You can create a perfect LoRA, but the 'formula' probably won't work for another dataset, it always needs some adjustments. And the threshold between something very good and something catastrophic is tiny.
I personally gave up on training faces and started using Faceswaplab, which produces much better results.
The only concrete things I can tell you are, avoid adaptive optimizers, because you can't put a lower rate on TE, be very careful with the captions, and also test the captions on the model you're going to use as a training base.
For example (note: my native language isn't English), I discovered after a long time that TE interprets better if I use 'laying' instead of 'lying' in the case of 'laying in bed', because lying is ambiguous.
Yes, the LoRA is probably 'undertrained' if you need to increase its weight to generate the desired effect. Depending on the situation, you can use more epochs or even increase the learning rate.
I would say that a number of 50 repetitions/epoch for each image is a good starting point. I don't know which training script you use, but in Kohya you can increase the repetitions by adjusting the number in the dataset folder name or by increasing the number of epochs.
The difference between repetition and epoch is that in repetition, the neural network sees the image more than once within the same epoch. It is another trial and error issue to find a balance between repetitions and epochs. 🥲
@IsnAI I see! Yeah, I've been messing with things.. I will say from my testings I have noticed cosine and cosine with restarts seems to work better for training a concept completely Alien to a model, and constant is good for a model that sort of kind of understands the concept but doesn't always get it. It's very frustrating indeed. I think if it wouldn't take so long to trial and error each batch that would make it more viable. I appreciate your input on the matter though! For adaptive are you referring to adafactor? If so, I've been using that well enough actually, but only seems to work well with sdxl. 1.5 I've bounced back and forth between AdamW and Prodigy with arguments
works perfect up to 512x1024. higher resolutions ended up in strange results with many limbs and stuff.
Hey there!
There's an extension for A1111 and ComfyUI (and probably others too) called Kohya Hires Fix. Using this should fix the issue you're having with large resolutions. SD 1.5 was trained on 512x512, and my LoRA on 768x768. If you go much higher than that, things start to get weird in the images, like extra heads, legs, etc. This extension fixes that.
😊
@IsnAI sorry, i did confuse you. i've just teste it which resolution work and which not. and i wanted to share this result before someone else may be asking.
i really don't understand what this does, is it a detail-adder or whta...
Hey there!
This is a Style LoRA, which means you can use it to generate images with mixed styles from various digital artists. It works with almost any checkpoint, and the styles vary depending on your prompt and the checkpoint you use. There's no activation word needed, just use the booru tags. I've also included a list of all the tags used in the training at the end of the description.
😊
I'm disappointed that there's no 'Bukkake' trigger word listed.
At least I didn't see it among the cool list you made.
Ahh, sorry, man! Totally forgot, I will put it in the next version. 😁
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.