CivArchive
    EnvyBetterHands LoCon - alpha3
    NSFW
    Preview 577258
    Preview 577322
    Preview 577261
    Preview 577276
    Preview 577271
    Preview 577277
    Preview 577353
    Preview 577389
    Preview 577417
    Preview 577323

    This model is a LoCon. You MUST install the Lycoris extension for it to load.

    I'm using Lora Block Weight. I believe you can also use Additional Networks and SD Webui Lycoris.

    UPDATE 4/27/2023: I've hit a training plateau so I'm in the process of adding a bunch more images to the dataset, including some more complicated stuff like intertwined fingers. I'm probably going to have to drop the learning rate some more, so things may be slower from here on. I'll keep everyone posted as things progress.

    UPDATE: Prompting advice for beta 2:

    • This is a completely new train on top of vanilla Stable Diffusion 1.5. I did this based on the advice of a fellow enthusiast, and it's surprising how much more compatible it is with different model. It doesn't mess with the style of your model at all as far as I can tell, and it really only affects hands and occasionally arms, leaving everything else untouched.

    • It seems to work best at a strength of 1, although turning up higher than that (1.5, 2, etc) can help it on some images at the cost of making it worse on others. No need to mess with your CFG scale, as it doesn't cause things to overcook at these levels.

    • Freely mix it with other LoRAs.

    • I've had best results putting "nice hands, perfect hands" in the positive prompt (increasing the weight makes things worse), and "(extra fingers, deformed hands, polydactyl:1.5)" in the negative prompt. This is on EnvyMix v1 (and probably RevAnimated), but YMMV for other models.

    • "Bad hands" negative embeddings appear to make it worse, although I haven't tested this extensively.

    • As usual, this won't work miracles, but I do find that over a large number of images, it does make things generally better on average. Hopefully this will continue to improve with a few more nights of training.

    Prompting advice for alpha 3 and beta 1:

    • Note that this advice is for RevAnimated 1.2. YMMV with other models.

    • It overcooks things a bit, but you need the strength set to 1.0 for it to really work well. You can work around this by reducing the CFG value to 5 or 6 or so. I've had good luck with enabling the dynamic thresholding extension and setting it to mimic CFG 5, and then I can set my CFG value to 9 or 10 and things come out fine.

    • I tried using it with another LoRA and got some pretty strange results, so YMMV there as well. Right now I'm just trying to get it to work consistently in a simple use case.

    • Oddly, I think it's regressed a bit on hands in neutral positions, but it's noticeably better at more complicated interactions, such as holding objects (which is why I have so many pictures of blacksmiths and librarians in the example images).

    • Keep your prompts simple and it tends to do better.

    • With RevAnimated, I tend to get 1 or 2 usable images out of every 8, with a bunch of other ones that are pretty close and can probably be fixed with inpainting.

    Prompting advice for alpha 2:

    • It's getting stronger now, and it works best around strength 1. Setting it to 1.3 like the previous version will make things look bad.

    • My negative prompt is still "(extra fingers, deformed hands:1.15), (worst quality, low quality, poor quality, bad quality:1.35)"

    • I had good luck just putting "nice hands" in the main prompt.

    Prompting advice For alpha 1:

    • Your prompt should contain these words: "beautiful hands, perfect hands, fingernails". I've had the best luck with them towards the middle, and at no emphasis.

    • The alpha1 LoCon seems to work best at a strength of around 1.3 (on RevAnimated 1.1, where I'm testing it right now -- YMMV for other models)

    • Don't use negative embeddings for improving hands. When I removed badhandv4 from my negative prompt, things improved noticeably. You may want to try without any negative embeddings at all. I haven't used them for a while now.

    • My negative prompt is: "(extra fingers, deformed hands:1.15), (worst quality, low quality, poor quality, bad quality:1.35)", which I arrived at through a lot of experimentation adjusting strengths and terms one at a time. It should work decently well.

    • This all gives me hope that there's a real shot at solving hands on SD 1.5. Even with good prompting, I'm generally not getting perfect results, but things are close. I'll consider this done when it creates well-formed hands without having to add anything to the positive or negative prompt.

    Now back to your regularly scheduled readme...

    I'm testing the theory that maybe the reason MidJourney's hands are so much better now is that they just took the time to specifically train a network on a high quality set of pictures of hands, and literally nobody else has actually tried. This LoRA definitely isn't at MidJourney levels yet, but I've been training it over night for several nights now and adding to the dataset where it appears deficient, and quality seems to be steadily improving. As such, I'm going to post this now so people can start using it. Consider this an early alpha -- I'll only stop updating once it stops getting better.

    Example images are cherry-picked. Please don't expect this model to make all of your hand generations better. It may even make some of them worse, so you should evaluate its usefulness on a large number of images and not just one. If it works for you like it does for me, the a lot of your results should be the same or better quality (some will just be bad in different ways).

    Description

    I lowered the learning rate a bit and added a bunch of new images to the dataset and have had another jump in quality. In particular, it seems to be getting better at rendering hands that are holding things or interacting with objects, which have never really worked before.

    This model is getting stronger, and I'm finding that it tends to overcook things at strength 1.0 (at least in RevAnimated), but you can fix that by using the dynamic thresholding extension and setting it to mimic a low CFG value (like 6 or so).

    FAQ

    Comments (16)

    kwikApr 23, 2023
    CivitAI

    I think this looks extremely promising, and I wonder what would happen if you trained this on the SD1.5-Inpainting model. I would try it myself if you would want to share your dataset. I could see myself going over and fixing hands on thousands upon thousands of my images by writing a script that masks hands for inpainting.

    Ideally, something like this (but with greater flexibility in terms of style) should be merged into 1.5 to create "1.6", which we could all train on going forward. It feels like good hands and control of arm positions is the last major hurdle for AI art.

    Keep on keeping on, Envy! You're doing a great thing with this model.

    ToxicBotApr 23, 2023· 1 reaction

    Yes an inpainting model would be great!

    _Envy_
    Author
    Apr 24, 2023

    Are you on Discord? I can contact you there.

    kwikApr 24, 2023

    @_Envy_ Sure, I'm there: kwik#4616

    ash0080Apr 24, 2023· 9 reactions
    CivitAI

    I hope that it is a TI solution instead of Lora. Based on the pain points that you mentioned in the readme, almost all of them are related to Lora, and these issues do not exist in TI solution.
    For example, with TI, you can easily mix even 10 different TIs without any screen fragmentation, whereas with Lora solutions, it can be difficult to mix even one other Lora without encountering problems.
    So it's in the wrong direction.

    nicetry20010Apr 24, 2023· 5 reactions

    agreed, TI are seperior than lora, they don't influence artstyle in major way. so things like face, body posture, clothes are on optimal form when trained on TI.

    _Envy_
    Author
    Apr 24, 2023· 10 reactions

    There are already multiple textual inversions out there that improve hands somewhat. The trouble with TIs is that they can't really impart meaningful new information to the network; they can only help find things that already exist in the latent space.

    Judging by how much time and how much data I've had to use to train this to make hands better, more information is necessary in order to help it draw hands.

    That being said, if you can come up with a TI that reliably fixes hands, by all means, do it.

    ash0080Apr 24, 2023· 1 reaction

    @_Envy_ good luck

    _Envy_
    Author
    Apr 25, 2023

    @ash0080618 You too. Seriously, if someone solves this with a TI, that would be awesome. I just don't think it's going to happen.

    ash0080Apr 26, 2023· 1 reaction

    I added a review, and then I shut up

    recoilmeApr 24, 2023
    CivitAI

    Alpha3 not working for me: output with shape [1, 320, 1, 1] doesn't match the broadcast shape [1, 320, 3, 3] lora.pyline 176

    recoilmeApr 24, 2023

    same for alpha1

    _Envy_
    Author
    Apr 24, 2023

    Install the Lycoris extension on automatic1111 if you haven't already. That should fix it.

    damnedcomic451Apr 25, 2023

    @_Envy_ I have the Lycoris extension installed and get the same error.

    prolmaApr 25, 2023

    @_Envy_ This should be filed as a Locon, not a LORA, in Civitai, yes?

    _Envy_
    Author
    Apr 25, 2023

    @prolma They only have LoRA and Lycoris.