Hi, this is my first post here.
My name is david and I'm a professional concept/3D artist who currently is doing AI workflows (currently in Substance Designer and Photoshop mostly). I started using AI in my works around 6 months ago when discovered Disco Diffusion and later MidJourney till I stall with Stable Diffusion. Since then I have been daily finetunning and training models for me and other people (mostly AI instagram artists) and recently I opened a patreon (with 2 other fellas) so people could support us in the model finetunning/making and experiments/workflows. If you think what we are doing is good, feel free to support us. Thanks!
Also if you want to check my works or just my daily basis feel free to check my IG at @davidsayszawarudoagain
--
This is the result of the mini tutorial I have put on our patreon for free.
https://www.patreon.com/posts/workflow-how-to-80636187
EDIT: Settings since probably someone want to test them or just know how was trained:
200 repeats per image, 30-40 images, epoch 6-8, lrate/urate default in this case, ClipSkip 2, max vector token 1, FP16 in both areas, constant without warm ups and trained over a custom model in DIM/NET 8x8 . Nothing to hide here. Input is more important than this "magical settings".
A bit different of what people use to do when training but I think it's lot of fun. Feel free to test a bit this char and tell me if works or not. As a quick difference this wasn't trained on SD 1.5 or NAI/Anything/Abyss, rather than that I used a custom finetunned model that did a decent job.
Lora weight works even at higher than 1 but recommended is 0.7-1. 0-5-0.6 should give you some char design style. Keyword is "reinbolt" and if somehow you can't see his green armor with lightings put "green" or "armor", it should appear.
By the way, as everything from my part, feel free to sell, transform, mix or whatever you want. Just have fun and throw some feedback if possible.
Description
FAQ
Comments (6)
What software did you use for training the Lora? The results look great and having a 9MB lora is a lot better than a 144MB lora that doesn't work any better, but I'm missing the very first step in replicating your results!
Hi! Thanks for your comment! I trained on my computer using Kohya_SS (you can find it here: https://github.com/bmaltais/kohya_ss).
In my opinion not because training 2000 images will look better than 40-50 images properly tagged, cropped and clean/redrawn or simply using personal illustrations.
@LDWorksDavid Hey David! i'll check out your Patreon :) I know it copes with higher rez and uncropped images, do you find theres a threshold or point where it gets confused?
@halo Hi! you mean this embedding or in general while training? If you can be more specific I will try to help you. For uncropped images and even having bucketing aspect ratio I think it's better to manually crop those images to 512x512 and put it off. And for inferencing I think it depends on the cropping and res of the data. I like to train always with 512x512 since I use mostly 512x768+ Upscale with inpaintings. That's why it's better to train a low number of custom images but with everything looking good than having 3-4 images of a 100 set that may "corrupt" the results.
@LDWorksDavid awesome, thanks!
@halo No worries, glad to help. Some people wondered why I was doing 2-3 pretraining to get a even more curated dataset and the reason is just that, the AI probably will overfit your training if it mix too much or too low degree so this way you take what works and not, retrain, etc. and repeat 2-3 times till you have a dataset you know AI will understand and will give good results. Wrote some posts in Reddit regarding that (and in Patreon, for FREE is was all explained)., Check them out if you still curious. You will realize very soon that this never was about settings or numbers, it's all inputs, reworking inputs, guiding the AI the best as possible and repeat. Simple as that.
This was just a showcase of a curated personal dataset and works working in SD as simple prompts:
https://www.reddit.com/r/StableDiffusion/comments/12635ck/surreal_sd_images_inpainting/
And all the logic behind there is here (putting this cause it's relevant to what you asked):
https://www.reddit.com/r/StableDiffusion/comments/125xhpp/finetunning_test_testing_stable_tunner_first_time/
(I dont know if links are visible, if not tell me). Peace!