I wrote in the tutorial the reasoning about the sks and why it is no longer a bad token

you can even see it in the last section of the video: if it is not trained it indeed generates a person with a weapon, but when it is trained - the weapon does not appear at all, pretty much never

as for the other part, I've already written in another discussion about it, if you have hundreds of models it is much easier to have just one same token:

+ You don't have to memorize or write down unique tokens for each model (which becomes a problem if you have hundreds of them)

+ You can save your prompts with the token and you don't have to modify those prompts (for example, in the styles dropdown in A1111)

- you are right that it is not possible to use multiple people in the same generation, however, to use it properly you would need to train both concepts into a single model from the get-go, otherwise, they blend in (other people already mentioned that if you use LORAs that touch the same part of latent space then you may get weird results); in my opinion, it is much better to do the scene first with one model and then load the second model and use inpainting to fix the second character

"and change this in your tutorial" - this part I agree with, I could change from sks to zwx or what was the other famous token commonly used? had 4 letters and there was x somewhere, but can't remember it now

@WalterB anyway, I have decided to make a test, please pick one existing lycoris model of mine, I will retrain it with another token (`zwx` or one of your choosing)

then you (or other people) would make multi-people generations (without inpainting) to see what could even be made

if we see that those generations are easy to do, then perhaps I will change my mind about this :)

@WalterB good choice, i will retrain her over the weekend :-)

btw, quick question: someone on reddit mentioned that the main usage of mixing models is not to have two or more people but one person that is a mix of the used models - is that a thing for you as well?

i've read in the docs that for LyCORIS the network should not exceed 64 otherwise there might be problems and the convolution dim should be less

this was by trial and error and 64/32 work rather well in this case

you need to click on the export button and there you will have templates/settings for exporting, you edit one or create another to fixed width/size of 512 and make it active, you select not to upscale it

as long as that preset is the only one (or the first) selected then you will have the 512x512 selection box

also there you can select format (I pick png)

it is the elevenlabs

i trained it on 5 minutes of morgan freeman speaking :)

it was 11labs voiceover :)

as for dreambooth, I followed the nerdy rodent guide which I've linked, there would be no point in me redoing his steps when his guide was pretty much superb

if you follow his guide, you will have the same base of dreambooth as me, then just use the scripts with my parameters and you should be golden :)

-> fight with installing ubuntu on windows

to install ubuntu on windows is very simple, just select wsl2 from the Microsoft store and just install it (ubuntu is the default linux i believe)

-> then fingers cross to install overlay d8ahazard over installed DB

i do not use it so, can't really help you with that, but the improved version that I'm using was fairly easy to install

well, the whole process took me a weekend back then, but that pretty much is dreambooth for you

however the point of this tutorial was to showcase a different way to make lora/lycoris, starting from a dreambooth

hey hey, i missed your comment somehow

I used eleven labs, provided 5 minutes of his voice and after a short moment I was able to use it :)

Yes indeed! That is what I do.

Well, it was mentioned briefly, because it is a small thing. All the scripts are pasted in the written guide.

To summarize it quickly: I use the kohakuBlueLeaf script "extract_locon.py"

For me the easiest way was to just install kohya_ss and use the same venv for the locon.

The example usage has all the parameters set the way I do it.

If you need some more help, just let me know :)

@technerd in my opinion - yes, this method is better

Nerdy Rodent made a nice test comparing quality of dreambooth and lora trainings: https://www.youtube.com/watch?v=gw2XQ8HKTAI

Turns out that dreambooth is better ( using same dataset )

Not sure what exactly, but extraction makes an improvement over that. Yes, it takes more time and needs more VRAM but training a model is a single action so it is worth spending a bit more time on that to achieve greater results :)

@truefalse I have 11 GB VRAM and I use most of it

there is a setting to lower it to 8 but i never tried that one (also it says that it is at a cost of quality, allegedly)

@technerd "Technically" it shouldn't matter as we extract only what we've trained (in layman's terms) but it indeed matters what is used as a base model, judging from my experience

at the moment I use Realistic Vision 2.0 as a base and it serves me well.

previously I used my own model merge and it served me well, but RV is better so I switched

there are several great photorealistic models so if you find something that suits you best - you can that :-)

@truefalse i'm not using that dreambooth version so it would be hard for me to do a video on it, however i can recommend Olivio's guide: https://www.youtube.com/watch?v=9Nu5tUl2zQw

He usually does it in a way that is great for everyone (advanced and beginners)

@semafor good question, this might be something I should put (expand) in my guide. But for now, I'll write here. Yes, it is like you said.

I installed kohya_ss (the requirements.txt file)

As a second step, I open the kohya_ss venv with this command: .\venv\Scripts\activate

and then i go one directory higher and I git clone the kohakuBlueLeaf lycoris repo, still being in the venv of kohya. I go inside that new LYCORIS folder and I run it its requirements.txt which is very small (i think only one thing is installed? the rest already is in kohya_ss)

And then I run the script.

You could install it separately but I did not want to duplicate the venv with the same stuff, and since it is advertised as kohya extension I figured it would work on the same versions of those python libs and it does.

@semafor hey hey

1. by requirements I meant this: https://github.com/bmaltais/kohya_ss/blob/master/requirements.txt

but you used the setup.bat and it already calls it in line 42 so no need to do anything without it anymore, you're good ( pip install --use-pep517 --upgrade -r requirements.txt)

2. exactly! after you run this: .\venv\Scripts\activate

you will see the (venv) before your path name, this means you're in the context now :)

steps 3 and 4 look good :-)

in step5, as long as you are still in that (venv) then you can just write: pip install -r requirements.txt

and it will install the remaining stuff needed (if there is something still needed at this point)

step 6 -> Almost all good, that script is in the subfolder tools so before running that script just do: "cd tools"

7. Exactly! :-)

but in order to not type everything all the time, it's better to do some batch files, for example, I have run-venv.bat with this content: .\venv\Scripts\activate

and go-lycoris.bat

for you, it should have this content: cd ../kohakuBlueLeaf/LyCORIS/tools

one thing to note, the script you pasted: python extract_locon.py --safetensors --device=cuda --mode="fixed" --linear_dim=64 --conv_dim=32 C:/Development/StableDiffusion/stable-diffusion-webui/models/Stable-diffusion/realisticVisionV20_v20.ckpt F:/Development/StableDiffusion/SDModels/Models/2023-04-17/model_morgan_v1_realistic2_3000.ckpt C:/Development/StableDiffusion/stable-diffusion-webui/models/Stable-diffusion/locon/locon_morgan_v1_from_v1_64_32.safetensors

-> You need to change some values there, first is a path to the base model, then is the path to the dreambooth model and third is the path to the output lycoris :)

(also I suggest putting that command into batch file so you can just run it and change the values )

@technerd wow, great for the tip!

i will inspect that over the weekend and i guess i will need to update the guide, thanks!

I've had some success with that, but please let me know more details what do you have in mind?

For specific stuff like that, I change the class of course, it is no longer "woman" or "person" but perhaps "pose" or "position".

You piqued my interest, so let's explore this idea further!