V.3.0 - Negative Embedding ( Not a LoRA! )
Just wanted to get this out because i will probably use it for some pictures. Needed it to be a bit more aggressive cause i know the effect it has, especially in combination with some LoRAs that have tendencies to add fog, haze, washed out colors and clutter. Sometimes that is a wanted effect and adds atmosphere and some of the examples i posted actually look better without it imho. I know those pictures aren't the highest quality in the first place, just wanted to show the effect.
The main reason to do this one was because of a test i did with a LoRA that creates a really strange style ( might going to release, depends ) which i combined with several other LoRAs. Wasn't so successful at first cause one LoRA always overpowered the other and less strength for one meant the style or effect was gone... for some reason though this embedding made the combination much clearer.
Have to say though that was a pretty "push it to the limit" test where i knew the outcome would be terrible.
Like with pretty much all of my stuff, it's more based on luck then anything else. What i can say though is you might wanna lower the CFG scale if you use this one. I got better results with a lower CFG and by the look of it ( and pure speculation ) it's almost like it has the effect of a higher CFG scale and going to high might burn your retina.
Like last time, first picture is always the one without the Embedding
Kinda works on Pony, but with less effect. Produces funny things sometimes, especially if you don't use the score_up stuff, which actually forces particular styles that may all look the same, but not using it tends to also fkup anatomy.
V2.0 - Negative Embedding ( Not a LoRA! )
Behold this massive 5kb file that almost made me buy a whole data center so i can store this behemoth.
Maybe i should have posted this thing separately so it isn't in the LoRA category but... mehh. Could only select LoRA as an option while uploading the file.
First time trying to use textual inversion. Even though i like the LoRA version, because it has some weird effects, i was still looking for a solution for what i was really trying to do. Embedding seems like the better solution, even though it was for some reason harder to train ( and used more VRAM )
Now it kinda does what it should do, like getting rid of some annoying stuff ( like my arch enemies haze, fog, and artifacts... for the most part), make the images clearer, more contrast and for some reason it also adds random details, like the LoRA does.
I tested a few models and some have some weird reactions. I used a, even by my standards, pretty weird prompt that had an idea but it was impossible to produce anything useful. As soon as i used this Embedding it was like the difference between night and day. The image changed completely and was following the prompt a lot more. It was still a mess but i didn't expect something like this.
Again, i will use this thing for now to see what it really can do. Also will test more TI stuff.
The first image is always the one without the Embedding, same seed, same model.
The one with the flower really shows some of the strange behavior
V1.0
Nothing to see here, literally. The images this was trained on had an idea behind them, but where all a blurry and noisy over-saturated mess with scratches, blotches and over the top lighting effects like lens flare, chromatic aberration and caustics... and a lot of other things.
It's just an experiment that went into a completely different direction then original planed. I just post it here because i will probably use this LoRA for some images and if someone is interested to recreate something you might need this. Also, images using this will be cross posted here i guess.
Don't ask what it does because i'm not quite sure yet, but i got some ( for whatever reason ) positive results, even though it was meant for the complete opposite... or let's say as a helper.
Before someone goes like "Uhh, mystery LoRA clickbait", it somehow enhances effects of other LoRAs and makes everything a bit sharper and realistic ( or blurry and unrealistic ). It also can change something completely from strength 1 - 1.2 and negative strength to like -0.5. I don't know if that's just a placebo effect, but tests so far were pretty consistent with the same prompt and seed. One might assume setting a negative strength would be better, but it's the complete opposite. Actually extremely baffling.
Will be pretty much of no use to anyone but me for quite some time, till i figure out what to do. If it doesn't do what i assume it does it's gonna get deleted.
Hmm, playing a bit with it and after testing with the same seed but different strengths from positive to negative it went either in a total different direction, stayed within what was prompted but with mayor differences or it helped actually following the prompt, even crazy stuff.
Description
Added Negative Embedding Version
V2.0 is a Textual Inversion not a LoRA!
FAQ
Comments (5)
Yup, in kohya_ss at least, I’ve found that when training a TI and a LoRA on the same training images with as-near-as-poss the same settings, the TI needed more VRAM and took longer. I’ve been fiddling with SDXL style TIs for a while now and find that the before-after effect can get pretty weird, even discounting the primary effect of the style. You get that with LoRAs too of course, but not to the same extent as TIs I think. I hadn’t considered trying to make a negative TI to force the issue, but you’ve got me thinking... especially after seeing that flower pic. Did you use hazy/messy/mutant images for training?
In case it’s of interest I did some TI before-after galleries where the only difference is one letter in the prompt to disable the TI to make the before image. You can see them as the 2nd last galleries here:
https://civitai.com/models/243600/dark-world-dreams
and here:
Yeah, i use the same setting as i do in my LoRAs. Haven't tried to train on something more, let's say, artistic where it's more important to capture the concept.
The dataset has nothing in it what i would call recognizable ( no humans, animal, buildings etc ) just a bunch of blurry, weird mess... but the pictures did show a lot of stuff i wanted to get rid of ( lens flare, glare, overexposure, motion blur, depth of field, hazy, dust... and the list goes on and on )
The last three picture are also interesting, because every time i use the embedding the subject changes into a woman. I take a wild guess here that the AI didn't know what to do with the images and decided for itself how to interpret something and a lot of models are biased to produce good looking woman.
For the LoRA i didn't use any captions except for the "N-EEYBLCH" thingy, even though i never use it. The embedding has the "Init word" and the "Token string" ( at least that's what it's called in Kohya ). the init word's are a bunch of stuff i wanted to get rid of and the token string is also "N-EEYBLCH", but i never use that word, so i guess i just let the embedding free-roam.
After looking at your stuff i have to test something. I guess the token string is the actual trigger word. At first i thought you managed to somehow put 2 trigger words in the embedding ( on for on and one for off ), but that was more off me misreading what you wrote with changing only one letter. It's interesting that an embedding is so sensitive with the trigger word, a LoRA doesn't really care about that.
If i get the time i will train something else that has more style with both LoRA and embedding to compare them.
@TijuanaSlumlord The trigger is just the filename before .safetensors, so you can change it to whatever you want... but usually trying to avoid a word SD "knows" is good. Token string is just what kohya adds to the front of the saved .safetensors - no other effect. The thing with LoRAs not needing the trigger(s) I didn't know until I accidentally left it off once😃 Now I only add the LoRA trigger for extra emphasis - or maybe I'm fooling myself. For TIs the init word field in kohya is really important - use the wrong one and you can bleep up your training before you start. A lot of writings say just use one word, but actually you can use up to the number of tokens = number of vectors you've picked in kohya. So either I use a single word like "pattern" or "painting"... or I try out some prompts in a1111. Say I've picked 8 vectors... then I try out prompts in a1111 making sure the token counter is showing 8. Looking for a prompt that somehow relates to the concept I'm trying to train. Oh, and as with LoRAs, captions can make a big diff, but sometime the style template in kohya is fine and you can forget about captions.
I had the same thought about comparing LoRA .vs. TI using same training images and as close to same settings as I could. Results are here:
https://civitai.com/models/290258/paper-marbling-lora-sdxl
https://civitai.com/models/154898/marblingtixl
Training images and some info are also at the TI link. The TI seems to keep the sense of the training images kinda, and the LoRA more specific to the training images. I've only done that one test so who knows if that's the general result🤔
@chromesun Hmm, that's even more confusing. There are so many variables that can make or brake something it's ridiculous. I'm currently doing a test with a single image to train a LoRA with different settings but always with 1000 steps. Will probably do the same for TI once i figure out what works and what doesn't. Thinking about ever switching from SDXL to a new model is giving me a headache, because switching from 1.5 to XL was already bad enough when it comes to which settings to use.
@TijuanaSlumlord Interesting test 👍