This Model Increases "Quality"
Now, you might be wondering "What is 'quality'" and I, unfortunately, can't directly answer that question, however, I can tell you that this model was trained on images that were generated with the abhorrent quality tag monstrosity that Pony Diffusion V6 XL has, but the captions didn't include the quality tag. this means it was trained to emulate the output of adding the quality tag, without adding the quality tag.
Why would I do this?
Well, I was very dismayed to find that the quality tag was so absurdly long and very poorly controllable, so I decided to make a LoRA to control the quality with a slider. this allows a more fine-tuned and varied approach to quality control and additionally saves on tokens in the prompt (which is useful to avoid unwanted BREAKS in attention)
How did I do this?
I used the LECO training script on GitHub by P1atdev based on the LECO paper to train this model, the LECO training process generates an image at an arbitrary denoising strength, and then trains on the difference between the output of the model when prompted for a concept, and when unprompted. this allows the model to alias tags, words, concepts, or phrases to any arbitrary prompt, in this case, I aliased
score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_upthe "quality paragraph" to
so I effectively was training it to always produce images that looked as though it had been prompted for the "quality paragraph"
Donations
Speaking of training, training models is expensive and I run training on my own private server, if you like what I do, consider supporting the development here!
Key Benefits
One of my favorite key benefits of this approach is that it makes "quality" a modular and controllable thing. adding weight to the quality tag has a somewhat enigmatic effect on the output however, this LoRA/LECO has very clearly defined and comprehensible changes that you can control the severity of by altering the weight of the LoRA/LECO which is an intended operation (as opposed to weighted prompts being a hack applied to the attention layer that doesn't always have the desired effect)
The other benefit is that this LoRA/LECO doesn't use up the 33 tokens that the "quality paragraph" uses! that's like half of an entire context window! eating up the context window forces the backend you are using, whether it be A1111, InvokeAi, or ComfyUI to add invisible BREAKs in attention that damage the overall coherence of the prompt you are constructing and can lead to other unintended consequences.
Quirks
v1 of this model is quite impotent and seems to operate stably with a weight between 2-3, but it continues to produce outputs that are recognizable up to weight 6
v3 has standard weight behavior, use it like you might a normal LoRA
v3 requires you to use a rating and source tag in your prompt
This model was only trained and tested on PonyDiffusion V6 XL! I do not guarantee compatibility with other models!
v1 of the model definitely alters backgrounds to appear more "painterly" and it means the backgrounds will fall apart significantly faster than the subject at higher strengths. if you care a lot about the background of an image, you might want to only use this model supplemental.
I have yet to fully test v4, please let me know of any strange behavior exhibited by it.
While I have tried to isolate the "quality" concept in the model and remove it from similar concepts, it is only possible to go so far with it. It may alter the content of your generation in unintended ways. if you wish to discuss these abnormalities, please go to the furry diffusion server on Discord to bring them to my attention.
discord.gg/furrydiffusion
Once you have joined, I have a thread made for it here:
https://discord.com/channels/1019133813105905664/1214131180572639312
if you are wondering where v2 is, it was so bad I couldn't disgrace the community with it's publishing.
Description
Trained at a larger guidance scale so it is more potent
FAQ
Comments (8)
I don't understand. If I use this lora, do I like not have to write score 9,8,7 and so on?
that is correct for v 3.3, although you do have to specify a source and rating, which I would recommend most people do anyway.
@Yolup a source and rating?
@omegablast20023899
Source tags: 'source_pony', 'source_furry', 'source_cartoon', 'source_anime'
Rating tags: 'rating_safe', 'rating_questionable', 'rating_explicit
Finally someone else is trying to make Lora's like this as well.
The best thing about this, besides not having to use up all those tokens, is that it enhances the quality without destroying the look of other LORAs.
I train my own character LORAs pretty often and the default quality tags tend to completely ruin them. This maintains both style and subject while adding much needed visual fidelity.
Thanks. :)
How did you train this exactly? I'm trying to make another slider but I am unsure what "target, positive, unconditional" means.
it might be easier to talk over discord about this, but for transparency's sake, I had to read the papers to understand what target, positive, unconditional are. but I think it is best summed up as follows:
positive: concept/tokens you want to inject
target: when you want to inject the positive/unconditional
unconditional: the negative tokens you want to inject (unconditional is just the official term for negative prompt)
neutral: the tokens you want to ignore
for the sake of the example, I am using the "enhance" method, not the "erase" method, but the concepts apply similarly.
so if I want to train a model to make large breasts, but only when prompted for "kobold" and I want it to never make clothing when prompted for "kobold" then I could prompt for
positive: "large breasts"
target: "kobold"
unconditional: "clothes"
neutral: ""
the neutral is used to "disentangle" concepts, which is described in the sliders paper. basically, if for some reason the concept "large breasts" is more likely to create a red body, then you can do something like
positive: "large breasts"
target: "kobold"
unconditional: ""
neutral: "red body"
I use a more advanced version of this in my training setup, I have modified the script to generate wildcard prompts on the fly for training so that it disentangles several concepts at the same time and also retains and do take this with a grain of salt, while I am attending uni for AI stuffs, I am currently very sleep deprived and I didn't reference the papers directly when writing this, perhaps I should write a guide sometime on it that is proper if you have any more questions, I will respond publicly here, or if you want quicker responses, I am active on my discord account: yolup