Long story short, for the first ever I needed a turbo lora (extracted as diff) to be used on the base Klein 9B model and I was not satisfied with the ones I found so I tried making my own with twice the rank (256).
Thus, I'll be using this one and I'm uploading it even though its quite late to do so since Klein is pretty much a very old model by AI community standards. Its still the best open source image editing model - which is why I'm still using it.
You can see in the comparison image - the output of the rank 256 BF16 turbo lora is closer to the original output of the Turbo model itself VS the rank 128 FP32 one.
Recommended Settings
0.25 Strength ➔ CFG: 3.0 | Steps: 16
0.50 Strength ➔ CFG: 2.0 | Steps: 12
0.75 Strength ➔ CFG: 1.3 | Steps: 8
(check the included workflow in 'Optional Files' which shows how you can set a workflow to automatically handle those settings - and the ConditioningZeroOut node - for you, from a single subgraph widget input. Requires: https://github.com/bananasss00/ComfyUI-SP-Nodes)
ABOUT THE NEGATIVE PROMPT:
when using turbo strength >= 1.0: 'ConditioningZeroOut' node must be used - which means your negative_prompt is completely ignored.
when using turbo strength < 1.0:
'ConditioningZeroOut' node CANNOT be used.
negative_prompt should be completely EMPTY for regular text-to-image tasks because otherwise your output quality drops dramatically. This is a flaw with the Base model itself - nothing to do with the turbo lora.
short, non-empty negative_prompt is only useful in some edge-cases in Image Editing mode
Description
FAQ
Comments (8)
Could you provide example image of cases where you were not satisfied and cases where you were satisfied?
I was planning on doing editing tasks that require as much quality as possible because the outputs are meant for AI training and they will have to be iterated multiple times.
Specifically what I need to do is to remove speech bubbles and text from cropped manga panels then I have to colorize them into a anime-like style by stacking a lora I previously trained on that manga (to make it easier when filling in obstructed faces/clothing details as well as getting the right colors without having to retry 1 million times).
Using just the Base model would be overkill and using the 4-step Turbo model felt like I would be rushing through this without much care - so tweaking a turbo lora however I like would be the best approach. So I searched and found I think it was a rank 64 one on huggingface (which I immediately ignored and there were even some performance warnings written by the author himself so that one was a instant no go) then I found another at rank 128 here on civit. But I know that for a 9B model even a rank 256 lora wouldn't surpass the limit where you start to not see any difference by having a higher rank - or at very least, that limit is definitely past rank 128. On top of that the lora was saved with FP32 precision even though BF16 would have been just fine and translate to roughly half file size.
I couldn't find a rank 256 BF16 lora - which is what I was looking for - so I made it myself.
To summarize, my displeasure stems from my disagreement with rank and precision choices by the authors who made the other turbo loras I found - not by lack of output quality because I didn't even try the rank 128 one until I made my own and only tried it to make a 1:1 comparison - which, mine managed to beat in what might or might not have been a fluke result but I have what I wanted now so I'm just sticking with it.
EDIT: I'll see if I can do some comparisons later today
Did a more deterministic comparison - see for yourself: https://imgur.com/a/oXwfI3y
(obviously, same prompt and seed used)
@Winchester99 I have checked the images. Regardless of which is better or worse, there does appear to be a difference. Instead of using an external site, you can add the comparison images directly to this page by clicking the three-dot button located to the right of the blue "v1.0" label (which indicates the LoRA version) at the top of the page.
@fantaseed I know I just did not feel like that would be meaningful but yeah I'll do it. The test at full strength is a conclusive comparison that leaves no room for doubts - you said 'Regardless of which is better or worse, there does appear to be a difference' so I think you may have missed the whole point. At full strength (1.0) - a turbo lora should output exactly the same image as the turbo model does (as close as possible), and you can see mine is much closer.
Is there a particular use case for each of the 3 recommended settings?
Just trying to understand the thought process.
The turbo model was distilled with RL alongside from base - meaning even though it only requires 4 steps, it will still output better looking images overall but its often less creative and not really better at everything. plus, it requires CFG=1 which means you cannot use negative prompt with it.
Add in loras trained on the base model for many steps (or multiple loras stacked together) then suddenly the equation further leans in base's favor.
So, in those cases should we just use base instead and call it a day?
Not necessarily. Turbo influence can still bring in benefits in the form of improved composition and - obviously - speed.
This lora allows you to control said influence in cases where you may feel compelled to use the base model.
ex 1: you are doing sensitive editing tasks where the outputs are meant for monetized projects or AI training and so you really want them to have good quality.
ex 2: you are using a lora trained on mostly monochrome inputs so its biased towards those but you want colored outputs. adding something like 'colored' in the prompt shifts the style you are aiming for too much so you found out that using 'monochrome' in negative prompt actually yields best results.
This so far only explains why using base model+adjustable turbo lora gives you much more freedom than just sticking and/or switching between full base or full turbo models - it still doesn't explain the 3 settings I've recommended.
About those settings there's not really much to say - they are just well balanced settings for 3 different turbo influence percentages (that I found through testing) but you can obviously use different values if you want. Using higher CFG means both your prompt and negative prompt will be more respected but since that requires less lora strength - it will also require higher steps, which means slower inference.
You choose whatever fits you better for whatever specific task you are doing.
@Winchester99 Thanks for the elaborate explanation. Appreciate it!