This is trained on a set of DPO pairs in order to try and push Klein 9B away from the waxy skin and arm/leg/hand issues that it can suffer from. No special prompts, just try from weight 0.5 - 1.0, you can try negative weights as well and see what happens.
v0.1 - Test version
As a courtesy, if you'd like to merge it into one of your models, please ask first.
Description
More, but better
FAQ
Comments (12)
I may be ignorant in the ways of humans...
But what does DPO stand for?
I will say, it does seem to help some of the washed out images I have gotten in Klien.
Other than "days past ovulation", this is what I found:
Direct Preference Optimization (DPO) uses pairs of images (preferred vs. dispreferred) to train the model to increase the probability of generating better images.
Amazing work! This does help a lot.
Do you mind sharing details on how did you train Klein with DPO?
Which tools did yuo use, how much compute, etc?
I generated a large set of images from Klein 9B using a script in batches of 6 per prompt, then had a local VLM (qwen-2.5) run through the each batch and pick the 'best' and 'worst' according to a set of criteria I gave it.
Each pair is captioned with the prompt used to generate them, then I used AI toolkit to build a lora 'slider' with the worst images as the -1 on the slider and the best as +1 on the slider. You'll find if you go negative you start getting many more 'bad' images.
@anyMODE you should consider trying it again with a more recent vision model of qwen. I could see it making better judgement since Klein uses it as a text encoder.
Thanks for the inspiration, I've been piling up my to-do list and now you've motivated me to attempt making a Stabilized and Improved Preference Optimization (SIPO) LoRA. https://arxiv.org/pdf/2505.21893v3
can you do SRPO? it seems more robust than DPO
At that point you're better off using Z image since it SRPO would be VRAM exhaustive. SIPO trains from the same chosen and rejected pairs as DPO, so it maps much more naturally to a general DPO-style LoRA. That would make more sense being used as a way to counteract DPOs flaws.
I've did bunch of tests and it actually fixes the scene so things make more sense (still not all), limbs, rooms, etc (+anatomy fixer lora even better).
Skin? Not really a problem in klein.
Hair. I wish it would fix hair. But it's the same "greasy" klein hair as always.
Must have lora for sure.
Limbs are the hardest and seem baked in to the model as a result of being able to edit things. Using this seems to reduce the number of issues with them though.