While trying to generate some images for Halloween, I discovered that the models I tried didn't know what a headless horseman was. On top of that, most models couldn't even be prompted to make a headless person. Flux, in particular, had issues when I tried to convince it that it was okay for someone to ride a horse without having a head. So, I decided to see if I could teach the character to Flux with a LoRA.
Trigger the LoRA with "headless_horseman."
This LoRA is made as part of the KI Welten Discord Group Collaboration. For the other Great Halloween-themed Flux LoRAs, follow the link to the KiWelten Halloween Collection.
Full Version 1.1
After some testing and rebaking, the previous issues with version 0.9 are resolved. Use the full version at a strength of 1 (+/- 0.1 or 0.2, depending on your prompt's difficulty). Use simply by substituting "headless_horseman" for headless horseman in your prompt:
headless_horseman galloping across a vast desert at dusk, the orange sky fading to deep purple. The rider holds a jack-o-lantern with a mischievous grin, and the horse kicks up clouds of sand behind them. Western with warm, muted tones and a cinematic feel to the vast landscape.
The LoRA sees the horse and rider as a single character, so expect some resistance if you try to remove the horse or change it into something else. It's possible but not easy. Future versions may fix this as I experiment with editing the captioning.
Beta Version 0.9
I used my normal LoRA training settings, but the model may be underbaked. At least, that's what I think happened. It works pretty well with a strength of 1.3 to 1.5 but still gets confused if the prompt mentions features like faces, teeth, hoods, etc. However, I did get some pretty good generations out of it. I'd appreciate it if anyone could try the model out, post what you can make, and give any feedback on how the LoRA functions. I'll be working on the next version in the meantime.
Description
After some testing and rebaking, the previous issues with version 0.9 are resolved. Use the full version at a strength of 1 (+/- 0.1 or 0.2, depending on your prompt's difficulty). Use simply by substituting "headless_horseman" for headless horseman in your prompt.
FAQ
Comments (8)
I should have read the notes! I posted sample pics, which are cool, but not quite the headless horseman. Those were at strength of 1 so I will retry at higher strength as you suggest, and if I get good output, will post those as well.
It's not a problem; any data is good data. I'm glad to see more images and prompts at 1.0 to troubleshoot. This is my test case for a difficult character LoRA on Flux, so its to learn more and for fun. Version 1.0 will hopefully be out today; I've got some good test images after leaving it to bake overnight. I'd be great if you can retry your earlier prompts once it is released and see how they work.
@Dark_infinity I will, thank you! That's a great attitude btw, I appreciate that you took the information I gave in a constructive manner!
I completely agree this is a very difficult character. Flux definitely wants to map a head to a body. I have run into similar issues with my loras where an alien body doesn't have the same location of things as a human... Flux fights it.
I just posted a few at 1.5 strength with the current .9 beta for you to see as well. Definitely headless! I got probably 3-4 unusable gens at that strength though just FYI.
@EnragedAntelope Those you just posted are great! But I agree. Using the LoRA creates the character, but at 1.4 strength, it also causes some distortion and other issues. I've gotten really good results with the newer version using only 1.0 strength, so hopefully, it'll solve the issue and provide some guidance for other trainers.
@EnragedAntelope And it's up! Let me know if version 1.1 works better at 1.0 strength and gets you better quality generations.
Here's an absolutely crazy thing I've discovered with your lora... I have a workflow that turns out very blurry images - unless I'm using your lora. Even when the image has absolutely nothing to do with the headless horseman, the results always turn out dramatically better when I use this. I have no idea why. 😆
That's really neat but also odd. The only thing I can think of is that I tried to get a variety of art styles in the dataset, so maybe it has something to do with the combination of images I put together. Can you post some before and after pictures? I'd love to see the transformation.