Loras, whose only changes are their breasts. Sometimes they can change their Asian face to a European one.
Description
FAQ
Comments (14)
Hey This looks amazing, how did you manage a lora that doesn't change anything else than the subject.
Unnecessary layers of information, which are always present during training, have been removed. Some of the breasts were obtained by extracting them from other models that were originally intended for anime stylization.
@Viennar are you removing blocks?
@sekoseko Yes, different blocks/layers are responsible for different things, environment/location, style, texturing
How do you pick out the blocks? Comparatively, or do these tend to be the same blocks for different loras (as long as the content is similar)?
Thanks for the reply and sharing your insight.
@firemanbrakeneck I run the generation for each block separately, there are 30-36 of them in total, then I leave the ones I need and adjust the strength.
@Viennar There is a comfyUI tool for that. It never made any sense to me, but it was pretty easy to ID which blocks had no effect on a lora, I just couldn't do anything with that info.
Which training are you using? And is there a guide to specify blocks for training?
@brnfd24434343d In 99% of cases, all 30 blocks influence the lora, but only 2-6 blocks influence exactly what the model was originally trained on. https://github.com/destinyfaux/Z-Fuse
@Viennar You mean with all of the other blocks turned off? Isn't there complex interaction between blocks sometimes, such that eliminating what appears to be a redundant block actually unbalances the output on a whole? Or do blocks somehow just naturally tend towards independence? Does that also happen with other bases and types of loras (say, within the "modifier" category, or even in characters or styles), that you've seen?
@brnfd24434343d Which tool would that be? I can maybe think of some ways (more reliably when given the dataset), but curious to see how it does that exactly.
@firemanbrakeneck That's why I generate each block separately and see what happens. The sampler preview helps, so I can see what's changing in the first steps. And I also think that all the lora is just instructions for restoring noise, because I don’t see a single lore that would bring fundamentally new information.
@firemanbrakeneck The one I know of is RealtimeLora comfy node. The user is ShootSomething. It works really well. Just use it as a single lora loader and adjust the slides and turn off layers. I still don't really know how @Viennar is doing it because it's not only a SLOW process but it's hard to determine what is positive change on a PER LAYER basis... I tried with groups of layers like 6 at a time (Qwen has like 60 layers) and it was slightly ambiguous at times.
@brnfd24434343d thanks for the realtime lora node pack suggestion. I wasn't able to make z-fuse work. Realtime lora worked like a charm.