Recommendations: 1) don't train on realistic vision, it becomes an inflexible model, should use the base sd-v.1.5-pruned

2) dataset is key of course.

3) prodigy is an optimizer that handles its own learning rate, unet and text encoder rates, it might help you not have to worry about those things, provided your vram can do it.

4) when you describe the images for the lora training, captioning, describe everything that isn't normal or isn't the character. example: don't say she's blonde, because that's her normal self, describe her hair being curly, wavy, tied up - and only say its color if its of a different tone.