Look at prompt examples. Negatives is important too!

I tried to collect best dataset, but i'ts very hard.

I captioned "parted lips" because they looks bad in most cases and it's on most of the dataset, i used 0.5 weight in prompt to avoid it in every generation.

If you need to lower weight of lora you may use "Curly hair" to avoid change hair to "default". Do not forget to use Adetailer.

Try to use different samplers, some make messy details on face at the end.

Try to use "extra lids" in negatives of Adetailer, i think it helps in some cases.