Besides lowering the strength of the lora you can try adjusting it from the prompt. Avoid words like "muscular", "toned", etc. and use more words and description for the body type you want. You can lower the lora strength more drastically to about 0.5 but you must compensate in description of the prompt.