This DoRA is trained on a 1.5k images of varying belly shapes and belly clothing states for noob_vpred. I felt like it was better to combine and improve my existing belly shape LoRAs into one for ease of use and training. Not to mention noob_vpred training results are just better than pony.
Update 2025/02/17:
forgot to upload v4 weeks ago, but this new version is just better at all shapes. Use the same tags as v3.
This was really just a test of a new optimizer ADOPT mixed with learned_timestep_loss_weighting, trying to find the best combination to make it work. There is still room for improvement, but this was the best of 1.5 weeks worth of test runs, lol
Important tags (same as Hyperfusion tags):
torpedo belly
sagging belly
distended belly (when the sides or ribs are being stretched by belly)
spherical belly
stuffed belly (the top half of the belly is same or larger than the bottom)
wide belly (only works with straight-on poses)
to a lesser extent:
low navel (navel is placed low on the belly)
hanging belly (unlike sagging, the character must be bent over in some way)
exposed belly
clothed belly
innie navel
outie navel
prey shaped belly (didn't learn well)
Probably others I haven't tested yet from the hyperfusion tags list
belly_sizes:
medium belly
big belly
huge belly
gigantic belly
hyper belly
hyper pregnancy
obviously some belly tags the model already knows were also included:
pregnant belly
bloated belly
chubby belly
etc...
Description
This is the first version of an all-in-one DoRA for belly shapes. I may add more shapes in the future, but 200 images per shape adds a decent amount of training time for each additional.
1.5k images, extracted from my hyperfusion dataset, so you can use most belly related tags from that.
Training notes:
Kohya's trainer
optimizer ADOPT
optimizer_args
"decouple=True" "weight_decay=0.01" "betas=(0.9, 0.999)" "eps=1e-7"
DoRa LoCon
frozen text encoder (increased training time, but I prefer to not touch the TE if possible)
lr 4e-4
dim 16
alpha 8
conv_dim 8
conv_alpha 4
batch 8
GA 4
1.5k images
flip
bucket
resolution 1024
tag dropout 0.1
dropout 0.2
caption_dropout 0.1
scale_weight_norms 10
ip_noise_gamma 0.02
min_snr_gamma 2
zsnr
v_pred
Extras:
soft_min_snr instead of the default formula
learned timestep loss weights, a small network to learn the loss scale for each timestep. similar goal to min_snr
sort important tags to the front and sort separately from others
tag implication dropout for all common implied tags ~40% drop