dataset composition with a ratio about the same as equal weight
128/128 network and convolution dimension/alpha
5e-5 u-net and 1e-5 text encoder learning rate
polynomial learning rate scheduling with a power of 0.5
learning rate cycle reset every 500 steps
no min snr gamma
batch size of 1
nesterov adamw optimizer (torch.optim.nadam) with args of "betas=[0.9,0.99] eps=1e-08 weight_decay=0.1 momentum_decay=0.004 decoupled_weight_decay=True"