prodigy 和 none ฅ/ᐠ｡ᆽ｡ᐟ \
丹炉 https://github.com/WalkingMeatAxolotl/AnimaLoraStudio

默认参数 改了 factor 到 4
https://github.com/WalkingMeatAxolotl/AnimaLoraStudio

I don't quite remember the exact data. I think it should be about 70 images with 2 repeat, 70 reg images, and eventually 76 epoch. ฅ/ᐠ｡ᆽ｡ᐟ \

@DualChimerra I don't remember the bs and ga for this lora, but I normal used as higher bs as I can to speed up the training (like 3 when use cloud computing and 1 in local). I will try to maintain bs * ga less than 4 so the resolution won't impact too much within ARB bucket.

Based on my understanding, reg images works when the train set has very unbalanced tag distribution. For example, pantyhose takes 100% of chen_bin. Without reg images, lora might not work well generating non pantyhose images. However, I didn't do a precise experiment to compare them ฅ/ᐠ｡ᆽ｡ᐟ \

@DualChimerra Hi, the most effective change I think is refilter the training set. I remove some images and add new ones. I croped some images into only feet, so 1024 resolution can still learn the pattern of pantyhose context. Scale up is also used during processing data set.

For training config, it is hard to say if which of them is better, because I think lora training is with high randomness. For this version, I used lora with weight decomposition instead of lokr; add noise iteration with 6 + 0.5; change to optimizer lion with starting learning rate 2e-5. I also test some dim reg during v3.0 - v3.8 but v3.9 doesn't.

Being honest, I still prefer V2.2 more than V3.9, that why v3 trained on anima base 1.0 has been delayed such long time /ᐠ𝅒▿𝅒ᐟ\ﾉ

@DualChimerra so happy to hear you tried anima lora studio. Is there anything you think is not a good experience and need improvement /ᐠ ᵒ̴̶̷̥ ‸ ᵒ̴̶̷̥ ᐟ\ﾉ

@DualChimerra this is one used for Onineko. I don't have a consist config recently. I change config every time to test if studio has some bugs. The data set is 57 images repeat 2 and 57 reg images repeat 1.
```

resolution: 1024

reg_caption: null

reg_weight: 0.3

shuffle_caption: true

keep_tokens: 1

flip_augment: true

tag_dropout: 0.0

prefer_json: true

cache_latents: true

lora_type: lokr

lora_rank: 32

lora_alpha: 32.0

lokr_factor: 4

lora_dora: true

lora_rs: false

lora_dropout: 0.0

lora_rank_dropout: 0.0

lora_module_dropout: 0.05

lora_reg_dims: null

epochs: 80

max_steps: 0

batch_size: 1

grad_checkpoint: false

grad_accum: 2

learning_rate: 1.0

lr_scheduler: none

lr_scheduler_t0: 500

lr_scheduler_t_mult: 2.0

lr_scheduler_eta_min: 1.0e-06

optimizer_type: prodigy_plus_schedulefree

prodigy_d_coef: 1.0

prodigy_safeguard_warmup: true

ppsf_d_coef: 2.0

ppsf_prodigy_steps: 0

ppsf_beta1: 0.9

ppsf_beta2: 0.99

ppsf_split_groups: true

ppsf_split_groups_mean: false

ppsf_use_speed: false

ppsf_fused_back_pass: false

ppsf_use_stableadamw: true

weight_decay: 0.0

kv_trim: false

noise_enhancement_type: pyramid

noise_offset: 0.0

pyramid_noise_iters: 6

pyramid_noise_discount: 0.5

timestep_sampling: logit_normal

timestep_shift: 3.0

timestep_mix_low_prob: 0.0

timestep_schedule_shift: 1.0

infonoise_enabled: false

infonoise_K: 64

infonoise_N_warm: 0

infonoise_M: 100

infonoise_B: 256

infonoise_beta: 0.9

infonoise_N_min: 50

loss_type: mse

huber_c: 0.15

loss_weighting: none

min_snr_gamma: 5.0

weight_cap_ratio: 0.0

detail_inv_t_min: 1.0

detail_inv_t_max: 5.0

grad_clip_max_norm: 1.0

mixed_precision: bf16

attention_backend: flash_attn

num_workers: 0

output_dir: G:\AnimaLoraStudio\studio_data\projects\21-onineko\versions\v2.2\output

output_name: onineko_v2.2

save_every_epochs: 4

save_every_steps: 0

save_state_every_epochs: 0

save_state_every_steps: 1000

seed: 42

resume_lora: null

resume_state: null

```

@DualChimerra LOVE YOU ฅ/ᐠ｡ᆽ｡ᐟ \ ♡