Trained for PonyXL with anime screencaps - thanks to https://fancaps.net/ - using kohya_ss. I would mostly recommend a weight of ~0.9. You can spawn the high value potion with "aphrodisiac".
Description
Trained on 96 anime screencaps. The metadata is preserved in the .safetensors file for exact kohya_ss settings.
FAQ
Comments (22)
Do you recommend using hires fix?
I use it all the time so yes.
@dude_ Interesting thank you. I thought it wasn't as useful in XL but I'll try it now.
excellent work,will you consider make a XL lora of Fern?
Thanks. Probably but she is not at the top of the list right now.
Could you share your training settings, like how many repeats, how many epochs and training rate, please?
You can check the metadata of the .safetensors file. I don't know why Civitai doesn't show them here. It's not exactly a secret 🤔
@dude_ I don't have a way to check the metadata, i don't have a computer that's good enough, I intend to train with google colab
@dude_ nvm found an online tool to scan metadata, thank you for your time.
Hey, really nice Lora.
Could you do one for Maomao in the future?
Thanks, yes I was planning to :)
Hi! Can you please explain which kohya settings you used for train, I have already tried a bunch of them and still cant get a good result (I would be very grateful for json if possible)
Hey, you can download the .safetensors file and check the metadata. It contains all kohya_ss settings.
@dude_ I ve already tried this, but I got a bad result anyway. As I saw you are using the batch size 4, on my rtx 3080 this causes out of memory crah with an same settings, despite the fact that I tried to reduce the batchsize, the error did not disappear, I suspect that this is due to an incorrect lr rate, which needs to be selected manually. Although maybe thats probably not the case, so the metadata didnt help me much, which is why I asked for your config. Its sad if you cant share it(
@degurshaft Hey, ya that might be a memory issue on your end. Try using `fp8_base = true` with the settings. Regarding the config file: It's the same as the metadata so technically I shared it already but here you go:
```toml
pretrained_model_name_or_path = "/models/ponyDiffusionV6XL_v6StartWithThisOne.safetensors"
train_data_dir = "your_training_dir"
shuffle_caption = true
caption_extension = ".txt"
resolution = "1024,1024"
cache_latents = true
enable_bucket = true
min_bucket_reso = 512
max_bucket_reso = 2048
bucket_reso_steps = 256
output_dir = "your_lora_output_dir"
output_name = "lora_name"
save_precision = "bf16"
max_train_epochs = 30
save_every_n_epochs = 5
save_last_n_epochs = 999
train_batch_size = 4
max_token_length = 225
xformers = true
seed = 0
gradient_checkpointing = true
mixed_precision = "bf16"
fp8_base = false
clip_skip = 1
optimizer_type = "AdamW8bit"
optimizer_args = [ "weight_decay=0.01", "betas=0.9,0.99",]
lr_scheduler = "cosine"
min_snr_gamma = 5.0
unet_lr = 0.0002
text_encoder_lr = 5e-5
network_module = "networks.lora"
network_dim = 16
network_alpha = 16.0
scale_weight_norms = 1.0
persistent_data_loader_workers = true
```
I don't use fp8_base although it reduces vram usage by a lot, is faster and results should be mostly comparable. However I experienced more broken image generations (still rare though) - might also have been a coincidence 🤔 this setting is new ~1 month old and impacts aren't yet fully known. Alternatively you could try a train_batch_size of 1 with the prodigy optimizer:
```toml
pretrained_model_name_or_path = "/models/ponyDiffusionV6XL_v6StartWithThisOne.safetensors"
train_data_dir = "your_training_dir"
shuffle_caption = true
caption_extension = ".txt"
resolution = "1024"
cache_latents = true
enable_bucket = true
min_bucket_reso = 256
max_bucket_reso = 2048
bucket_reso_steps = 64
output_dir = "your_lora_output_dir"
output_name = "lora_name"
save_precision = "fp16"
max_token_length = 225
xformers = true
save_every_n_steps = 250
max_train_steps = 3000
seed = 0
train_batch_size = 1
gradient_checkpointing = true
mixed_precision = "fp16"
fp8_base = false
clip_skip = 1
optimizer_type = "Prodigy"
optimizer_args = [ "decouple=True", "weight_decay=0.01", "d_coef=0.8", "betas=(0.9, 0.99)", "use_bias_correction=True", "safeguard_warmup=True",]
lr_scheduler = "linear"
min_snr_gamma = 5.0
learning_rate = 1.0
network_module = "networks.lora"
network_alpha = 16.0
network_dim = 16
scale_weight_norms = 1.0
persistent_data_loader_workers = true
```
Edit: Also keep in mind that the data you train on is the most important part. A good selection of images for the dataset and correct tags are key.
@dude_ Which card do you use for training, and how long does it take to train Lora? An ordinary Lora with 1200 steps with batch 2, being cooked for 4 hours or an 10 hrs with diff settings on my rtx 3080, which feels abnormally long even for sd xl.
@degurshaft Uh ya sounds painful, I'm using 4090. How long it takes really depends on the settings, especially the number of steps and batch size, from 30 minutes to 3 hours usually. I think you should stick to prodigy since it requires less steps on average for good results and combine it with fp8_base.
@dude_ think I ll probably have to buy at least 3090...
@degurshaft That would certainly be a big help. From what I've read the 4090 isn't even much faster for training due to bad optimizations for 4090. Image generation is faster though.
smash
It would be nice to have the ability to =ω= she makes in the show. There's not really an easy way to get it without mixing expressions since Frieren appears to be the only character who makes that face.
Have you tried this LoRA?
https://civitai.com/models/420063/w-frieren-elf-expression-or-concept-lora-or-pony-xl
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.