New version is out: <a target="_blank" rel="ugc" href="https://civarchive.com/models/628865/sotediffusion-v2">https://civarchive.com/models/628865/sotediffusion-v2</a>Anime finetune of Würstchen V3.This release is sponsored by <a target="_blank" rel="ugc" href="https://fal.ai/grants?rel=sote-diffusion">fal.ai/grants</a>Trained on 6M images for 3 epochs using 8x A100 80G GPUs.This model can be used via API with <a target="_blank" rel="ugc" href="http://Fal.AI">Fal.AI</a>For more details: <a target="_blank" rel="ugc" href="https://fal.ai/models/fal-ai/stable-cascade/sote-diffusion">https://fal.ai/models/fal-ai/stable-cascade/sote-diffusion</a> Please refer to Huggingface for SD.Next UI, Diffusers or UNet models: <a target="_blank" rel="ugc" href="https://huggingface.co/Disty0/sotediffusion-wuerstchen3">https://huggingface.co/Disty0/sotediffusion-wuerstchen3</a> CivitAI page has only the ComfyUI checkpoint models. <h3 id="inference-parameters:-h0i2a8fq2">Inference Parameters:</h3>Download the Main model (8.14 GB file):<a target="_blank" rel="ugc" href="https://civarchive.com/api/download/models/563950?type=Model&amp;format=SafeTensor&amp;size=pruned&amp;fp=fp16">https://civarchive.com/api/download/models/563950?type=Model&amp;format=SafeTensor&amp;size=pruned&amp;fp=fp16</a> Download the Decoder model (4.24 GB file):<a target="_blank" rel="ugc" href="https://civarchive.com/api/download/models/563892?type=Model&amp;format=SafeTensor&amp;size=pruned&amp;fp=fp16">https://civarchive.com/api/download/models/563892?type=Model&amp;format=SafeTensor&amp;size=pruned&amp;fp=fp16</a>Positives:<pre><code>newest, extremely aesthetic, best quality,</code></pre>Negatives:<pre><code>very displeasing, worst quality, monochrome, realistic, oldest, loli,</code></pre><h3 id="main:-e3o1xo1nd">Main:</h3>Sampler: DDPM or DPMPP 2M with SGM Uniform CFG: 7 Steps: 30 or 40<h3 id="decoder:-s5ikrpa4j">Decoder:</h3>Sampler: Euler a Karras CFG: 1 or 1.2 Steps: 10Compression: 42 (or 32 to 64)Resolution: 1024x1536, 2048x1152.Anything works as long as it's a multiply of 128.<h1 id="training:-pf661l1ds">Training:</h1>Software used: Kohya SD-Scripts with Stable Cascade branch. <a target="_blank" rel="ugc" href="https://github.com/kohya-ss/sd-scripts/tree/stable-cascade">https://github.com/kohya-ss/sd-scripts/tree/stable-cascade</a>GPU used: 8x Nvidia A100 80GB GPU hours: 220<h2 id="base-gvk49eumc">Base</h2>parameters | value<ul><li>amp | bf16</li><li>weights | fp32</li><li>save weights | fp16</li><li>resolution | 1024x1024</li><li>effective batch size | 128</li><li>unet learning rate | 1e-5</li><li>te learning rate | 4e-6</li><li>optimizer | Adafactor</li><li>images | 6M</li><li>epochs | 3</li></ul><h2 id="final-75m2n5zdv">Final</h2>parameters | value<ul><li>amp | bf16</li><li>weights | fp32</li><li>save weights | fp16</li><li>resolution | 1024x1024</li><li>effective batch size | 128</li><li>unet learning rate | 4e-6</li><li>te learning rate | none</li><li>optimizer | Adafactor</li><li>images | 120K</li><li>epochs | 16</li></ul><h1 id="dataset:-cxh4lqwko">Dataset:</h1>GPU used for captioning: 1x Intel ARC A770 16GB GPU hours: 350Model used for captioning: SmilingWolf/wd-swinv2-tagger-v3Model used for text: llava-hf/llava-1.5-7b-hfCommand:<pre><code>python /mnt/DataSSD/AI/Apps/kohya_ss/sd-scripts/finetune/tag_images_by_wd14_tagger.py --model_dir "/mnt/DataSSD/AI/models/wd14_tagger_model" --repo_id "SmilingWolf/wd-swinv2-tagger-v3" --recursive --remove_underscore --use_rating_tags --character_tags_first --character_tag_expand --append_tags --onnx --caption_separator ", " --general_threshold 0.35 --character_threshold 0.50 --batch_size 4 --caption_extension ".txt" ./</code></pre> dataset name | total images<ul><li>newest : 1.85M</li><li>recent : 1.38M</li><li>mid : 993K</li><li>early : 566K</li><li>oldest : 160K</li><li>pixiv : 344K</li><li>visual novel cg : 231K</li><li>anime wallpaper : 105K </li><li>Total: 5.628.499 images</li></ul>Note:<ul><li>Smallest size is 1280x600 / 768.000 pixels</li><li>Deduped based on image similarity using czkawka-cli</li><li>Around 120K very high quality images got intentionally duplicated 5 times, making the total image count 6.2M</li></ul><h1 id="tags:-jzbj7tx2d"> Tags:</h1><h3 id="tag-format:-b15c2cmis">Tag Format:</h3>Model is trained with random tag order but this is the order in the dataset if you are interested:<pre><code>aesthetic tags, quality tags, date tags, custom tags, rating tags, character, series, rest of the tags</code></pre><h3 id="date:-o7yxmif0j">Date:</h3><ul><li>newest : 2022 to 2024</li><li>recent : 2019 to 2021</li><li>mid : 2015 to 2018</li><li>early : 2011 to 2014</li><li>oldest : 2005 to 2010</li></ul><h3 id="aesthetic-tags:-2zs1t8sgj">Aesthetic Tags:</h3>Model used: shadowlilac/aesthetic-shadow-2<ul><li>score &gt; 0.90 : extremely aesthetic</li><li>score &gt; 0.80 : very aesthetic</li><li>score &gt; 0.70 : aesthetic</li><li>score &gt; 0.50 : slightly aesthetic</li><li>score &gt; 0.40 : not displeasing</li><li>score &gt; 0.30 : not aesthetic</li><li>score &gt; 0.25 : slightly displeasing</li><li>score &gt; 0.10 : displeasing</li><li>rest of them : very displeasing</li></ul><h3 id="quality-tags:-6p4f4nab5">Quality Tags:</h3>Model used: <a target="_blank" rel="ugc" href="https://huggingface.co/hakurei/waifu-diffusion-v1-4/blob/main/models/aes-B32-v0.pth">https://huggingface.co/hakurei/waifu-diffusion-v1-4/blob/main/models/aes-B32-v0.pth</a><ul><li>score &gt; 0.980 : best quality</li><li>score &gt; 0.900 : high quality</li><li>score &gt; 0.750 : great quality</li><li>score &gt; 0.500 : medium quality</li><li>score &gt; 0.250 : normal quality</li><li>score &gt; 0.125 : bad quality</li><li>score &gt; 0.025 : low quality</li><li>rest of them : worst quality</li></ul><h3 id="rating-tags:-i4ucnneao">Rating Tags:</h3><ul><li>general</li><li>sensitive</li><li>nsfw</li><li>explicit nsfw</li></ul><h3 id="custom-tags:-xdvkprq26">Custom Tags:</h3><ul><li>image boards: date,</li><li>text: The text says "text",</li><li>characters: character, series</li><li>pixiv: art by Display_Name,</li><li>visual novel cg: Full_VN_Name (short_3_letter_name), visual novel cg,</li><li>anime wallpaper: date, anime wallpaper,</li></ul><h2 id="license-iqyvf9v5p">License</h2>SoteDiffusion models falls under <a target="_blank" rel="ugc" href="https://freedevproject.org/faipl-1.0-sd/">Fair AI Public License 1.0-SD</a> license, which is compatible with Stable Diffusion models’ license. Key points:<ul><li>1. Modification Sharing: If you modify SoteDiffusion models, you must share both your changes and the original license.</li><li>2. Source Code Accessibility: If your modified version is network-accessible, provide a way (like a download link) for others to get the source code. This applies to derived models too.</li><li>3. Distribution Terms: Any distribution must be under this license or another with similar rules.</li><li>4. Compliance: Non-compliance must be fixed within 30 days to avoid license termination, emphasizing transparency and adherence to open-source values.</li></ul>Notes: Anything not covered by Fair AI license is inherited from Stability AI Non-Commercial license.

SoteDiffusion Wuerstchen3

Trained more. Currently trained on 704K images in total.Ran LLaVa on the images that has "english text" tag in it. This adds <code>The text says "text"</code> tag. If LLaVa has no idea what the text is, it describes the image instead.

Files with SHA256: ef5a36ec7181487256498bf5e064f029c3e88249589a314f1096b1fc7eea1608