This is an experimental attempt to convert a standard SDXL model to V-prediction one with Zero Terminal SNR schedule with a fixed colors using a relatively small diverse dataset of some hand-picked images.<h1 id="v6-a7mfmfe42">V6</h1><h2 id="you-must-set-sampling-type-to-v-prediction-and-apply-zero-terminal-snr-patch-otherwise-you-will-get-noise.-h90rfanzu">You must set sampling type to V-prediction and apply Zero Terminal SNR patch, otherwise you will get noise.</h2>This checkpoint was continued from <a target="_blank" rel="ugc" href="https://civarchive.com/models/435207/artiwaifu-diffusion">ArtiWaifu</a> using a modified version of <a target="_blank" rel="ugc" href="https://github.com/kohya-ss/sd-scripts">https://github.com/kohya-ss/sd-scripts</a> on ~1750 images with text encoder frozen. Compared to 5.X, switching to an actual finetune fixed weird and uneven loss scaling across layers, which allowed the model to learn how to show the full color range.<h3 id="unlike-v5-and-5.1-v6-is-a-&quot;normal&quot;-checkpoint-not-a-lora.-umlj3pgya">Unlike V5 and 5.1, V6 is a "normal" checkpoint, not a lora.</h3>The flaws are:<ul><li>Doesn't work well that great if merged to Neta 2.</li><li>Some concepts and poses appear to be overfit.</li><li>Some styles, poses and other knowledge isn't transferred well (if at all).</li></ul><h1 id="v5-and-5.1-9n2w1f7s7">V5 &amp; 5.1</h1><h2 id="to-use-it-you-must-apply-it-as-a-lora-to-another-artiwaifu-based-checkpoint-such-as-artiwaifu-or-neta-art-2.0-set-sampling-type-to-v-prediction-and-apply-zero-terminal-snr-patch-otherwise-you-will-get-noise.-fqvi2hdsw">To use it, you must apply it as a lora to another ArtiWaifu-based checkpoint such as <a target="_blank" rel="ugc" href="https://civarchive.com/models/435207/artiwaifu-diffusion">ArtiWaifu</a> or <a target="_blank" rel="ugc" href="https://civarchive.com/models/410737">Neta Art 2.0</a>, set sampling type to V-prediction and apply Zero Terminal SNR patch, otherwise you will get noise.</h2>This checkpoint was continued from <a target="_blank" rel="ugc" href="https://civarchive.com/models/435207/artiwaifu-diffusion">ArtiWaifu</a> using <a target="_blank" rel="ugc" href="https://github.com/KohakuBlueleaf/LyCORIS">https://github.com/KohakuBlueleaf/LyCORIS</a> and a modified version of <a target="_blank" rel="ugc" href="https://github.com/kohya-ss/sd-scripts">https://github.com/kohya-ss/sd-scripts</a> on 1066 images.You can use this sample workflow for ComfyUI: <a target="_blank" rel="ugc" href="https://files.catbox.moe/cd3ao0.png">https://files.catbox.moe/cd3ao0.png</a>Someone was confused about whether this is a lora or not, and it's certainly should not be called a "LoRA" as in <a target="_blank" rel="ugc" href="https://arxiv.org/abs/2106.09685">https://arxiv.org/abs/2106.09685</a>. As to what exactly is this, you should ask kblueleaf.The flaws are:<ul><li>Full color range isn't learned due to OUT5 layer frying out. I blame it on the way LyCORIS trains individual layers.</li><li>It's very unstable and fails to work properly outside of 1024x base resolution.</li><li>Some styles, poses and other knowledge isn't transferred well (if at all).</li></ul>

artiwaifutistic

<ul><li>switched to standard unet finetune</li><li>refined dataset</li></ul>

Files with SHA256: 37b1158fc182ec250b5bd34bc4791c7113c4aef341dd05a96fa0d98e78f0fab0