Kohaku XL beta
An anime SDXL model trained on 1.5M images.
Notice: Further experiments have showed that beta7 is generally better than beta7pro. Beta7pro have some mosaic artifact (more significant than beta7) and beta7 have better txt-img alignment. So I rearrange the order of models.
Introduction
This model is resumed from [Kohaku-XL alpha](Kohaku-XL alpha - nyan | Stable Diffusion Checkpoint | Civitai) with 1.5M images and then merged with other models.
Usage Details
This model is very flexible on resolution, you can use the resolution you used in sd1.x/2.x to get normal result(like 512x768), you can also use the resolution that is more native for sdxl(like 896*1280) or even bigger (1024x1536 also ok for t2i).
recommended negative prompt for anime style:
photorealistic, 3d model, bad, worse, worst, ugly, bad anatomy, blurry, close-up, disembodied limbphotorealistic, text, icon, artist name, signature, twitter username, naked, nude, monochrome, blurry, bad anatomy, watercolor, oil paintingwatercolor, oil painting, photo, deformed, realism, disfigured, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry
Training Details
Kohaku-XL beta5
This model is trained on new-danbooru (danbooru images with id from 5,000,000~6,600,000) which have 1.48M images.
This model is resumed from kohaku-xl alpha7 and then merged with NekoRayXL.
Kohaku-XL base4 (haven't publish yet)
This model is trained on new-danbooru (danbooru images with id from 5,000,000~6,600,000) which have 1.48M images.
Thie model is resumed from sdxl-0.9 (due to some bad property in sdxl-1.0 which will affect finetuning). In the plan this model will be trained with 2epoch (about 94.5k steps).
I haven't published this pretrained model yet.
Kohaku-XL beta7
This model is merged with base4 and beta 5, the formula is:
beta(5+n) = beta5 + (n/4) * (base4 - sdxl0.9)So beta7 is beta5 + 0.5 * (base4 - sdxl0.9)
Kohaku-XL beta7.1(7pro)
as same as beta7 but use the finished base4 and 0.25 weight.
Note: the base4 here is the 50k step version!!
Future Plan
I will run training on base4(after it finish) in Mynefactory dataset or CyberMeow(alea31415)/Narugo1992's reg dataset.
Acknowledgements
Models
Description
Use finished base4 to merge with beta5
(beta7.1 = beta5 + 0.25 * (base4 - sdxl))
FAQ
Comments (40)
🤲 👍
You are the pioneer of the 2D SDXL model.
1024*1024图片的细节很多
muy buenas ilustraciones
I am still in awe this model can generate really unusual scenes and compositions, most of which i'd probably have a hell of a time to generate with any other of the XL models i have.
And i'm not even using utility loras (most of the time: you'd better not, degrade faces) or ControlNet 😮
At worst ADetailer can fix the little wonky bits here and there.
Thank you again for it ^^
I can't seem to get it to finish the generation. It always gets right to the end and then turned into a deep fried mess, no matter what sampler or settings I use. Even when copying the prompts and settings for the above images exactly, it still doesn't work.
1, set VAE to None if you are using sd-webui
2, use exponential or polyexponential scheduler if you are using DPM++ series sampler
Sounds like you're using a Non-XL VAE, get yourself the SDXL VAE and try with that.
Why aren't you uploading the next version?
WIP
cannot load this on Comfyui
Really need the SDXL Turbo version of CheckPoint
I think you can just "merge" them.
青叶大佬的模型~
When will the next version be uploaded? Don't you have any plans?
I'm doing experiments on my new dataset system. Which will be the beta8 (if it do show something better).
And I will start making a better dataset (up to 2~4M imgs) and train a Kohaku XL 1.0
The thing is no one know if my dataset is better
and since I don't have A100 or A6000, I cannot use fp32 weight to train SDXL. which means I will suffer from the precision problem in some optimizer.
It is really hard to say when will be the next version
What I could say is I already waste over 500$ on electric bill after publish the Beta7.
No update bcuz all of them are trash, not bcuz I give up.
琥珀青叶你能不能自己练一个模型？不要用我的模型微调了？我的模型标注了禁止融模的。你自己练的模型效果差，跑过来融我的模型干什么？
Amber green leaves, can you practice a model yourself? Don't use my model to fine-tune it? My model is marked as forbidden to melt. The model you practiced yourself is not effective. What are you doing here to melt my model?
Announcement
I'm doing experiments on my new dataset system. Which will be the beta8 (if it do show something better).
And I will start making a better dataset (up to 2~4M imgs) and train a Kohaku XL 1.0
The crucial thing is no one know if my dataset is better
and since I don't have A100 or A6000, I cannot use fp32 weight to train SDXL. which means I will suffer from the precision problem in some optimizer.
(but if my experiment show some evidence that my dataset is ok, I may rent some A100)
It is really hard to say when will be the next version
What I could say is: I already waste over 500$ on electric bill after publish the Beta7.
No update bcuz all of them are trash, not bcuz I give up.
Thanks a lot for working on this! Your model is one of my favourites and I'm happy to hear that you are trying to improve it even more.
Have you considered starting training from new (3.0) animagine model? It's style is a bit meh, especially backgrounds, but from my initial tests it's stability and prompt understanding is phenomenal compared to other SDXL anime models. Tails don't turn into mess, asking for "wariza" or "the_pose" works reliably, etc. So maybe it would be a cheaper way to get something that looks like Kohaku but with better prompt understanding.
Even if you don't want to start training from it, maybe some techniques used by them will be useful for you.
@Hirumilio Fun fact: Linaqruf and Me actually use each others' model for training base (or other usage) frequently, or, in general speaking, we are collaborating with each other. Next version of kohaku-xl will also utilize the result of aniamgine xl 3.0. And anxl will use my result in the future too.
@Hirumilio Fun fact 2: there are some of techniques used in anxl3 is inspired by me.
Thank for your great work , we all appreciate you sincerely because it is your model improved the drawing ,and a lot of chinese uploaders have publicize your name on bilibili.
@lin40423 UwU/♡
Please use sdxl_ Origin_ v1.0 source model was trained, and the Kohaku XL model was trained on the origin_v1.0 model. However, since the origin model is not an SDXL source model, the derived model is not suitable for further training. I have not publicly disclosed the structure of the origin model, so please use the source base model for fine-tuning
Kohaku XL is finetuned with the sdxl_Origin_v1.0.the model is not a native SDXL model, its a modified model. If you want to finetune a sdxl model, please use the native sdxl_Origin_v1.0 version, otherwise it will damage the Origin infrastructure and cause a significant decline in model quality.
The characteristics of the Origin model can lead to the fusion of Origin models only being applied to the Origin's own ecosystem, isolated from other SDXL model ecosystems. When refining Lora, please use the source version of SDXL_ Origin_ V1.0.
琥珀青叶你敢不敢用自己的版本继续训练？融我的模型干什么？自己练自己的模型去。
Do you dare to continue training with your own version? Why melt my model? Practice your own model.
琥珀の青葉は自分のバージョンでトレーニングを続ける勇気がありますか？ 私の模型を溶かして何をしますか？ 自分の模型を練習して行きます。
所以您的意思是10月21号发布的模型融了10月31号发布的模型是吗……没别的意思啊,就是问一下
@FcCat i think you made your point. if you can prove any illegitimate use of your model, then you are ofc free to take legal action against the uploader. until then please refrain from further spamming the comments section with the same issue. thx
为什么在这个模型下用不了controlnet的openpose功能呢？
如果你用的是sd1.x的controlnet那是因為這是SDXL模型
如果你用的是SDXL的controlnet那代表那個controlnet的transferebility是廢物。
Love this model. It's like an XL version of Sudachi. Easy to get beautiful results with anime appeal.
Hello, kblueleaf! I was experimenting with different XL VAEs I have downloaded and noticed that the VAE I got from this page is in some cases twice as fast as other VAE. What is the reason?! Is there a newer or better version of this VAE available someplace? I can't find any information on it but the speed is noteworthy.
The VAE in my model is FP16fixed version of SDXL vae.
other SDXL VAE cannot work in FP16 (can only work in bf16 or fp32). And I assume you are using sd-webui which will automatically fallback to fp32 which result in half speed.
@kblueleaf I thought maybe it was FP16. I am using sd-webui you're right. I have also using other FP16 VAE but at least based on my tests yours was faster than those, "FIX FP16 Errors" VAE.
What would cause it to fallback to fp32?
For context, I am running the latest sd-webui with an RTX 4090. It's fast but I feel I should be getting more it/s than I do which is why I was playing with VAE.
beta, alpha, delta都有啥区别
希腊字母α，β，Δ，理解成机器人1号，二号，三号...就行了
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.