Research Model - How to Build Protogen
ProtoGen_X3.4 - Enbrace the ugly, if you dare...
By Downloading you agree to the
Seek Art Mega License, and the CreativeML Open RAIL-M
Model Weights thanks to reddit user u/jonesaid
Running on Apple Silicon devices ? Try this instead
Trigger words are available for the hassan1.4 and f222, might have to google them :)
Seriously, removing off (ugly) on negative prompts brings out some really detailed shots of what real life consist of, decay, rubble, grass, worned clothing...Have fun and keep it fluffy!
Live Demo at Available on Hugging Face
Comprehensive guide to camera control
This is Protogen_v2.2 with 5% of the following
roboDiffusion_v1.ckpt
openjourney-v2-unpruned.ckpt
analog-diffusion-1.0.ckpt
rpg_v2Beta.ckpt
The result is Photorealism, with RPG elements, Scifi, and some creative flow from OpenJourney model.
Let me know how are the outputs on The Dicussion Below, and I will continue to release more models based on the feedback...Enjoy!
Description
FAQ
Comments (78)
Can you please add trigger words so it would be easier to know how to use the model properly?
Thanks!
looking at the sample generations it seems to be "modelshoot style, (extremely detailed CG unity 8k wallpaper)"
It's a mix of models that have no particular trigger words, the exception is modelshoot, so "modelshoot style" should have some sort of effect.
Thanks!
Don't use for commercial?
Updated it to research model
what VAE you suggest?
Can you please point to the best yaml and vae files please
Was generating images of batman with other models, changed to this one and learned that it has no idea who batman is. interesting. Doesn't know superman either.
how you removed negative prompts from model?
I'm having a bit of trouble with faces... I tried copying the generation data from the images posted here and the faces still look weird. Any tips?
Could you post one of your images? I could take a look at all the parameters and see what I get when using those settings.
I'm having the same issue, I think it is the lower resolution that is the problem, But I can't test any larger than 768x768 so I don't know.
Edit: It is the lower resolution that is causing problems, you can upscale images with fixres and faces look as amazing as the preview images.
New to SD, any tips on some prompts that can help get the sort of models I'm seeing? :o
if you click the i symbol on bottom right corner it shows the prompts used
hei! nice work! but which "promt words" you are used? for this mans, and womans ?
The images on the model preview are on the far lower right, its a white I system, it has all the information on the prompt
anyone have thoughts on why nmkd would show incompatible when the 22 version works fine
Hello, can you give me some manual to instal?
do you recommend using x5.3 over this or this one still has some merit? it's not clear from the descriptions :)
Hello, can you give me some manual to instal? HELP PLEASE
Great tutorial
thank you, but there the video is in .ckpt format, and I downloaded it here in .safetensors format. Should I download everything again?
no the safetensors is all you need. you pick one or the other. Safetensors > .ckpt unless you have some technical reason for doing so.
Hi if you are not running this in Stable Diffusion Automatic or similar convert the Safetensors file to .ckpt. The easiest way to do this is to downdoad NMKD Stable Diffusion GUI this is a one click install easy to setup and use. when installed top bar: Convert Models: Allows you to convert model weights between Pytorch (ckpt/pt), Diffusers, Diffusers ONNX, and SafeTensors formats. point this to the folder containing Safetensors file, ensure Safetensors is selected in the drop down menu then convert it will save the ckpt file in the same folder. Move ckpt file to "wherever installed"\stable-diffusion-webui\models\Stable-diffusion this answer is not specifically for you but this question keeps on being asked in various forms and being answered by technically competent people who tend to miss steps out because they always expect others to be as capable as them. Huw
If the model is creativeml-openrail-m, you can't arbitrarily say "no hosted services". That's already be explicitly allowed by the license. You need to pick a new license for that to be applicable and enforceable. Though considering what it's based on, that isn't possible to relicense it.
is this for hugging face ? because its fixed on what can only be selected
Yeah, listed in the license that it's linking: https://huggingface.co/spaces/CompVis/stable-diffusion-license
I can't use this in Dreambooth, keep getting an error when trying to create a model, HELP!
also the rest are listed here too
https://huggingface.co/darkstorm2150/Protogen_x3.4_Official_Release/tree/main
It's great but literally doesn't make black people....
give this prompt a try (came from reddit), also its a bit nsfw, so careful prompting at work :)
beautiful ebony woman, highly detailed, detailed realistic dark skin, detailed realistic rasta hairs, detailed realistic expressive black eyes, detailed realistic thicc lips, intricate detailed dress, intricate, detailed, 4k, hdr, masterpiece, african tribal style, ((full body view)), detailed ancient mayan temple as background, bokeh
I trained the model with dreambooth with photos of my Rwandan girlfriend and I had very good results afterwards
Please! Create a model to create paintings in the style of Dan Mumford! At the moment, the best AI that can do this is MidJourney :(
Does this work to produce dark skinned, or black people? I am trying to find one that isn't white-washed.
Try brown skin as a modifier. Works with 2.2 and get a brown to black tone based on other things like name,ect. Also try by nation of origin "ghana, mexico, ect" as a modifier. You can also get African American characters by using certain styles of naming common in American culture. Gl.
@Jetro30087 thank you!
In my experience, f222 does pretty well about it; you could also try "dark skin" as part of your prompt, I've found that helps.
Detected: Trojan: Win32/Sirefef!cfg
Almost All that I downloaded
Use an offline version like “invoke Ai.” Or in a Virtual machine
Misunderstanding or not I am going on what I have seen.
from windows Defender after scan
Detected: Trojan: Win32/Sirefef!cfg
Status: Remove failed
This treat or app might not be completely remediated.
Date: 2023/01/21 12:38
Details: This program is dangerous executes commands from an attacker. “Severe”
Hey bro, I use paid Kaspersky antivirus and it doesn't report any viruses
@Pikaro - https://civitai.com/user/claulis - Gave an explanation - will keep it in mind, that falls positives are a possibility, and generally Happens with window Defender
thank you for the follow up.
Claulis response was.
(1) For the pickled and infected file issue, it is a false alarm, the developer who did the finetune of Anything-v3.0 (on hf) confirms it as a false positive. Some versions of SD get false flagged.
(2) As the detection footprint for this particular trojan is a mere 2048 byte date/timestamp footprint as reported by trend micro, I'm inclined to think false positive as it is very feasible for byte conversion of floats to match such a small heuristic test over a large number of gigabytes.
Microsoft Defender AV is typically the only one that reports it. We have tested with Bitdefender, Kaspersky, NOD32, and ClamAV here and nothing.
* (1) https://huggingface.co/Linaqruf/anything-v3.0/discussions/15
* (2) https://www.trendmicro.com/vinfo/us/threat-encyclopedia/malware/troj_sirefef.cfg
@Pergetoryofdream there is safetensors available, the ckpt are there because people use them outside of automatic1111, and are requested even with false positives..
never rely on windows defender.. just dont
screw this license
I can post the photos generated by me, on my social midia,instagram, facebook? non-profit, I don't need to mention anything?
of course, the license is usually for websites for profit and such..
@darkstorm2150 oh I understand, thank you very much
Can someone please share prompts!!
My generated content is nowhere like any of yours.
Click on any image and you will see hints
What are the best resolution settings for this model?
Please answer this question: The way I read the license, the only thing I'm allowed to do with this model is use the model to generate pretty pictures for my own personal use? I can't generate artwork and print it on a canvas that I then sell at a local market, I can't design character art or cover art for products? Basically, any art I make I'm not allowed to sell in any way? If this is true, what is the point of using any Protogen models?
Just use other models with commercial opportunities
i just saw this. omw. been wondering if I should keep using protogen as the outputs are very off sometimes now I have a reason to wipe this model
From the License:
"Personal use Exception. As an individual unaffiliated with any commercial entity or business, you may use model output (images) for commercial purposes, respecting section 5.
5. Use-based restrictions. The restrictions set forth in Attachment A are considered Use-based restrictions. Therefore You cannot use the Model and the Derivatives of the Model for the specified restricted uses. You may use the Model subject to this License, including only for lawful purposes and in accordance with the License. Use may include creating any content with, finetuning, updating, running, training, evaluating and/or reparametrizing the Model. You shall require all of Your users who use the Model or a Derivative of the Model to comply with the terms of this paragraph (paragraph 5)."
https://huggingface.co/coreco/seek.art_MEGA/blob/main/LICENSE.txt
PROMPT BUG ISSUE
Hey there,
Initially gr8 work with the Protogen models, I love em.
I'm writing here as There is no other means to communicate with you as the developer. I'm sure you are aware of the prompt bug with Protogen
Has there been any solution to that or at least an intention to solve this issue?
If not, I'm currently writing the prompts so that they lead with a comma or underscore, would that be enough to solve the issue?
Hi, the solution is to sacrifice a char, for example "a beautiful portrait of a cat" should be "_a beautiful portrait of a cat" using _ , should suffice.
Thank you for this absolutely essential information! It means all my tests were wrong and I have to retake them...
Normal and Pruned versions are completely different models. (both models have same hash: ef8629e2c8)
For other models I get exactly the same results on fixed seed but for this one I'm getting two completely different images. It would be great if someone tests it
I want to train my own checkpoint model. I have about 10,000 high-quality pictures. How can I do it? Does anyone know? If you provide relevant technical documents or solutions, I will be very grateful
try dreambooth
@thushanthan thanks，let me try
nice!
Hi there,
These are lovely results - I'm trying to recreate them as a way to calibrate my local setup. I'm using (presumably) the same model, and getting very different results - still lovely, just different lovely. I'm using exactly the provided prompt, seed, steps etc. The only thing potentially missing is the resolution used for generation (I'm using the local web ui from https://github.com/AUTOMATIC1111/stable-diffusion-webui). Can you share the resolution used for generation, and what machine/setup was used? again, this is a sanity check to calibrate a local setup, as I'm somewhat new to this...
Is it necessary to use any VAE? In the demo on huggingface I often got blue spots on the persons.
Has anybody got it to run with the "Accelerate with OpenVino" script for Intel Arc?
The pruned version gives me errors: NotImplementedError: Could not run 'aten::_local_scalar_dense' with arguments from the 'Meta' backend.
And the full version gives me different results than what I get online with the model.
Can this model do NSFW? The mentioning of Hassan and F222 makes me think, yes!?
Hello, when I try to make an image using your model (512*512, 30 steps), the program generates the image for 10-15 seconds, and then is interrupted by the following “error: Could not load the stable-diffusion model! Reason: Cannot copy out of meta tensor; no data!". Although my friend has your model working stably. Please, can you tell me how to fix this?
Hi! Cool checkpoint! All example image are kinda anime/cartoonish which is nice if you are going for that. Would be awesome if you would like to share example image prompt of a realistic photograph render. :)
How do I know which version of the model to download and what is the difference between the four versions here?
3.4 stopped working for me. New A1111 1.9.0 installation.
It says "Error: Protogen... checkpouint not found: Loading Fallback. But I definitely have it. I re-downloaded. When selecting it and rebooting, it loads, but does not generate anything resembling my prompt. Just some paper stuff. And if I get it to generate after restart, the quality is abyssmal. Other models are OK. Something got messed up. Full model also as abyssmal quality suddenly.
Due to the fact that this model is already very old, it looks quite fresh ... :) Unusual faces, different composition and reaction to prompts. Of course, it is already very outdated, but in some places it looks good. Thanks! It was once a very cool model...
Any of Protocgen checkpoints works in KRITA..
Clip ERROR at generating start...¿?
It was developed for only AUTOMATIC1111/stable-diffusion-webui.
I didn't know KRITA uses stable diffusion models
... with a pluging named "Krita AI Diffusion" that installs a custom server inside with ComfyUI. ok, that should be the problem. Sorry. Thanks for yout time.
Details
Files
protogenX34Photorealism_protogenX34.ckpt
Mirrors
3666_protogenX34Photorealism_1.ckpt
ProtoGen_X3.4.ckpt
protogenX34Photoreal_1.ckpt
protogenX34Photorealism_1.ckpt
ProtoGen_X3.4.ckpt
protogenX34OfficialR_1 (2).ckpt
protogenX34Photorealism_1.ckpt
protogenX34OfficialR_1.ckpt
protogenX34OfficialR_1.ckpt
ProtoGen_X3.4.ckpt
ProtoGen_X3.4.ckpt
protogenX34Photorealism_protogenX34.ckpt
protogenX34OfficialR_1.ckpt
model.ckpt
protogenX34Photorealism_protogenX34.safetensors
Mirrors
3666_protogenX34Photorealism_1.safetensors
protogenX34Photorealism_1.safetensors
Protogen_X3.4.ckpt
protogenX34Photorealism_1.safetensors
protogenX34Photorealism_1.safetensors
ProtoGen_X3.4.safetensors
protogenX34Photoreal_1.safetensors
protogenX34OfficialR_1.safetensors
protogenX34Photorealism_1.safetensors
protogenX34OfficialR_1.safetensors
ProtoGen_X3.4.safetensors
protogenX34Photorealism_1.ckpt
Protogen_x3.4.safetensors
protogenX34Photorealism_1.safetensors
protogenX34Photorealism_1.safetensors
2.5D：protogenX34Photorealism_1.safetensors
protogenX34Photorealism_1.safetensors
protogenX34Photorealism_protogenX34.safetensors
Mirrors
protogenX34Photorealism_protogenX34.safetensors
protogenX34Photorealism_1.safetensors
ProtoGen_X3.4-pruned-fp16.safetensors
ProtoGen_X3.4-pruned-fp16.safetensors
protogenX34Photorealism_protogenX34.safetensors
protogenX34Photorealism_1.safetensors
pgen_34_pruned.safetensors
Protogen x3.4.safetensors
protogenX34Photorealism_protogenX34.safetensors