First, please take the time to watch the video, you wont regret it.
I am proud to introduce to you an innovative workflow solution for all of your prompting needs! Prometheus is a custom model along with 30 Hypernetworks I like to call Latent Later Cameras. These are virtual cameras that have been embedded in to the latent space which allow you to choose the camera angle and camera shot for your subject. These cameras put you in to the directors seat so that you always get the shot you envisioned in your mind. Besides being very flexible, the latent cameras also tend to be very cohesive and produce very good results through natural language prompting. That means there is no need for a word salad in your positive or negative prompts. In fact you can use the Hypernetworks without any negative prompts at all and still get very good results. As long as you follow very basic rules outlined in the video guide, on average you will get very good results. Thank you!
Camera shot and angle cheat sheet. https://postimg.cc/gallery/V645cMS
Huggingface link. https://huggingface.co/doctorderp/prometheus.safetensors/resolve/main/prometheus.zip
Please note the Hypernetworks work with all models...that are based on the 1.5 architecture... sorry if I misled in the video. Sorry no 2.0 or 2.1 support.
Don't forget to enable your dynamic prompts by checking the enable box, otherwise you can load the Hypernetworks manually, just know that the strength of the Hypernetworks should be at 0.55 by default, otherwise you will not get good results.
[Tip Jar](https://www.paypal.com/donate/?business=9B4V7D6BBUNPL&no_recurring=0¤cy_code=USD)
Discord if you have any questions = dr.derp#7202 or just dr.derp
Description
FAQ
Comments (27)
`there is a token that corresponds to a certain image "render angle"` as i understood.
Looks mad, much clearer than not precise "from x", "looking at x". Though using tokens is as complicated as playing ASCII game.
This is really innovative and hopefully will be adopted and improved upon wth all models going forward. Great job.
Can you please upload Hypernets and Wildcarts separately from the model?
Great work. You need to make more of a point about keeping the CFG at 3.5. I was getting weird artifacts on the faces when using this, I re-watched the video and saw you mention the CFG should never be moved. Some people won't watch (all) the video, so mentioning it in the text would be better :) Thanks for your hard work.
Doesn't work for me. The instruction video wasn't much help either.
It works great!
Thank you for sharing your hard work with us.
This is a game changer tool, it shoes the possibilities not yet explored. I can only hope that someone can expand the idea for head positions and so many other things.
Again, thank you!
I made some wildcards to use the hypernetworks more intuitive:
This is great, but it will be amazing if it ever includes rotation, so you can choose looking down on something vs looking up.
I don't have those specific angles trained but I've found that interpolated angles work well in some instances. try adding "from below" or "overhead view" at the very beginning of the prompt, you get decent results sometimes.
hey man this is great, could we get a sdxl version?
I would love to port my Latent Layer Cameras to SDXL. That was the first thing I tried to do, but Hypernetwork training does not work for SDXL and I have not been able to replicate this with Lora's so far.
@drderp this kinda work on sdxl tho have you tried it?
if you can share training dataset we can work on a lora together if you want
@MrNoir After reading your comment I felt that maybe I did not spend enough time working out the kinks on the SDXL model side of things. So I decided to do a lot more tinkering with the Lora's to make sure I could bring these cameras to SDXL. With renewed effort I've had good success so far, so I think I will spend this week working overtime to brink this project to SDXL. Hopefully within 2 weeks they will all be ported to SDXL.
Any plans to make an inpainting version of this model?
The inpainting actually works really well with the hypernetworks. So you would load up the 1.5 inpainting model as base and throw the accompanying hypernetwork on top at 0.55 strength and go to town.
When I try to open the zip-file on linux, it always gives me an error... Zip maybe broken on last update?
I got the same issue
My most absolute gratitude goes ahead for such a good idea of being able to control the camera angles to generate our images. That said, I have a problem. Sometimes, you know SD is arbitrary, when using your camera wildcards, I get an Out of Memory CUDA error. I suppose it will be a problem with my GPU (3Gbs VRAM) when loading the hypernetworks, because with the other wildcards that I use I have never gotten that error, but I would like to know your opinion and if you have any advice to solve it. Because I really love using your camera control in Auto1111. Very thankful.
3 gigs is very little so I can see how you might get this when loading the hypernetwork. As these hypernetworks are on average a larger then your average 1.5 lora. theres a thread here https://www.reddit.com/r/StableDiffusion/comments/12fmefm/help_for_low_vram/ for someone with a 4 gig card might help you there. Id try the launcher settings he suggests there. --xformers --listen --api --no-half-vae --medvram --opt-split-attention --always-batch-cond-uncond
@drderp Thank you very much for the help. I'm going to try that suggested configuration and comment, in case it can help someone else. Thank you, wonderful work.
@drderp So far, after several images generated with your wildcards, it works perfectly. I have added the suggested settings and it is working fine for me, even when Hires.Fix is activated. As I have read, with this version 1.6 of auto1111, xformers are no longer necessary, because with version +2.0 of Torch you can now use the 'sdp scaled dot product' optimization, which seems to be better. But with xformers so far it is optimizing well for me. I'm going to try the 'sdp' optimization to see if it improves VRAM usage. Thank you so much.
Fantastic! One small niggle - it doesn't always produce the specified camera angles, but the checkpoint is by far and away right up there with the best - many thanks
The hypernetworks are very sensitive to settings, make sure that the settings are exactly as described in the video, so the cfg scale, sampler type, step count, and everything else especially the resolutions and the 4 w's should be as described. Ive done a lot of tests with the Prometheus model and it has been giving the angle 85 percent of the time. If getting the angle less then that somewhere the settings are off or not using the 4 w's when prompting. Besides that I know the hypernetworks aren't as responsive with some other checkpoints but should still be above 65% of the time. Cheers!
@drderp yep! - I wasn't always using the 4 w's in testing....
I'm curious, there doesn't seem to be any documentation regarding all the extra wildcards included with numbers after: a2-2, a2-3, etc
those are lower strengths of same latent camera. what they do is lower stability of the camera but increase the interpolation ability within that angle. basically it makes the latent camera more flexible wit less stable