360 Diffusion v1
Posted first on HuggingFace.
This LoRA model was finetuned on an extremely diverse dataset of 360° equirectangular projections with 2104 captioned training images, using the Stable Diffusion v1-5 model.
This model was finetuned with the trigger word qxj. If using the AUTOMATIC1111 WebUI, then you will have to append <lora:360Diffusion_v1:1> to the prompt as well in order to activate the model.
Rendering Tips
When rendering, it is recommended that you use either a 1:2 ratio or a perfect square. Rendering as a 1:1 square can help improve concept coherence (like the walls of a room).
Details can lose coherence at large sizes with txt2img, so it is recommended that you initially render a smaller version with at least one dimension near 512px, and then upscale it with img2img (with denoising set to 0.5) or a built in high-res fix feature.
Details can sometimes be improved by looping the output back through img2img multiple times, with a denoising of 0.5 and seed resizing.
Seam Handling
As Stable Diffusion only renders squares and rectangles, any equirectangular projections will have edges that may not fully match the other side. While these seams are generally pretty minimal, there are multiple ways to deal with them:
* Using the asymmetric-tiling extension's x-axis tiling feature can help eliminate seams entirely, but the extension can significantly degrade output. It is recommended that you set the 'Start tiling from step N' setting to start at around 50% in order to minimize the impact (ex: start at 9 if using 20 steps).
* Inpainting can be used across the seam after shifting the image horizontally to the right or left.
* GIMP (potentially with G'MIC) or Photoshop can be used to remove the seams.
Useful Tags
In order to improve usability of the model, various words and phrases were used to tag objects, scenes, styles, and content. Note that these lists are based on the training data and do not include things added by the base model. These lists are also not comprehensive.
Styles
- "photo", "photobash", "render", "architectural rendering", "illustration", "digital illustration", "painting", "digital painting", "drawing", "watercolor painting" "concept art", "charcoal drawing", "sketch", "rough sketch", "fractal art", "crayon drawing", "anime", "pixel art"
Camera Locations
- "underwater", "aerial view", "interior", "exterior", "pov", "street level", "above the clouds", "low earth orbit", "underground"
Locations
- "library", "bedroom", "bathroom", "hallway", "corridor", "bridge", "helm", "cockpit", "driver's seat", "street", "road", "forest", "city", "train station", "railway", "greenhouse", "residential street", "dock", "hanger", "landing pad", "ferry", "cave", "observatory", "amusement park", "waterpark", "tunnel", "mine", "tropical", "beach", "desert", "steep slope", "cliff", "ocean", "body of water", "river", "mountain", "space", "underground bunker", "space station"
Skies
- "aurora borealis", "cloudy", "overcast sky", "blue sky", "stars"
Time
- "sunset", "sunrise" "night", "sunny day", "winter", "twilight", "fall"
Weather
- "rain", "raining", "snow", "snowing", "fog", "haze", "smoke", "storm", "stormy", "lightning", "flooded", "arid"
Lighting
- "bright", "dark", "dimly lit"
Themes
- "futuristic", "cyberpunk", "historical", "messy", "scifi", "minimalism", "minimalistic", "simple", "simplistic", "video game", "surrealism", "surrealistic", "cartoon", "comic", "black and white", "smooth", "ancient", "medieval", "vector art", "abandoned", "horror"
Humans & Animals
- "people", "women", "woman", "man", "men", "cat", "dog", "horse", "group of", various dinosaurs, "zombie", "fish", "shark"
Viewing 360 images
The images created with this model are meant to be viewed by 360° viewers and thus will have weird distortions when viewed in 2D. Therefore, the following viewers are recommended:
Website (supports VR headsets): https://renderstuff.com/tools/360-panorama-web-viewer/
AUTOMATIC1111 WebUI Extension: https://github.com/GeorgLegato/sd-webui-panorama-viewer
WebUI Extension for converting your renders to stereoscopic 3D images: https://github.com/thygate/stable-diffusion-webui-depthmap-script
Example Image Models
The example images used the following models:
- Landscape renders used: https://civarchive.com/models/4384/dreamshaper
- Renders of people used: https://civarchive.com/models/4823/deliberate
Description
FAQ
Comments (17)
Please consider triggers that relate to the model name; I'm never going to remember all these obscure triggers for all these loras
The trigger word is meant to have as minimal an impact on renders as possible. Previous attempts with trigger words closer to the model name resulted in everyone wearing VR headsets or other issues.
@progamergov even something like "virtpano" or "p4nor4ma" if you're worried about it would help people like me, (not that I feel entitled to a change), because then at least I can change the name of the lora myself to match the keyword. But if I see a lora named "qxj" later, I won't know what it is
personally I never remember trigger words too so I use the civitai helper extension
@greenersonman thanks I'll check that out
@calvin You could try appending something like "tw_<triggerword>" to the names of all the LoRA's you have in order to make it easier to know their trigger words
@progamergov yes, true, and I suppose I'll do that too, but ideally (for me) they would be something logical so I could just use a simpler lora rename
it's much better to do things this way, just use the extension helper or rename the lora to include the trigger inside the filename, it's much better in the long run to avoid any sort of conflict/adjacency in the tokenization
really great work here, thanks
that works quite well, only the closure at the top is not working. but that has potential.
Yeah, the top and bottom can sometimes have issues, but unfortunately those seem to be related to LoRA model themselves. When training finetunes on the same dataset, those issues disappear entirely.
Someone came up with a trick for cleaning up top and bottom issues: https://github.com/GeorgLegato/sd-webui-panorama-viewer/issues/23
You mentioned that the tag lists are based on the training data. Does it mean the training data contains images of different styles, e.g. painting, drawing, illustration and sketch?
@zxsd299 Yep! The training dataset contains all of the content listed in the tags!
@progamergov I'm curious about how to get so many training images of different styles. ^_^
Any thoughts on a XL model version?
There are plans in the works for better models with even more training data
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.