Convert flat videos to VR workflow: here
PLEASE READ: for 2.0: all you have to do is upscale and apply depth via Custome nodes
for 1.0: Download my Custome nodes and use the workflow thats in it to fix the video so its usable in VR. Despite my best efforts this dosnt work right out of the box but the good news is you can just drop it in the video cropper and then apply the Depth. you'll see the nodes in my workflow.
also i created a git because i plan to update it with essential tools to make VR videos so follow me on here or my patreon for updates.
TL/DR:
my custom nodes in repo
use workflow to fix video in said repo
crop in half then apply depth.
meta data already applied if used SBS
????
profit in VR
Anyway, as always, if you like what I do and want to support the work, feel free to buy me a coffee ☕
https://ko-fi.com/aidepository35
or support me on patreon
https://www.patreon.com/c/aitrepreneur/posts
So many of you have been asking for a 180 degree version of this lora so i made one. i had a difficult time finding flat panoramas to use for the training data but there was an abundance of them in cross eyed format.
for the videos made you can just crop the video in half, upscale it, then re-apply the cross eyed depth. I'm sorry i couldn't make this an easy one click but i do plan on improving this lora in the future if it dose well.
Actually from testing it out i kind of realized i can literally just crop half of it and train another model on that. maybe i'll give it a shot later.
important links:
https://github.com/google/spatial-media/releases
https://github.com/Ragamuffin20/MuffinsVRFixes/tree/main
Rant:
Dude wtf man this lora took 4 god dam days to train and i almost gave up because my run pod timed out and i thought i lost all the data for a second. Like you guys have no idea how mad i was. 4 days in and $50 almost down the drain but luckily everything was saved on the volume disc. i really whish it would automatically create identical cross eyed videos and tbh it probably would have with more training but i cant afford it tbh. 0.40 cents an hour adds up to alot if your training something for days straight. actually funny thing, i was using the A40 to trian it and still had to toggle low vram and offloading. but yeah here yall go, hope you can make some cool things.
Description
FAQ
Comments (19)
noice
Looking good! Got some feedback: for moving/walking videos there is a hand and go pro visible in the frame. I'm guessing that was common in any video tagged "walking"
makes sense, some of the training data was of a CGI character and what your probobly seing is whats supposed to be "your body's arm". probobly needs to train for longer but thanks for the feedback though!
I tested this in my VR headset and it's AMAZING!!! Great job. It reminds me of online service called https://www.owl3d.com/ that is able to add depth to images and videos. But the biggest issue is that it does not add the extra warped space around the main subject which ruins the whole VR feeling. Is there any chance you would create a workflow that supports transferring already existing images / videos into a 180 video? Overall awesome job!
me and others have been trying to do this for months now. https://www.patreon.com/cw/hybridworkflow has actually done this but his method require an extreamly high amount of vram, youl have to use runpod to use it. I could make one but i need an inpainting checkpoint for LTX otherwise the outpainting wont blend as smoothly. thanks for the feedback though, i really do apprecaite it!
I think VLC media player has panorama mode.
Trigger words? Strength? Would be nice to get a bit more info
An ultra-wide 180-degree cross-eyed panoramic depth VR scene
could we have an example workflow and a space where we can discuss progress things and tester like me could give feedback ik civitai and reddit can do that but for more immersion and focus, discord & telegram are better ngl
does it only work for text in the video? is there no way to do it for the image in the video?
it can animate panoramic videos yeah but it cant convert regular videos to VR. for that you need this: https://civitai.com/articles/28978/make-any-video-into-vr-with-muffins-flat-2-vr
@Ragamuffin20 Thanks
amazing work! works great, used the outputs in my quest 3 and it's legit af
Has anyone actually found the VR180 workflow for this because whenever anyone asks the creator only links to his patreon which has the same 360 seam fix and upscale workflow for videos that have already been created. in his custom nodes folder its the same upscale workflow. No where is the actuall VR180 workflow. In his youtube video he links to aitrepreneur patreon but again there is nothing in there. Does anyone have a direct link to the VR180 workflow?
this page is for the 180 degree lora for LTX-2.3. not the flat 2 VR workflow which is on my patreon for my members which is why i linked my patreon.
I have always had the doubt if this works with image-to-video. Input an image, then the LoRA builds the 180- or 360-degree surroundings, doesn't it?
it actually dose suprisingly well if the video is already a 180 degree panoramic. but this is old stuff, ive made a workflow that can convert any video into a 180 or 360 VR video if your intrested https://civitai.com/articles/28978/make-any-video-into-vr-with-muffins-flat-2-vr
im also working on a updated version wich will be out soon so please follow the patreon and spread the word if u can
I used the 180-degree lora with LTX lora loader and 2:1 ratio. But it still produced 2:1 flat videos, unlike the 1:1 ratio of 180-degree VR videos. Anything I'm missing here?
Details
Files
LTX-2.3_180 Degree.safetensors
Mirrors
LTX-2.3_180 Degree.safetensors
LTX-2.3_180 Degree.safetensors
180 Degree VR - LTX2.3.safetensors
180 Degree VR video - LTX2.3 - An ultra-wide 180-degree cross-eyed panoramic depth VR scene.safetensors
180 Degree VR - LTX2.3.safetensors
180 Degree VR video - LTX2.3 - An ultra-wide 180-degree cross-eyed panoramic depth VR scene.safetensors
ltx-2.3-180-degree-vr-video.safetensors
180 Degree VR - LTX2.3.safetensors
180 Degree VR video - LTX2.3 - An ultra-wide 180-degree cross-eyed panoramic depth VR scene.safetensors
180 Degree VR - LTX2.3.safetensors
180 Degree VR video - LTX2.3 - An ultra-wide 180-degree cross-eyed panoramic depth VR scene.safetensors
180 Degree VR - LTX2.3.safetensors
180 Degree VR video - LTX2.3 - An ultra-wide 180-degree cross-eyed panoramic depth VR scene.safetensors