krpano Stereoscopic 3D / Oculus Rift Support

  • This thread is OUTDATED - for up-to-date information about 'Stereoscopic 3D / Oculus Rift Support' please see this new thread there:
    krpano 1.19 (Pre Release) - WebVR / MobileVR / Stereoscopic 3D Support (Oculus Rift, Google Cardboard, GearVR, ...)


    Hi,

    I recently got my Oculus Rift (btw - a very great device ) and I of course tried to use krpano with it.

    Here are the results of that try:
    https://krpano.com/stereo3d/#top

    The panos can be viewed in any HTML5 capable browser that supports WebGL and the HTML5 Fullscreen API.
    To be able to use Oculus Rift it would be necessary to install the NPVR browser plugin - this plugin connects to the Oculus Rift and provides the head tracking data to HTML/Javascript (=to krpano).

    Additionally there are also demos for viewing the stereoscopic panos on common 3D TVs. By using a Side-by-Side (SBS) rendering its possible to view the panos also on all common 3D TV screen that are supporting are 3D mode (should be almost all).

    A special thanks to Robert Kooima for providing the Stereoscopic Spherical Panoramas!

    For all Oculus Rift owners - have fun watching the panos!

    Edit: Update for DK2 and Mobile-VR:
    krpano Stereoscopic 3D / Oculus Rift Support

    Best regards,
    Klaus

  • Really a fantastic day for me - awesome features...
    "i specially thanks for the Krpano Stereoscopic 3D"

    Expecting the vtour droplet for the Stereoscopic 3D!

    regards,
    Suresh

    Edited once, last by view360 (August 22, 2013 at 7:09 AM).

  • Hi,

    another interesting geek-feature *g* Could you give us some more informations about the Oculus Rift itself, Klaus? Is it worth the price? How heavy does it get after a while? How exact is the head tracking? Did you already test it with other apps/games than krpano?

    BR
    Nupsi

  • Hi,

    Quote

    I'm drooling here. Hope to be able to get one

    I had to wait more than 3 months for it, but the waiting was it worth - it's an awesome device! *wink*

    Quote

    Could you give us some more informations about the Oculus Rift itself, Klaus?
    Is it worth the price?

    I would yes, definitely! *wink*

    Quote

    How heavy does it get after a while?

    The weight is absolutely no problem at all, it feels rather light than heavy.

    Quote

    How exact is the head tracking?

    The tracking itself is very exact and very good working, the bigger problem here is the latency. When it gets to high, fast head moves can feel irritating.

    Quote

    Did you already test it with other apps/games than krpano?

    Yes, of course, I think I have downloaded and tested already more than 20-30 different demos for the rift, but one of the best demos is already the default Oculus SDK Demo (Tuscany) - it already shows what is possible and the experience with it is very impressive.

    Another absolutely must see/must try is the 'RiftCoaster' demo - search Youtube for it *wink*
    e.g. http://www.youtube.com/watch?v=GoQ0OXJCbaE

    I think the biggest problem at the moment is the resolution, the Developer Kit has only 1200x800 - that would need to be MUCH higher. It will be interesting to see that the final Oculus Rift will offer here.

    Btw - viewing panos (especially stereoscopic ones) with it, is also very impressive! *g*

    Best regards,
    Klaus

  • Hi Klauss !!! This looks like really promising !!! I'll keep an eye on that and try to get an oculus as soon as possible !!

    By the way, to make the stereographic panos, we just have to make the same pano with a little displacement ?? About 20 cms ?? Or there's any special workflow ??

    Thanks and congrats for bringing your engine to the lastest hardware tecnology

  • By the way, to make the stereographic panos, we just have to make the same pano with a little displacement ?? About 20 cms ?? Or there's any special workflow ??

    Sorry, but I can't answer that, I have never shoot or stitched a stereoscopic (not stereographic) pano by myself, but it seems shooting and stitching is a bit more complex here.

    Would it be possible to use it with video 360 ?

    Not with the current setup, loading and playing two video streams at the same time might be difficult to sync. Eventually later with a special version and only one video stream, but then when the resolution will be probably a bit low - e.g. when putting two full spherical videos side by side into one 1920x1080 full hd video, that would be only 960x1080px per pano which equals a roughly a cube-side resolution of only 306x172px and this is very low, probably too low...
    Beside of this - I don't know if recording stereoscopic pano videos is possible...

    Best regards,
    Klaus

  • I think the biggest problem at the moment is the resolution, the Developer Kit has only 1200x800 - that would need to be MUCH higher. It will be interesting to see that the final Oculus Rift will offer here.


    Thanks for the insight, Klaus *smile* The last time I did try VR was back in the 90´s and I´m waiting since then for the next VR-Generation. Hopefully this time VR will become a much bigger thing as it was back then.

    But I think I´ll wait for the next Oculus Rift-Version, supporting FullHD-Resolution *g*
    http://www.youtube.com/watch?v=KSboqL82fmw

  • Hi Klaus!

    I got my Oculus Rift yesterday and played around with some of my panoramas:
    http://www.panotour.at/panos/oculus

    It is really very nice and has a lot of potential. But, as you said, the resolution need to be MUCH higher!

    Now I need some hotspots to link the panoramas together. Maybe with a joystick or a controller. Or even with the keyboard and mouse.
    Do you have already some code for that?

    Best regards, und viele Grüße aus dem Salzkammergut!
    Gerhard

  • Sorry, but I can't answer that, I have never shoot or stitched a stereoscopic (not stereographic) pano by myself, but it seems shooting and stitching is a bit more complex here.

    Not with the current setup, loading and playing two video streams at the same time might be difficult to sync. Eventually later with a special version and only one video stream, but then when the resolution will be probably a bit low - e.g. when putting two full spherical videos side by side into one 1920x1080 full hd video, that would be only 960x1080px per pano which equals a roughly a cube-side resolution of only 306x172px and this is very low, probably too low...
    Beside of this - I don't know if recording stereoscopic pano videos is possible...

    Best regards,
    Klaus

    Not with the current setup, loading and playing two video streams at the same time might be difficult to sync. Eventually later with a special version and only one video stream, but then when the resolution will be probably a bit low - e.g. when putting two full spherical videos side by side into one 1920x1080 full hd video, that would be only 960x1080px per pano which equals a roughly a cube-side resolution of only 306x172px and this is very low, probably too low...
    Beside of this - I don't know if recording stereoscopic pano videos is possible...

    Hi Klaus,

    Can you explain further how bad the lag when playing a 360 video with the new script? The loading time of your 360 still demo was quite slower than it's usual. Sometimes ago I tried displaying a 4k 360 video on a SBS splitted screen as the same way as told in the javascriptsync demo, the lag was 'ok' for a dual 1920x1080 display but very bad on a triple display setup. Not sure on the rift.

  • Hi,

    Can you explain further how bad the lag when playing a 360 video with the new script?

    I haven't tried playing pano videos yet, but technically loading and playing two video streams is bad. There is no guarantee for sync.


    I got my Oculus Rift yesterday and played around with some of my panoramas:
    http://www.panotour.at/panos/oculus

    Great panos! I like the 'traktor.xml' one


    Now I need some hotspots to link the panoramas together. Maybe with a joystick or a controller. Or even with the keyboard and mouse.
    Do you have already some code for that?

    I have no examples for such yet, but it's the normal krpano viewer, so all possibilities are there - using distorted hotspots to load others panos should work fine - the only thing is that this need to be done 'indirectly' via Javascript to load the same pano in both viewers.

    Best regards,
    Klaus

  • Hello Klaus and everyone,

    Thank you for these Oculus Rift stereo demo's, they are very exciting!

    In 2010 I began working on a Stereo-3D viewer using Krpano's flash browser to create anaglyphs and side by side, over/under stereo panorama viewers. StereoPan.com is the resultant site dedicated to 3D-Stereo Panoramas with beautiful spherical and high resolution gigapixel panoramas to explore.

    I have found that deep-zooming into stereo content to be most interesting and unique.

    For this site, I created a method for displaying two krpano Flash panorama views within the same parent Flash container, and was able to create a communication mechanism for the two views to coordinate movements in each eye view to eliminate eye-strain upon zooming in.

    The Stereopan.com viewer is unique in that it coordinates each eye view as one zooms in and out, actively keeping both eyes looking within the same frame window without eye-strain, even if a user zooms in all the way on a multi-gigapixel stereo-3d panorama! This coordination method keeps objects of interest from going out of frame (off screen) for one eye and prevents eye strain from mis-registration of the left and right eye views upon zooming in to the image by using a depth-map. Without this coordination, mis-registration as the viewer zooms in destroys the 3D effect.

    The Stereopan.com flash viewer does not work for iPhone, iPad, Android, etc. (as Flash does not work here), so I have been working on creating a Stereo-3D viewer that will work well on these systems.

    I have developed a stereo-3d panorama viewer using HTML5 flat panorama viewing technology that outperforms my experiments with stereo krpano multiresoution HTML5 and flash setups in multiple ways:

    1. There is no lag between the left eye and right eye views, the two eyes are always perfectly in synch as the viewer script is drawing both to screen at the same time, not one eye after the other. Both eye views receive movement information at the same time, so one eye is not playing catch up with the other.

    2. It does not crash out the iPhone or iPad, which I have found running two multiresolution krpano html5 viewers within the iOS environment definitely does within a few seconds or minutes of start-up.

    My viewer is not spherical, so I would be very excited to switch my development focus of Stereopan.com to use the krpano multiresolution html5 viewer for Stereo-3D panorama viewing, but at this time, through my experimentation with the multiresolution krpano in a stereo environment, I find the results unsatisfactory (there are synch delays between the views, iOS Safari crashes...)

    Viewing these panoramas on an iOS device is really amazing! An Oculus Rift is not necessary for great 3D viewing, one can use an iPhone or iPod touch with the Hasbro My3D viewer for a high resolution experience (and yes, it can use the gyroscope for head tracking) Get one for only $5 on Ebay . Viewing anaglyphs on the iPad is great fun with excellent resolution (just need 50 cent red, blue glasses) I would imagine when the next iPad Mini comes out with retina resolution, it would be perfect for making an Oculus Rift type device for viewing panoramas with head tracking and super high resolution.

    I would hope in the future for there could be some way with krpano html5 multiresolution to have the two eye views stay perfectly in synch with each other without update lag and of course to turn down the memory usage of krpano-html5 multiresolution somehow when two viewers are present in a window at the same time so as not to crash iOS safari. This would allow me to fully use krpano html5 multiresolutiuon in a stereo environment.

    Here are links to my various Stereo-3D Viewer experiments:

    Cross Eye 3DTV krpano html5 multiresolution
    Parallel Eye 3DTV krpano html5 multiresolution
    Over/Under 3DTV krpano html5 multiresolution

    Multiple View Options krpano FLASH multiresolution

    Stereopan.com Viewer is my in-browser anaglyphing flat gigapixel 3D-stereo panorama viewer using HTML5.
    It is a synchronized flat viewer and will work without issue in iOS(anaglyphing doesn't work in Internet Explorer)

    Another for my Stereopan Deep Zoom stereo-3D viewer:
    Philipsburg Refinery by Ron Schott, can be zoomed in for incredible 3D detail!

    If you are interested, please take a moment to try the different viewing methods and technologies, everyone's feedback would be most welcome!

    Shooting 3D-Stereo panoramas is quite a lot of fun. Of course, it doubles some of the work, but the results are very satisfying, and you get a lot of questions out in the field.

    Thanks

    Jason Buchheim *smile*

    Edited 3 times, last by odysseyjason: spelling,grammer (August 29, 2013 at 9:51 PM).

  • Hi All


    I have a 'basic' question re shooting stereo panos...

    My understanding is that to 'capture' 3D panos' one needs to shoot two 360's from the exact same location but offset by a set distance, ie equating to two 'eyeballs'.

    Assuming I've got this right *confused* my next question is what is the 'recommended' offset distance between the panos and at a practical level what do you use when on location to get the offset?

    Would something like this manfrotto rig be suitable: http://www.manfrotto.co.uk/accessory-arm-for-4-heads

    thanks in advance


    Tim

  • Hello Tim,

    To shoot stereoscopic panoramas you do need a method to create a view for each of the viewers eyeballs, and the simplest way to do this is to shoot two panoramas with an offset. There are other computationally challenging methods to produce stereo panoramas that solve some issues with the two eyeball-offset setup, but they come with their own issues that prevent one from causally dipping their toes in to the subject, so the two viewpoint method is by far the best to get started (the advanced method requires two cameras where one is in the center and one is offset, used to create a depth map, and a computer graphics system to warp and synthetically create the two eye views on the fly from the obtained data...).

    It is quite possible to make fantastic, fun stereo panoramas without going to the full 360 degree route by just shooting a static panoramic scene and then moving the camera a distance and shooting again. This is not good for action or when clouds are moving and casting shadows, but otherwise it is easy and enjoyable. The stereo separation between each eye will change from the edges to the center of the scene as the two cameras would not be rotating around a common center, but I find when viewing these that it is really not a problem (we are, of course, used to things going flat as that is how tv's have been for ages) When shooting a distant scene for 120 degree view, it really is not a problem. I shoot many handheld stereo panoramas in this method.

    If one wants full 360 degree view, then the cameras need to rotate around a common center. I use a Gigapan Epic Pro robot with a custom bar that I mount two cameras on. This allows me to freeze action and clouds within the same eye view and to shoot in a full circle. I had to develop a simple circuit to fire both cameras from the Gigapan Robotsimultaneously. Of course, this requires two similar camera systems. The stereo separation can NOT be significant or stitching will become difficult to impossible as each camera is not rotating through its lenses entrance pupil (again, not a problem for distance scenes, but trouble with close up elements).

    The distance between the eye views of the cameras (the stereo base) is actually something that depends on the minimum subject distance of a scene. A scene of farway mountains can actually have a much wider stereo separation base to create a normal stereo scene for a viewer, while one that has close up subjects needs the camera separation to be much closer for things to look normal or 'Ortho' stereo.

    The general rule for separation to make things look normal to the viewer is to take the minimum distance to the subject/30 = stereo base separation.

    There is also Hyperstereo and Hypostereo. Hyperstereo is a result of a larger separation between the cameras and actually results in objects in a scene appearing smaller than they should. Hypostereo is when the separation was smaller, causing a scene to appear larger than in reality. Sometimes these effects can be used creatively to your advantage, such as to bring out the stereo depth of a far mountain scene that does not have close up elements.

    Here is a link to a poster presentation I presented at Carnegie Mellon University in 2010 on this subject http://www.3d-360.com/gigapan/?id=64651</a> It explains these concepts in detail with photo examples. John Toeppen is by far the best stereoscopic panorama creator I have met, with incredible vision. I found his work so inspiring that we wrote a paper for SPIE conference this year concerning making and displaying stereo panoramas, feel free to read it here http://www.holographics3d.com/hg/demo/spie/I…Panoramas14.pdf

    Yes, your bar would work well for providing a stereo separation on cameras, you will need to firgure out how to attach it to a panoramic tripod head.

    Hope this helps, after you shoot some stereo panoramas, I would be happy to assist in getting them aligned and presented on stereopan.com if interested!

    Jason Buchheim

  • Hi Jason

    Many thanks for your detailed answer. I have a client shoot this Friday and will try and capture some stereo panos. For now I will just move the tripod 200mm 'sideways' as I need to do some testing with the manfrotto bar to ensure a) its stable and b) its not in the shot!

    I tried the oculus rift at the Virgin Media Game Space in London, the 3d experience is impressive, resolution less so, but I understand a higher res version is in the pipeline.

    Tim

  • Hi,


    Quote

    I've been trying to setup the oculus in a mac mini without succes, it has the especifications below :

    What exactly do you mean with 'without success' - too slow rendering?


    Quote

    2.4GHz Intel Core 2 Duo/2 GB DDR3 SDRAM (PC3-8500)/320 GB HD/SuperDrive/GeForce 320M/AirPort & Bluetooth Built-in
    Should it be possible to make it work ?

    Not necessarily - regarding to some tests on the web the Gefore 320M isn't a very good GPU (and it uses system-memory) and the GPU is the most important thing here, 2GB RAM is also a bit few...

    Here some Geforce 320M Tests/Infos:
    http://www.videocardbenchmark.net/gpu.php?gpu=GeForce+320M
    http://www.notebookcheck.net/NVIDIA-GeForce-320M.28701.0.html

    Any cheap PC with an average but dedicated graphics-cards might be a better solution for Oculus Rift usage. Onboard-GPUs are often much too slow.

    Best regards,
    Klaus

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!