Oculus GO & KRpano

  • did anybody of you try to use the js api ?

    https://developer.oculus.com/documentation/…ts/vrweb-intro/

    Not yet ;)


    Not that Klaus needs defending or anything but KRPANO is (righfully so IMHO) implementing WEBVR (https://webvr.info/) which, like any browser based technology, sacrifices some functionality that is available to native apps in order to be platform agnostic and run on the widest number of devices possible.

    I think in the early "wild west" days of VR, that is absolutely the right approach because nobody knows who the winners and losers are going to be yet. We are all investing time and energy on a "platform" and if that platform dies, we lose the time and money investment we have made on that platform. It is sort of like Blue Ray vs HDDVD or to us older folks, VHS vs Betamax.

    I do think that with the release of the GO, Oculus has made a large enough leap forward with regard to saturation that I am now personally reaching a point where leading our studio towards a native Oculus App may begin to make some sense and I suspect the industry as a whole, including the WEBVR team, may be beginning to reach the same conclusion.

    I'll also add that performance seems pretty good on our end, much better than the GEAR running on a phone and the battery and heating/cooling is far superior. Obviously all VR devices are suffering from a resolution limitation at the moment and I think until we get at least 2k per eye from the actual displays, that is going to be one of the biggest factors holding things back on the hardware side.

    On the software side of things, we're using different resolutions of content depending on the platform and that now includes stereo content for VR. We're also pointing the VR devices towards their own URL now that strips out a lot of the unnecessary KRPANO code to keep things as lite as possible although that is really more to keep development easier and doesn't really seem to be affecting performance all that much. We're using plenty of hotspots for menus, navigation, and various other things and performance all seems very good considering this is basically an untethered mobile device redrawing a 2,560 x 1,440 display at 60fps with two 5k panoramic images.

    Edited 3 times, last by bhh (May 10, 2018 at 6:22 PM).

  • I have also just got a Go and agree that the controller makes a HUGE difference from GearVR. I also like the display better even though it is LCD rather than OLED. And battery life is way better.

    While playing with the Go I made the "discovery" that on an HMD, krpano's webvr really shows immersive spheres. I may have been the last panographer who did not know that. Anyhow it was an 'aha' moment, and now I am keen to make some webvr tours with stereo panos.

    I have also noticed a slight discrepancy of stereo disparity between Gear and Go, and on the Go it seems to vary with image resolution. I have tested 360 Photos with Otoy cubes of face size 1536, 1920, and 2048. The first two seem to have identical stereo geometry, with the 1920 a bit sharper. The last 2 have same sharpness but the 2k image seems to give a wrong depth impression. This might be due to some image shift. Since it is likely that Oculus browser uses the same rendering engine as 360 Photos, we need to be alert for such effects in webvr.

    Which brings up a question: how do we know/control the size of the working 'display cube' on a webvr device?

  • Hi,

    I have also got my Oculus Go now and I have to say it works pretty well. I would say its basically the same as the new GearVR, just with a different screen and a smaller fov, but therefore a bit sharper (more pixels/degree). And way more comfortable (at least for me).

    The WebVR-API support is there and VR rendering works correctly as it should - the browser and the headset are doing all the tracking and lens-distortion and VR-rendering.

    Personally I think the lens-distortion is not as good corrected as in the HTC-Vive or in the Windows-MR-headsets, but that's a general Oculus thing for me. It fells like the screen gets slightly bended when looking around, but that's the same also in all other Oculus headsets (Rift, GearVR). But that can be also only my personal perception depending on my headshape and eyes of course.

    When using VR-optimized pano-images (e.g. build by using the 'MAKE VTOUR (VR-OPT) droplet') there should be normally no performance problems.

    The there could be only smaller performance problems when 'directly' using multires-panos in VR. 'Directly' means without extra VR images. But that was also the case for the GearVR and other VR devices and the reason why using special optimized VR images are recommended. These devices simply doesn't have the power for rendering a lot of tiles in stereo and quickly loading and unloading them without any interruptions (at least with the current brower-tech).

    Using VR-optimized images (by default 1536px per cubeface) should solve performance problems here.


    When you have a Oculus Go (or GearVR or other good VR headset) I would recommend trying these examples:

    First the stereoscopic example because stereoscopic panos are especially great in VR:
    https://krpano.com/examples/usage/#demotour-stereoscopic

    and the normal WebVR example:
    https://krpano.com/krpanocloud/webvr/


    -"Enter VR" in the middle of the screen stands for "Undifined Enter VR", anyway it' not very handy as we already have the vr icon in the toolbar. is there a way to disable this button?

    When you get a 'undefined Enter VR' text, then you had manually removed the 'title' attribute in the tour.xml.
    See in the vtourskin.xml in the 'skin_webvr_onavailable' action for the related part and you can anytime modify that of course.


    -In non VR mode the pointer of Oculus is very handy to navigate, but is disappear as soon as we enter VR mode, is there an option to keep it in VR mode?

    No, that's not possible, there is no API for that, but VR-controller support will come with one of the next krpano releases.


    -Distorted hotspots are not displayed in VR mode :(

    All hotspots that can be already rendered by WebGL work also in VR. Today these are all hotspots except hotspots with children layers and polygonal hotspots.


    -A tour with distorted hotspots is very jerky, although they are not show, but VR is OK with no distorted hotspots

    Please post a link to your tour, because that's definitely not normal (or you're using a too old krpano version or way too much or too huge hotspot images for the device...).


    1. The tours look like they are rendered at much lower resolution in WebVR mode. Like 720p instead 1440p.
    2. There is no controller support. It seems to me it should work like a mouse and not a touch device (in web mode). In VR it's not recognized.
    And performance is not at it's best...

    Is your krpano version up-to-date?
    And are you using mulitres-panos directly for VR?

    I ask because some time ago the Oculus browser (in the GearVR at this time) got an update that changed how it was working - it was reporting a window-resolution of only 200x150 pixels (or something like that) and a rendering resolution of 2048x1024, normally these sizes should be the same and the krpano was using the window-resolution for the multires-level-selection. And that leaded to a too low resolution. But that's already corrected since version 1.19-pr14.

    And beside of this, multires-panos itself are generally not optimal for VR-performance, try building the tour using the VR-OPT droplet.

    VR-Controller support will come and it's not just like a mouse or touch *wink* - the controller is a position+rotation matrix and it's necessary to render the controller (and/or a pointer-ray) and do hit-testing in 3d-space manually (nothing special but still a lot of work).


    did anybody of you try to use the js api ?
    https://developer.oculus.com/documentation/…ts/vrweb-intro/

    There is basically only the WebVR API itself and krpano already uses that of course.


    I have also noticed a slight discrepancy of stereo disparity between Gear and Go, and on the Go it seems to vary with image resolution. I have tested 360 Photos with Otoy cubes of face size 1536, 1920, and 2048.
    ...
    Which brings up a question: how do we know/control the size of the working 'display cube' on a webvr device?

    The VR rendering itself should be completely independent of the pano image size. I would recommend a cubeface-size between 1500 and 2000px for VR today. A cubesize of 2048px itself might be not that optimal for VR, because images with a power-of-two size like 2048 will get mipmapped by default (although that could be disabled manually) and that can lead to a slightly blurred rendering - and with disabled mipmapping that size might be already too high for the screen and you will get too much aliasing.

    Best regards,
    Klaus

  • Quick reply
    Yes, I used multires but on v19.16.
    The main concern though is that distorted hotspots are rendered at lower resolution.

    Maybe I should provide an example...

  • Thanks, Klaus.

    I did a little experiment on resolution with Oculus 360 Photos on Go, with a stereo pano. Otoy cubes of 3 different face sizes, the orignally recommended 1536, 1920 which has worked well on GearVR since 2 years, and 2048; also over/under equirectangulars of width 6144, 8192, and 12000. The results are pretty clear.
    The two large images had trouble, both slightly incorrect stereo and crashes. The other 4 were handled well.
    The 1920 and 8k images showed distinctly sharper than the 1536 and 6k ones. The large images were no sharper.
    So I's say the ideal cube face size is probably 1920x1920.

    On a different issue, many of us have noticed severe "shimmer" due to motion aliasing on images displayed in VR by the Oculus browser. This is completely absent when the same image is displayed by 360 Photos or Gala360. So clearly Oculus has some work to do on their WebVR rendering. This not krpano's problem, certainly, but I was wondering if you know of any tricks we could use to mitigate it?

    -- Tom

  • Another serious conern about WebVR.

    Did I read you correctly that the WebVR API in Oculus browser does not give access the the primary pointing device? That seems hard to believe, given that WebVR is all about control. If true, it must be fixed.

    I have viewed websites on Go where the trigger and/or trackpad switches worked in VR, if not the pointer. And the trackpad itself, if I recall right. The VR mode of Sketchfab seems to have some response to motion of the hand controller, though there is evidently no proper drag function.

    I think this is a critical issue for two reasons, one general, the other more panocentric.

    1) A pointer connected to a mouse or wand is much better to use than a "gaze cursor" -- faster, more accurate, no false triggering. If available,it should always be preferred to the gaze cursor.

    2) VR image display UIs need a way to move the view direction around the horizon, independent of the gyros. This makes it possible to view a 360 photo comfortably while seated, for example on a plane where you can't stand up, or on a couch where you don't want to. The obvious way to implement that is dragging with the pointing device.
    Some krpano sites -- notably 360Cities -- do provide finger-pan on "cardboard", and that is a really welcome feature.


    Are you really telling us that it can't be done in WebVR?

  • On a different issue, many of us have noticed severe "shimmer" due to motion aliasing on images displayed in VR by the Oculus browser. This is completely absent when the same image is displayed by 360 Photos or Gala360. So clearly Oculus has some work to do on their WebVR rendering. This not krpano's problem, certainly, but I was wondering if you know of any tricks we could use to mitigate it?

    I think the reason here is that the Oculus Browser only provides a 2048x1024px rendering buffer in WebVR. That means the maximum that krpano can render/output per eye is a 1024x1024 image, and that's BEFORE lens-distortion, which additional takes some resolution, so the resulting/effective resolution might be only something 800-900px per eye.

    I think native apps don't have such limitation and changing the that 2048x1024px output limitation is not possible (I've tried with several ways). Here the Oculus Browser would need to get improved/updated (but that was already the same in the GearVR and any requests here were unfortunately ignored).


    Are you really telling us that it can't be done in WebVR?

    No, I've never said that - please see above - there is no VR-controller support in krpano yet, but that will come of course.

  • Thanks, Klaus.


    We should all try to get Oculus to fix the browser rendering issue. Now that they are pushing VR into the mainstream, they should hear a lot of complaints. The difference in render quality between the browser and the apps is obvious to anyone.


    I'm looking forward eagerly to full controller access in krpano webvr!

    -- Tom

  • By the way, Oculus support is very responsive. It is nice idea to report them all that stuff.

    I have sent them report about wrong color reproduction. Also there should be reports about screen flickering sent by some users on Reddit...

  • Klaus,

    I am really keen to understand all technical issues that are preventing fully satisfactory viewing on Oculus Go/GearVR Browser with krpano WebVR.
    Part of the trouble is no doubt due to defects in Oculus Browser, and I want to keep pressure on them to fix those things. Another part is due to the preliminary nature of the WebVR plugin -- which is perfectly understandable, given all you do. However I want to keep some pressure on you too.

    So I hope you are willing to answer a few more questions, or tell me why they are stupid.

    1) In a quick look at Oculus WebVR documentation including sample code package "Carmel" I found no explicit limits on webGL texture size, nor any way to query for such limits. So how do you know the size of the render buffer?

    2) There is a general MSAA (multi-sample anti-aliasing) setting in the Oculus APIs, which they highly recommend be used though it is off by default. I have no idea whether this applies to webGL rendering in the Go Browser. Have you investigated that?

    3) The plugin has a function for re-orienting the view azimuth, independent of the gyro. Yet somewhere you say that this should not be done in VR, and it seems likely that the plugin does disable re-orientation in "real webVR" mode. Would you re-consider whether that is really necesssary?

    4) In "desktop fake webVR" as well as "cardboard" modes the web'vr plugin does support dragging the view azimuth with the pointing device. Why is that not possible also in real VR?

    Thanks again,
    -- Tom

  • 1) In a quick look at Oculus WebVR documentation including sample code package "Carmel" I found no explicit limits on webGL texture size, nor any way to query for such limits. So how do you know the size of the render buffer?

    There is no texture size limit, but the size of the renderbuffer is given by the browser and set to 1024x1024 per eye (=2048x1024 total) by default.

    Technically it's the 'renderWidth' variable from the WebVR API - see here:
    https://developer.mozilla.org/en-US/docs/Web…ers/renderWidth

    The WebVR API reports its recommend rendering size to the application (=to krpano) this way.

    By using the oversampling setting of the krpano WebVR plugin it's possible to scale that resolution. That means krpano will internally render with a higher resolution and pass that higher-res image back to the browsers WebVR API.

    The last time I've tried increasing the resolution this way, it wasn't working in the Oculus Browser in the GearVR (or was it the Samsung VR browser?), the browser was still only rendering with 2048x1024 - BUT - I've tested again now - and it it seems that limit has been removed and using custom higher rendering resolutions is possible now (at least on the Oculus Go, need to retest on GearVR).

    That means by using higher oversampling setting, it would be possible to make the VR view sharper, but that will definitely also cost performance of course. The device will need to render a lot more pixels.

    (Btw - in the desktop WebVR browsers (Firefox, Edge, Chrome) with HTC-Vive, Windows-MR and Oculus Rift increasing the resolution was also possible this way and can also increase the image quality).

    Here the stereoscopic example with different oversampling settings:

    oversampling=1.0 (=rendering at 2048x1024 on the Oculus Go):
    https://krpano.com/examples/119/krpano.html?xml=examples/demotour-stereoscopic/tour.xml&html5=webgl+only&plugin[webvr].oversampling=1.0

    oversampling=1.5 (=rendering at 3072x1536):
    https://krpano.com/examples/119/krpano.html?xml=examples/demotour-stereoscopic/tour.xml&html5=webgl+only&plugin[webvr].oversampling=1.5

    oversampling=2.0 (=rendering at 4096x2048):
    https://krpano.com/examples/119/krpano.html?xml=examples/demotour-stereoscopic/tour.xml&html5=webgl+only&plugin[webvr].oversampling=2.0

    oversampling=2.5 (=rendering at 51206x2560):
    https://krpano.com/examples/119/krpano.html?xml=examples/demotour-stereoscopic/tour.xml&html5=webgl+only&plugin[webvr].oversampling=2.5

    Surprisingly it seems working well even up to oversampling=2.5...
    But the images in this example are only 1536x1536 per cubeface, maybe the performance will differ with higher-res panos...


    2) There is a general MSAA (multi-sample anti-aliasing) setting in the Oculus APIs, which they highly recommend be used though it is off by default. I have no idea whether this applies to webGL rendering in the Go Browser. Have you investigated that?

    That doesn't relate to the texture sampling. That's more important to the edges of hotspots.

    At the moment the browsers default setting is used (yes, in the Oculus Browser WebGL disabled by default).

    But I agree enabling it can make sense - the only problem it would need to get enabled already at startup (changing it dynamically is not possible) and it can cost performance. That means when enabled by default would effect all viewers, even ones with lower-end devices that might not be interested in VR and only want to view a simple pano...
    But I will check this, it should be possible to detect the Oculus Browser already during embedding and then only enable by default in this case.

    Manually enabling WebGL anti-aliasing in krpano would be possible in the html embedding code this way:

    Code
    embedpano({..., webglsettings:{antialias:true});


    3) The plugin has a function for re-orienting the view azimuth, independent of the gyro. Yet somewhere you say that this should not be done in VR, and it seems likely that the plugin does disable re-orientation in "real webVR" mode. Would you re-consider whether that is really necesssary?

    Sorry, but I don't understand what you mean...
    Do you mean the lookat / resetsensor action?
    https://krpano.com/plugins/webvr/#lookat
    You can use that to change the view of course.

    Maybe you mean 'animating' the view (e.g. looking around by script) - that means taking control away from the user - and that feels very bad in VR and is definitely not recommended.


    4) In "desktop fake webVR" as well as "cardboard" modes the web'vr plugin does support dragging the view azimuth with the pointing device. Why is that not possible also in real VR?

    Because there is no mouse-pointer control there anymore.

    Best regards,
    Klaus

  • That's more like it!
    The 2x oversampled 2241 cube image looks nice and sharp on both Go and GearVR/Galaxy S6, with no trace of shimmer. There is a bit of judder on the S6 at 2x oversampling, but on Go it moves perfectly smoothly. The 1.5x image moves smoooth on S6.

    I think there is a sweet spot for resolution on Oculus Go at just under 2k x 2k cubeface size (~ 8K equi width). So maybe 1920x1920 cube at 2X oversample is a good formula. I hope Roundme and 360Cities can soon replace their players with one that does it this way.

    I was asking about the lookat / resetsensor action. Even with only a gaze cursor, there could be a useful widget to jump-pan the view using that action. Unless you tell me that resetsensor is disabled in VR mode, I intend to build such a widget.

    Quote

    Because there is no mouse-pointer control there anymore


    That was my first question: Why is it not possible to use the primary pointing device inside webVR? I intend to keep asking you and Oculus until I get a satisifactory answer, namely "of course it is possible to use the primary pointing device in webVR".

    That is critically important for the whole virtual tours industry. To build usable UIs, we must have a real pointer. You know that as well as anyone. I really don't understand this resistance to having the universally accepted point-and-cliick paradigm work in VR like everywhere else.


  • I hope Roundme and 360Cities can soon replace their players with one that does it this way.

    Why even hosting services?
    Just use your own site with krpano...
    When everyone would only use hosting services, there would be no krpano anymore in the long term...


    I was asking about the lookat / resetsensor action. Even with only a gaze cursor, there could be a useful widget to jump-pan the view using that action. Unless you tell me that resetsensor is disabled in VR mode, I intend to build such a widget.

    That's an action of the WebVR plugin itself, so it isn't disabled in VR of course ;-).

    But you can also just call the normal lookat() action to change the view.
    That action has internally already an automatic connection to the WebVR plugin.
    I would say that should be the best and recommend way to change the view in VR.


    Why is it not possible to use the primary pointing device inside webVR? I intend to keep asking you

    Controller tracking + controller rendering + pointer rendering is something that need get implemented manually - and that will come of course. It's just a lot of work and needs time (and there are also a lot of other features and things to work on).

  • ohhhh yeaHHHhhh...

    Got my GO today, and i works nice!!
    Would be great indeed if we could have access to the mouse pointer thingy.

    love it!
    Tuur *thumbsup*

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!