You are not logged in.

jeromebg

Professional

  • "jeromebg" started this thread

Posts: 985

Location: Angers - France

Occupation: 360 experiences creator

  • Send private message

1

Tuesday, March 9th 2021, 5:34pm

Matterport Cortex AI

Searching solutions to build depthmap virtual tours I tried Everpano (still Beta and no updates) Blender (Much better but really time consuming when trying spaces with furnitures) and saw solution like Leica BLK360 (too expensive)...

I never liked Matterport business model, but gave a try with a simple Insta360 OneX camera and a free account and must say I've been quite impressed with their Cortex AI feature to automatically build a 3D space from a few 360 panorama : https://matterport.com/ja/node/712
Was simple to use, and fast rendering, but of course IQ from Insta 360OneX sucks and I don't want to host my work on Matterport ;)

I wish we could have such a tool to use with any 360 (DSLR ones) and the player we want (Krpano of course) !

Tuur

Sage

Posts: 3,463

Location: Netherlands

Occupation: Krpano custom coding / Virtual Tours / Photography / Musician / Recording engineer

  • Send private message

2

Tuesday, March 9th 2021, 6:01pm

Hi,

Obviously it would be great when krpano could handle such + a tool.
We all are hoping for such a work flow and execution, although Blender 'chops' is always good to have/learn..
Why not put it in feature requests?

Tuur *thumbsup*

jeromebg

Professional

  • "jeromebg" started this thread

Posts: 985

Location: Angers - France

Occupation: 360 experiences creator

  • Send private message

3

Tuesday, March 9th 2021, 6:09pm

Why not put it in feature requests?

Because I don't think this can be done by only one guy, even if Klaus is a genius ;)
An interesting article on Cortex AI : https://www.spar3d.com/news/hardware/mat…omputer-vision/

4

Wednesday, March 10th 2021, 1:44pm

Interesting read regarding MP Cortex AI.

I too like many others have tried EasyPano & Blender both of which from a commercial stand point are too time consuming for most run of the mill tours. Metareal seems to be quicker but it still takes too long - I did query with Metareal a while back whether I could export the model and textures to use as I required and they said yes but I don't think this has come to fruition yet.

I have also tried various photogrammetry software with limited success.

Perhaps the iPhone 12 will be the poor mans BLK360 solution?


Tuur's recent blender example was super but I guess the time to produce that was fairly significant.


Not really sure what the solution will be ?

Posts: 71

Location: Tilburg, The Netherlands

  • Send private message

5

Friday, March 12th 2021, 3:11pm

I tried BiFuse (Github / Paper) with promising, but messy results:



It's a python script that estimates depth in monoscopic 360° panoramas. Of course there are numerous glitches in each photo, and it doesn't combine multiple depthmaps to a single 3D model. I guess those drawbacks could be solved with software.

jeromebg

Professional

  • "jeromebg" started this thread

Posts: 985

Location: Angers - France

Occupation: 360 experiences creator

  • Send private message

6

Friday, March 12th 2021, 4:22pm

Thanx for the info, gonna try that ASAP !

Tuur

Sage

Posts: 3,463

Location: Netherlands

Occupation: Krpano custom coding / Virtual Tours / Photography / Musician / Recording engineer

  • Send private message

7

Saturday, March 13th 2021, 6:39pm

Quoted

Tuur's recent blender example was super but I guess the time to produce that was fairly significant.


Thanks *thumbup* , you could leave fairly away.
It was my first try.
I have a second one, from last year, to show.
I will do in another thread.

Already working / started thinking about the next three projects/studies.
This Ai cortex thing would be great + krpano when the result is acceptable.. and manipulatable afterwards would be even better.

Tuur *thumbsup*

jeromebg

Professional

  • "jeromebg" started this thread

Posts: 985

Location: Angers - France

Occupation: 360 experiences creator

  • Send private message

8

Monday, March 22nd 2021, 1:13pm

I've been playing with BiFuse to auto generate depthmap from 360 panos.
Results are not so great, but hey it's full auto !
Maybe it could work alright only for 3D transitions for spaces without too much furnitures.
https://360images.fr/barebone_auto_depthmap/

9

Thursday, March 25th 2021, 11:04am

Hi,

great example!

Actually not that bad for automatic generated depthmaps!

But all depthmaps images in the example have the same/similar artifacts at the bottom and especially in the left-bottom corner. That causes these 'spikes' at the nadir. Maybe some bug in the BiFuse software (or maybe because the 'training data' was always without nadir). Without that, the images would be really good.

Additionally also a slightly larger depthmap.offset value might be good, e.g. try '0.05' or something around that:
https://krpano.com/docu/xml/#depthmap.offset

Best regards,
Klaus

jeromebg

Professional

  • "jeromebg" started this thread

Posts: 985

Location: Angers - France

Occupation: 360 experiences creator

  • Send private message

10

Thursday, March 25th 2021, 11:54am

Updated with offset="0.05" and it's better indeed !
Yes, spikes are due to the training data based on Matterport3D which has no depth ground truth on the top and bottom area, but I guess this could be solved easily.

Similar threads