Sie sind nicht angemeldet.

Matterport Cortex AI

jeromebg

Profi

  • »jeromebg« ist der Autor dieses Themas

Beiträge: 1 120

Wohnort: Angers - France

Beruf: 360 experiences creator

  • Nachricht senden

1

Dienstag, 9. März 2021, 17:34

Matterport Cortex AI

Searching solutions to build depthmap virtual tours I tried Everpano (still Beta and no updates) Blender (Much better but really time consuming when trying spaces with furnitures) and saw solution like Leica BLK360 (too expensive)...

I never liked Matterport business model, but gave a try with a simple Insta360 OneX camera and a free account and must say I've been quite impressed with their Cortex AI feature to automatically build a 3D space from a few 360 panorama : https://matterport.com/ja/node/712
Was simple to use, and fast rendering, but of course IQ from Insta 360OneX sucks and I don't want to host my work on Matterport ;)

I wish we could have such a tool to use with any 360 (DSLR ones) and the player we want (Krpano of course) !

Tuur

Erleuchteter

Beiträge: 3 839

Wohnort: Netherlands

Beruf: Krpano custom coding / Virtual Tours / Photography / Musician / Recording engineer

  • Nachricht senden

2

Dienstag, 9. März 2021, 18:01

Hi,

Obviously it would be great when krpano could handle such + a tool.
We all are hoping for such a work flow and execution, although Blender 'chops' is always good to have/learn..
Why not put it in feature requests?

Tuur *thumbsup*
https://www.virtualtuur.com
Panimator: https://krpano.com/plugins/userplugins/panimator/#top
ToolBox: https://bit.ly/3TTnd6U
Measure in 360 photos : https://pame.virtualtuur.com
Moving Hotspots V2: https://krpano.com/plugins/userplugins/movinghotspots/#top
Donations
https://bit.ly/3NnpynC

jeromebg

Profi

  • »jeromebg« ist der Autor dieses Themas

Beiträge: 1 120

Wohnort: Angers - France

Beruf: 360 experiences creator

  • Nachricht senden

3

Dienstag, 9. März 2021, 18:09

Zitat von »Tuur«

Why not put it in feature requests?

Because I don't think this can be done by only one guy, even if Klaus is a genius ;)
An interesting article on Cortex AI : https://www.spar3d.com/news/hardware/mat…omputer-vision/

GeoK

Schüler

Beiträge: 91

  • Nachricht senden

4

Mittwoch, 10. März 2021, 13:44

Interesting read regarding MP Cortex AI.

I too like many others have tried EasyPano & Blender both of which from a commercial stand point are too time consuming for most run of the mill tours. Metareal seems to be quicker but it still takes too long - I did query with Metareal a while back whether I could export the model and textures to use as I required and they said yes but I don't think this has come to fruition yet.

I have also tried various photogrammetry software with limited success.

Perhaps the iPhone 12 will be the poor mans BLK360 solution?


Tuur's recent blender example was super but I guess the time to produce that was fairly significant.


Not really sure what the solution will be ?

360gradenfotografie

Schüler

Beiträge: 71

Wohnort: Tilburg, The Netherlands

  • Nachricht senden

5

Freitag, 12. März 2021, 15:11

I tried BiFuse (Github / Paper) with promising, but messy results:



It's a python script that estimates depth in monoscopic 360° panoramas. Of course there are numerous glitches in each photo, and it doesn't combine multiple depthmaps to a single 3D model. I guess those drawbacks could be solved with software.
360gradenfotografie.nl

jeromebg

Profi

  • »jeromebg« ist der Autor dieses Themas

Beiträge: 1 120

Wohnort: Angers - France

Beruf: 360 experiences creator

  • Nachricht senden

6

Freitag, 12. März 2021, 16:22

Thanx for the info, gonna try that ASAP !

Tuur

Erleuchteter

Beiträge: 3 839

Wohnort: Netherlands

Beruf: Krpano custom coding / Virtual Tours / Photography / Musician / Recording engineer

  • Nachricht senden

7

Samstag, 13. März 2021, 18:39

Zitat

Tuur's recent blender example was super but I guess the time to produce that was fairly significant.


Thanks *thumbup* , you could leave fairly away.
It was my first try.
I have a second one, from last year, to show.
I will do in another thread.

Already working / started thinking about the next three projects/studies.
This Ai cortex thing would be great + krpano when the result is acceptable.. and manipulatable afterwards would be even better.

Tuur *thumbsup*
https://www.virtualtuur.com
Panimator: https://krpano.com/plugins/userplugins/panimator/#top
ToolBox: https://bit.ly/3TTnd6U
Measure in 360 photos : https://pame.virtualtuur.com
Moving Hotspots V2: https://krpano.com/plugins/userplugins/movinghotspots/#top
Donations
https://bit.ly/3NnpynC

jeromebg

Profi

  • »jeromebg« ist der Autor dieses Themas

Beiträge: 1 120

Wohnort: Angers - France

Beruf: 360 experiences creator

  • Nachricht senden

8

Montag, 22. März 2021, 13:13

I've been playing with BiFuse to auto generate depthmap from 360 panos.
Results are not so great, but hey it's full auto !
Maybe it could work alright only for 3D transitions for spaces without too much furnitures.
https://360images.fr/barebone_auto_depthmap/

klaus.krpano

Administrator

Beiträge: 13 499

  • Nachricht senden

9

Donnerstag, 25. März 2021, 11:04

Hi,

great example!

Actually not that bad for automatic generated depthmaps!

But all depthmaps images in the example have the same/similar artifacts at the bottom and especially in the left-bottom corner. That causes these 'spikes' at the nadir. Maybe some bug in the BiFuse software (or maybe because the 'training data' was always without nadir). Without that, the images would be really good.

Additionally also a slightly larger depthmap.offset value might be good, e.g. try '0.05' or something around that:
https://krpano.com/docu/xml/#depthmap.offset

Best regards,
Klaus

jeromebg

Profi

  • »jeromebg« ist der Autor dieses Themas

Beiträge: 1 120

Wohnort: Angers - France

Beruf: 360 experiences creator

  • Nachricht senden

10

Donnerstag, 25. März 2021, 11:54

Updated with offset="0.05" and it's better indeed !
Yes, spikes are due to the training data based on Matterport3D which has no depth ground truth on the top and bottom area, but I guess this could be solved easily.

PanoramicWEB

Fortgeschrittener

Beiträge: 192

Wohnort: Turkey

Beruf: 360 Software and Hardware Developing

  • Nachricht senden

11

Donnerstag, 29. April 2021, 12:47

Zitat von »klaus.krpano«

BiFuse software (or maybe because the 'training data' was always without nadir). Without that, the images would be really good.
Hi Klaus, i mean in mail before this problem, laser scanners or 3dstereo panorama has problems to make ground and a big black hole , can you make the mesh render in ground and top user selectable for example depth not work in ground 10 degrees, and it will make a mesh hole and only render 360 photo in ground?.
Maybe my english is problem to explain you what i mean *unsure*
PanoramicWEB Solutions

http://www.panoramicweb.net - info@panoramicweb.net

Ähnliche Themen