Blog post - building the Aggstein Ruin 3D virtual tour

kme · April 20, 2023 at 9:37 PM

Hi all,

Those who have been here for the last couple of months surely noticed that I've been very active with krpano, especially with the 3D possibilities. But up to a couple of weeks ago, I did not have any experience in building a real tour and my focus was more on building small extentions for krpano to make tour building more accessible.
Thanks to these experiments I have had contact with some really nice people and had a chance to exchange ideas, share experiences and learn from eachother.

In one of those contacts, we explored how krpano could handle a larger 3D model. The model was a photogrammetry scan of a ruin and provided in glb format.
Some data:
- vertices: 482.736
- faces: 413.744
- glb size: 14.6Mb excl textures
- textures: 6x8K textures (total: 65Mb)

Wireframe:

So I started building a tour simular to the one online now at https://www.ruineaggstein.at/en/castle-ruin/360-degree-tour. As I didnt want to put 30+ panos as hotspots on the map so I looked up the most interesting points and came across 2 reference points:

I really like the style of the second one. Cirlces and numbers - looks like my annotations plugin
In the end, I decided to go for the content of the first one, mapping all points (1-16) on the 3D model with the annotations plugin. Looking back, it may have been too much (it is a bit distracting when you open the 3D model the first time).

Some observations:

krpano is really good at handling these files. And when using 8K textures, the obj always loaded fast (at least locally), and in the browser, the most time is spent re-mapping the textures to the model when loading the scene. With 4K, this goes a lot faster so I opted for that resolution in the final tour
The converted obj file is about 60mb and baked textures for the day/sunset views are 4K and 20Mb in total. Especially the obj file format is a lot more bandwidth to transfer...
building tours is hard work with krpano! Organising the panos, linking them together, find a good way to navigate... I underestimated all aspects of the workflow
while eating my own dogfood with the annotations, I found many things that could be improved so I did those along the way - an update is coming soon (adjustable colors, some workflow improvements, ...)
Building the menu from scratch was also a challenge, but I'm happy with the result.
I still miss some key features for these types of tours, more specifically an easy way to link the panorama's to the model/depthmap and a nice transition from the model to that panorama. if anyone has any tips/tricks, do reach out...
I need to pay more attention to mobile when creating the tour (menu needs a good mobile version, could also load different versions of the models when using mobile to save bandwidth, ...)

All in all this was a very nice learning experience. Special thanks to Sebastian from https://360studios.at/ for the mentoring and providing the models/pano's.

And if you are curious: here is the tour:

https://krpano.kri-soft.be/aggstein/
Screenshot:

Would love to hear your feedback!?

gr,
Kristof

lastromantik · April 20, 2023 at 10:14 PM

Very impressive! Well done!
This is exactly the direction you need to move the krpano)
Hopefully soon I will finally finish my castle) and we will compare)

360studios.at · April 20, 2023 at 10:47 PM

This is the way! :)

spacerywirtualne · April 21, 2023 at 8:10 AM

kme · April 21, 2023 at 8:17 AM

Fixed.
Case sensitive server Thanks for reporting

Could you try again, it is well worth it

ramirox3 · April 21, 2023 at 3:18 PM

wonderful, intuitive and easy and enjoyable tour.
Congratulations¡¡¡¡¡¡¡

indexofrefraction · April 21, 2023 at 5:10 PM

Quote from pano-erbauer

This is the way! :)

This is the way.

s-maier · April 21, 2023 at 7:21 PM

Quote from indexofrefraction

This is the way.

This is the way! :)

klaus.krpano · April 21, 2023 at 9:12 PM

Hi,

Quote from kme

https://krpano.kri-soft.be/aggstein/

Wow! Wonderful!
Very well done!

But I have also to say - 'a large 3D model plus several normal panos' - that would be exactly what I'm looking for *love*;-).
On the one hand, also for making a nice demo and on the other hand, as help for developing/improving the creation tools and for improving/optimizing the viewer. Good data to work with is always a good help.

Quote from kme

The converted obj file is about 60mb and baked textures for the day/sunset views are 4K and 20Mb in total. Especially the obj file format is a lot more bandwidth to transfer...

Here are some improvements possible to speed up the download and startup time:

On the server the gzip compression could be enabled/added for .obj files.
That should reduce the actual transfer/download size to something between 14 and 18MB.
The 'aggstein-logo-baked.png' image is pretty large (~6MB) - and actually not used in the 3D-model - so it can be safely removed from the .mtl file.
With that large 3D-model, some time is also spend on creating the internal hit-testing data-structure.
You could try using hittestmode="gpu" instead.
This should improve the startup-time and reduce the memory-need (can be important on mobile devices, especially on iOS devices).
E.g. try this setting:
Code
```
<depthmap ... hittestmode.mobile.and.no-mobilevr="gpu" />
```
This will enable the GPU mode for mobile devices, but not for VR-headsets like the Oculus Quests, these are unfortunately very slow when using GPU-hit-testing.

Quote from kme

building tours is hard work with krpano! Organising the panos, linking them together, find a good way to navigate... I underestimated all aspects of the workflow

This is unfortunately true... but known and I'm already working on that.

Quote from kme

I still miss some key features for these types of tours, more specifically an easy way to link the panorama's to the model/depthmap and a nice transition from the model to that panorama. if anyone has any tips/tricks, do reach out...

Linking the 3D-model and panos - by using the 3D-model also for panos and additionally using nice 3D-transitions - This is the way! I'm ready to help on that.

Best regards,
Klaus

Tuur · April 21, 2023 at 11:41 PM

Nice model!

Quote

'a large 3D model plus several normal panos' - that would be exactly what I'm looking for

Yes I would also be pleased with such a model + panos to play around a bit.
..
ah gzip! Nice! didn't know that.
Would that also work on a video file?

What is the connection with 'This is the way' ?? missing that.

Tuur

360studios.at · April 22, 2023 at 9:43 AM

Hey! :)

So about a month ago I started creating my first 3D models.
I was looking for the right tool to display my models and use it for my business.
of course first you come across sketchfab but is limited to 3D only.
my plan was to combine the 3D models and 360°.

then i came across kristof's post in an fb group where he presented his plugin and i got in touch with him.
I found out that krpano can now also display 3D models.
as a long-time krpano/panotour user, i was of course immediately enthusiastic.
Unfortunately, it quickly became clear to me that krpano, for me as a non-coder (only a little bit ;) ), and in general as far as the possibilities for 3D are concerned, is still in its infancy.

After a bit of research I found out that 3DVista has also been supporting 3D since the end of last year, and i have to say vista has the edge when it comes to 3D.
so i started to create my first demos and tours in vista.
at the same time i was in close contact with kristof, we exchanged ideas every day and inspired each other in our process of developing the possibilities of the various tools.

so right now i have built my first finished tours in vista, also the burg ruine and i am working on a second one right now.
here are the tours:
Aggstein - https://360studios.at/3d/aggstein/
Stift Herzogenburg - https://360studios.at/3d/stiftherzogenburg/
All my 3D models, no tours only a showcase of my models if you are interested - https://360studios.at/3d/alle/

for the tours in vista, as you can see, the experience is very fluid and the whole tour is a dream to use.
this is the way ;)

Dear Klaus, I would be happy to provide you with the data of the castle so that you can use them to advance the development of krpano.
i love krpano and i would like the tool to develop in the right direction with you and the community, so that it doesn't have to hide behind the big boys ;)

Best regards, Sebastian

Tuur · April 22, 2023 at 9:58 AM

the Stift Herzogenburg model is also really nice!
Interesting stuff that photogrammetry. Seen a lot of that already, but never tried.
Would love to play with a model.

Tell me .. what is that about that 'this is the way'.. I understand that it is a series or so.. but what is the fun?

Tuur

indexofrefraction · April 22, 2023 at 4:37 PM

Quote from 360studios.at

Best regards, Sebastian

hi, just FYI, safari warns of visiting your site (certificate problem)

klaus.krpano · April 22, 2023 at 9:05 PM

Quote from 360studios.at

Aggstein - https://360studios.at/3d/aggstein/
Stift Herzogenburg - https://360studios.at/3d/stiftherzogenburg/
All my 3D models, no tours only a showcase of my models if you are interested - https://360studios.at/3d/alle/

The 3D-models are very well done!
Very clean and good homogeneous lighting! I assume shooting when cloudy?
Would you like to share details on the creation? Shooting details? What photogrammetry software?
But I could understand, when not of course!

Quote from 360studios.at

Dear Klaus, I would be happy to provide you with the data of the castle so that you can use them to advance the development of krpano.

Thank you Sebastian! I will contact you via email for more details.

klaus.krpano · April 22, 2023 at 9:08 PM

Quote from Tuur

Tell me .. what is that about that 'this is the way'.. I understand that it is a series or so.. but what is the fun?

It's a pop-cultural quote from the Star Wars Mandalorian series.
They say it to each other when they agree on something.
See this clip to get an idea:
https://www.youtube.com/watch?v=Mw7zSQ7ja7Y

Tuur · April 22, 2023 at 9:48 PM

ah!
hahaha..
Thanks for clearing that up!

Tuur

360studios.at · April 22, 2023 at 11:53 PM

Quote from klaus.krpano

The 3D-models are very well done!
Very clean and good homogeneous lighting! I assume shooting when cloudy?
Would you like to share details on the creation? Shooting details? What photogrammetry software?
But I could understand, when not of course!
Thank you Sebastian! I will contact you via email for more details.

thank you :) yes all are shot on cloudy days.
i shoot with a mavic pro 2 , in raw, lightroom for editing the pix and reality capture for creating the model :)

Tuur · April 22, 2023 at 11:53 PM

kme

would you be so kind to share the whole navigation stuff, mouse and gesture behaviour etc?
Maybe in an small example with a small dummy model, a football or so, haha.. to share?
I think it works great and is so common/standard.. it would help many people, including me obviously .. hehe.

sebastian
ah! thanks for sharing!
There is also quite a large Facebook group with quite some info.
I think Jeffrey Martin also was quite active with photogrammetry.

Tuur

kme · April 23, 2023 at 1:56 PM

Tuur

For the Aggstein tour I off course used some of my own plugins :

You probably have seen the annotation plugin, but may have missed the orbit control plugin. As that last plugin is not completely productised yet, I have not announced it here on the forums. A brief mention was made in a youtube tutorial, but other than that, not much is visible on this.

Around the same time as I was starting this plugin, San7 posted something simular. I tried to get a discussion/collab going in this thread, but no response so I went ahead and implement it in the way I thought it was generally useful/accepted. It took some time to get my head around the different gesture events but now I think I finally get it. I also learned that each tour will have its own "optimal" settings for all of the movements/controls so I adapted the plugin even more to accept different parameters.

Both of the used plugins were vastly improved during the creation of the tour and i will release updates to them later this week. You can already try the current version if you are interested.

Tuur · April 23, 2023 at 2:16 PM

Hi,

thanks for the reply Kristof.
To be honest I would like to see, need and use an opensource of this.
Like the depthmap navigation xml or so.
As it is nice and seems to work great, and with rapidly extending krpano functionality, I think it should be shared opensource as it is quite a mainstream functionality... and we all like to fiddle.. and experiment.. evolution.
I'm quite new to this whole 3d thing. In 2021 der-mische and I made https://panimator.virtualtuur.com/example/ for our plugin. I did some 3d modelling before, but it takes me too much time to get it nice. Ohhh I wish I had some parallel lives.
But with the photogrammetry stuff and models as well coming soon to krpano I guess this navigation is important.
As this whole 3d stuff is quite new, and you did a great job also with Sebastian, you can't ignore that it will be 'common' practise soon I guess. Like the whole 3d models as we used to make before 1.21.
I am mostly interested in the mouse/key functionality as on the castle. (the Sketch fab way) .. also for mobiles etc..
Unfortunately I'm a bit too busy to spend much time on this specific to figure out, like many others I think. And it can be tricky.
I was fiddling a bit in the depthmap navigation xml to see how etc.. but didn't come to proper result.. yet.

So.. I hope you'll re-consider this.
I would pay a fair price for the opensource if that would change your mind.
Enough for you to 'shine with' left.

Tuur

Participate now!