krpano 1.23 - 3D Gaussian Splatting Support

Quote from tksharpless

I've been quite impressed by the triangle splatting project (https://github.com/trianglesplatting/triangle-splatting) and I'm wondering if you will consider also supporting that way of displaying radiance fields?

This method creates a 'soup' of 3D triangles, each with a spherical harmonic radiance field, that can supposedly be rendered more efficiently than a Gaussian 'soup', and with better visual fidelity. The training process definitely runs faster than for a Gaussian splat, I can do it in 30 minutes on my little gaming laptop.

Despite the title of their paper, that team has not produced a usable viewer, and their native renderer runs much slower than real time. It is possible to build one-off viewers by exporting their triangles to a game engine, so far only via the primitive .off file format, with fully opaque triangles. However if you would like to experiment with rendering triangle splats, I would be glad to help by implementing exports in other formats.

Another project implementing triangle splatting is https://gaoderender.github.io/triangle-splatting/ They do have a web viewer :) Have not experimented with it yet, but looks very simular to the triangle-splatting project you mentionned (though formats are probably not interchangeable).

Wow, thanks, kme. I'll give it a try.

I'm looking for information on splat file formats, especially for examples of real, working files, in the formats that krpano can load, but interested also in others.

My aim is to create scripts for exporting models from the triangle splatting engines, that will be close to what krpano is already comfortable with.

In fact I have a notion that, pending Klaus implementing native triangle splats, it may be possible to convert triangle splats into bogus Gaussian splats that will play on krpano 1.23.

So, please, if you have a .splat or .ply file that works with krpano, please drop me a link.

Hi Klaus and Jerome,

Thank you both for your help. I finally have my first splat ready to publish!

Here’s the link:
krpano

It looks okay for a first try, though there’s definitely still a lot of room for improvement.
For example, I’d like the model to appear exactly in the center of the screen when it loads.

Best,

Gregory

Thank you, will try

Hi,

the good news - the splat opens in center of my desktop screen ty="-160".

Thank you,

Gregory

I hope krpano will soon support spherical harmonics for spat rendering. This is a very important element of the superior image quality of splats vs 3d models, because it enables lifelike motion of highlights. Please compare the highlights on this gold-covered statue as rendered by the SuperSplat player, with krpano's current rather dull rendering. SuperSplat version: https://3d3sixty.com/tksharpless.ne…tml?noanim=true . Splat data to drop into the krpano demo: https://drive.google.com/file/d/1VeeEky…?usp=drive_link

Not sure Spherical Harmonics will be very usable for web display, keeping those informations makes the file 4 time bigger for a small benefit which can be almost compensated adding a little contrast IMO.

I agree that spherical harmonics aren't needed for many images; their use should certainly be optional. But they do contribute a lot to image quality for shiny surfaces. It might be possible to apply them selectively, for example by marking the splats where they are needed.

Hi,

I need some advice from the group. What command /action I could to add to Gaussian viewer, to make it play view points automaticaly, let say after 10 second after a splatter file is open. The same like autorotation in panoramas.

I tried <events onloadcomplete="start_viewpoints_playback();"/>

or onstart="play_viewpoints();" but it did not work out - well, I am just started to learn.

Thank you,

Gregory

start_viewpoints_playback or play_viewpoints is not actions embedded into krpano like autorotate is.
I guess you used splattools to generate your viewpoints, the code to play the viewpoints is also is that file : splattools_playviewpoints()

Thank you, Jerome.

It works !

klaus.krpano any news for ply support? not splats.

Yes! Great update!
Okay, I'm a little late to the party..
..I haven't played around with it yet because I wanted to create a few splats first.
Now I have a few and am looking forward to using the update!

It would be nice if we could load an additional collision model (.stl/.DEPTH) for the scenes in addition to the splats.
I've seen Tuur's solution with two krpano layers, but this would be much more convenient.

Quote from klaus.krpano

Hi,

your example was not about ply support, it was more about using a point cloud to create a depthmap surface for a panoramic image. That's a whole different topic than Gaussian Splats rendering. Connecting arbitrary points to create a surface is a non-trivial topic I haven't worked on yet. But maybe in future...

Yes, in one of the next releases.

Best regards,
Klaus

Maybe not making depthmap, nearest point finding and calculating overlap can be. It can be a way to make it faster.

Hi,
I just tried the VR mode of the examples in a Quest 3.
The way "house" opens make me think this is still under progress (as I can not move and it seems like the viewpoint is inside the house). Bicycles and others are not playing well in vr mode.... Is it because more ressource then the Quest 3 will be required for VR Headset ?
In the mean time I will soon try to do my first few splats.... Interesting new tech.
Looking at KR's evolution is so cool. Thanks for the amazing work.

Quote from tksharpless

I'm looking for information on splat file formats, especially for examples of real, working files, in the formats that krpano can load, but interested also in others.

My aim is to create scripts for exporting models from the triangle splatting engines, that will be close to what krpano is already comfortable with.

In fact I have a notion that, pending Klaus implementing native triangle splats, it may be possible to convert triangle splats into bogus Gaussian splats that will play on krpano 1.23.

So, please, if you have a .splat or .ply file that works with krpano, please drop me a link.

https://poly.cam/capture/eab383…16-fd2d252b9ce0 you can download

Hello,

tour.js is not included in the example. You can generate tour.js using the krpano tools: launch krpano tools and go to the protect tab. There you can use « generate viewer » to generate tour.js using your license. Put the tour.js in the same folder as the files from the tutorial.

gr,

Kristof