Posts by Eric Chen

For this use case, I think threejs is a better fit.

3D Earth Demo

Quote from indexofrefraction

it is certainy an impressive demo, congrats.

can you explain a bit the technical details?

when you send the question, the visible view is upload and is part of it?

btw. it might have an use for visually impaired people,

if the view can be described by text or even voice

ps.

interesting would be also the other way, to create a panorama by text.

i did some tests with that 1-2 yrs ago but

sd wasn't able to create a proper equirectangular image and

the max resolution was no where near to what we would require.

Display More

Yes, attach the currently visible view when asking a question.

Voice interaction is feasible, just one more step, voice to text.
However, this increases response time and incurs additional STT and TTS costs.
As a small experiment, I don't want to put in the cost yet.

In fact, I did another experiment in threejs to add body movements and expressions to character models. Interactions are performed via AI, matching body movements and expressions based on content feedback. It also replies with voice, and goes one step further by generating mouth motions based on speech.
But this way, each response takes 5~10 seconds.
Of course, it may also have something to do with the fact that I use free plan for some products. I'm stingy about doing some non-profit projects, haha.

create a panorama by text
There are already some companies doing it, like SKYBOX.

Yes. Everyone has different thoughts, different viewpoints and the collision of ideas.

This was just a flash of inspiration for me, and then I spent two days making an example. I've just taken the first step.

Although my current work focus is not on 360 panorama or krpano, I still hope to see more examples of AI integration with krpano this year.

Quote from kme

Very nice proof of concept, but I feel like the restrictions that have been implemented are a bit too strict...

where am i => the kitchen?

and the plant is clearly related to the content... :)

Would love to try it out more though, nicely done!

Display More

Thanks for pointing it out, it's been fixed.
The original logic, which answers questions based only on the image, doesn't think of the image as where the user is.

But not all questions can be answered, such as what is the size of the object, how much is the object worth, etc. Nor can AI know.
As for the name of the plant, this would probably require domain-specific AI to recognize. It's just a general AI, not a botanist.

Quote from Fernando

Hi, thanks for sharing the example. In my opinion it would be very interesting if you could direct the camera to the subject of the question, for example if you say "wooden door" the camera would turn towards that door.

There is indeed AI that can do this and can add bounding boxes to the identified objects. If you're interested in doing your own research, you can check out Google Cloud Vision and Microsoft Azure Computer Vision.
I don't think this is very interesting though, if there was time to type it, it would have been done quickly by mouse.
Usually this is only suitable for special projects, where objects need to be annotated.

Hi everyone,

Happy New Year.

Demo
For Chinese without VPN (same content)

This is an attempt to integrate AI into krpano. Image recognition technology based on AI is used to analyze what the user sees in the panorama.
You can ask all kinds of questions, such as what objects are there, what is it, what color, how many, etc.

AI sees what you see.
I made some restrictions, such as the content length of the response, etc. The main reason is to avoid incurring too many bills.

tips：You don't have to use English. You can also communicate with it in other languages.

Dear Turr,

Please don't get mad. The ecological status of krpano cannot be changed in the short term.

For most developers, krpano is not a career, or a primary job. As a result, developers spend most of their time working or studying. Sharing knowledge or resolving issues in a forum is a matter of personal preference at the time.
Personally, although I've gained more knowledge over time, I don't have the passion to make plugins anymore. I think other former krpano plugin developers are similar. But seeing YOU active in the FORUM makes me feel familiar and warm here. Please don't doubt yourself.

Here's how I can help:
When you run into this problem, from a developer's perspective, the first thing you should think about is how to add the right mouse click event to hotspot.
Note: Do not set capture=false for hotspot.

		<hotspot ... onloaded="add_right_click_event"/>

		<action name="add_right_click_event" type="js"><![CDATA[
			krpano.actions.trace('>>>loaded');
			caller.sprite.addEventListener('mousedown', function(event) {
				krpano.actions.trace('>>>add mousedown event listener');
				event.preventDefault();
				if (event.button === 2) {
					// Write your logic here
					// krpano.call("Drag3DDown");
					krpano.actions.trace('Right down detected!');
				}
			}, true);
			
			caller.sprite.addEventListener('mouseup', function(event) {
				krpano.actions.trace('>>>add mouseup event listener');
				event.preventDefault();
				if (event.button === 2) {
					// Write your logic here
					// krpano.call("Drag3DUp");
					krpano.actions.trace('Right up detected!');
				}
			}, true);
		]]></action>

Quote from klaus.krpano

Are the custom keys embedded in the viewer?
There is check for this in the make vtour droplet and in the protect tool.

The test revealed some interesting phenomena and found that the problem came from id. When using lowercase letter combinations such as "zlp|123456789" will not work, but "Zlp|123456789" will work.

The ios crash is mainly because the model is too complex, it contains many parts, such as a basketball hoop contains 10+ parts. It's essentially Three.js, with no relation to Krpano. R3F just makes the code shorter because it packages some components. I just said I don't like the native Three.js because it's too complex, with many lines of code just to implement one function.

Hope that Krpano can also do something similar, by creating components with certain functions, similar to R3F Drei.

Bug: Encryption using custom keys does not work properly. Using a license to encrypt is fine.

I prefer to use R3F over the current krpano+threejs development approach. A physics engine can be used, and the code is more concise. Dislike native threejs, it takes too much code to implement a feature.
Threejs Demo

Klaus, I found that after upgrading the krpano License, if I updated krpano.js in the old projects, the encrypted xml files would not be resolved.

I think you should add some way to make old License encrypted files still parsable. As a player, not a standalone project, it is not possible to re-encrypt all files with the new License.

If you want to use it like JS, use js methods.

<action name="test" type="Javascript"><![CDATA[
    const name = args[1];
	const attr = args[2];
	const val = args[3];

	const button = krpano.get(`layer[${name}]`);
	button[attr]=val;
]]></action>

I am trying to replace the tiles with the webp image with transparent background, and there is a serious jaggies around the image in krpano. Looking forward to krpano's support for transparent tiles.

Yes, I also think webp support is important. krpano tools needs to be upgraded, which shouldn't take much time. Expect Klaus to update soon with the next version.

Excellent work. Such high resolution, unbelievable.

The compatibility of H265 in browsers is not good. Streaming is a good option for longer videos, as you only need to pre-load a few "segments" to start playing. However, for higher resolutions such as 8K, it is still difficult to load on lower versions of the iPhone.

Hi Klaus，
I found that in the <flat>, the tow-finger pinch operation on trackpad doesn't work and I couldn't zoom. But it works fine in <cube>. I ran the same test with your examples, and the results were the same.

https://krpano.com/releases/1.21/…rtment/tour.xml
https://krpano.com/krpano.html?xm…untain/flat.xml


events.ongesture wasn't triggered on trackpad.

wheeldelta_touchscale Once pinched on the trackpad, the value is persisted. After using the mouse wheel, the value will remain the same.

Klaus
One bug：
krpano.focus() is works, but not in a krpano layer.

layer[krpano_layer].krpano.call( 'focus()' )

Klaus Another bug： Flat video won't work in v1.21, but it works in v1.20.
flatpanohfov in the webvr plugin doesn't seem to work with flat video.

<flat url="plugin:video" />