So exited to hear more about it! It's a very good idea!

I don't have an answer but I saw on blender maybe the begin of an answer :https://www.video-d.com/blender-vr-immersive-3d/ (this site is in french but there is demonstration videos that maybe orientate you if blender will be usefull or not for this purpose)