This is done already quit a few times. I did it my self 4-5 years ago.. in the early days of krpano.
It is a nice way to present your site but it is most of the time hard to maintain and SEO is also not that great...
It depends offcourse how you organise things in the background.
One thing i really liked was using the cube faces as bakground of the pages.
I'll try to recover it to show you..
Tuur