About this Project
Within the project, we investigate novel methods to interact with large dense 3D surface reconstructions while being immersive within them using virtual reality. The research covers streaming of large dense reconstruction to enable live exploration, navigation and selection of (occluded) reconstruction patches.
Streaming & Exploration of Dense 3D Surface Reconstructions in Immersive VR
We introduce a novel framework that enables large-scale dense 3D scene reconstruction, data streaming over the network and immersive exploration of the reconstructed environment using virtual reality. The system is operated by two remote entities, where one entity - for instance an autonomous aerial vehicle - captures and reconstructs the environment as well as transmits the data to another entity - such as human observer - that can immersivly explore the 3D scene, decoupled from the view of the capturing entity. The performance evaluation revealed the framework´s capabilities to perform RGB-D data capturing, dense 3D reconstruction, streaming and dynamic scene updating in real time for indoor environments up to a size of 100m2, using either a state-of-the-art mobile computer or a workstation. Thereby, our work provides a foundation for enabling immersive exploration of remotely captured and incrementally reconstructed dense 3D scenes, which has not shown before and opens up new research aspects in future.
Data Set - Viennese Flat
For testing, we employed precomputed camera trajectories using [Choi et al.] to stabilize the camera pose estimation and ensure a well aligned model.[Download] [Citation]