Interactive Media Systems, TU Vienna

Immersive Interaction in Dense 3D Surface Reconstructions

Lab project in the area of Virtual and Augmented Reality.

Keywords: Virtual Reality, Immersive, Selection, Navigation, Dense 3D Surface Reconstructions, Point Clouds, Perception, Occlusion Management.

About this Project

Within the project, we investigate novel methods to interact with large dense 3D surface reconstructions while being immersive within them using virtual reality. The research covers streaming of large dense reconstruction to enable live exploration, navigation and selection of (occluded) reconstruction patches.

Streaming and Exploration of Dense 3D Reconstructions in Immersive Virtual Reality

We introduce a novel framework that enables large-scale dense 3D scene reconstruction, data streaming over the network and immersive exploration of the reconstructed environment using virtual reality. The system is operated by two remote entities, where one entity - for instance an autonomous aerial vehicle - captures and reconstructs the environment as well as transmits the data to another entity - such as human observer - that can immersivly explore the 3D scene, decoupled from the view of the capturing entity.

The performance evaluation revealed the framework´s capabilities to perform RGB-D data capturing, dense 3D reconstruction, streaming and dynamic scene updating in real time for indoor environments up to a size of 100m2, using either a state-of-the-art mobile computer or a workstation. Thereby, our work provides a foundation for enabling immersive exploration of remotely captured and incrementally reconstructed dense 3D scenes, which has not shown before and opens up new research aspects in future.

System Overview

[PNG] intro.png