Realistic Rendering in Mobile Augmented Reality
Thesis by Johannes Unterguggenberger
Supervision by Hannes Kaufmann and Peter Kán
Abstract
Augmented Reality (AR) applications combine a view of a physical, real-world environment with computer-generated objects and effects in real-time. Depending on the application, it is desirable to maximize the visual coherence of the virtual objects compared to the real-world image. To achieve this goal, virtual objects have to be rendered as realistically as possible. This thesis presents an image-based lighting (IBL) technique for realistic rendering of virtual objects on mobile devices which uses lighting information from the real-world environment. In the first step, the presented technique uses a mobile device´s camera and motion sensors to capture an omni-directional image of the surrounding in high dynamic range (HDR) and stores it in an environment map. In the second step, the captured environment map is prepared for rendering with different materials by calculating a set of maps. During rendering, the most suitable of these maps are selected for each material and used for shading a virtual object with the specific material. The map which contains diffuse illumination information is called irradiance map, and the maps which contain glossy or specular illumination information are called reflection maps. The calculation of the maps corresponds to a weighted convolution. The weighting is determined by a reflection model which takes the correct amount of incident lighting from all directions into account. How these calculations can be performed efficiently on mobile devices is the main focus of this thesis. Multiple approaches to perform the calculations are described. Their properties, results, strengths and weaknesses are analyzed and optimizations are proposed. We describe three different approaches for the calculation of irradiance and reflection maps in this thesis: the accurate calculation, a MIP-mapping based approximation method, and calculation via spherical harmonics (SH) frequency space. We provide detailed implementation instructions, analyses, and discussions for each of these approaches with regard to the properties and limitations of mobile devices. Furthermore, we describe how the calculated maps can be used with IBL rendering and be combined with established rendering techniques to achieve a high degree of visual coherence of virtual objects in AR scenes. The main novelty of this thesis is its focus on the capabilities of mobile devices. We describe how to do all steps on a single commodity mobile device: From capturing the environment at a certain point in space, to calculating the irradiance and reflection maps, and finally rendering virtual objects using the calculated maps in an AR scene.
Reference
J. Unterguggenberger: "Realistic Rendering in Mobile Augmented Reality"; Supervisor: H. Kaufmann, P. Kán; Institut für Softwaretechnik und Interaktive Systeme, 2016.
BibTeX
Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.