Depth-guided disocclusion inpainting for temporal consistent virtual view synthesis

Thesis by Thomas Rittler

Abstract

Depth Image Based Rendering (DIBR) has become an important tool in the field of free-viewpoint video and three-dimensional television as this technique allows to generate virtual views of a scene from a single view and its associated depth map. However, a common issue in DIBR are disocclusions, i.e. areas that are uncovered in the newly synthesized perspectives. This thesis addresses the problem of filling disocclusions in the context of stereoscopic images and three-dimensional videos. To synthesize visually plausible content in the disoccluded regions of the novel views, a texture-oriented inpainting method is presented based on the randomized correspondence algorithm PatchMatch of Barnes et al. As missing time-consistency results in disturbing flicker artifacts, our proposed inpainting approach exploits temporal information by propagating both patch correspondences and color information to subsequent frames. Moreover, the depth information available in the field of stereoscopic imaging is incorporated to avoid the interference of foreground color in the restored image background regions. To further account for artifacts caused by the image warping process as well as inaccuracies in the underlying depth map, several enhancements are introduced including unilateral dilation, adaptive patch sizes and a matching threshold. An extensive evaluation using objective quality assessment metrics and a subjective user study shows that the spatial and temporal enhancements of the proposed image and video completion framework clearly outperform existing inpainting tools such as "content-aware fill" from the professional graphics editing program Adobe Photoshop.

Reference

T. Rittler: "Depth-guided disocclusion inpainting for temporal consistent virtual view synthesis"; Supervisor: M. Gelautz; Institut für Softwaretechnik und interaktive Systeme, 2014; final examination: 04-28-2014.

BibTeX

@mastersthesis{TUW-234957, author = {Rittler, Thomas}, title = {Depth-guided disocclusion inpainting for temporal consistent virtual view synthesis}, school = {Institut f{\"u}r Softwaretechnik und interaktive Systeme}, year = {2014} } Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.