PatchMatch Stereo - Stereo Matching with Slanted Support Windows
By Michael Bleyer, Christoph Rhemann, and Carsten Rother
Abstract
Common local stereo methods match support windows at integer-valued disparities. The implicit assumption that pixels within the support region have constant disparity does not hold for slanted surfaces and leads to a bias towards reconstructing fronto- parallel surfaces. This work overcomes this bias by estimating an individual 3D plane at each pixel onto which the support region is projected. The major challenge of this approach is to find a pixel's optimal 3D plane among all possible planes whose number is infinite. We show that an ideal algorithm to solve this problem is PatchMatch \cite{Barnes09} that we extend to find an approximate nearest neighbor according to a plane. In addition to PatchMatch's spatial propagation scheme, we propose (1) view propagation where planes are propagated among left and right views of the stereo pair and (2) temporal propagation where planes are propagated from preceding and consecutive frames of a video when doing temporal stereo. Adaptive support weights are used in matching cost aggregation to improve results at disparity borders. We also show that our slanted support windows can be used to compute a cost volume for global stereo methods, which allows for explicit treatment of occlusions and can handle large untextured regions. In the results we demonstrate that our method reconstructs highly slanted surfaces and achieves impressive disparity details with sub-pixel precision. In the Middlebury table, our method is currently top-performer among local methods and takes rank 2 among approximately 110 competitors if sub-pixel precision is considered.
Reference
M. Bleyer, C. Rhemann, C. Rother: "PatchMatch Stereo - Stereo Matching with Slanted Support Windows"; Talk: British Machine Vision Conference BMVC 2011, Dundee; 08-29-2011 - 09-02-2011; in: "British Machine Vision Conference 2011", (2011), 1 - 11.
Additional Information
Supplementary Material: You can find a video illustrating PatchMatch Stereo here. Alternatively, you can watch the video on YouTube below. A document describing the global method used in the paper can be obtained here. You can also download the BMVC 2011 presentation here.
This work was supported by the Vienna Science and Technology Fund (WWTF) under project ICT08-019.
Downloads
| Download Patch Match Stereo | 90.4 MB | Zip archive | Download |
| Patch Match Stereo | 49.7 MB | AVI video | Download |
| Patch Match Stereo Global Method | 120 KB | PDF document | Download |
BibTeX
Click into the text area and press Ctrl+A/Ctrl+C or ⌘+A/⌘+C to copy the BibTeX into your clipboard… or download the BibTeX.