The availability of quantitative online benchmarks for low-level vision tasks such as stereo and optical ﬂow has led to signiﬁcant progress in the respective ﬁelds. This paper introduces such a benchmark for image matting. There are three key factors for a successful benchmarking system: (a) a challenging, high-quality ground truth test set; (b) an online evaluation repository that is dynamically updated with new results; (c) perceptually motivated error functions. Our new benchmark strives to meet all three criteria. We evaluated several matting methods with our benchmark and show that their performance varies depending on the error function. Also, our challenging test set reveals problems of existing algorithms, not reﬂected in previously reported results. We hope that our effort will lead to considerable progress in the ﬁeld of image matting, and welcome the reader to visit our benchmark at www.alphamatting.com.
Christoph Rhemann, Carsten Rother, Jue Wang, Margrit Gelautz, Pushmeet Kohli, Pamela Rott. A Perceptually Motivated Online Benchmark for Image Matting. Technical Report TR-188-2-2009-03, May 2009.