Learning disparity for dense stereo matching

Motivation

Dense stereo matching can be denoted as the central step in any photogrammetric 3D reconstruction. The core task of dense stereo matching is to find dense correspondences, i.e. one for each pixel, between images, and thus calculating the disparity of corresponding pixels between images. Current traditional stereo algorithms treat disparity estimation as a similarity measurement problem, which is, measuring the similarity between corresponding patches of two or more images. They measure similarity by using the hand-crafted matching cost metrics, which often have difficulty with areas of poor texture, reflective surfaces, thin structures and repetitive patterns. In this program, we construct an end-to-end deep learning model for directly predicting dense disparity map without post-processing.

Goal

The goal of this program is to develop an end to end deep architecture to predict disparity directly. As a CNN has a natural "deep" architecture, we expect it to have a stronger modeling ability to learn disparity directly from images. Also, we believe that learning dense correspondence representation is easier and more effective if the architecture can be structured to leverage the inherent geometric properties. We thus plan to use geometric relations which define the structure and shape of the world to simplify the learning problem. Thus, we strive to at least partly reach an explain ability of the otherwise big black box nature of deep networks.