High Fidelity Image-Based Modeling

Опубликовано 6 сентября 2016, 16:31

This talk addresses a novel multi-view stereo algorithm that takes a set of calibrated photographs and outputs a (quasi) dense set of rectangular patches covering the surfaces of an object or a scene of interests visible in the input images. Unlike many other multi-view stereo algorithms, our approach does not require any initialization such as a bounding volume, valid depth ranges, a visual hull model and etc., and it detects and discards automatically outliers and obstacles (e.g., pedestrians in front of a building where we want to reconstruct a building). It does not perform any smoothing across nearby features, yet is currently the top performer in terms of both coverage and accuracy for four of the six benchmark datasets presented in the Multi View Stereo Evaluation Project (vision.middlebury.edu/mview/). The keys to its performance are effective techniques for enforcing local photometric consistency and global visibility constraints. A simple but effective method for turning the resulting patch model into a mesh appropriate for image-based modeling is also presented. The proposed approach is demonstrated on various datasets including objects with fine surface details, deep concavities, and thin structures, outdoor scenes observed from a restricted set of viewpoints, and ``crowded'' scenes where moving obstacles appear in different places in multiple images of a static structure of interest. I will also briefly talk about our more rescent project on markerless dense motion capture.