LabelMe: online image annotation and applications

Опубликовано 17 августа 2016, 20:54

Central to the development of computer vision systems is the collection and use of annotated images spanning our visual world. Annotations may include information about the identity, spatial extent, and viewpoint of the objects present in a depicted scene. Such a database is useful for the training and evaluation of computer vision systems. Motivated by the availability of images on the internet, we introduced a web-based annotation tool that allows online users to label objects and their spatial extent in images. To date, we have collected over 500K annotations that span a variety of different scene and object classes. In this talk, I will show the contents of the database, its growth over time, and statistics of its usage. In addition, we use the collected user-provided object annotations to extract the real-world 3D coordinates of images in a variety of scenes. Important for this task is the recovery of geometric information that is implicit in the object labels, such as qualitative relationships between objects (attachment, support, occlusion) and quantitative ones (inferring camera parameters). We show that we are able to obtain high quality 3D information by evaluating the proposed approach on a database obtained with a laser range scanner. Joint work with Antonio Torralba (MIT) and William T. Freeman (MIT)