Natural Scene Categorization in Humans and Computers

Опубликовано 6 сентября 2016, 6:21

For both humans and machines, the ability to learn and categorize natural scenes as well as the objects within is an essential and important functionality. The bulk of this talk will focus on a computer vision model we developed recently to tackle to the problem of categorizing complex real-life images. To motivate this topic, I will present a series of recent human psychophysics studies on natural scene recognition. All these experiments converge to one prominent phenomena of the human visual system: humans are extremely efficient and rapid in capturing the overall gist of natural images. We can categorize a scene as a beach image or a rock concert image in literally a split of a second. Can we achieve such a feat in computer vision modeling? We propose here a generative Bayesian hierarchical model that learns to categorize natural images in a weakly supervised fashion. We represent an image by a collection of local regions, denoted as codewords obtained by unsupervised clustering. Each region is then represented as part of a `theme'. In previous work, such themes were learnt from hand-annotations of experts, while our method learns the theme distribution as well as the codewords distribution over the themes without such supervision. We report excellent categorization performances on a large set of 13 categories of complex scenes.