Unsupervised Discovery of Objects and their Interactions for Common-Sense Physical Reasoning

Опубликовано 29 мая 2018, 21:52

Common-sense physical reasoning is facilitated by representing sensory percepts into discrete objects. By decomposing a complex visual scene into distinct objects, humans can describe relations between the objects and reason about their dynamics as well as the consequences of their interactions. What is the program that performs such reasoning, and how do we reverse-engineer this prior knowledge of physics into machines? One approach has been to use a physics simulator whose parameters are inferred from raw sensory observations. Another approach has been to directly learn to map raw observations to predictions. However, the advantage of one is the disadvantage of the other: the source code of a physics engine is typically difficult to adapt to real-world scenarios, but traditional unsupervised learning approaches for modeling physical interaction in pixel space do not generalize to different scenes or respect commonsense intuitions about object permanence. I will discuss a line of recent work that seeks to bridge the gap between the two approaches. With the Neural Physics Engine (paper: arxiv.org/abs/1612.00341, webpage: mbchang.github.io/npe/), we first assumed object state information and proposed a learnable model that can generalize to different numbers of objects and scene configurations the way symbolic physics engines do. Then with Relational Neural Expectation Maximization (paper: arxiv.org/abs/1802.10353; webpage: sites.google.com/view/r-nem-gi... we removed the state-space assumption with a method for obtaining object representations from pixels. The result is an end-to-end system that learns to discover objects and model their physical interactions from raw visual images in a purely unsupervised fashion, exhibiting simple notions of object permanence, attention, and compositionality that characterize human-level common-sense physical reasoning.

See more at microsoft.com/en-us/research/v...