Learning Language from its Perceptual Context

Опубликовано 17 августа 2016, 2:30

Current systems that learn to process natural language require laboriously constructed human annotated training data. Ideally, a computer would be able to acquire language like a child by being exposed to linguistic input in the context of a relevant but ambiguous perceptual environment. As a step in this direction, we will present systems that learn to sportscast simulated robot soccer games and to follow navigation instructions in virtual environments by simply observing sample human linguistic behavior in context. This work builds on our earlier work on supervised learning of semantic parsers that map natural language to a formal meaning representation. In order to apply such methods to learning from observation, we have developed methods that estimate the meaning of sentences from ambiguous perceptual context.