Tightly Connecting Vision and Language

Опубликовано 15 сентября 2021, 18:52

Remarkable progress has been made at the intersection of vision and language. While showing great promise, current vision and language models may only weakly "connect" the two modalities and often fail in the wild. In this talk, I will present our recent efforts aiming to bridge this gap along two dimensions: informativeness and controllability. In particular, I will describe how we can leverage large-scale datasets, including our recently-released CC12M and Localized Narratives, to benefit existing vision-and-language tasks as well as to enable new applications.

Speaker: Soravit (Beer) Changpinyo, Google

MSR Deep Learning team: microsoft.com/en-us/research/g...