Enriching Speech Translation: Exploiting Information Beyond Words

Опубликовано 6 сентября 2016, 17:13

Current statistical speech translation approaches predominantly rely on just text transcripts and do not adequately utilize the rich contextual information such as prosody and discourse function that are conveyed beyond words and syntax. In this talk I will introduce a novel framework for enriching speech translation with prosodic prominence and dialog acts. Our approach of incorporating rich information in speech translation is motivated by the fact that it is important to capture and convey not only what is being communicated (the words) but how something is being communicated (the context). First, I will present various techniques that we have developed for automatically detecting prosody and dialog acts from speech and text, and will survey some of the most important results of our contribution. I will then describe techniques for the integration of these rich representations in spoken language translation.