ParaEval: Using Paraphrases to Improve Machine Translation and Summarization Evaluations

Опубликовано 6 сентября 2016, 6:22

The machine translation and text summarization communities have both benefitted greatly from the introduction of automated evaluation procedures. Automatic evaluation methods facilitate faster turnaround for system development and assessment cycles, thus making them an essential part of the research problem. However, at the present, methodologies used in the evaluations are limited to lexical identity matching. The lack of support for word or phrase matching that stretches beyond strict lexical matches has limited the expressiveness and utility of these methods. In this talk, I present ParaEval, an automatic evaluation framework that uses paraphrases to improve machine translation and summarization evaluations. A large collection of paraphrases is collected through an unsupervised acquisition process, inspired by phrase-based statistical machine translation. I will focus on ParaEval's support for paraphrase/synonym matching, recall measurement, and correlation with human judgments.