Machine Learning Work Shop - A Minimax Entropy Method for Learning the Wisdom of Crowds

Опубликовано 12 августа 2016, 3:11

Machine Learning Work Shop-Session 4 - Dengyong Zhou - 'A Minimax Entropy Method for Learning the Wisdom of Crowds' In many real-world machine learning applications, we have training examples labeled by a crowd of workers with varied expertise rather than by an expert. Collecting labels from a crowd is usually cheap, but the label quality is low. For solving this problem, we propose a minimax entropy principle to simultaneously estimate worker expertise, task ambiguities, and true labels with a game theoretic interpretation. Moreover, we suggest an objectivity requirement for reasonably measuring worker expertise and task ambiguities, and show that the proposed method is unique in meeting the objectivity requirement. Experiments on real crowdsourcing data show that our minimax entropy approach dramatically outperforms states of the arts. It is a joint work with Sumit Basu, Yi Mao and John Platt.