Steering and Capturing Human Insight for Large-Scale Learning of Visual Objects

Опубликовано 17 августа 2016, 21:57

An important factor determining our success on object recognition and image search problems is how machine learning algorithms solicit and exploit human knowledge. Existing recognition approaches often manage human supervision in haphazard ways, and only allow a narrow, one-way channel of input from the annotator to the system. We propose algorithms that can steer human insight towards where it will have the most impact, and expand the manner in which recognition methods can assimilate that insight. The underlying goal is to use manual effort cost-effectively for robust visual learning. More specifically, I will present an approach to actively seek annotatorsΓÇÖ input when training an object recognition system. Unlike traditional active learning methods, we target not only the example for which a label is most needed, but also the type of label itself (e.g., an image tag vs. full segmentation). Further, since annotations should be fielded by distributed, uncoordinated annotators, we develop cost-sensitive selection algorithms to compute far-sighted predictions of which batches of data ought to be labeled next. Finally, beyond ΓÇ£askingΓÇ¥ annotators the right questions, I will show how we can ΓÇ£listenΓÇ¥ more deeply to what image-taggers unknowingly reveal in their annotations, by learning implied cues about object prominence from lists of ordered keywords. Using these cues, we improve state-of-the-art object detection and image retrieval results on benchmark datasets. This is work done with Sudheendra Vijayanarasimhan and Sung Ju Hwang.