Natural Language Processing From Scratch

Опубликовано 7 сентября 2016, 17:59

We will describe recent advances in deep learning techniques for Natural Language Processing (NLP). Traditional NLP approaches favour shallow systems, possibly cascaded, with adequate hand-crafted features. In this work we purposefully try to disregard domain- specific knowledge in favor of large-scale semi-supervised end-to-end learning. Our systems include several feature layers, with increasing abstraction level at each layer, that is, a multi-layer neural network. We will describe training techniques that easily scale to a billion of unlabeled words. We will discuss multi-tasking different tasks and end-to-end structured output learning. We will demonstrate state-of-the-art accuracies with considerable speedups.