Energy Based Models: From Relational Regression to Similarity Metric Learning

Опубликовано 6 сентября 2016, 17:19

In this talk I will first give a brief introduction to Energy Based Models. I will then discuss in detail the two areas where I have applied these techniques, namely Relational Regression and Similarity Metric Learning. In a number of real world problems, such as, web-page classification, social networks, and house price prediction, the samples are related to each other in complex ways such that the values of unknown variables associated with one sample depends not only on its individual features but also on other samples. Furthermore, these variables could be continuous and the relationships could be hidden (not given as part of the data). In such situations rather than treating the samples as I.I.D, one must resort to a form of collective prediction. We present a novel factor graph based framework that is capable of doing regression in such relational settings. The method is applied to the problem of real estate price prediction and is shown to be superior than other existing techniques. Knowledge of an appropriate distance metric over the data which is capable of capturing the complex dependencies exhibited by it, is an important step in designing algorithms for more advanced tasks such as information retrieval, contextual classification, and data visualization. However, most previous techniques either depend on a meaningful and computable distance metric in the input space, or they do not compute a function that can accurately map new input points whose relationships to the training data is unknown. Can we learn a metric that is solely dependent on the arbitrary neighborhood relationships in the data and is also faithful to new unseen samples? We present a method - called Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) - that answers this question. We apply this technique for dimensionality reduction and for face verification and show some interesting results.