Layers of appearances and motion

Опубликовано 7 сентября 2016, 16:09

Visual scene analysis when posed as an inference problem in a generative model offers efficient and principled integration of various causes of variability such as motion, lighting and appearance changes, all in a single framework. In addition, joint modeling of multiple causes enable each part of the model to capture some of the variability in the data until all is decomposed and explained using modeled causes. In this talk, I will present a layered generative model that describes an image as a composition of appearance, shape, global transformation and non-uniform deformation of objects occupying each layer. I will describe a variational EM algorithm for inference and learning. Inference reveals the distribution over all the hidden variables (appearance, shape and deformation field for each layer) for each frame in the sequence. I will then present epitomic representation as a viable alternative for modeling appearance and shape when we have only small amount of training data. Results on video summarization, image segmentation and frame interpolation will be presented.