Lessons from the Wild: How Context Can Shape Consumption in Content Recommendation Systems

Опубликовано 15 апреля 2013, 10:27

Paul Ogilvie, ECIR 2013

Cranfield style evaluations have the advantage of reproducibility but ignore many factors that can shape user interaction. It is common in industry to augment static test collections with tests performed on live user traffic. Split or A/B testing can be used to help control compounding factors, but nevertheless they can have a large impact on online metrics. How users interact with the product also provide a valuable source of data for understanding a product and forming hypotheses. For example, users following an email link messaged "Top content, tailored for you from the people, industries, and companies you're following" may have a different expectation about the content recommendations than when following a link from the homepage of LinkedIn titled "LinkedIn Today recommends this content for you." In this talk, we share lessons and observations about patterns of user interaction with the content recommendations we provide in LinkedIn Today. We pay special attention to how members arrive at the product, the different contexts in which the product may be consumed (website, mobile, email), and what they do next.