The Microsoft SenseCam and Other Lifelogging Devices

Опубликовано 6 сентября 2016, 16:36

The SenseCam is a personal, wearable camera developed by Microsoft Research in Cambridge, UK, and used as a lifelogging device in projects like MyLifeBits. Its use in applications like MLB and and the memory aid research for Alzheimer suffers at Addenbrooks hospital in the UK, is based on wearing the SenseCam for lifelogging of 'events' during your day, and generating a fast-forward movie of the event as the memory recall interface. At Dublin City University we have been working with SenseCams for 15 months and our use of the device as a lifelogger is based on wearing it for the whole of your day (one of our users has over 1 million SenseCam images of his life). In the first part of this talk we will outline the work we are doing with SenseCam images, which begins with detecting the boundaries between events in a wearer's day, for example moving from breakfast to travel to work, to sitting at a desk using a computer, to moving to a meeting, to having coffee with a colleague, to walking to another building, etc. To do this we use a variety of image-image similarities from global image features to regional SIFT features, combined with sensor readings from the SenseCam. Once event boundaries are detected, we then create a calendar of the wearer's lifelog where s/he can browse his/her recorded lifelog. We allow a user to search for events similar to a given event, and in cases where a wearer is also carrying a GPS device we are able to populate the lifelog with other information culled from the internet such as images and place information. We are also starting work on automatic detection of (semantic) features from SenseCam images such as 'in car' or 'outdoor' or 'people present' with a view to categorising detected events into things like driving, shopping, in-meeeting, etc. Our lifelogging work uses other recording devices besides the SenseCam, including off-the-shelf ones to take biometric readings, standard ones to record GPS and audio recorders, and RFID readers for objects tagged in the home or office. We also use some of our own wearable sensors which are the fabrics used in garments, and we can use these to measure any kind of body movement. We have further developed devices to measure the amount of bending or flexibility or the wearer which is useful for measuring things like back posture. The second part of the talk will include details of our work using these other lifelogging devices.