Camera-based non-contact health sensing

Опубликовано 29 марта 2021, 18:33

The SARS-CoV-2 (COVID-19) pandemic is transforming the face of healthcare around the world. One example of this transformation can be seen in the number of medical appointments held via teleconference, which has increased by more than an order of magnitude because of stay-at-home orders and greater burdens on healthcare systems. Experts suggest that particular attention should be given to cardiovascular and pulmonary protection during treatment for COVID-19. However, in most telehealth scenarios physicians lack access to objective measurements of a patient’s condition because of the inability to capture vital signs.

In this webinar, Microsoft Principal Researcher Daniel McDuff and University of Washington PhD student Xin Liu will present an overview of computer vision methods that leverage ordinary webcams to measure physiological signals (for example, peripheral blood flow, heart rate, respiration, and blood oxygenation) without contact with the body. Learn about some examples of state-of-the-art neural models that enable on-device sensing even in resource-constrained settings and understand some of the challenges and exciting research opportunities in this space. This webinar will frame the application of these methods in the context of telehealth; however, non-contact physiological measurement also holds promise in broader health, well-being, and affective computing settings.

Together, you’ll explore:

■ The optical and physiological basis for camera-based sensing.
■ State-of-the-art, on-device algorithms for fast, scalable, and privacy-preserving measurement.
■ Future challenges, open research questions, and opportunities in this space.

Daniel McDuff is a Principal Researcher at Microsoft Research AI in Redmond, where he works on scalable tools to enable the automated recognition, analysis, and synthesis of human behavior, emotions, and physiology. His work has received nominations and awards from Popular Science, South-by-South-West (SXSW), and The Webby Awards and has been reported in The Times, The New York Times, The Wall Street Journal, BBC News, New Scientist, Scientific American, and Forbes magazine.

Xin Liu is a PhD student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, Seattle. His research interests lie in the intersection of machine learning, sensing, and healthcare. He received a bachelor’s degree with honors in computer science from the University of Massachusetts Amherst in 2018.

𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲 𝗹𝗶𝘀𝘁:

■ Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (Publication): microsoft.com/en-us/research/p...
■ MTTS-CAN (Github): github.com/xin71/MTTS-CAN
■ iPhys-Toolbox (Github): github.com/danmcduff/iphys-too...
■ Xin Liu: homes.cs.washington.edu/~xliu0...
■ Daniel McDuff: microsoft.com/en-us/research/p...
■ rPPG Demo: vitals.cs.washington.edu
■ Physiological Sensing (Project page) microsoft.com/en-us/research/p...

This webinar originally aired on November 23, 2020

*This on-demand webinar features a previously recorded Q&A session and open captioning.