Project Abstract:

We will investigate methods of exploiting the availability of different types of signals such as video, sound and physiological signals for improving the performance of classifiers and for enabling semi-automatic adaptation to the environment.

One key requirement for monitoring systems such as the elderly health monitoring system is to reduce the occurrences of false positives as users will turn off systems that “cry wolf” too often. The use of multiple types of signals can reduce false positives by“explaining away” some of the false indicators e.g. a video showing a person walking around while his heart rate sensor measures zero heart rate probably means that the heart rate sensor is not functioning properly rather than a heart failure. One challenge is hence to develop models and learning methods that can increase specificity while maintaining sensitivity by exploiting multiple types of signals. The difficulty of this problem is compounded by the fact that the events that we want to detect are rare. We expect that most of the measurements using the different signals would be done independently (without the other sensors being present) with only a small amount of combined measurements available. Another requirement for adaptive systems would be to adapt to the new environment without (or with very little) explicit feedback from humans. A system that is tailored to a particular individual or environment will be able to outperform a system that is designed to operate robustly across different individuals or environments. By exploiting the independence of errors in classifiers based on the different signals, it may be possible to adapt the system “blind” (without explicit human feedback) or in a semi-supervised (with very little human feedback) manner. The main sources of signals available will be video streams, physiological signals and audio signals. Processing of video streams will require tracking of human subjects using algorithms from the field of computer vision. We intend to develop machine learning algorithms that will be able to take into account evidence from multiple sources of information and incorporate them in a meaningful way to achieve the objectives of a monitoring system.