Research Outline

Mobile robots have the potential of being broadly useful to humans. One particular application where they can make a huge impact is in serving as companions and caretakers for the elderly. An important technical problem is for a mobile robot to maintain, using its sensory input, a model of the state of its environment, including the location of the robot and the locations and activities of the other people in the environment.

Our goal in this project is to equip robots with the capability to monitor two or more moving people. Since the objects may move in different directions, there is no guarantee that the robot can keep all the objects in its view. The robot has to use its limited sensing resources such as a video camera and figure out how to move around in its environment in order to do a good job of tracking the people in the environment.

Deciding where to move in order to be as effective as possible is a complex reasoning problem. In addition to navigation problems, the robot has to take into account the substantial uncertainty about how people behave. Our strategy is to use the techniques of reinforcement learning, which allows the robot to learn from its interactions with the environment. The robot maintains state estimates of its own location and the location of the people, which it may not have seen. The robot’s goal is to keep the entropy of the estimates low. Since the robot has no control over the movement of the objects, it has to proactively look for one or more of the objects.

Fundamentally, it is not easy to reach an optimal solution to the problem due to the high-dimensional state space needed to express the robot’s uncertainty about people’s behaviour. We are investigating approximation solutions that will give our robot an effective good policy of where to move given a reasonable amount of training time.