MIT Humans and Automation Lab
Multimodal Interface Toolkit for UAV Systems (MITUS)
The visual channel is the primary modality for displaying information to
unmanned aerial vehicle (UAV) operators. The focus of the MITUS research
has been to explore alternative modalities and combinations of modalities
for displaying information to UAV operators. Are there certain pieces of
information that are better portrayed over the audio or haptic channel? What
are the effects on performance and workload when information is parsed out
or repeated over various modalities? This research project has evaluated
the use of continuous audio (sonifications), a tactor wrist vibrator, and
a waist pressure band. All testing has been completed on HAL’s Multiple Aerial
Unmanned Vehicle Experiment (MAUVE) simulator. The results of an experiment
with 44 military personnel have shown that using continuous audio in conjunction
with visual displays does enhance operator performance. Further, preliminary
studies have shown that continuous haptic feedback can also enhance performance,
in particular for monitoring of events that are continuous in nature (e.g.,
UAV course conformance).
Collaboration
and distributed decision making is a critical component for
network-centric operations like those needed for first-response teams,
air traffic control, and military command and control. In these complex systems,
allowing remotely located individuals and/or groups the ability to leverage
information both locally and globally to come to decisions is critical.
However, since these systems necessarily contain high levels of automation,
it is a fundamental human supervisory control problem to determine what roles
or sharing of roles is effective, and how intelligent autonomy may improve
or degrade time sensitive team decisions. This research effort involves the
development of technology to support collaborative decision making between
both humans and humans and computers. In particular, this project focuseson
supporting teams of operators interacting with highly autonomous unmanned vehicle
systems (UVSs) during time-sensitive intelligence, reconnaissance, and surveillance
(ISR) missions. In addition, several assistive collaboration technologies
are currently under development, including activity awareness interface technologies and interruption assistance
technologies to facilitate the planning and coordination activities of
both individual team members and team supervisors.
Participants: Boeing
Phantom Works, Thales, Air
Force Research Lab, Charles River
Analytics, Office of Naval Research, University of Central
Florida's MIT2 Lab
Complex
task domains such as emergency response and command and control often involve
collaboration between operational personnel in the field and tactical personnel
in a central command centre responsible for coordinating the efforts of those
operational personnel. The asymmetries of their respective work environments,
job responsibilities, available information, and situation constraints produce
distinctly different technological requirements for potential support systems
for these different personnel. A tactical actor in a command centre may exploit
the benefits of a large powerful computing system, but the operational actor
in the field is restricted to using a small handheld device. We use a tabletop
display to support the planning and coordination duties of the tactical actor,
and a handheld device for the simpler map and schedule reading of the operational
actor. The acute difference in display sizes, coupled with the other role
related differences, create a very asymmetric type of remote collaboration.
Our research focuses on the design of interfaces for tabletop and handheld devices for synchronous time-critical collaboration. We are using urban search and rescue as a scenario. Our task analysis has lead us to create interfaces for sharing three classes of information: maps, schedules, and forms. Maps provide spatial information about the incident, schedules present a temporal view of the team's plan, and forms allow essential data to be captured and disseminated. We treat these classes as shared workspaces that ground conversation between team members, and we provide workspace awareness mechanisms to support collaborative gesturing and editing. Continuing advancements in tabletop displays, mobile computing, and wireless networking set the stage for this work. Synchronous collaboration in mobile contexts offers significant benefits over asynchronous message passing, and the asymmetry we are considering raises new issues that are not apparent in conventional groupware.
Sponsored by Thales and European Commission FP6
In current operations in Iraq and elsewhere, the supervisory control of unmanned vehicles (UVs) involves multiple operators controlling single vehicles. As the demand for UVs increases with little or no gain in the number of trained operators available to control them, current research efforts are studying how a single operator can effectively control multiple, heterogeneous unmanned vehicles. However, there is a limit as to the number of UVs a single operator can effectively control and at some point, teams of operators, each of whom control teams of UVs, will eventually need to share a common area of interest and work together to obtain their goals. The question becomes, how will working in teams affect the operator’s performance? The goal of the research is to develop a model of how teams of operators, each of whom are responsible for controlling multiple, heterogeneous unmanned vehicles, coordinate and collaborate, under time-critical, life-critical scenarios. Previous research at MIT has successfully modeled the supervisory control of multiple, heterogeneous unmanned vehicles by a single human operator based on queuing theory and implemented with a discrete event simulation (DES). This research will expand the existing single operator queuing model to describe multiple human operators collaborating and coordinating in a team environment, as they each control multiple, heterogeneous unmanned vehicles. From this model, performance metrics will be generated that will explain how teamwork affects the supervisory control of multiple, heterogeneous unmanned vehicles and recommendations will be made for team allocation and / or interface design.
Related papers:
Nehme, C.E., (2009), Modeling Human Supervisory Control in Heterogeneous Unmanned Vehicle System, Ph. D. Thesis, MIT Dept. of Aeronautics and Astronautics, Cambridge, MA.
Nehme, C. E., Mekdeci, B., Crandall, J. W., Cummings, M. L. The Impact of Heterogeneity on Operator Performance in Future Unmanned Vehicle Systems, The International Command and Control Journal, (2008), Vol. 2(2).
In many important applications, current technologies require multiple human operators to control a single unmanned vehicle (UV). However, in order to (a) reduce costs and (b) extend human capabilities, it is desirable to invert this ratio so that one human operator can control multiple UVs. A human operator involved in one-to-many interactions does not have sufficient cognitive resources to perform low-level tasks on all UVs. Thus, low level tasks must be offloaded to the automation, which allows the human operator to focus on more high level tasks. In order to design systems to promote this futuristic control, we are currently investigating:
Sponsored by Lincoln Labs, AAI Corporation and Charles River Analytics
