MIT Humans and Automation Lab
Replan Understanding for Heterogeneous Unmanned Vehicle Teams
In order for a single operator to supervise multiple unmanned vehicles remotely,
the operator must interact with the vehicles via a mission manager, with lower
level cognitive tasks like actually flying and navigating the vehicles relegated
to the automation. Automation is required because of the large amount of information
that needs to be processed under time pressure, but human judgment and experience
is needed because of the significant uncertainty inherent in the system. In
order to allow a single operastor the ability to manage multiple vehicles,
we have developed a display design that processes the plethora of sensor data
coming from the multiple vehicles, and presents it in an easy-to-understand
format, which engages the user in high-level tasking decisions as well as contingency
planning.
This
project is a continuation of research for the Combat System of the Future,
a Naval Submarine that will be operational 20-25 years down the road. The
main focus of current research is the design of a Mobile Situational Awareness
Tool (MSAT) to aid the commander in collision avoidance for surface operations.
A Cognitive Task Analysis has been completed, and the informational requirements
were used in the design of the MSAT. In order to determine the usefulness
of this tool, testing is currently underway to determine the added benefit
of this tool when compared with current navigation methods, including the
usefulness of an automatic path planner for path planning/re-planning.
» see video
Related paper:
Carrigan, G. P., (2009), The Design of an Intelligent Decision Support Tool for Submarine Commander, S. M. Thesis, MIT Engineering Systems Divisvion, Human Systems Engineering Track, Cambridge, MA.
Sponsored by Rite Solutions
The Cooper-Harper Scale is a 10 point quasi-subjective scale developed in
1957. The scale was developed to get standardized reporting from test pilots
on the controllability of the aircraft being evaluated. The focus of MCH-UVD
project is to develop a similar scale for evaluating unmanned vehicle displays.
The objective is to have a standard tool for troops in the field to identify
unmanned vehicle display deficiencies. Modified Cooper Harper Scale is intended
to serve as an instrument for the DoD to compare displays when acquiring
new software and to provide designers with further guidelines on what the
troops in the field need. An experiment is underway to validate this scale
using unmanned ground vehicle (UGV) and unmanned aerial vehicle (UAV) simulators,
in collaboration with the Robotics Institute at Carnegie Mellon University
and Aurora Flight Sciences Corporation.
Sponsored by U.S. Army Aberdeen Test Center
Multimodal Interface Toolkit for UAV Systems (MITUS)
The visual channel is the primary modality for displaying information to
unmanned aerial vehicle (UAV) operators. The focus of the MITUS research
has been to explore alternative modalities and combinations of modalities
for displaying information to UAV operators. Are there certain pieces of
information that are better portrayed over the audio or haptic channel? What
are the effects on performance and workload when information is parsed out
or repeated over various modalities? This research project has evaluated
the use of continuous audio (sonifications), a tactor wrist vibrator, and
a waist pressure band. All testing has been completed on HAL’s Multiple Aerial
Unmanned Vehicle Experiment (MAUVE) simulator. The results of an experiment
with 44 military personnel have shown that using continuous audio in conjunction
with visual displays does enhance operator performance. Further, preliminary
studies have shown that continuous haptic feedback can also enhance performance,
in particular for monitoring of events that are continuous in nature (e.g.,
UAV course conformance).
Computers are a critical element of the command and control information synthesis and decision making process, but the human element is equally important. Both humans and computers bring different strengths and limitation to problems solving in large problems spaces with many variables, some changing dynamically, like what is experienced in command and control resource allocation problems. The goal of this research is to determine how humans and computer optimization algorithms can complement each other to provide viable solutions in time critical command and control resource allocation scenarios. Developing a collaborative model of human-computer decision making for resource allocation is critical for a futuristic complex command and control systems that involve humans who must integrate temporal and spatial elements, as well as solve problems, manage assets, and perform contingency planning in a high workload environment.
This research is sponsored by the Office of Naval Research, NASA and Perceptronics
HACT is an innovative taxonomy aimed at providing a new information-processing model for collaborative human-computer decision-making, defining specific collaboration roles (moderator, generator, and decider) and their characteristics, and representing collaboration in a direct-perception visualization. HACT can be both used to describe and compare command and control system(s) from a collaboration standpoint. Future research based on HACT will incorporate design trade-off characterizations in order to provide system designer with a cost-benefit analysis tool.
S. Bruni, J.J. Marquez, A. Brzezinski, C. Nehme and Y. Boussemart (2007). Introducing a Human-Automation Collaboration Taxonomy (HACT) in Command and Control Decision-Support Systems, Accepted, 12th International Command and Control Research and Technology Symposium, Newport, RI, June, 2007.
The "Tracking Resource Allocation Cognitive Strategies" tool
(TRACS) allows for post-hoc visualization of the cognitive steps exhibited
by a human operator while interacting with a multivariate resource allocation
decision-support interface. This tool was applied to both mission planning
for multi-criteria resource allocation for military strikes, and also multi-variable
geospatial path planning problems for astronaut moon traversals. Both domains
involve a human operator interacting with an automated decision-support system
in order to find a solution to a complex planning problem involving multivariate
and constrained optimization for a cost function. With the help of TRACS, clear
patterns of behavior were identified that could be correlated to performance
in both applications.
Related papers:
S. Bruni, Y. Boussemart, M.L. Cummings, and S. Haro . Visualizing Cognitive Strategies in Time-Critical Mission Replanning. In Proceedings of HSIS 2007: ASNE Human Systems Integration Symposium, March 19-21, 2007, Annapolis, MD, USA, 2007.
Bruni, S., Marquez, J., Brzezinski, A., & Cummings, M.L., Visualizing Operators’ Cognitive Strategies In Multivariate Optimization, Proceedings of HFES 2006: 50th Annual Meeting of the Human Factors and Ergonomic Society, San Francisco, CA, USA, October 16-20, 2006.
Bruni, S., & Cummings, M.L., Tracking Resource Allocation Cognitive Strategies for Strike Planning, COGIS 2006 - Cognitive Systems with Interactive Sensors, Paris, France, 2006.
To complete a high level system acquisition decision, decision makers must process
a large amount of data to determine which system best fits a given mission
or purpose. This project investigates what type of decision support tool could
provide decision makers the greatest understanding of the system acquisition
trade space, thus allowing them to make more informed decisions. As part of
this project, a new configural display named Fan Visualization (FanVis) has
been conceived, designed and developed. FanVis takes the system acquisition
trade space data, and by using emergent features, naturally maps the data for
the decision maker. An experiment has shown improvement in performance using
FanVis over a set of Excel bar and line charts, which represent the tools
used currently in system acquisition decisions.
Related paper:
Massie, A. E., (2009), Designing a Graphical Decision Support Tool to Improve System Acquisition Decision, S. M. Thesis, MIT Aeronautics and Astronautics, Cambridge, MA.
Air Force Office of Scientific Research
StarVis: A Configural Decision Support Tool for Schedule Management of Multiple Unmanned Aerial Vehicles
As unmanned aerial vehicles (UAVs) become increasingly autonomous, current
single-UAV operations involving multiple personnel could transition to a single
operator simultaneously supervising multiple UAVs in high-level control tasks.
These time-critical, single-operator systems will require advance prediction
and mitigation of schedule problems to ensure mission success. However, actions
taken to address current schedule problems may create more severe future problems.
Decision support could help multi-UAV operators evaluate different schedule
management options in real-time and understand the consequences of their decisions.
This thesis describes two schedule management decision support tools (DSTs)
for single-operator supervisory control of four UAVs performing a time-critical
targeting mission. A configural display common to both DSTs, called StarVis,
graphically highlights schedule problems during the mission, and provides projections
of potential new problems based upon different mission management actions.
This configural display was implemented into a multi-UAV mission simulation
as two different StarVis DST designs, Local and Q-Global. In making schedule
management decisions, Local StarVis displayed the consequences of potential
options for a single decision, while the Q-Global design showed the combined
effects of multiple decisions. An experiment tested the two StarVis DSTs against
a no DST control in a multi-UAV mission supervision task. Subjects using the
Local StarVis performed better with higher situation awareness and no significant
increase in workload over the other two DST conditions. The disparity in performance
between the two StarVis designs is likely explained by the Q-Global StarVis
projective “what if” mode overloading its subjects with information.
This research highlights how decision support designs applied at different
abstraction levels can produce different performance results.
