Since joining the MIT Robotic Mobility Group in 2007, Sterling has been developing a new framework for autonomous and semi-autonomous control of vehicles. This framework is explicitly designed to identify, plan, evaluate, and selectively enforce constraints, rather than the reference trajectories common to traditional planning and control frameworks. This basis in constraints provides a rich new context for shared, human-machine control; one in which the human is no longer restricted to a specific, automation-determined path (which is often arbitrary, non-intuitive, or over-restrictive to the human operator), but is free to maintain significant control of the machine when, and for as long as his/her control inputs are safe.
Instead of forcing a vehicle to track a specific line on the roadway or a particular path through an open field, this system strategically limits the range of available inputs to ensure that the operator retains as much control freedom as possible without risking collision or dynamic instability. As driver performance degrades or the threat of collision or instability increases, the controller gradually takes over. In the limiting case, when nothing short of an optimal control sequence is capable of keeping the vehicle safe(ly within its stability envelope), the controller retains full (autonomous) control and navigates the vehicle to safety before returning control to the human operator.
The videos that follow demonstrate a subset of this control framework's capabilities. For a more accessible description of this technology and its potential impact on both the safety and energy needs of passenger vehicles, please see this four-part blog series. For more detailed information, please see related publications. Sterling developed the system shown on this page during his graduate work in collaboration with Steven Peters and Karl Iagnemma and with the sponsorship of Ford Motor Company. It has been demonstrated experimentally in over 800 trials on a Jaguar S-Type and has been the subject of a journal paper, several conference publications, a master's thesis, four patents (pending), and various press articles. In addition to the manned Jaguar system, Sterling has more recently developed an unmanned/teleoperated variant in collaboration with Quantum Signal, LLC. This system has been tested in over 1,200 trials with over 20 different drivers, and has also been discussed in publications and press pieces, but is not explicitly described on this page.
The video below overlays the results of two simulations (shown in lieu of their experimental analogues for IP considerations). Both vehicles (blue and gray) are being operated by the same driver model traveling at 20 m/s (~44 mph). The driver of the gray vehicle operates alone — without the assistance of our semi-autonomous framework. The blue vehicle is equipped with the semi-autonomous system. The bar at the right edge of the video shows the percentage of control authority the computer is allocating to the controller to ensure stability through the maneuver. Notice that without ever taking more than 50% of the total available control authority, the system not only keeps the vehicle stable, but it also moderates the driver's inputs in the process. Whereas the unassisted driver oversteers and loses control of the vehicle, the assisted driver notices that the vehicle is responding as desired and is thus more measured in his steer commands. Moreover, allocating less than 50% of the available control authority to the controller (see green bar on the right) is sufficient to keep the vehicle on the navigable roadway and within 0.4 meters of the line the driver is trying to track. The combined effect of both inputs (driver and controller) is a vehicle trajectory that more closely tracks the path the driver is trying to follow than the driver could accomplish on his own.
In this next test, the driver of fails to steer around an impending threat (as if drowsy or impaired). Notice that in this scenario as well, the controller intervenes only enough keep the vehicle safely on the navigable road surface. Once the threat has been reduced, it returns control to the driver.
The videos below demonstrate the semi-autonomous controller's ability to avoid moving hazards. This avoidance is accomplished by predicting the motion trajectory of perceived obstacles and constraining the motion of the host vehicle to avoid projected collision states. In both videos shown below, the driver input is identical: he maintains his steering angle at 0 degrees as though he does not see the impending hazards. Notice that the host vehicle (blue with assistance and gray without) maintains a constant velocity (~44 mph) while other vehicles (red and yellow) accelerate and decelerate.
Finally, the simulation below demonstrates how, in the absence of control inputs from the human driver, the semi-autonomous controller effectively behaves much like an alert driver would - seeking first to pass, then pulling back in behind the other vehicle as it accelerates.