Dr. Thomas E. von Wiegand & Dr. Jonathan D. Pfautz Massachusetts Institute of Technology, Research Laboratory of Electronics Fifty Vassar St. 36-755 Cambridge, MA tew@mit.edu jpfautz@mit.edu

The "Hand of Death," a distributed haptic interface for signalling exposure to virtual enemy fire. Abstract In the following paper we describe the evolution of a bodyworn tactile interface through three iterations. The first iteration illustrates a simple "single pod" system in which a tactile cue is presented to one location on the body based on the position of the pod in space. The hardware and software required to implement this prototype are the seed for subsequent multi-pod systems. Keywords Body-worn haptics, distributed haptics, tactile array, tactile interface. 1 Introduction Distributed bodily contact with objects in the environment is a common occurrence in the real world. From such contact arises our sense of corporeal solidity, as well as our impression of the solidity of the objects that we encounter. In addition, such contact often serves as a warning, and the pattern of contact can reveal information about the source even before the attention of other sensory resources is focused on the impinging stimulus. Unfortunately, it is technologically impossible to construct a suitable interface to accommodate and match the convoluted, expansive, and multi-sensory qualities of the skin. When imagining such a hypothetical interface for use in a VE system, the difficulties are compounded because the VE model ideally would need to know the position in space of each point on the skin surface, leading to a significant tracking problem as well as the stimulus generation problem. By way of contrast, audio/hearing is one of the most straightforward interfaces because to a first approximation the stimulus is a one-dimensional pressure-vs-time function, applied to a localized orifice (the ear canal), where meaningful information can be transmitted even if the position of the orifice in space is ignored. Despite the foregoing caveat, there are many examples of more modest interfaces (limited to a small area and specific modality, and generally lacking a position tracking function), used for the purposes of prosthetic sensory substitution (White 1970), extension of proprioception to enhance the piloting of aircraft (Rupert 2000) or other vehicles, research on the skin senses (Tan 2000) for maximizing information transfer, etc. Much less work using these simplified interfaces in VE's has been reported, though there are many useful applications in that context. The work described here arose out of an attempt not only to signal collisions with virtual objects, but also to signify more abstract conditions or events directly related to the goals of infantry training in a VE. The ultimate purpose of the work was to develop a relatively un-encumbering, non- mechanically grounded, wearable, tactile stimulator system that can be used to signify situations that would arise in close-quarters-battle, such as bumping into virtual objects, exposure to enemy fire, or being hit by a bullet [don't worry, a very soft bullet]. 2 Uni Pod We first developed a simple system utilizing a Polhemus tracker, a pager motor vibrator, and a BASIC Stamp microcontroller (to help delineate a standardized, extensible interface to the VE) (see figure 1). A simple VE was then constructed (see figure 2) in which a virtual sniper is lurking behind a doorway, and as the pod (worn like a wristwatch) comes into the line-of-sight of the sniper, the wearer is alerted with a shake: the grim reaper's warning of a brush with virtual death. Figure 1. Photograph of the Uni Pod. The Polhemus receiver and pager motor vibrator are affixed to the wristband. The wires from the wrist lead to the Polhemus control box and the BASIC Stamp respectively. The driver circuit for the motor uses a single transistor and back-EMF protection diode as shown in Figure 4, the software running on the Stamp is listed in the appendix. Figure 2. Photograph of the Uni Pod in use. The display screen shows a "first person" view of an environment in which a sniper (note the blue sphere representing the sniper's head) is located in a room ahead, the yellow hand symbol represents the position of the Uni Pod in the VE. The red plane (not normally visible in the first person view) represents the plane of vulnerability. As the hand moves into the exposed zone (or alternatively, as the sniper moves his line-of-sight to include the hand) the Uni Pod will vibrate. 2.1 Hardware Notes The hardware comprising the Uni Pod is straightforward. Tracking is accomplished by a Polhemus Fastrak 6DOF tracker, which communicates with the host computer via a serial interface. The receiver unit is attached to the wristband to which the vibrator is also mounted. The proximity of the Polhemus receiver to the motorized vibrator presents a potential problem due to the Polhemus' sensitivity to stray magnetic fields and metals which can distort the nutating magnetic field produced by the Polhemus transmitter. However, the main problem that we encountered is that the position readings became somewhat more erratic during periods in which the vibrator was activated. For purposes of our initial testing we were able to ignore this "jiggling," but if this approach was to be used on a more permanent basis some filtering or gating would be appropriate. The vibrator used in the system is based on a miniature motor with an eccentric weight attached to its shaft. Motors (like all inductive actuators) can be difficult loads to drive from solid-state circuitry. The major issues are that the startup and stall currents are often an order of magnitude higher than the running current, and upon turn-off, the collapsing magnetic field in the inductor creates a high-voltage reverse spike that, without the proper precautions, destroys (immediately or eventually) the controlling transistor. The schematic of Figure 4 shows how these issues can be handled in a simple and straightforward manner. The driver circuit includes a ZVNL110A MOSFET transistor that features a logic-level sensitive gate. This transistor is able to fully switch on and off when driven by the output bit of a microcontroller such Pin0 of the BASIC Stamp. The transistor is wired such that it provides a current path to common ground when it is activated. The vibrator motors generally run at a lower voltage than the rest of the logic circuitry. The schematic shows how a zener diode is used in conjunction with some local filter capacitance and a reverse polarity protection diode to provide a nominal 3 volt power supply that is sufficiently decoupled from the rest of the power system that the startup transients do not affect the operation of the of the other powered circuitry. The motor is connected between this +3v power supply and the drain of the transistor. An all-important "back EMF protection" diode is connected across the motor terminals to shunt the turn-off spike safely away from the switching transistor. There are somewhat more sophisticated "snubber" circuits that can be constructed to address the back EMF phenomenon, however for the switching speeds and power levels involved in this circuit, a simple "backwards diode" is sufficient. The serial interface is provided via a Parallax Inc. BASIC Stamp microcontroller. Given the forgivingly wide range of voltages constituting "mark" and "space" in an RS232 serial connection, it is possible to make a simplified connection to the host computer by inverting the output and utilizing some current limiting resistors (this approach is described in the documentation that comes with the BASIC Stamp. An alternative that is almost as simple and conforms to the standard is to use a "no-external-parts" buffer circuit (such as the Maxim MAX233) to translate the logic levels to standard RS232 levels. The source code to be run on the BASIC Stamp is exceedingly simple (a version of this source code is listed in the appendix). The main program loop consists of waiting for an input character, performing a test, then turning the appropriate control bit on or off. For the UniPod, the protocol is limited to sending an ASCII character "0" (zero, 48) to set the output pin controlling the vibrator motor low, and "1" (one, 49) to turn the motor on by setting the pin high. 2.2 Software Notes The commented listing for the program that we used as a sample "sniper exposure" VE is contained in the appendix. The main thing worth noting is that the exposure computation was highly simplified. To accomplish this, we picked values for the layout of the world and the location of the sniper that would make the computation trivial. A more general way to compute exposure would be to exploit selection or "picking" techniques in OpenGL. Selection in OpenGL re-renders the scene from the current viewpoint and lists objects that fall within a unit cube (say 2 pixels by 2 pixels). The location of the viewpoint when the scene is re-rendered could be chosen to be that of the enemy or enemies and the list could be searched for any polygons that belong to the representation of the user. Alternatively, the "unique color selection method" could be used. Either of these methods would provide a fairly efficient and general mechanism for calculating exposure. Our prototype work was done under Linux, on a newer USB-equipped PC. One of our goals was to use serial-to-USB adapters (such as the Keyspan USA-19) and multiport hubs in order to have dedicated "comm ports" for each item of hardware in the VE suite. In this simple case the suite consisted only of the Polhemus tracker and the Uni Pod controller, but we also foresee using the system with many other devices. One of the concerns that we had was that under Linux there would be some glitch in opening serial ports through USB adaptors. Fortunately, USB-to-serial adaptors under Linux are supported in most newer kernels (see http://www.linux-usb.org for more details on supported devices). Linux treats the adaptor as a standard tty serial port off the USB bus. Using the device is then identical to using a serial port and no unexpected difficulties arose. 3 Quadra Pod Elimination of the connecting wires and increasing the number of pods were the next goals that we explored in developing the Quadra Pod. The first step in eliminating the wires to the individual pods was to change our tracking technology to use an IR-triggered ultrasonic tracker system, the Lipman VScope. This tracking system unfortunately is limited in the number of transponders it addresses (ours tracks four maximum) but in theory it can be extended to a large number of units (as discussed further in section 4). To unfetter the individual vibrators, we utilized a set of RF transmitters and receivers to control each of the four pods. The circuitry within the pod consists of a Laipac receiver, tuned to a unique frequency, matched to a corresponding transmitter driven by an output pin of a BSAIC Stamp. The output of the receiver controls the vibrator through the driver/buffer circuit previously described. The BASIC Stamp output bits control the "CW" transmission on four frequencies. The only complication is that due to the design of the detector and AGC circuitry in the receiver, periodic polling of each channel is required to keep the automatic level control set to reject random noise. In practice, this periodic momentary toggling of the state of the vibrator ("blip") can be done such that it isn't noticeable to the wearer. Essentially, there is enough inertia in the vibrators that if the duty cycle of the "blip" is short enough it doesn't move the vibrator enough to be felt. A final hardware consideration for the Quadra Pod configuration is that of the antenna requirements. Initially we attempted to follow guidelines published in the literature and on the web (Smith 2000), however, it turned out that due to the extremely close range of our application, an un-critical length of wire was sufficient to enable the system to work reliably. Figure 3. Photograph of the Quadra Pods in use. Each enclosure contains an RF receiver tuned to a separate frequency. The output from the receiver controls the current to the motor through the single-transistor driver circuit shown in Figure 4. A BASIC Stamp directly triggers each of the four corresponding transmitters using the Stamp software listed in the appendix. The tracker transponders must be attached to the enclosure in such a way as to enable direct view to the beacons. 4 Centi Pod Ultimate goal is to have many tens of pods, perhaps up to 100, distributed over the body. Achieving this requires that information be coded on the RF carrier and subsequently decoded using a local microcontroller in each pod. There are many issues that arise in establishing a reliable radio link to transmit this data, it is certainly more involved than the CW link of the Quadra Pod. Some of these issues that we have begun to explore include balanced codes (so that the average appearance of "mark" and "space" events appear at about 50% probability in order to maintain optimum detectability by the ALC circuit), error detecting code (to ensure that noise bursts do not result in erratic triggering of the vibrators), and definition of locally generated temporal patterns of vibration to signify particular events or to generate novel cutaneous sensations. Some additional research has been started by our lab to include other modalities, especially electrocutaneous stimulation, to signify both mild (exposure/warning) as well as very intense conditions such as being struck or shot. Also, as the schematic of figure 4 shows, multiple vibrators per pod are an obvious extension of the system. In the Centi Pod prototype we utilized two types of vibrators mounted so as to create vibration in two orthogonal planes for normal and tangential stimulation. A third output was included for future expansion. The limitations inherent in the un-modified Lipman VScope tracker are essentially due to their method of addressing each transponder using a 1-of-8 bit position scheme. This approach enabled the original design to economize on the hardware contained in each transponder, however, with the advent of extremely cheap microcontrollers it is possible to fully utilize the 8-bit code to address 256 transponders. We have not yet attempted this redesign, but it is clearly a viable approach to providing tracking for each pod of a Centi Pod system. Figure 4. Schematic of Centi Pod. This unit includes three vibrator motors, diagnostic LED displays, user-selectable address, and auto-shutdown feature. A Ubicom SX28 microcontroller is used (instead of the BASIC Stamp of the Uni Pod and Quadra Pod) for improved performance (click to enlarge). Figure 5. Photo of Top of Centi Pod, showing diagnostic LEDs, rotary address selector (right), receiver module (front), header pin connectors (for programming and connecting additional vibrator). Lipman tracker transponder is held at left. Figure 6. Photo of rear of Centi Pod, showing two types of vibrator (lower right corner) and wraparound wire antenna. An important consideration is that as the number of tracked locations (and Pods) increases, the representation of the user in the VE must increase in complexity, in turn increasing the complexity (and required computation) for calculating exposure. Some account must be taken of the user's real body; for example, a point may be obscured by the user's body, meaning that it is not exposed. To ensure that exposure is properly computed, the user's body must be modeled in the VE. The complexity of this model will be a function of the computational power available and the desired update rate. An additional advantage of modeling the user in the VE is that fewer tracked pods could be used and the body's position could be interpolated so that other, non-tracked pods could be correctly activated. This alternative to the "brute force" approach of exhaustively tracking each individual pod would be more practical, especially in a commercial system where cost is an issue. These improvements and alternate realizations of the Centi Pod system are areas of future planned research in our Hand-of-Death program. Acknowledgements This work was performed with support from ONR grant N00014-01-1-0197. References Rupert, A. H. (2000). "Tactile Situation Awareness System: Proprioceptive Prostheses for Sensory Deficiencies." Aviation, Space, and Environmental Medicine, Vol. 71, No. 9, Section II, September. Smith, K. (2000). Antennas for Low Power Applications. Technical Report: RF Monolithics Inc. http://www.rfm.com/corp/appdata/antenna.pdf. Tan, H., Lim, A., & Traylor, R. (2000). "A Psychophysical Study of Sensory Saltation with an Open Response Paradigm." DSC-Vol. 69-2, Proceedings of the ASME Dynamic Systems and Control Division - Volume 2 ASME 2000. White, B. W., Saunders, F. A., Scadden, L., Bach-Y-Rita, P., Collins, C. C. (1970). "Seeing with the skin." Perception & Psychophysics Volume 7 (1), pp 23-27. Appendix Code listing for BASIC Stamp 1 interpreter (click here). Code listing for proto1 demo (click here).