Machinechild: Ground Truth Essay

Introduction

Recent global developments confirm the superior performance of the American way of life. The Gulf war and the collapse of communist Russia, the soaring performance of the US Stock Market, the leading US role as peacemaker in the Middle East and the Balkans are examples, not to mention the recent helping hands of brotherly financial underwriting for devalued Asian currencies. It is no coincidence that economic well-being at home is paralleled by financial outreach to those less fortunate abroad. America is home to the technological and industrial shepherds for the world’s free market flock. The global presence of the internet and satellite communications systems is enabling distributed and remote cultures to join the fold, to subscribe to America’s extended family.

The caretakers of this "good will web" recognize the need to keep it freely accessible to all members, and secure. The American recipe for financial and political security is one of long-range investment. Instruments are being invented and installed today which will return benefits to their makers for years to come. It is only natural for less fortunate cultures to bond with the systems that help them get back on their feet. Today there is an investment parallel between private sector "global shepherd" causes and a bullish stock market on the one hand, and the increased Congressional funding for military information systems on the other. Government agencies that once shied away from defense funding are now embracing the climate change toward defense allocations of taxpayer dollars. The Advanced Research Projects Agency (ARPA), which funded the CAVE™ and other computer interfaces, was recently promoted from ARPA to DARPA with the blessings of Congress — (the "D" is for "Defense"). The National Center for Supercomputing Applications (NCSA), America’s leading-edge site for high performance scientific computing and Internet 2, recently established a contract office for DoD Mod (Department of Defense Modernization) projects. And institutions of Higher Learning including MIT, UC Berkeley, U of Michigan and U of Illinois, have recently formed research consortia with Lockheed-Martin, Rockwell International and other military systems integrators. These consortia are sponsored by the ARL (Army Research Laboratory) Federated Laboratories program [] (FedLab).

Since the Gulf war the Armed Services have been incubating a modern identity to replace the role they aborted in Vietnam. US armed forces are the logical global defenders of our 21st-century technodemocracy. This rebirth is capitalized by a funding strategy whereby public monies are divested from legacy military hardware and software. Instead, tax dollars are invested in "Dual Use" technologies, modular, lightweight, and distributed, with commercial intellectual property rights that are not usurped by military R & D. Dual Use doctrine ensures that military technology also meets the needs of civilian industry. US consumers, under the protective investments of the private sector, will reap the benefits of technologies developed initially for military applications. These technologies will be eventually transformed into home appliances or other attractive commodities. In this way Congress maintains Investment Equilibrium between military and civilian economies. Dual Use is a striking example of the tactical merger of the marketplace and the battlefield.

Consider extending this scenario to the funding of artistic projects. In May 1998 DARPA published a Program Solicitation for proposals to develop a Command Post of the Future (CPOF) []. The DARPA CPOF BAA (Broad Agency Announcement) advances a vision of an extended cyber-commander, VR-empowered, sensing, comprehending and responding to instantaneous conditions on the battlefield. This capability is known as Battlefield Visualization (BV). To achieve a seamless BV will require cognitive engineering of sensors, transmitters and information displays such as computer graphics and computer-generated sound. Cognitive engineering describes the crossover from soft intelligence to wet intelligence (from artificial to biological information processing). If DARPA is reaching its R&D audience, then content providers, even artists, will recognize this opportunity to leave their creative mark in history while serving their country. The CPOF BAA anticipates the realm of artistic possibilities. On page five the BAA suggests "developing general principles of visualization from principles of human perception and experience in graphic art, advertising, news broadcasting, story telling and entertainment." Cinematography, animation, computer games and location-based entertainment may prove worthy of Dual Use. Visionary artists could leverage Dual Use to accommodate the decency-in-funding requirement of the NEA (National Endowment of the Arts). Decency-in-funding was prescribed by Congress and recently upheld by the US Supreme Court. The questionable morality of art funded by taxpayer dollars has proven a sore point for certain American constituencies. The morality of public funding for military projects, however, and the fundamental decency of defense-related research, are beyond question.

An artistic perspective is important for the solution of complex representational problems. Representational skill is necessary to clarify the display of excessive amounts of information. Supported by military funding, artistic techniques could constitute a Representational Strike Force to maintain Information Dominance through Knowledge Management. In time-critical maneuvers and in training, representation tools might include live performance to demonstrate the coordination of an ensemble of assets and devices. A live performance by an instructor demonstrating a hypothetical virtuoso commander in action, might be described as a literal theater of war.

literacy and technology

When we speak of technology we often focus on tools and techniques. They are easier to refer to, due to the conventional ways of making references by turning to the namable objects and well-defined methodologies. However the primary inquiry of technology is in forming questions, generating cases, and creating problems that may not fit in existing paradigms. It is this aspect of technology that makes it an open discourse in a broader context in which scientific and engineering practices can be revisited. Technology includes "illiterate" practitioner's inventions as well as theories. For example the people who invented the compass were illiterate, meaning in the practice domain of compass as a tool, they were not scientists. The word "illiterate" is applied to people here. Let's note the term also applies to the practice for which descriptive language is not yet available. When the practice is indescribable we are illiterate to the need for the solutions to the problems. When we are illiterate to such problems, we call them ill-defined problems. Bringing ill-defined problems to well-defined problems is a beginning of an involvement with technology, necessarily involving our linguistic practices.

The Installation

Ground Truth is a semi-automated installation and a real-time performance in distributed virtual reality. It demonstrates the application of visual and auditory representations in a multi-modal decision-making environment. The environment is a dynamical multi-agent simulation. Multiple workstations provide visual, auditory and tactile interfaces. Movement sensors allow an observer to schedule state changes in system components. Gesture-based interactive queries may be performed. State changes automatically propagate from one region to another. Visualization and sonification of current states of numerical simulations determine the display.

Ground Truth simulates the immutability of economics as combat. In the Ground Truth simulator, Economy is at war with Uncertainty. Information must be reduced to generate profit. Market share must be reduced to refresh the source of raw information. Participants may choose sides, and vote for Prosperity or for Uncertainty. The core simulation is composed of scientific and military software driving an economic simulation, acted upon by observers from a distributed workstation array. Each workstation is a Tactical Operations Center (TOC) where participants experiment with military simulations and numerical models of uncertainty. Battles are fought in economic arenas using military force, financial instruments and media webcasts. TOC’s allow participants to select military courses of action, to make financial investments, to edit and webcast multimedia, and to negotiate uncertainty in a physically-based model of resource delivery.

The multiple models in Ground Truth are arranged in three layers of real estate. Each layer visualizes a system that commanders contend with to orchestrate a combat mission. At bottom is the Battlefield or Marketplace terrain layer. At top are the Cloud Cities and Factories, a resource renewal and repository layer. Between them is the Uncertainty layer, a turbulent flow field that can obscure the terrain and divert resources transmitted between the Cloud Cities and the terrain. The layers interact by exchanging 3D tokens that represent information quanta. The tetrahedron is the information atom. Geometric bonding of quanta forms three classes of information currency: media, military and economic resource tokens. Rigid body and particle dynamics simulate mass, velocity and acceleration of information quanta. Currency tokens are affected by gravity and simulated flow. They collide with other currencies and with bounding surfaces. Avatars provide an immersive view for each TOC workstation, where participants shepherd the information into desired repositories. Tokens are expended in a battle or invested in a Cloud City to fortify the civilian economy. Currencies descend from the Cities to the Terrain in a Resource Rainfall Cycle of taxation-as-precipitation. Battles are executed using FOX, an automated Course-of-Action planner that applies genetic algorithms to calculate optimal troop maneuvers. The Media TOC transmits resources up to the cities in the form of publicity, determined by the newsworthiness of the conflicts on the battlefield.

Information and Uncertainty

In military doctrine, "ground truth" is the technical term for an elusive state of complete information. It is the absolute knowledge of the momentary state of a battle. Ground truth is unattainable, but it determines which party best meets their objectives in armed conflict. In war, military forces periodically contend with one another, whereas they immediately and continuously contend with uncertainty. At Bell Labs in 1949, Claude Shannon demonstrated that information is the statistical measure of uncertainty in a communications channel []. Every communication involves a ratio of signal to noise. Meaning must be encoded to resist decay under this ratio. The decay of a relevant message into noise is inherent to the entropy of transmission. There are many forms of noise, some recognizable, others masquerading as content. Uncertainty arises when meaning cannot be distinguished from noise in a communication. To the significant degree that communications are required to conduct a military operation, uncertainty plays a role in the outcome. Predictions have been made that our children will wage war in cyberspace [], but in terms of information, noise, and entropy, war is conducted in cyberspace now, today. Whether the combatants are soldiers, or stock brokers, or news networks and politicians, global turf is staked, defended and conquered in a binary-coded alphanumeric medium. The digital medium is regarded as an almost-physical substrate for decision-making. In the ebb and flow of the binary stream, the forces that master uncertainty carry the day.

To bring uncertainty within grasp, military scientists and engineers are designing systems to approach ground truth on an infinite arc. The mastery of uncertainty follows a path that can be likened to a curve approaching but never intersecting its asymptote. As working technology advances along this curve, a transformation of the battlefield is taking place. The machines that are devised to sense, evaluate, and report ground truth, are becoming a substitute battlefield. The machines are becoming a replacement for the conditions they are designed to measure. The physical state of war is evolving into a staging ground for informational battle. Compression algorithms and search engines reduce geography, forces and events to a pure form of asynchronous message consisting of pointers to previous messages. The new battlefield relies upon temporal mechanics for articulation in a chain of reference without closure, the event horizon for a recursive theater of war.

medium and content

For resisting the description of an artist as "content provider":

when we wish certain content to be present

we must prepare the condition

from which the content could be accounted for.

The term "medium" has specific implications associated with the technology such as mass media, where the term presupposes a quality of being transparent and a premise to simply mediate with no interference. Through the observations of mass media practices we have learned the implication of such premises is no longer feasible for supporting any medium as an objective representation tool for the world. Similarly, we wish to revisit whether any medium can be an objective tool for artistic expressions. Can a compositional idea be independent from the medium it predicts? One might say it doesn't matter what an artist uses to create a work of art as long as he or she gets the result. This premise may be compatible with the description of an artist as a content provider often for an industrial demand. The role of an artist is reduced to the role of demonstrator for proving the utility of certain tools and software for commercial promotions. The conceptual ground for such premises, however, is irrelevant for practitioners and artists in a computation environment, particularly when one faces the machines that take nothing for granted beyond the specifications of their internal states, inputs, and product set. The specifications of the machines are unique to the compositional problems. With the prevalence of multimedia there is a similar concern in presenting a work of art using certain tools as objective tools for achieving certain effects. We wonder how much tools shape the final results, when we note how certain packaged products keep our students of art busy collecting data and sampling. Not to eliminate the possibility of many tools, what has to be examined and included in our discourse is the composability of, and with, the tools and conditions that surround us.

Tools leave a trace in works of art and this is not necessarily undesirable. It simply means the composability of the tools has to be taken into compositional criteria, and this is desirable. Thus we attribute the distinctions between "a composition for a medium" and "the composition of a medium". For the former the composition assumes a well-defined medium for its presentation, for the latter the composition explicitly addresses the construction of its medium of presentation. The word "medium" is applicable in a wide range, from the substances appealing to our basic senses, to instruments, genre, and tonality. I do not intend to categorize different uses of the term. The discussion starts from the following two statements:

1. A medium requires an observer’s action for its presence to be witnessed.

2. Let the action be a performance.

A composition will involve the construction of a medium as a sphere circumscribing subsystems and structures of action, by which the systems will behave. With this task definition of composition, what do we mean by content? When we input content such as soundfiles or images and retrieve them it will be nothing more than utilizing technology of data storage and sampling. When we speak of dynamical models as subsystems and complex interaction we do not think of content in terms of what to be input or provided. Content is not what we put into the composition, rather we prepare the system and structure in ways the content will be brought up by ways of representing systems behaviors. The systems behavior again is not the content-oriented goal itself. It is the product of complex interactions involving internal structures of the complex system in which the interaction takes place.

Time-critical Performance

Physical relativity assures us there must be a temporal gap between "now" in one location and its report elsewhere. Transmissions of signals cannot overcome this interval. A military operation occurs across a sequence if such intervals. The commander’s intuition masters time by anticipating future events, as if bringing about the expected in advance of its occurrence. In the case of physical transmissions, information overload results from the latency between ground truth and information display. This is due to the accountability of the variable n. If a commander could be n places at once, where n > 1, then interpretation and decision would receive timely execution. However, if n = 1, the commander acts in real-time. In military doctrine a commander may substitute n-time execution with recursive time, in the form of a Prioritized Information Request (PIR). A PIR is a recursive query. It asks a big question, the big question in turn asks smaller questions, and so on down the chain of command. Eventually the smallest answers report to bigger answers, and so on. When enough answers return, a decision is made at the top. But there is rarely enough time during battle to complete the recursion. Decisions are made with less than complete knowledge. In place of knowledge, the commanding officer relies upon information display. Information display may be thought of as a representation of time as knowledge. But this knowledge is implicit, as it is a representation of uncertainty. The difference between available information and knowledge required for a decision, is the unknown quantity in the display.

In the Ground Truth installation, uncertainty is represented. This uncertainty is the asymptote of the research curve. The military term for the uncertainty curve is Situational Awareness.

Political Ideals and their Application

There is a history of artists who step forward to support the cause of battle, such as those who shouldered arms against Franco in the Spanish Civil war. There is also a history of artist-activists opposed to the formerly-named "military-industrial complex". The climatization of artistic to military practices reminds me of an anecdote told by a professor in a documentary film class. The class had just finished screening The Battle of Anzio by Frank Capra (It’s a Wonderful Life and other films). Anzio was part of the "Why We Fight" cinema newsreel series, produced during World War II by the Department of Defense. Various Hollywood directors produced installments in this series as part of the war effort. Narration and raw footage was provided by the DoD. According to the story, officials were unhappy about an apparent time offset in Capra’s work, between the narration and the silent newsreel footage. The timing of the narration, they protested, unduly emphasized the soldiers’ difficulties. The results were less than inspiring to prospective soldiers. Capra is said to have replied, "You didn’t expect me to make a movie promoting war, did you?"

This research is supported by the Army Research Laboratory and the Army Intelligence and Security Command via the Advanced and Interactive Displays Consortium. Consortium sponsored by the U.S. Army Research Laboratory under the Federated Laboratory Program, Cooperative Agreement DAAL01-96-2-0003. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied of the Army Research Laboratory or the U.S. Government.

survivability

We live in a society with the generally accepted definition of adaptation (Lorenz, 1965)

As computer technology is advanced enough to update various simulations based upon complex phenomena, the logical processes based on biological explanations or "real-world" understandings come to the front line of discussions. Subsequently the metaphors such as "environmental fitness", "mutations", "memes", "adaptation", and "survival" cross over many disciplines. Let’s not forget metaphors are often used for explanatory designations. The uses of metaphors that are heavily dependent on explanatory powers tend to be often mistaken as prescriptions for the subsequent thoughts. The production of art differs from the production of explanations and I can not elaborate this here. However, as a composer I stand closely by the following reference. Not necessarily contradicting the preceding reference, as an alternative, in the early 1960’s Ross Ashby stated

How absurd it appears to be, at least at first sight, this statement seems to contradict our common acceptance of nature as unpredictable in detail and uncontrollable by its nature. Note that the assertion adaptability is "good" often shadows the examination of the condition under which certain adaptability is "good". Further the value association obscures the fact that a generative process starts from the stipulation of constraints. The stipulation of constraints gains freedom and generative power. Performance may generate metaphor but is not constructed on explanation-based metaphor. In composing a medium, generating performance cases is preferred to explanations for examining the explainer’s conditions under which the performance is generated. The criterion for composing a medium is to create the conditions such that, certain human capacities are facilitated otherwise the particular capabilities to observe might not be attainable. In creating this condition, performability is more of our concern than survivability.

One might have to refuse to survive under certain

condition in order to construct an alternative.

"Survive!," advised he, who has survived. He could be.

the dreamer’s nightmare