A Skeptic's Guide to Computer Models

John D. Sterman
Sloan School of Management, MIT

Abstract

Computer modeling of social and economic systems is only about three decades old. Yet in that time, computer models have been used to analyze everything from inventory management in corporations to the performance of national economies, from the optimal distribution of fire stations in New York City to the interplay of global population, resources, food, and pollution. Certain computer models, such as The Limits to Growth (Meadows et al. 1972), have been front page news. In the US, some have been the subject of numerous congressional hearings and have influenced the fate of legislation. Computer modeling has become an important industry, generating hundreds of millions of dollars of revenues annually. As computers have become faster, cheaper, and more widely available, computer models have become commonplace in forecasting and public policy analysis, especially in economics, energy and resources, demographics, and other crucial areas. As computers continue to proliferate, more and more policy debates—both in government and the private sector—will involve the results of models. Though not all of us are going to be model builders, we all are becoming model consumers, regardless of whether we know it (or like it). The ability to understand and evaluate computer models is fast becoming a prerequisite for the policymaker, legislator, lobbyist, and citizen alike. During our lives, each of us will be faced with the result of models and will have to make judgments about their relevance and validity. Most people, unfortunately, cannot make these decisions in an intelligent and informed manner, since for them computer models are black boxes: devices that operate in completely mysterious ways. Because computer models are so poorly understood by most people, it is easy for them to be misused, accidentally or intentionally. Thus there have been many cases in which computer models have been used to justify decisions already made and actions already taken, to provide a scapegoat when a forecast turned out wrong, or to lend specious authority to an argument. If these misuses are to stop and if modeling is to become a rational tool of the general public, rather than remaining the special magic of a technical priesthood, a basic understanding of models must become more widespread. This paper takes a step toward this goal by offering model consumers a peek inside the black boxes. The computer models it describes are the kinds used in foresight and policy analysis (rather than physical system models such as NASA uses to test the space shuttle). The characteristics and capabilities of the models, their advantages and disadvantages, uses and misuses are all addressed. The fundamental assumptions of the major modeling techniques are discussed, as is the appropriateness of these techniques for foresight and policy analysis. Consideration is also given to the crucial questions a model user should ask when evaluating the appropriateness and validity of a model.


You can download the full document in PDF format by clicking the link below.
  PDF full version

In order to view the PDF file you must first have a copy of Acrobat Reader. If you do not have one installed on your system, please follow this link to receive your free copy of the LATEST version of Acrobat Reader. If you have problems opening the file, it may be because you have an older version of Acrobat Reader. Please follow the link below to update your Reader.

   Free Adobe Acrobat Reader

Back to SDG index page
Back to John Sterman's publications and research page

A printed version of this paper is available by snail-mail from John Sterman at the MIT Sloan School of Management, 50 Memorial Drive, E53-351, Cambridge MA 02142, or jsterman@mit.edu.

  • System Dynamics '96 Virtual Proceedings.