8.2.2 Model Accuracy



Given equally useful sets of performance measures, models are often compared on their predictive accuracy. This is fine if not taken to extremes. Model accuracy should be judged in terms of its decision-aiding utility. If a simple M/M/N queueing model will suffice for determining a preferable scheduling of service personnel, an elaborate simulation is not required. [Simple models were dictated in the 911 case study, in which results were needed in one month.] All too often we see a model criticized for not depicting to the finest detail all of the operational idiosyncrasies of a system; this is a key motivation for builders of simulation models to err by inclusion much more often than by exclusion. As model creators, we may tend in our enthusiasm to transfer our priorities from the decision maker to the model itself. A model's comparative advantage in decision aiding is, in our opinion, more relevant than its predictive accuracy. In this light-to overstate the case-a model could be a factor of 2 in error on the primary performance measure, but (as long as the factor remains constant) the model would be fine for rank-ordering alternatives and assessing their relative merits. Of course, we do not advocate factor-of-2 errors. But we believe energies directed at concerns for multidecimal accuracy might be better expended in other parts of the implementation process.