Metrics
affect research decisions, research efforts, and the researchers
themselves. From a review of the literature, interviews at ten
research-intensive organizations, and formal mathematical analyses,
the authors conclude that the best metrics depend upon the goals
of the R, D&E activity as they vary from applied projects to
competency-building programs to basic research explorations.
For applied projects, market outcome metrics (sales, customer
satisfaction, margins, profit) are relevant if they are adjusted
via corporate subsidies to account for short-termism, risk aversion,
scope, and options thinking. The magnitude of the subsidy should
vary by project according to a well-defined formula.
For R, D&E
programs that match or create core technological competence,
outcome metrics must be moderated with "effort" metrics. Too
large a weight on market outcomes leads to false rejection of
promising programs. The large weight encourages the selection
of lesser-value programs that provide short-term, certain results
concentrated in a few business units. This, in turn, leads a
firm to use up its "research stock". Instead, to align R, D&E
with the goals of the firm, the metric system should balance
market outcome metrics with metrics that attempt to measure
research effort more directly. Such metrics include many traditional
indicators.
For long-term
research explorations, the right metrics encourage a breadth
of ideas. For example, many firms seek to identify their "best
people" by rewarding them for successful completion of research
explorations. However, metrics implied by this practice lead
directly to "not-invented-here" attitudes and result in research
empires that are larger than necessary but lead to fewer total
ideas. Alternatively, by using metrics that encourage "research
tourism," the firm can take advantage of the potential for research
spillovers and be more profitable.
Download
Paper (pdf)
