Department of Urban Studies and Planning, Massachusetts Institute of Technology, Fall 1998
Session Summary -- November 16, 1998
John de Monchaux
Rating Place-Ratings
We have seen a proliferation of reports which rate towns and cities and such ratings are intended to influence or inform decisions about the place. They tell us about the attributes of the city and they tell us something about the role of the media in constructing or affirming the images we have of places. Can we as planners learn anything from these ratings in terms of both the theory and practice of planning?
Though the ratings phenomenon may seem fairly recent, documents exist as far back as the 17th century that can be seen to fulfill this purpose. The more recent growth spurt has been attributed to the demands by state and federal government to evaluate the effectiveness of services and systems, and the increased mobility of populations. Their competitive format also plays a part.
Rating endeavors are undertaken for a variety of purposes. They vary widely in terms of their focus, the seriousness of their approach, and the stringency of their methodology. They serve to "booster" particular places and are part of an important industry. There are a range of publications rating places from the strictly analytical to the commercial and operational and to the more literary comparisons of the affective properties of place.
The place ratings literature tends to fall into three broad categories:
1) Location and moving guides: The most widely recognized guide is the Places Rated Almanac. It rates and ranks 343 metropolitan areas on 10 factors such as cost of living, job outlook, housing, transportation, education, health care, crime, the arts, recreation, and climate. In another vein is FunkyTowns USA which also looks at physical attributes such as public hangouts, whether a place is pedestrian friendly, etc.
2) Do-it-yourself guides: These guides allow the reader to fill in forms and questionnaires, such as those from Money Magazine.
3) Good reading: These make comparisons in essay form and resemble travel literature, such as A Guide to Good Places to Live.
There are a number of shortcomings to these guides:
i) Quality of life indicators: Although the usual indicators (crime, health, education, transportation, etc.) appear on almost all the lists, there are no indicators to cover even the broadest concept of quality of life. It is assumed that these factors are important to people.
ii) Congruence across ratings: Even where different authors use the same indicators, there is often little congruence in the outcomes.
iii) The measure used: The theoretical basis of many of the measures used is often non-existent or debatable.
iv) The statistical basis of the ratings: The rankings are not based on any meaningful fieldwork. On the whole they use statistical data that is collected for other purposes by other agents.
v) Aggregating the data: Because the areas across which rankings are made are determined by the availability of data, they aggregate across areas for which the degree of internal variations cannot be accounted for.
vi) The balance and weighting of attributes: The indicators themselves are hard to balance collectively in terms of influence, accuracy and meaning. If is difficult, for example, to evaluate indicators on weather and temperature against indicators of crime or residential density.
vii) Lack of longitudinal comparisons: Most rankings represent a single moment in time.
viii) The meaning and interpretation of indicators: It is not always clear what a particular indicator is assumed to indicate.
ix) Perception and attitudes: Decisions about place are often affected by factors which cannot be fully captured by statistical indicators. Popular or user perceptions of the quality of life in towns and cities are rarely sought.
x) Physical attributes: The physical features of towns and cities are not included in the ratings. Without some understanding of the way in which quality of life issues relate to physical infrastructure and setting, the value of the rankings remains limited.
xi) Causality: Perhaps the most critical is that the rankings embody no causal theories or proposition. They offer no explanatory or causal relationships between the indicators or between indicators and particular urban conditions or policies.
Given this list of doubts, what can we do with these ratings? One way is to see them as a version of "Cliff Notes." But trying to glean some operational value to planners from the ratings, there are a number of important points to take into account. There is a lack of relationship between the evidence of a city's quality of life indicators and the city's urban stress factors. And the quality of life does not necessarily equate with quality of place. If we look to rating as some kind of aid to planning, we can use rankings that are undertaken by outsiders. Towns and cities should develop their own accounting system that relate to their own interests and need.
There is a suggestion in the ratings literature that one effect of rankings is that they shape the images held by people and businesses of particular cities and desirability of locating in them. Virtually all the factors used by raters and rankers to analyze towns and cities are "imageable" at least in a behavioral sense, but these images cannot be easily fused and become the single image of place. The choice of factors chosen by rankers give us an implicit, general, sense of how cities and towns are expected to perform and what conditions in them are likely to be valued. Cities and towns are dynamic. They change in spite of rankings and ratings. An interesting question is how far the knowledge provided by the rankings has in fact changed the very conditions of those rankings.
In conclusion, rankings and ratings are simply part of the often vast and detailed information that the planner assesses when analyzing problems and designing and defending what they propose. Planning on the whole has a long-term focus, and rankings and rating are too immediate to be significant without a clear and persuasive account of cause and effect. We should develop far more sophisticated systematic ways and reliable ways to do so.
Questions, Answers, Comments
C: I want to break them into different categories. These ratings are done by different magazines and they have to be different each year or they won't be able to sell them the next year. They change because they have to change. And these ratings are invented in part for the reason of selling magazines so that they can sell advertising.
Humans have a tendency to simplify complex situations and it's worth asking about that.
C: One of the more popular ratings is that of schools. The US News and World Report has propelled itself into a separate publication and it would be interesting to see what the impact is on the schools.
Q: You haven't talked much about the physical form and the physical image of the city. You've talked about how these ratings are important to us as planners, but how do we relate these rating with physical form?
A: Let me go back to Kevin Lynch and ask how place rating relates, if at all, to the qualities which he described as the city should have -- vitality, sense, fit, access, control, and meta-criteria -- efficiency and justice. I could not map any of the usual ratings criteria against any of Lynch's. I think it's a much bigger exercise.
C: First, there's the whole middle-value problem. Most of the cities are going to be in the middle, and what that means is totally obscure. The process of using metropolitan data is troublesome because it flattens out the whole region. And I think reviewing the history of the census would be useful too. The earliest variable used by the census was of course population for the distribution of Congress and they were very interested in productivity versus tax valuation. During the second half of the 19th century came the demographic data on mortality, mobility, age distribution, and immigrants. And later came labor statistics and occupation and hours of work. In the 30s came the housing inventory. Most recently have been the additions of health, traffic, and transportation. You quite rightly are looking for variables that might help planners improve the quality of life. Its critical and reasonable to compare one place with another. It seems that in your paper you're saying behind image is a serious question of valuation.
A: The middle-value problem is very serious but in a sense uninteresting because it doesn't tell you much. But developing indicators of planning effectiveness is interesting.
Q: Do these ratings have an effect on cities?
A: Not that I'm aware of any studies that have looked at the impact of a given place with a place's ranking or attribute. It would be interesting to find out if it occurred.
C: What would be a good value of these variables? With crime, is a high number good or bad. A higher number could indicate that people have more confidence in the police and are more willing to report crimes. That's not the only explanation, but it's one that's taking seriously.
C: Until recently, when the Reagan administration closed down the Bureau of Economic Statistics in the Department of Commerce, the Bureau used to take a sample of social security records and they would follow people and you could get information on where people moved, their occupation, and whether their new jobs paid more than the old place. If that had continued, we would have been able to get a lot of behavioral information from it.
Terry Szold, real-time rapporteur
I'm going to go from the serious to the humorous... In 1983, I worked as a planner in the city of Nashua NH and I was involved in the city's master plan's first update in 20 years. So I did a lot of data collection. During my second year there, Money Magazine rated Nashua as one of the top-rated places in the US to live. And I remember the Mayor and dignitaries having a press conference talking about this. Little was said about the externalities of Nashua's warm embrace of commerce and growth, including suffocating traffic, bland architecture, and unappealing signage. The city did not direct it's growth well and did not do enough for its inner city neighborhoods. Worrying about whether Nashua was the best, I thought was not as important as worrying about making it better.
John's overview of ratings provides a good framework for looking at them, but nonetheless, he seems to think that developing a rating system provide fertile ground for further research and could be of use to the planning profession. I have grave reservations about the ratings exercise itself. I think our society is obsessed with ranking and evaluation. This obsession is inflicted on children and institutions. At it's most benign, rating helps consumers choose restaurants, movies, beers, hotels, etc. Less benign examples are GRE and SAT exams. Ratings are often a way of evaluating conformity and have little to do with excellence.
Even if the liabilities of place-ratings models were overcome, I believe we can never come up with criteria that do not import biases. Another problem is that there is an underlying assumption that there is an ideal place to be fashioned. Why isn't striving for improvement enough? Ratings may help to describe attributes of place that we value, but it will not bring us to genuine remedies or solutions. Ratings tell us little about what can be discovered.
I decided to go on the web and return to Money Magazine to find my best place. So I did that. Not surprisingly, New York New York was my number one. My number three was Washington DC. But do you know what my number two was? Trenton New Jersey! I'm sorry, but I can't reconcile how Trenton NJ slipped in there between NYC and DC. But I thought about how I ranked affordable housing as important and wasn't obsessed about crime, so they put me in Trenton instead of the nearby Princeton. I was worried that they might put me in Jersey City. So I don't know where my next voyage will be, but you can look for a P.O. box in Trenton NJ.
| Home | Overview | Schedule | Speakers | Sessions | 11.947 Graduate Seminar |