PISA 2012: Results as expected
From Mackenzie Hird
 A few weeks ago, the results of the 2012 PISA (Programme for International Student Assessment) test were released, causing quite a stir across the country. You can read the full results here, but in short, the United States performance has actually declined in Math and Reading, and only improved slightly in Science, since the test was first administered in 2003. More troubling is that other nations have shown an upward trend, with 16-22 countries now performing statistically significant above the US score, depending on the subject.
Of course, both sides of the debate around education reform were very vocal in interpreting these results. Opponents of standards-based reform believe that these results show that the reforms of the last decade have and little or no impact, and that it is time for us to get rid of standardized tests all together. Proponents of these reforms explained away these results, saying that correlation doesn’t imply causation and that instead this shows the continued need for reform. Proponents also frequently mentioned the results of the 2011 TIMSS (Trends in International Mathematics and Science Study) exam, where the United States has shown continued improvement each testing cycle, and ranks above the average. Further, only 6-12 countries score statistically significant above the US score on TIMSS, depending on the subject and grade level.
The United States is not the only country whose score has differed significantly between these two international tests. Finland, long held up as a high-performing education system based on its results from the PISA test, performed only marginally better than the United States on the TIMSS exam. So how should we interpret these results? Is the US catching up to Finland, or are we really not improving at all?
My answer to this question comes from examining the questions from each. The PISA exam is based on demonstrating the deeper reasoning skills that students have acquired, and thus it uses complicated word problems that often require students to transfer their knowledge to new situations or types of questions they might not have seen before (Sample Question Here). TIMSS, on the other hand, is more heavily based on testing specific concepts, and is more reminiscent of a typical standardized test that we are all familiar with (Sample Questions Here).
Armed with this knowledge, these results are now unsurprising. The United States, which for the last decade has focused on developing students knowledge of facts and concepts that will be tested through high-stakes standardized tests, has shown improvement on the test which assesses progress in knowledge of specific memorized concepts. However, this concentration on memorizing concepts has, unsurprisingly, not translated to any improvements in the higher order skills tested by PISA. Similarly, Finland, a country which emphasizes developing critical thinking skills over memorizing concepts, does very well when these skills are tested by PISA but performs only average when memorized concepts are tested by TIMSS.
These test scores reveal the flaw in reasoning behind our mandated standardized test usage in this country. Our students are obviously performing better at memorizing concepts and regurgitating them on a standardized test: our current system makes it a priority. However, memorizing concepts should not be prioritized over development of critical thinking and reasoning skills for our students. These students will be able to look up any such facts instantly on a small device in their pocket, and success in the future will instead come from their deeper analytical abilities. We should instead be preparing our students to take real-world authentic assessments of their ability, which will equip them with the tools for the complex tasks they will be doing throughout a successful career.
Mac Hird is a Ph.D. student in the Engineering Systems Division at MIT and is studying education as a complex socio-technical system.
Back to newsletter |
