PISA: Is testing dangerous?
The OECD’s Programme for International Student Assessment (PISA) is an internationally standardized test undertaken by random samples of 15-year-old students in 57 countries around the world. It is the best known international comparative study undertaken regularly in education today, and its results have a significant impact on education policy in participating countries and beyond. As such, PISA is increasingly important for education policy development, but it is just the tip of the iceberg of a bigger process: the trend towards evidence-based policymaking as a force in politics on international scale.
In the PISA process, students are randomly selected to perform pencil-and-paper tests in 45 minutes. This makes it similar to many tests that are a key part of schooling methods nowadays. However, PISA’s influence goes far beyond merely taking a snapshot of how well students have acquired certain knowledge or skills. Tests are becoming powerful policy tools, both nationally and internationally. In 2008, EI undertook a survey analyzing the impact of PISA 2006 on the education policy debate, with a specific focus on how the mass media reported on PISA, how governments drew from its conclusions and how unions reacted to them. In other words, how education is affected by the most important international comparative survey. The findings show that PISA has a tremendous impact on the way in which educators, parents and governments understand education. The analysis of press materials demonstrates that the overwhelming majority of articles on PISA make a simple reference to PISA as a “measurement” of the quality of education. Making comparisons to distant places such as Poland and Singapore governments use the testing to justify their reforms. “As shown by PISA, our education system is performing well behind our neighbors, and that is why we need to act,” is a common refrain. These actions usually lead to the introduction of more measurements, of national testing systems based on the PISA model and methodology, of more scrutinized teaching procedures, and ultimately, to linking teachers’ performance and pay to students’ test scores. This trend is very dangerous and unions are increasingly concerned about it because it fundamentally changes teaching and learning and, indeed, the overall meaning of education. International comparisons can produce fascinating data, but it should not be forgotten that the basis for this data is rather simple, and it does not convey the complexity nor the breadth of education. The same limitations apply to any other mass standardized test. PISA 2006 reveals interesting data on correlations between the performance of 15 year-old students in science as well as reading and mathematics, their socioeconomic backgrounds, and the organization of schools. However, it does not transmit anything close to the total picture of education quality. It can help to stimulate debate about education, but any attempt to use the PISA results to support already-developed political agendas would be a misuse of the report and the data it contains. In this, PISA is similar to many national tests. As far as they help understand students’ progress, strengths and weaknesses in learning, they can be very helpful tools, but when they purport to evaluate the overall quality of education systems, they inevitably tend to become instruments of injustice. In fact, what makes PISA different, and more dangerous than the other international comparative surveys in education, is its clear policy orientation led by the principle of increasing school efficiency. This renders it a powerful tool for political influence, as the OECD is able to exert a sort of peer pressure and “soft governance” on national governments, by virtue of its status as an authoritative impartial source of evidence. Given the OECD’s capacity to develop international comparisons based on sound data gathering mechanisms that are globally recognized, staying out of international comparative research and education indicators is not an option for teacher unions. This said, we should not deny the possibility of using PISA reports in an integral, yet creatively positive way. The wealth of data and the variety of conclusions of PISA allow space for sound interpretation from a unionist point of view, as well. A better understanding of the real role and scale of PISA’s potential impact, which EI has developed over recent years, leaves no doubt that the use of PISA for informing policymaking is not only unavoidable, but can be beneficial for the teacher unions’ cause. Laura Figazzolo.
