Ei-iE

This week the 6th edition of the International Summit of the Teaching Profession will gather in Berlin, organised by the German Länder Ministers of Education, OECD and Education International. The main theme is teachers’ professional learning and growth, and country delegations made up by education ministers, union officials and teaching professionals will discuss which policies help foster teachers’ competencies and how to implement them.

The OECD background report for the event captures what is possibly the key issue across those debates when stating that in building teacher professionalism “formal, measurable skills are necessary but not sufficient; they must be complemented by the intangible qualities that are difficult to quantify, including motivation and self-efficacy.”

The recognition of the complexity of teachers’ work and the required competencies poses a challenge to policy-makers, researchers, and everybody else involved in policy design. The OECD references to the limits of quantification is especially important in the light of the pervasiveness of the school effectiveness regime in contemporary education research and policy globally.

School effectiveness research has been instrumental in exposing patterns of outcome among student groups. While engaged with professional development in Denmark, I remember when PISA 2000 helped to highlight the comparative failure of the Danish school system in supporting pluri-lingual students. This illustrates that one thing is the exposure of ‘facts’, another is the use of data for political purposes. Because in the case of Denmark, the ‘facts’ were not addressed by policies to make the most of the lingual resources in the student population but to contain and align them to the preferred norm of Danish as the first language among students. The political use of research data in this case had enormous implications for the content and objectives of teachers’ professional development.

We should also note that the distinctive hope in the power of education that comes with school effectiveness might verge into a politics of distraction when the use of statistical tools as policy instruments is taken too far.

The most obvious example is the use of value-added models (VAM) in school and teacher evaluation frameworks. These have become widespread especially in the US and England during the last decade. VAM is based on the assumption that it is possible to create complex statistical models that capture the essential and universal factors in what makes some schools and teachers more effective than others without sacrificing the complexity of education, teaching and learning.

According to its proponents, VAM helps to make education systems more efficient. However, leading researchers and commentators have for some years made clear that VAM is deeply flawed in terms of reliability, validity, bias, and fairness. VAM effectively paralyses school systems in a sort of short-term thinking that is in stark contrast to the general consensus that wide-ranging educational reform takes time and dedication.

VAM - in spite of the lack of evidence and extensive critique by many including the American Research Association (AERA 2015)- is likely to be promoted by development agencies in middle and low-income countries due to the moves towards results-based financing, or by private enterprises for profitmaking.

VAM is clearly not intended as a politics of distraction. Effectively it is. It is hard to ignore the fact that VAM has been taken furthest in two political entities with some of the highest levels of inequality among high-income countries. VAM focuses our attention on all the good that teachers and schools should strive to do. However, factors beyond the control of schools and teachers continue to be much more important for student performance. The exposure of allegedly failing schools and teachers, hence, disguise that the most serious challenges to ensuring educational opportunities are related to poverty and disadvantage.

We can take this point to a more general level. Looking back over the last 30 years, it is remarkable that in the US and England where school effectiveness research emerged and has been pushed the most, inequality has risen dramatically. What does that tell us about education as ‘ the great equaliser’ and school effectiveness research as a means to that end?

This is just a simple correlation that does not explain anything. Yet, it spells out that the original positive vision of school effectiveness is severely corrupted when the use of statistical tools are taken too far towards bounded rationality in education policy.

VAM, if it has any use at all, must expose the misleading use of statistical mumbo jumbo that effectively #VAMboozles teachers, schools and society. This could help to spark some much needed reflection on the basic propositions of school effectiveness, the negative effects of putting too much trust in numbers, and lead us to start holding policy-makers to account for their misuse of data in policy formation.

Here is the discussion paper. It will be updated and finalised later shaped by the feedback we receive. If you have any questions about VAM, any comments or suggestions about how it works or might work in your context, or you just want to contribute to the debate, please do not hesitate to get in touch. Email us at: [email protected]