Ei-iE

by Antoni Verger, Adrián Zancajo and Xavier Bonal

In the context of the post-2015 debate, many stakeholders are advocating for the introduction of a learning-related goal as the most appropriate way to monitor educational progress in low-income countries. Of course, good quality education systems should provide all children with the same opportunities to learn, as well as to enjoy a satisfactory and safe schooling experience. Learning is one of the most important outputs in all education systems, but, from a development perspective, educational learning cannot be measured and monitored in a vacuum.

A good measure of educational development should be holistic in essence. It should combine learning with other indicators that cover both those conditions that, precisely, foster student learning, as well as the strengths and weaknesses in the different dimensions that, beyond outputs, form education systems. The emphasis that is too often placed on learning results neglects the fact that there are other elements in education systems that are indispensable for understanding student development and cognition. Therefore, a comprehensive measure of educational development should pay attention to the ways in which different educational dimensions are mutually constituted as well as the ways in which they influence each another.

This article aims at contributing to the post-2015 education agenda debate by defining a methodological and conceptual framework that can help to envision a comprehensive measure of educational progress. Before we get into technical details we reflect, on the basis of international literature, on the main notions that the construction of such a measure should contemplate:

• Equity:The OECD/PISA report, in its numerous editions, shows that there is a strong relationship between equity and effectiveness. In other words, those education systems that are more equitable are also those that are more effective. The balanced distribution of students and, specifically the “peer-effect,” explains why promoting equity at the system level has great potential towards improving educational results ( Gorard, 2006).
• Quality Teachers:According to the UNESCO Global Monitoring Report (2005), teachers “are the strongest influence on learning”. For teachers to develop and practice their roles effectively, teachers’ training, both pre-service and in-service, needs to adhere to high quality standards; teachers’ selection processes need to be sufficiently demanding; the teacher profession needs to be attractive in terms of labour conditions; teachers, especially in the first years in-service, need to enjoy good support structures; and learning environments should be safe, stimulating and invite collegial interaction and exchange.
• International evidence shows that thestudent/teacher ratioalso impacts on the schooling experience of children. This does not apply to all students equally: evidence shows that students from low-income households benefit further from lower student-teacher ratios than do those students from middle-income households ( Ready 2008). Paradoxically, however, the poorest students are usually those that experience overcrowded classrooms.
• Adult education:Adult illiteracy still affects almost a billion people in the world. Not only is adult education a human right, but it is also an effective measure in its capacity to increase basic education levels, since those children with educated parents are more likely to experience better educational opportunities ( Davis-Kean 2005).
• Finally, we cannot forget thatequal education access still matters.According to the Global Partnership for Education, there are still 57 million children out of school today, with that number still rising in some sub-Saharan countries. Thus, a biased focus on learning, without adequate attention to access, would be inappropriate for monitoring education progress and the effectiveness of education systems around the world. In terms of access, a measure of education development should also take into account early childhood education (ECE) since, as more and more evidence shows, ECE is instrumental to preventing early dropout rates and reducing inequality.

Toward a Comprehensive Education Progress Index

On the basis of the evidence presented above, the composite index we set out below might contribute towards a more holistic understanding of education progress. The challenge here is both comprehensiveness, and keeping the measure as simple as possible, in an effort to guarantee sufficient coverage.

The following table presents the index and its main components, in terms of dimensions and indicators. It is clearly evident that equity is a crosscutting element in the elaboration of the index, since we consider it should apply to all the dimensions of the education system. In fact, it can be considered that this new index has three sub-indexes: the Access index, the Process Index and the Output index. Likewise, it may also be broken down according to Effectiveness and Equity indexes.

Dimensions

Access

Process

Outputs

Effectiveness

• ANER Pre-primary

• ANER Primary

• ANER Secondary

• Student-teacher ratio

• % Trained teachers

• Adult literacy rate

• % of students with a minimum level of competency (according to test X)

Equity

• ANER Pre–primary / Primary / Secondary female - male ratio

• ANER Pre-primary / Primary / Secondary first - fifth quintile ratio

• Student-teacher ratio first – fifth quintile ratio

• Trained teachers first - fifth quintile ratio

• Adult literacy rate first - fifth quintile ratio

• Adult literacy rate female- male quintile ratio

• % of students with a minimum level of competency (according to test X) first - fifth quintile ratio

The indicators included in this framework are proxies of the different educational dimensions that need to be measured, although we are aware of the difficulties to capture the complexity of such dimensions with a limited number of them. In the following box, however, we justify our choices.

Indicators justification

The access indicators we have selected try to capture the level of enrolment at three basic levels of education: pre-primary, primary, and secondary. We prefer to use adjusted net enrolment rates (ANER) to net enrolment rates because the latter may result in over-evaluating educational progress in certain countries.

The process indicators refer to two elements that, as previously mentioned, condition the quality of education: the good quality of teachers and the student/teacher ratio. The student/teacher ratio is, moreover, a proxy to ‘inputs’, since teachers’ salaries represent the biggest proportion of running costs in all education systems.

For the output dimension, we propose the adult literacy rate as a proxy of the situation of adult education in each country. If the results of an international assessment test were to be included in this dimension, we would select PISA. Despite its presence in low-income countries is still modest, PISA is the program with the highest number of participants and its focus on competences facilitates comparability (although other may argue that learning measures that focus on curricular content, such as those applied by the International Association for the Evaluation of Educational Achievement, are more appropriate). Specifically, we would select the percentage of students above level 2 in PISA as a proxy to the effectiveness of an education system, since this is the level that is considered “ as a baseline level of proficiency, at which students begin to demonstrate the reading skills that will enable them to participate effectively and productively in life” (OECD, 2010).

For all of these dimensions, equity indicators(based on the differences between females and males and/or income quintiles) have been also included. Differences between urban and rural populations, or by ethnic origin, have been dismissed to ensure the comparability of the index between countries with different social and territorial configurations.

Main application limitations

The above-proposed index framework represents a way to measure educational quality from a multidimensional approach; however, some comments need to be made about its applicability:

• Availability of indicators:Some of the proposed indicators, such as the adjusted net enrolment rate for the secondary or pre-primary level, are not available in the main existing international education databases [1]. Because of this reason, the international community usually ends up adopting less accurate indicators, like net enrolment rates, which affect the validity of the measure.  It is also well-known that there is a big gap in globally comparable data on learning, which is something that clearly complicates the application of the outputs sub-index.
• Equity indicators:In most cases, it is not possible to measure some indicators according to income inequalities. Using national household surveys could be an option, but it would make the calculation, the cross-national comparison and the updating of the index difficult. In the case of process indicators (student/teacher ratio, or trained teachers), it is difficult to calculate inequalities on their basis due to the fact that they are not related to students’ characteristics.

Due to these data limitations, we cannot apply exactly the same index to particular cases and in consecutive years yet. However, in a previous work we did for UNESCO, which can be consulted here, we applied an equivalent index (with the same components, but not exactly the same indicators) to a group of low-income, middle-income and high-income countries.

In conclusion: More equity-sensitive data is needed

Beyond the suggestion of a specific index, this article aims at presenting a framework to think about the measurement of education progress from a multidimensional and more comprehensive perspective.

Obviously, elaborating on these types of indexes is an exercise that implies an over-simplification of reality; for instance, we are aware of the fact that differences in learning cannot be simply attributed to differences in education systems’ effectiveness, since a range of contextual and non-educational factors may influence educational results. We are also aware of the fact that, to enhance its comprehensiveness, the index should contemplate adding an educational outcomes dimension (i.e. what people are able to achieve when in possession of education: political participation, access to the labour market, reducing the gender gap in income and employment, etc.) to the access-processes-outputs framework. But again, this option would reduce the territorial scope and applicability of the index even more.

It is also important to acknowledge that although the framework suggested here could be applied globally, the specific index does not have a universal ambition. Its main components are, rather, suitable to those countries, usually low-income, that are targeted in the context of the post-2015 debate. Thus, if the territorial scope is to be widened, the indicators should be adjusted to different education standards and levels of development.

Nevertheless, the main message in our index proposal is that equity needs to be understood as a key component in the definition of educational quality, as well as in the measurement of educational development; however, the poorness of existing data on economic inequalities in international education databases makes it difficult for the equity component to be applicable in the short term. Such an important absence in international data reflects that, in practical terms, equity has traditionally had a low profile in global education agendas.

The post-2015 debate opens up a window of opportunity to revert such a trend and think about more comprehensive ways or measuring educational development. If we adapt the final goals to available data, we would be losing a great opportunity to correct a historical deficit in this terrain.

[1] We take the UIS-UNESCO database or Edstats – World Bank as a reference.