Ei-iE

As part of the broader 2015 landscape, various initiatives and processes are taking place. One of these initiatives, the Copenhagen Consensus Center, had the idea of carrying out cost-benefit analysis on the different targets that have been debated over the past year. One of these analyses was on education specifically and on which EI was asked to give feedback. Below is an adapted version of a response paper by Steve Klees in which he reflects on the education analysis carried out by George Psacharopoulos for the Copenhagen Consensus Center.

In their “Post-2015 Consensus” document, the Copenhagen Consensus Center argues that the way to “make sure that the best and most effective [post-2015] proposals” are selected is to determine “which proposals, based on economic evidence, will do the most good and should be prioritized over others.”  While this may seem reasonable in the abstract, it is not reasonable in practice.  Unfortunately, economic evidence, cannot offer very useful information for prioritizing public choices either within sectors or across sectors.

Of course, there are some economists who would disagree.  Most particularly, George Psacharopoulos does in his “Benefits and Costs of the Education Targets for the Post-2015 Development Agenda.”  Nonetheless, there is ample evidence for my position; the most significant piece of evidence supporting the position that economic evaluation offers little in the way of practical utility is historical.  Cost-benefit analysis, and its common manifestations in rate-of-return (ROR) analysis and benefit-cost ratios (BCRs), has been around since the 1930s but it has been mostly an academic exercise and not used much in public sector decision-making.  Although it has found a home in agencies like the World Bank, it is rarely used by developing countries and often contested by them as promoting results that are contrary to common sense (e.g., cutting investment in higher education).  In fact, even the World Bank and the regional banks do not have the temerity to use it to make cross-sector policy given its many flaws and fallacies.  And even developed countries rarely use these types of economic evaluation for decision-making.  The point in this reflection is to illuminate why the U.N. should not consider such flawed evidence in the very difficult task of choosing among competing priorities.

Let me start by pointing out what Psacharopoulos did well.  What was extremely important was his elaborating the sobering, failed history of international promises for educational improvement for at least the past 50 years.  An analysis of these promises and their repeated failure has not been a part of the post-2015 agenda.  But if we don’t understand why we have failed, how can we expect that postponing everything to 2030 will yield a different result.  (The failure is due mostly, despite good intentions, to the lack of serious effort in what are readily attainable goals.)  Now, Psacharopoulos comes to the wrong conclusion – that repeated failure means we should give up and pursue, as he says, “Education for Some,” not “Education for All.”  This is a problem inherent in taking the narrow view of economics that Psacharopoulos proposes and ignores the global transformation that has accompanied looking at education as a right for all, not as a commodity.

Methodological Issues

The ROR methodology is fundamentally flawed and has been critiqued by economists since its appearance in the 1960s.  I will summarize three principal problems.

Narrow view of benefits.  In theory, economics takes a very broad view of individual and social benefits, but, in practice, usually just looks at the increase in earnings resulting from education.  This yields many problems.  Most discussed is the fact that externalities – the broad effects of education on families, crime, welfare, economic growth and development, poverty, inequality, technological development, urban and industrial development, competitiveness, and much more – are usually ignored, in practice.  This is monumental!  As Psacharopoulos says, including externalities could completely change the priorities his benefit-cost analysis indicated.  There is considerable reason to believe that externalities can be more important than individual income effects.  There is no reason to believe that it is valid to make social priorities using RORs based on individual income effects.  It is not partial information for prioritizing, it is no information at all if priorities could be reversed if externalities were included.

A salient story is that for decades the World Bank hammered developing countries with ROR data (from Psacharopoulos and his colleagues) that purported to show that primary education was a better investment than higher education.  Many developing countries resisted since higher education was seen as essential to a balanced education portfolio.  Nonetheless, over decades, this rationale was used to cut government spending on and privatize higher education around the world.  Then, in the year 2000, UNESCO teamed with some ROR skeptics at the World Bank and produced a report highly critical of the use of ROR data and used new economic growth theory research to argue that the externalities from higher education were very substantial and its true ROR could be even higher than that of primary education.  Yet decades of policy had been made on simplistic ROR data, seriously disadvantaging many individuals and countries.

Earnings do not reflect productivity.  Earnings are a private benefit to the individual.  To economists, earnings are only a measure of benefits to society if earnings are an accurate measure of individual productivity.  Economists theorize this is true under a highly, economists say ‘perfectly,’ competitive economic system.  But this is not true.  Our economic system violates all the assumptions of perfect competition:  it has monopolies and oligopolies, imperfect information, discrimination in the workplace, patents and other barriers to economic activity, unions, minimum wage laws, and many other ‘imperfections.’  Under these circumstances, even if you could measure education’s effect on earnings and calculate RORs, they tell you nothing about whether education is a good social investment.

Moreover, even the effect of education on earnings is impossible to measure.  Psacharopoulos ignores one of the major flaws of ROR analysis: getting an estimate of how much education affects earnings.  Imagine you collected data on the earnings of 100 people selected at random.  Imagine delineating the literally dozens of factors that would explain why they had different earnings, only one factor being their educational differences.  Despite sophisticated statistical techniques, it is simply impossible to accurately separate out the impact of education from all other factors.

For all these reasons, the ROR data Psacharopoulos uses has no validity.

Psacharopoulos’ Priorities

Based on his assessment of the economic evidence – RORs and BCRs – Psacharopoulos recommends the following:

• The “highest BCRs” are for the expansion of preschool and primary schooling and for improving school quality;
• A “valuable target” is ensuring secondary school completion; and
• “Relatively ineffective” or too “uncertain” are providing vocational education or education and training programs for older workers.

For the reasons given in the previous section, Psacharopoulos’s delineation of priorities is fundamentally flawed.  The data proffered are so biased to an unknown extent that they do not even offer a partial basis for public decision-making.

For example, the preschool recommendation is based primarily on one completely unbelievable study in Kenya that purported to show that benefits are 77 times greater than costs.  Almost every economist who sees such results will believe that the study must be flawed.  77:1 does not happen in the real world.    And then to arbitrarily cut it in half, as Psacharopoulos does, (as opposed to one-quarter? one-tenth? one-hundredth?) simply shows the lack of serious attention to an empirical basis and sound reasoning in Psacharopoulos’ analysis.

The preschool, primary schooling, and secondary schooling are for post-2015 policies that imply substantial expansion.  However, economic analysis of past data, at best, only tells you what happens for small changes from present circumstances.  For large changes, costs and benefits will likely change so RORs based on past data are not good guides to future returns.

Returns to quality improvement are based on very few studies.  Those studies are all based on a very narrow view of quality as measured by a few test scores.  Moreover, the ability of those studies to establish what causes test scores to go up is minimal.

Psacharopoulos calls investment in vocational education or adult education “ineffective” or too “uncertain.” But, as should be clear from the analysis above, there is huge uncertainty is applying RORs anywhere. There have been a number of studies that show the effectiveness and efficiency of vocational education.  Moreover, the ones that show a low ROR are often based on an incorrect premise that the comparison should be with academic education.  To the contrary, the policy choice is often vocational education or, in its absence, students drop out of school, not continue with an academic education.  If dropping out were used as the comparison group, RORs to vocational education would likely be much higher.

Adult education is an area in which there are few ROR studies.  In the 1970s, adult and nonformal education were major investment areas around the world.  Starting in the 1980s, the World Bank and others, with Psacharopoulos in the lead, argued to cut their funding drastically, based on ideology more than empirical evidence.  Adult education and the training of younger and older workers offers important benefits to individuals and society.  We have been engaged in a triage focused on the very young.  Even instrumentally, adult education is important as it contributes greatly to the education of children.

Psacharopoulos’ Biases

Psacharopolous argues that his paper is based on “findings in the economics of education literature.”  To the contrary, I would argue that his paper is based on a very selective reading of that literature that conforms to his prior biases, ignoring contrary findings and ignoring fundamental methodological flaws.  Psacharopoulos has had a very public career and has been an open and passionate advocate of particular policies which he finds evidence to once again support today.  At the World Bank, he was known as “Mr. Rate-of-Return,” which was generally not a complement as he used his view of economic evidence as a bludgeon to attack higher education, vocational education, and adult and nonformal education, an attack which he continues in his paper that I examine here.  Psacharopoulos is seen by many in the profession as taking extreme and simplistic views.  He was known for arguing that context did not matter much and one-size fits all.  This view is echoed in his response to readers of his first draft:  “If aspirin reduced headaches in the Unites States, it must do so in Kenya or Zambia.”

Education is not aspirin and context does matter.  Simply the idea that we can determine global priorities across contexts using economic data is completely flawed.  Priorities must be determined by a messy, democratic, participatory, political process.  Not an economic one.  Priorities are very rarely determined in developing or developed countries using these economic methodologies – for good reason – they do not work.  This is much less a critique of Psacharopoulos’ study and more a critique of how the economic framework is so loose that all such studies allow the biases of the researcher to determine the outcome.  I am not arguing against these areas as educational investments, only that priorities cannot be illuminated by this economic methodology.  Nor am I arguing against the use of data.  For example, some valid studies of costs, effects, and benefits can and should be used in the political process but not as a substitute for it.  The U.N.’s post-2015 efforts must repudiate this economic dead end and take a new approach to the very important business of prioritizing policy directions.