Budget Brief No. 8 – Quantity Without Quality? Rethinking Education Goals

THE OPINIONS EXPRESSED IN THIS BRIEF ARE THOSE OF THE AUTHOR
AND DO NOT NECESSARILY REFLECT THOSE OF THE IBP

Quantity Without Quality? Rethinking Education Goals

Ruth Carlitz

This brief draws on the work done by the author while working at the Tanzanian NGO HakiElimu from March 2006 to June 2008.

In 1990 representatives from 155 countries and 150 organizations gathered in Jomtien, Thailand, and made a commitment to bring the benefits of education to all the world’s people by making primary education universal and boosting literacy rates by the end of the decade.[1] Ten years later, many countries were far from reaching these ambitious goals. The international community met again in 2000 and affirmed their pledge to achieve Education for All by the year 2015, identifying six key education goals. The global commitment to universal primary education was also made explicit as one of the eight Millennium Development Goals (MDGs).

Although the MDGs have come under fire for being overly ambitious or unfair, they have mobilized resources and helped to build political will to improve the education sector in countries around the world. Now, more than halfway to the MDGs’ target end date, concerns are being voiced that progress toward achieving the goals is off track, particularly in Africa. This has led to calls for increased foreign aid and greater local investment in education and other development priorities.  There is evidence that these calls are being met. In the majority of countries with data, public spending on education has increased since 2000.  There is also evidence that governments are prioritizing education spending—for instance, the median share of total government spending that goes toward education for sub-Saharan African countries is 18 percent.[2]

However, despite spending more on the education sector as a whole, governments inevitably have to make choices about what to prioritize within the sector. Unfortunately, it appears that in some cases additional spending to achieve the education MDG has been channeled disproportionately toward quantity (dramatically increasing enrollment), possibly at the expense of quality. This brief presents evidence of this phenomenon from Tanzania, as well as from the literature on international education goals. The brief further discusses why quality in education matters and suggests ways to monitor and achieve it.

Experience in Tanzania

In 2001 Tanzania launched the Primary Education Development Program (PEDP), which placed an explicit focus on increased access.  (PEDP notably abolished fees for primary education.) Increasing access was seen as a necessary prerequisite for improving quality. Government reviews indicate that Tanzania has made significant strides toward the access goal, boosting primary school enrollment rates dramatically. (See Figure 1.) A subsequent secondary education reform (the Secondary Education Development Program, or SEDP) launched in July 2004 has achieved similar successes in terms of increasing enrollment. (See Figure 2.)

The World Bank provided initial funding for the primary education reform in the form of a $150 million credit for the first three years.  The Government of Tanzania and a pool of donors provided the remainder of the needed funds.  As PEDP and SEDP have been implemented, the government has blamed irregularity in the flow of funds from donors for delays in implementing some activities.  This reflects the broader challenges associated with Tanzania’s dependence on foreign aid (which makes up around 40 percent of the nation’s total budget).

Perhaps as a result of the substantial donor presence, the MDG framework has come to play a significant role in how Tanzania’s education goals are measured and supported.[3] The achievements of PEDP and SEDP have led to Tanzania being considered an MDG “success story.” For instance, at a media briefing on a recent meeting of the UN’s MDG Africa Steering Group, UN Secretary-General Ban Ki-moon cited Tanzania as an example of progress on primary education.[4] The World Bank has also highlighted Tanzania’s “impressive results” in boosting primary school enrollment.[5]

However, Tanzania’s success masks some pernicious consequences of hasty efforts to boost enrollment and calls into question the longer-term impact of these reforms. An immediate consequence of increased enrollment has been overcrowded classrooms. While the government’s plans to address this problem include building classrooms and recruiting new teachers, pupil-teacher ratios remain high—an average of 54-1 in primary schools, and 37-1 in secondary schools.[6] These averages, as high as they are, mask glaring disparities—particularly between schools in urban areas and those in more remote, rural places where teachers are often unwilling to be posted or fail to report for duty. A 2006 progress report issued by the government of Tanzania noted that just 7,271 of the 10,510 teachers posted to various councils actually reported to schools.[7]

The impact of quickly expanding enrollment has been particularly hard on the secondary school system. With net enrollment jumping from 6 percent to 26 percent in just four years (2004 to 2008), there has been a scramble to accommodate the new primary school graduates, resulting in the recruitment of vast numbers of poorly qualified teachers.[8] Barely out of secondary school themselves, and sometimes given just weeks of training, these teachers are providing instruction of questionable quality and, some argue, bringing down the prestige of the teaching profession.[9]

The combination of overcrowding and underqualified teachers has been cited as the cause of rioting in secondary schools, which resulted in damage to school property and even loss of life.[10] In addition, the rush to expand secondary school infrastructure has had some negative consequences. Cases of intimidation and violence have been reported by people who were forced by local leaders to contribute money and time to build secondary schools in their communities.[11]

The zeal to expand access to basic education in Tanzania is clear. But the current situation begs the question of whether the current approach will ultimately have the desired outcomes.

Quantity ≠ Quality: Evidence from the Literature

The significance of education for economic development is widely recognized.  Education experts at the Center for Global Development note that in many poor countries an additional year of schooling translates into earning 10 percent higher wages, which, in turn, contributes to economic growth.[12] However, a growing body of evidence indicates that simply spending more time in school does not necessarily lead to improved economic conditions. Rather, various studies show that educational quality—particularly in terms of students’ cognitive skills—is a much more significant predictor of individual earnings, distribution of income, and economic growth.[13]

Progress toward the MDG education goal, however, is typically measured in quantitative terms (primary school completion) and thus may fail to capture the quality dimension. Indeed, the experience across many countries indicates that progress toward the education MDG has not translated into progress in terms of actual learning.[14] Analysis of the results of internationally comparable student assessments has shown that even in countries with universal primary completion, the majority of youths are not reaching even minimal competency levels.[15]

Furthermore, a recent evaluation of World Bank support for primary education concluded that countries that expand access too quickly could put learning at risk. The evaluation notes, “The Millennium Development Goal of universal primary school completion by 2015 has added urgency for countries to improve primary school access, but may be pushing some of them to expand too quickly. Some countries, such as Kenya, Malawi, and Uganda, have expanded so rapidly that they have been unable to deliver basic educational services effectively.”[16] This corresponds to the experience in Tanzania.

Strategies for Monitoring and Measuring Quality

Given the significance of quality in education, it is important to develop accurate ways of monitoring and measuring it, particularly to strengthen incentives and enhance accountability. As researchers at the Institute for Development Studies have noted, “rather than making the measurable important, a greater focus is needed on how to make the important measurable”—that is, on creating incentives for quality.[17] Donors, governments, and civil society organizations can all play a role in this endeavor.

Suggestions for Donors

Revising international goals for education is an important step in creating incentives for quality. Donors might consider an alternative to the education MDG that has been proposed by the Center for Global Development. The CGD researchers suggest judging progress in terms of outcomes of the educational system, or the capabilities of all children in a given cohort.[18] Assessing progress toward such a Millennium Learning Goal (MLG) would create incentives for improving quality in education, not just quantity.

In addition, this framework would address legitimate concerns that have been raised about a possible trade-off between quality and equity—in other words, increasing the quality, and consequently the cost, of education might lead to restricting enrollment. An MLG that is constructed to measure capabilities of children both in and out of school would create an incentive to draw more children into the formal schooling system, since such an action would presumably raise cohort learning achievement.

Donors might further contemplate the “cash on delivery” strategy, which has also been proposed by CGD.[19] Under this approach, additional aid would be linked to evidence of progress already achieved on the ground, measured by independent assessment. Unlike traditional donor “conditionality,” aid would not be tied directly to the implementation of any specific policies or reforms. Rather, the means of achieving progress would be left to the discretion of the individual government.  “Cash on delivery” could be integrated into the MLG framework, with a given learning goal linked to aid payments for education.

However, it should be noted that the widespread administration of internationally comparable tests of competency would have significant cost implications, including having to make tough choices about other education spending priorities. Initiatives such as a testing scheme could result in shifting incentives in curriculum development and teaching methods in undesirable ways—i.e., “teaching to the tests.”[20]

Suggestions for Governments

Planning and budgeting for increasing quality in education will require a fundamental shift from thinking about inputs to focusing on learning outcomes (what an “educated” person is able to do). Once they have identified their desired educational outcomes, governments should then work to determine the inputs needed to achieve these outcomes. Starting from inputs (simply directing more money to the education sector) will not guarantee improved outcomes. In particular, just increasing spending on physical infrastructure and other inputs has not been shown to lead to substantial increases in children’s competencies and learning achievement.

One input that flows more logically from a focus on learning outcomes is investment in teacher quality— one of the only school-related factors that consistently has been shown to influence student achievement. Many countries lag behind target teacher-to-pupil ratios and also suffer from chronic underinvestment in teacher training and professional development. Spending more on teachers implies a longer term view, as the benefits of such additional spending would not be realized immediately.  However, it could help to ensure that the newly constructed classrooms are not empty shells but, instead, fulfill their promise of expanding access to quality education.

Governments should also take a holistic view of their education systems, investing wisely along the entire education pipeline to obviate the strain on secondary and higher education systems that expanded primary enrollment can create.

Finally, governments could strengthen “value-for-money” auditing of the education sector to ensure that additional investment is having an impact. Facilitating public expenditure tracking studies would also help to ensure that money spent on education is spent well.

Suggestions for Civil Society

Civil society can also play an extremely important role in monitoring and measuring the impact of spending on education. Most important, civil society organizations can act as watchdogs and not let their governments and donors get away with checking the education MDG box without making substantive efforts to improve educational quality.

Specifically, CSOs can complement government efforts to perform value-for-money audits. For instance, the Uganda Debt Network (UDN) has supported community monitoring groups to track the impact of spending under the country’s Universal Primary Education Programs.  The UDN organized citizens, empowering them to gather relevant budget information and monitor the quality of expenditure and new services. The groups then held public hearings to raise concerns about poor quality school construction and other misspending.[21]

In Malawi, the Civil Society Coalition for Quality Basic Education has implemented public expenditure tracking surveys to monitor education spending at the district and school level.  The coalition’s efforts spurred the government to launch its own expenditure tracking survey and address issues raised by the coalition, such as the late or incomplete disbursement of teacher salaries.[22] In Tanzania, HakiElimu has engaged in similar efforts through the development of a PEDP Monitoring Tool—a questionnaire that community volunteers implement at the school and local government level.

CSOs also can help to measure the extent to which children are learning and building cognitive skills. For example, the CSO Pratham in India produces an Annual Status of Education Report (ASER) that uses data collected by a huge corps of citizen volunteers that  is dispatched across India’s rural districts, where they administer simple tests to school-age children in basic reading, simple comprehension, basic math, and English. The volunteers also visit schools to gather information on enrollment and infrastructure and record other general observations.[23]

ASER, with its comprehensive and systematic coverage, has emerged as a powerful tool for advocacy and education reform in India by clearly demonstrating the “real” situation in India’s schools and illustrating the dangers of equating enrollment with learning. A similar initiative—called Uwezo (“capability” in Swahili)—is planned for Tanzania, Kenya, and Uganda.

Conclusion

Revising international education goals in order to incorporate incentives for quality may pose a new set of challenges. Quality in education is a subjective concept. Even if we define “quality” in terms of certain learner capabilities—the skills and aptitudes that children develop in the education process—it is difficult to make broad policy and budgetary prescriptions that will guarantee it.

However, these challenges are worth addressing, given the significance of quality in education. Educational quantity without quality is a hollow promise. Rather than rush to meet targets in a superficial manner, the international education community should harness its collective imagination and devise policies and incentives that expand access and quality.