Reversing the trend: Improving toilet access for better test scores

Poor sanitation is associated with adverse health outcomes among children, which in turn are linked with poor educational outcomes. Leveraging the staggered implementation of ‘Swachh Bharat Mission’ across districts in India during 2014-2018, this article shows that exposure to the programme improves children’s test performance in the short term, with larger estimated benefits the longer districts were exposed to the programme.

Despite close to US$40 million invested in education in India between 2010 and 2014, the average achievement of school children (aged 5 to 16 years) in terms of test scores declined steadily during this period (Figure 1). Proposed factors underlying this trend include increased teacher absenteeism, universal promotion of students, and changes in the demographic composition of the school-going (and thus test-taking) population due to a greater influx of historically disadvantaged students into government and private schools through the 2009 Right to Education Act (Shah and Steinberg 2019).¹ Hence, it is of interest whether other programmes (such as those targeting health, sanitation, and nutrition) can complement existing education initiatives aimed at improving school access and school infrastructure, and subsequently result in improvements in child school performance on average.

Poor sanitation is associated with adverse health outcomes such as disease incidence and child malnutrition, which in turn are linked with poor educational outcomes due to reduced school participation and impaired learning (Berkman et al. 2002, Fischer Walker et al. 2012, Gilmartin and Petri 2015, Petri et al. 2008). Therefore, programmes designed to improve sanitation may potentially improve educational outcomes in addition to health.

Figure 1. Changes in children’s standardised test scores between 2010 and 2018

Source: Authors’ calculations using the Annual Status of Education Report (ASER) data.

Note: This graph plots average standardised math and literacy test scores during 2010-2018. A score of zero is equivalent to the nation’s average score in that test.

In a recent study (Karmakar and Villa 2023), we examine the impact of the Swachh Bharat Misson (‘Clean India Mission’), a large nationwide sanitation initiative, on children’s test scores in math and literacy. The Swachh Bharat Mission (SBM) was introduced by the Indian government in 2014 with the objective of reducing open defecation and improving toilet access and sanitation infrastructure throughout the country. Primary components of SBM include constructing household- and community-owned toilets, sewage facilities, and adequate community waste collection systems as well as eliminating manual scavenging (cleaning and disposing waste from sewage pits). By 2018, approximately 100 million toilets were constructed under SBM, and close to 84% of households, on average across districts, were covered by the programme (Figure 2).

Figure 2. Percentage of households with a toilet or latrine, by district

Source: Authors’ calculations using district-level data from Ministry of Drinking Water and Sanitation.

Data and methodology

For our analysis, we combine child-level math and literacy testing data from the 2010, 2011, 2012, 2013, 2014, 2016, and 2018 rounds of the Annual Status of Education Report (ASER) with annual district-level data on the number of toilets constructed under SBM during 2014-2018, obtained from the Ministry of Drinking Water and Sanitation website. Our working sample comprises 2,456,787 children across 588 districts. Of these districts, 519 first implemented SBM (that is, the first SBM toilet was constructed) in 2014 and an additional 58 did so in 2015. We refer to these districts as early adopters. Late adopters, comprising nine districts, are those who first implemented SBM in 2017, and non-adopters, comprising two districts, had no SBM-constructed toilets by the end of our sample period.

Empirically, we define district-level SBM intervention in two ways. The first is a dichotomous indicator of whether a child resides in an early adoption district or a late/no-adoption district. The second is a continuous measure of the number of toilets constructed per 1,000 population in the district.

To investigate the impact of SBM on math and literacy test scores, we compare changes in test performance in the years before and after SBM implementation across districts based on the timing of implementation and their exposure to SBM. We additionally estimate ‘event study’ and ‘staggered implementation models’, which allow the impact of SBM to change over time and differ across districts who were exposed to SBM earlier versus later (Wooldridge 2021).

The credibility of our methodological approach depends on the assumption that early- and late-/no-adoption districts would have experienced similar trends in math and literacy test scores in the absence of the programme. A common check for this assumption is to test whether these groups experienced similar test score trends in the years prior to SBM’s implementation. Pre-existing differences in test score trends between the two groups may suggest that our estimated programme effects may be, at least in part, due to differences between the two groups unrelated to the programme itself.

While we do find some evidence of pre-existing differences in math score trends, we also find that differences in post-SBM trends, not attributable to SBM, would have to be substantial in order to render the programme’s impact to a null effect. In fact, using an approach proposed by Rambachan and Roth (2023), we find that difference in post-SBM math score trends would have to be at least 1.5 times as large as the largest pre-SBM trend difference before we could no longer rule out a null effect of SBM. The estimated impact on literacy is much noisier and therefore we cannot consistently rule out a null effect on literacy. However, the SBM impact on math is consistently positive and statistically different from zero.

Toilet construction and its impact on child cognition

Our findings indicate that, in general, exposure to SBM does improve math and literacy scores. Specifically, we find that living in early adoption districts relative to late/non-adoption districts improves math scores by approximately 0.033-0.038 standard deviations (SD)² and literacy scores by approximately 0.026-0.035 SD, on average. These standard deviation estimates represent a 1.9-2.2% improvement in math scores, and a 1.5-2% improvement in literacy scores of the sample averages. We further find that each additional SBM-constructed toilet per 1,000 population in a district improves district-level average math scores by 0.001 SD or 0.05%. However, the corresponding estimated impact on literacy scores is small in magnitude and not statistically different from zero.

When we allow for the impact of SBM to differ depending on the timing of programme implementation and length of programme exposure (Wooldridge 2021), we find dynamic improvements in test performance: the positive impact of SBM increases the longer a district is exposed to the programme. For example, the estimated impact of residing in an early adoption state on math scores increases from approximately 0.03 SD or 1.3% in the first year of exposure to approximately 0.18 SD or 10.4% after 3-4 years of exposure. Similarly, the impact on literacy scores increases from approximately 0.052 SD to 0.138 SD (Figure 3). These literacy score improvements are from 3 to 7.5%, respectively.

Figure 3. Effect of early adoption of SBM on children’s math and literacy scores

Source: Authors’ calculations using data from the Annual Status of Education Report and Ministry of Drinking Water and Sanitation.

Note: This graph plots the treatment effect of residing in an early adoption state in 2014 (year of initial implementation), 2016 and 2018 (two and four years after initial implementation, respectively). It shows that estimated effects increase the longer early adoption districts are exposed to the programme.

We additionally find that the SBM math score impact is largely similar between boys and girls – however, the estimated literacy impact appears slightly larger for boys. We also compare programme impacts across younger (5-10 years) and older (11-16 years) children and do not find substantial differences across the two age groups in either test subject.

Finally, we examine potential mechanisms through which SBM might impact test performance. We find that living in a SBM-exposed district increases the likelihood that an individual household has a toilet, increases child-level school enrolment, and reduces the likelihood that a child’s home has a thatched roof. Thus, exposure to the programme improves household-level sanitation (and therefore potentially child health), school participation, and household socioeconomic well-being, each of which can plausibly improve child school performance.

Conclusion

The incidence of disease is usually more severe for children in regions with inadequate access to water and sanitation services. Being at the frontline of these issues, low-income countries have made massive improvements in drinking water and sanitation infrastructure since the early 2000s (World Health Organization, 2023). Even though it is well-documented that greater sanitation access reduces diarrhoea and improves nutrition outcomes (Augsburg and Rodríguez-Lesmes 2018, Kumar and Vollmer 2012), there is growing interest in the impact of these changes on children’s education (Adukia 2017).

Improved child health holds positive implications for child cognitive development and learning outcomes. Therefore, public investments in sanitation programmes can have benefits for multiple dimensions of human capital. Our results support this notion and indicate that improved sanitation infrastructure can complement educational policy in improving children’s test scores. We find that exposure to SBM improves children’s test performance in the short term, with larger estimated benefits the longer districts were exposed to the programme.

I4I is now on Substack. Please click here (@Ideas for India) to subscribe to our channel for quick updates on our content

Notes:

The Right to Education Act, 2009 is a legislation passed by the Indian government mandating free and compulsory education for children between 6 and 14 years of age. Principal provision of the Act include a no grade retention policy, school construction within neighborhoods and school infrastructure improvements (Karmakar and Villa 2022a, 2022b). Examples of school infrastructure are classrooms, toilet and drinking water facilities, separate toilets for boys and girls, playgrounds, and cooking facilities for midday meals.
Standard deviation is a measure used to quantify the amount of variation or dispersion of a set of values from the mean (average) value of that set. Standard deviations are converted to percentage estimates by dividing the product of the coefficient and standard deviation of the raw math and literacy scores with the respective means of the raw scores.