Published on: 27/06/2018
Presenting data differently can boost pro-poor policies and emphasise reaching everyone.
The new JMP WASHdata website offers a simple tool for visualising data on country-level achievement towards the Sustainable Development Goals (SDGs) for water and sanitation. The targets for water, sanitation and hygiene are broken down into service-level ladders with each rung representing the proportion of the population receiving that level of service. Evaluating these rungs for each country is useful in judging progress on a small scale, but it can be difficult to compare these ladders across countries or regions. Most often, we see comparisons between countries based on the proportion of the population receiving a “safely-managed” or “at least basic” service. This type of evaluation aids the sector in understanding how far a country lies from achieving 100% coverage, as advocated for in the SDG targets, but fails to shed light on where the rest of the population falls on that ladder—i.e. the population with no or very limited services.
In striving to achieve the SDGs for water and sanitation, should we place more emphasis on lifting those with basic services up to safely managed? Or should we instead focus on that portion of the population that is underserved or wholly unserved? Using an index that includes a weighting scale is beneficial, because it allows us to account for both, and to show whether progress is being made in either.
With this issue in mind, Stuart Kempster and Andres Hueso, in their co-authored draft WaterAid discussion paper, put forth a potential method to compare achievement levels in sanitation across countries and time. This new index, the Sanitation Progress Index (SPI), considers members of the population who fall at the lower rungs of the ladder. It does so by placing a weight at each rung, with “safely managed services”, the gold standard, assigned a weight of 1 and “open defecation”, or the lack of service, a weight of 0. By simply multiplying the proportion of the population at each rung by the weight assigned and summing these components, the SPI for each country/region is easily calculated. This score provides a more holistic picture of where a country lies in achieving the SDGs and makes it possible to acknowledge progress made even when it is not reaching the higher benchmarks of safely managed or basic services (for example, a country which has improved access via shared toilets or construction of traditional latrines in place of open defecation).
Using the method proposed by Kempster and Hueso, we calculated an SPI for each of the IRC focus countries over the 15-year time period from 2000-2015. We followed a weighting scale prioritizing pulling those practicing open defecation up to at least ‘limited’ services (Open Defecation: 0, Unimproved: 0.1, Limited: 0.6, Basic: 0.8, Safely Managed: see Note). While there is no truly objective method for weighting each rung, the bias inherent in different weighting scales can be used for positive political purposes, such as advocacy for reaching at least limited services for all.1 We then compared these index scores for each country to the proportion of the population receiving ‘at least basic’ services—the more traditional method of comparison. Presented below are the SPI scores for each country, as well as comparisons to the more commonly used ‘at least basic’ indicator. The SPI score is shown adjusted to a scale from 0-100 to provide for a clearer comparison between the two scores.
Graph 1. SPI scores compared to the proportion of the population receiving ‘at least basic’ services for four of the six IRC focus countries.
Using the same weighting system from the Sanitation Progress Index, we then calculated a “Water Progress Index” (WPI) for each of the IRC focus countries from 2000-2015. The results of this comparison are shown below, along with a comparison to ‘at least basic’ service levels for the focus countries.
Graph 2. WPI scores compared to the proportion of the population receiving ‘at least basic’ services for the IRC focus countries.
Comparing across countries within each index, as well as across indices, provides some interesting insight into the progress that each of our focus countries has made over the 15-year period.
Using these progress indices, as opposed to comparing on one service level, allows for a clearer preliminary study on the trends of progress in water and sanitation. These comparisons can aid researchers and monitoring professionals in determining which trends warrant further investigation, and they can be useful in advocacy and agenda-setting. Data and presentation of data can have a strong influence on public opinion, and an alternative index can bring a new perspective to the conversation that may have implications on where to focus and allocate resources. The index highlights the importance of reaching un- and underserved population segments, rather than just improving service quality toward safely managed services at the higher rungs. Through this method of data presentation, we can work to incentivise pro-poor policies and emphasis on reaching everyone.
Another caveat of using this method is that many countries do not have data available in the JMP database for safely managed services, therefore bringing down their index score and perhaps unfairly representing their progress. Thus, the limitation of one method is reflected in another. One possible way to deal with this issue is to calculate the index scores for countries without safely managed data by weighting “at least basic” as 1 and moving down from there. Kempster and Hueso do address this issue in their paper and explain that the index is used to provide comparison with the end goal of “safely managed” services for all. Therefore, the reduction in the SPI score resulting from the lack of data offers an incentive for countries to improve their monitoring systems.
For more information on the Sanitation Progress Index (SPI) please contact WaterAid's Policy Analyst for Monitoring and Accountability, Stuart Kempster at StuartKempster@wateraid.org
Since Kempster and Hueso developed the SPI, they have updated their methodology to calculate what they call a 'total service gap'. They describe this measure as a way to "combine data on each rung of the [sanitation] service ladder to quantify how far away each country is from universal safely managed services" based on a "sensitivity analysis" that assesses "the validity of using uniform service level weights". Read more in their November 2018 article "Moving up the ladder : assessing sanitation progress through a total service gap" listed below under Resources.
At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.