Published on: 02/06/2015
Ultimate success in water service delivery is defined by the service level received by households. There can be excellent infrastructure and impeccable administration, but if households don't receive enough water of good enough quality without spending an excessive amount of time collecting it, then it's a bit of a charade. To assess service levels you can't rely on data from service providers or community organisations; you really need to go house to house and talk to the people who are using (and paying for) the water. This blog reviews service level data from the Community Water Plus project in India.
By: Benjamin Harris and Matthias Javorszky (Cranfield University)
As part of the Community Water Plus project, data is being collected from nearly 2500 households in 80 villages across India on the water services they receive. The survey covers metrics such as the source type, quantity and quality of water, user satisfaction with service and socio-economic data. The overall aim of that part of the project is to validate that users of community-managed rural water supplies actually receive good services. As part of a student group project at Cranfield University we had access to an early subset of this data, and were able to start investigating whether the services were delivering for users, and what potential factors impacted on the success (or otherwise) of services.
Much of the methodology for assessing services levels was derived from the work carried out as part of IRC's WASHCost project. This defines water service levels by indicators for quantity, quality, accessibility, reliability and continuity. The basic acceptable service level is set with reference to the Indian Norms for rural drinking water, with services provided above this threshold rated as 'improved' or 'high'. Services below the threshold are similarly 'sub-standard' or 'no service'.
In our project we then went a step beyond this, looking at different ways of aggregating these separate indicators into a single, composite, indicator. This was primarily to enable comparisons between households: what is the overall service level of a household that receives a high quantity of water, but only gets water 30 minutes a day? It's like trying to compare apples to oranges. After looking at several methods, we settled on using the geometric mean to aggregate five individual scores into one. We assigned a score from one to five to each indicator (one being no service and five high), and calculated the geometric mean from this. As a method it's simple enough for practitioners and policy makers to use and understand it, whilst also offering a reasonably nuanced approach. It's not perfect, but it serves its purpose.
For example, a household which access a water source which is highly accessible (5) and provides a high quantity (5) of water, but has only basic reliability (3) and basic water quality (3) would be assessed as basic overall (a score of 3.9 being rounded down). Whilst the geometric mean may not be as immediately understood as the arithmetic mean, the manner in which it prevents high scores in one indicator compensating for low scores in another is an attractive feature in this application. We further added thresholds to prevent this compensability: if the score on any indicator is 'sub-standard' or 'no service', therefore failing Indian Norms, the overall service level is unacceptable too.
This might seem like a lot of unnecessary maths, but it does have a real implication for assessing the service level people receive. In our sample, over 98% of households access an improved source (as per JMP definitions), but only 47% receive an acceptable level of service. The main reason for this failure are the quantity accessed - over 70% of households are receiving an unacceptable service in terms of the quantity of water. Accessibility was also poor, with 35% of unacceptable services due to households spending excessive time collecting water. It should be noted that often there were multiple reasons for poor service - i.e. both quantity and accessibility were unacceptable.
Using our data this suggests that existing forms of monitoring may be overestimating progress made in providing drinking water. The success in achieving the MDG on drinking water five years early is rightly celebrated, but is this 'success' merely masking many of the underlying failings in rural drinking water supplies? People may be familiar with the sad statistics such as the 36% of handpumps in Sub-Saharan Africa that are non-functioning. But our data show that even of the systems that are functioning, a significant portion don't provide adequate services. This finding echoes other studies: in three districts in Ghana, it was found found that only 20% of supplies from handpumps and piped systems provided a basic level of service, largely due to failure to meet the accessibility indicator. Looking specifically at the water quality of improved sources Onda et al. estimated that rather than being met in 2010, there was a 10% shortfall in meeting the MDG in 2012 due to improved sources being faecally contaminated.
After assessing the service levels, the next step was to try and understand which factors affect the service levels users receive. To do this we analysed the available database of more than 800 household surveys. We identified possible links between the 43 variables in the survey and then conducted statistical tests to see whether there is a significant correlation.
Point sources such as standposts or handpumps generally fail to provide acceptable service to users
One of the results that stood out was that point sources such as standposts or handpumps generally fail to provide acceptable service to users. In our sample, more than 85% of point source users received an unacceptable service, mostly due to the very limited quantity used. Even though people have a standpost in their village, because of the time and effort it takes to collect water from there, they use less water than prescribed in the Indian National Norms for Rural Drinking Water from these sources. Household connections on the other hand provided good service to users, supplying sufficient quantities of water directly to their homes for 72% of the survey respondents. Point sources therefore provide poor service, while piped supplies achieve higher service levels. This is one of the reason why the Government of India is aiming at providing piped systems even in rural areas. It is also reflected in an indicator for the upcoming SDG target 6.1, which defines 'safely managed water', the highest level of service, as 'an improved water source that is located on premises'.
But isn't having access to a handpump better than no improved water source? Are we being too harsh on point sources? This argument is illustrated by one case study from West Bengal, where it appears that all households receive an unacceptable service. This is because people collect very limited quantities of water for drinking and cooking from handpumps (below 20 lpcd) and use surface water for all other domestic activities. Even though the quantity collected from boreholes is well below the national norms, this represents a significant step up from using surface water for drinking and cooking as well. If supplying piped water is not possible for economic or hydrogeological reasons, isn't a point source that provides at least a small quantity of drinking water already an improvement? Point sources are a step on the way towards realising the human right to water for everyone. The question is, how long do we remain on this level? At what stage of economic development do we say handpumps are no longer an option? At what stage should we insist on household connections as the only way to provide a truly acceptable service?
The other main result that emerged from our analysis was that underlying communal wealth is one of the most important factors affecting service levels. We found a very strong and significant correlation between the median income in a village and the village service level - defined as the percentage of households that receive acceptable service.
Looking at individual households, there is only a very weak correlation between income and the service level received. This supports results from last year's group project which reviewed 130 case studies of community-managed water supply projects and found that underlying wealth is one of the key success factors for rural water supplies.
So what have we learned from this exercise? Perhaps the first thing is a disclaimer: household data is very useful but it is hard to collect. We spent a significant amount of time cleaning the data to make it useable and sometimes had to make assumptions because of missing or unclear data, as we couldn't go back to the field to resolve these issues. Also it should be recognised that many of the scores are based on individuals' perceptions about the service they receive. Nonetheless, we could show that a significant level of analysis was possible using the household data: in total 17 statistically significant correlations could be found. Perhaps most importantly the data validated that where the right support is in place, community management can be successful: those villages identified as best practice clearly enjoyed better water services than those used as controls.
So where next? The Community Water plus project is continuing and will identify not just where community management is successful, but what is needed for success and what that costs. Community management still has a vital role to play in ensuring the human right to water is met for everyone, and the experience and evidence gained from this project can help ensure that fewer villages fall into the trap of slippage and failing services.
At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.