“Does help help?” is the wrong question

Published on: 12/12/2011

There is much less evidence available which shows that help helps, and most of that is not quantified, though there are exceptions, such as the work by Georgia Kayser from El Salvador. Recent work by IRC and CINARA at the Universidad del Valle has tried to quantify the relationship between post-construction support and sustainability of rural water supply in Colombia. Some of the findings finally provide the quantitative evidence to the assertion that help helps. But they also show that in fact “does help help” is the wrong question to ask.

Before presenting the highlights of the results, a couple of notes on what the study actually entailed. We looked at 40 rural water systems, all piped supplies with household connections, the standard type of supply in rural Colombia. Some of the community-based service providers who manage these systems were linked to a formal post-construction support mechanism, i.e. they receive regular support from an agency. We identified 7 of these mechanisms in the study area, including support by national, departmental and local government, as well as support by an NGO, an association of CBOs and by urban utilities. We looked at systems supported by each of them. In addition, we looked at a number of systems, which we believed not to be linked to any formal support mechanism. Sustainability of a rural water system is difficult to measure at a given point in time – you would need to study it over a time period. Hence we looked at both the service level, i.e. the characteristics of the water service people get (including water quantity, water quality, continuity, etc) and at the performance of the community-based service provider, i.e. how the CBO was doing in terms of its functions of administration, technical operation and maintenance and organisational management.

A first finding confirms work done by the World Bank: nearly all communities do get support. Even the ones we had selected as being “not supported” in fact did get such support. However, they do get it less frequently and from fewer agencies. In short, it comes down to the following. Communities not linked to a support model, do seek support when they need, for example from their municipality or another government agency or NGO operating in their area, and they do then get support in resolving a problem. Communities linked to a support model get a similar kind of support, but in a more formal way. Or, they are in a more regular support with their municipality, which helps them to anticipate problems. The municipality can then also source support from other agencies to address issues. For example, they may send the operator to a course at the vocational training centre, or source help from the environmental agency in catchment protection works. In short, they provide a more comprehensive support approach compared to the ad hoc support obtained in the communities not formally linked to a support model.

So what? One may say that as long as the problems get solved, it doesn’t really matter how the support is organised. Not exactly. The more comprehensive approach has the potential at least to anticipate problems before they become too big to handle. It may also lead to generally better and more professional management. Indeed, we did find that those systems with the more structural support have better performing CBOs, though the evidence was not statistically significant. One reason for that was that there was much difference in the quality of the support provided by the different models. Particularly, the CBOs with structural support scored better in their financial and organisational management. For example, they were found to have relatively low non-payment rates. No clear impact was found on the actual services delivered. The main reason for that is that we also didn’t find a very outspoken relation between the performance of the CBO and the level of service provided. A well-organised CBO doesn’t necessarily provide the best level of service; nor, do CBOs that perform relatively poorly provide the worst services. What did stand out though is that some support models had put lots of emphasis on water quality management, and the systems supported by them did score high on water quality indicators. These findings were similar to the results found in the same study by the World Bank (see below), where they also saw the impact of post-construction more in improvement if the administration by water committees than in the direct result of service delivery.

Finally, the study looked deeper into characteristics of the support that make a difference. What stood out there is that there is a positive correlation between the frequency of support and the performance of the service provider, and what could broadly be described as the quality of the support, which we tried to measure by factors such as whether the support agents use standardised methods and have staff with an adequate profile in their organisation.  Also a more multi-stakeholder approach to post-construction support was found to be more effective than providing support by one single agency. But, not all these factors were found to be statistically significant. Doing a more qualitative analysis of how support is organised may help generating an understanding of how the quality of support may impact on performance of service providers.

We concluded that help helps, but only to some extent. All community-based service providers do get help, but as they have different needs, not all the help they receive makes them improve performance, or improve service delivery, because not all support may be well-targeted to their needs, or the quality of the support is not adequate. This also means that for the future the research question should no longer be: does help help? Rather, the question should be what kind of help is most needed? And, how can support best be organised and targeted so that it does help? I hope that future research can delve into those questions.

For those who want to know more details on the current study results, it will be published soon, with kind support from the Inter-American Development Bank, who also supported the research in the first place.