Monitoring: critical mass or critical mess?

Published on: 10/09/2011

One of the issues observed and discussed during the study was monitoring of water and sanitation services. In this small municipality, water and sanitation services are monitored in at least 6 different ways:

1. Water committees (juntas de agua in Spanish) check their water systems on a daily basis, and review the administration (including issues such as account balances and default rates) also with a certain periodicity. This is done mainly through observations without any indicators, formats or whatsoever.
2. The juntas de agua in the municipality have formed an association (called AJAM), which acts as a coordination body between them and the municipality. The association also regularly monitors performance of its members and provides support. For example, if they find one of the members is not chlorinating, they try and help it with the dosification of chlorine. Again, no systems or indicators are used in this monitoring; it is purely based on the AJAM’s own experience.
3. A next institutional level up is the municipal water and sanitation technician. It is already a big feat that the municipality has such a technician; most rural municipalities in Honduras don’t. He is supposed to monitor the performance of all 15 water systems in the area. He doesn’t have any standard format or so for this either. He doesn’t visit the systems on a routine basis, but only goes and checks when there is a reported problem and tries to provide assistance.
4. There is also an Operation and Maintenance Technician (TOM), a staff member of SANAA (the national water  supply agency) who operates at municipal level tasked with post-construction support. This support is supposed to be provided based on regular monitoring, for which SANAA has a well-developed information system, called SIAR (Rural Water Supply Information System). Even though this may have some gaps, it is a reasonably good system, with relevant information on water supply performance,  upon which a TOM should be able to act. However, the SIAR is not regularly updated. Even though WFP has engaged him fully over the last few years in the municipality, he hasn’t updated the SIAR. Whether he will regularly monitor in the future remains to be seen. This is not unique to Chinda. Over the last few years, SANAA  has been seriously under-funded and the routine monitoring and support, for which it got famous, has  stopped.
5. There are line agencies, like the Health Secretariat and the Education Department who are also supposed to monitor certain bits and pieces related to water and sanitation, including specifically water quality tests and checking the status of WASH at schools. Reality is that this happens in a haphazard way, if and when the technicians of these line agencies have some money to cover their transport costs.
6. Finally, ERSAPS, the national regulator also keeps a database of all service providers, including rural juntas de agua. Every year a report is sent to the regulator with basic info on the juntas in the area.

Looking at the above, one would expect a critical mass of monitoring activities, to help identifying activities to improve performance at different levels, from the juntas de agua up to higher institutional levels. And indeed it is much better than the situation found in many other countries where there is no monitoring at all, or where it is limited to monitoring project implementation and not ongoing service delivery. Instead, it is more of a critical mess.

The informal, unstructured monitoring by service providers may be adequate to them, but there is a critical gap in the more formal monitoring systems by municipalities and national line agencies (arguably these do not merit the name monitoring, but rather reporting as at the moment little action is taken on the basis of data). This is made worse by the fact that these formal systems use similar, but slightly different  indicators. Reasons why this situation has evolved as it has, are manifold, and include for example changes in the mandate of an agency like SANAA, and with that a change in the use of tools like the SIAR. This example poses broader questions on the challenges in monitoring rural water and improving their performance. Where do some of the possible solutions lie?

• Use of ICT tools. Over the past few years a number of new tools have become available for mapping and monitoring water and sanitation services. These include amongst others: Water Point Mapper, h2.0 Monitoring Service to Inform and Empower Initiative  and FLOW, Field Level Operations Watch. The latter has been launched by Water for People. As it is planned to be put to use in Chinda only in a few weeks, its functioning couldn’t be assessed in this study. The obvious advantage of these kinds of tools is that they allow for a much cheaper way of carrying out monitoring and updating databases. The risk, though, is that it will end up being monitoring system number 7, adding to the already existing sets of indicators and monitoring systems. For it to be a useful technology, it should use sets of indicators that are used in the country (in this case either the ones of SIAR or of ERSAPS), and it should be clarified where responsibility of using and updating the information lies: with the juntas de agua, the municipal technician or one of the line agencies. The same applies obviously to any of the other tools mentioned above.
• Indicators. Another approach being promoted, amongst others by Triple-S, is to focus more on the sets of indicators themselves than on the technology for data collection. In several countries we have now been developing indicators for monitoring 1) service levels, 2) performance of service providers and 3) performance of service authorities (municipalities). Although they all follow a similar structure they are adjusted to the context of each country. For example, last week we worked on such a set of indicators in Colombia, which speak to the Colombian legal framework for community-based management and common service levels. In the various countries, this is a process of defining indicators with the relevant national authorities, to make sure the indicators are adopted nationally, and not remain a pilot. However, this approach doesn’t address some of the other problems, mentioned above on roles and responsibilities in monitoring, or the funding for monitoring activities by technicians.
• Other initiatives focus on developing large sector information systems, which use locally adopted indicators, may have fantastic  data collection tools, but many of which remain empty. Or, a mapping exercise is done once and then the system is updated infrequently, because there is nobody responsible for that, or there is no budget. This is the current situation with the SIAR. In other cases, the databases demand to much detailed information. In Colombia, a big bone of contention has been that rural community-based service providers had to provide information on literally hundreds of indicators to the national regulator. This made reporting too cumbersome and counterproductive, and many opted for not reporting anything. Whereas all these elements may offer part of the solution, the fundamental issue seems to be, what I would call, the governance of monitoring. This includes the way roles and responsibilities for monitoring are defined, how these are financed, how indicators are harmonized, or not, between different agencies, and how the monitoring is linked to post-construction support. As a colleague said last week “if there is no post-construction support mechanism in place, why do monitoring?”. We would then know the problems but nobody would be responsible for supporting water committees in addressing them. Addressing these governance issues will be hard and messy. This will touch upon vested interests; on institutions with their own systems that they do not want to let go; it will reveal in many places that post-construction support is not happening, even if it should according to policies and laws; and, it will probably show that in many places budgets are just way too low to carry this out as a routine task. Yet, without addressing the governance of monitoring, the situation will remain even messier.

More initiatives will be undertaken to address it, with the associated risks of duplication of efforts, parallel systems, or systems that will be an empty shell. Therefore efforts to introduce new monitoring technologies, sets of indicators or sector information systems, do also need to address the key related governance issues, to create the critical mass for better monitoring and support to rural water supply services and add to the mess there is already.