- What we do
- Where we work
This blog looks at the experience of using 'Payment-by-Results' in Tanzania's water sector, and how adaptive programming is key to bridging the gap.
Payment-by-Results as an incentive for local government performance
The UK’s Department for International Development (DFID) has worked with the Tanzania’s Ministry of Water and Irrigation (MoWI) over the past four years to develop a results based financing scheme based on the idea that Local Government Authorities (LGAs) could obtain much needed resources.
The scheme awards LGAs additional funding ‘bonuses’ based on improvements in local service delivery, and so is known as ‘Payment-by-Results’ (PbR). LGAs were benchmarked and rewarded against the functionality of water points with the hope that they would be motivated to increase their efforts on maintenance, and hence improve overall operational sustainability of water points in their jurisdictions. To date, however, DFID has only disbursed 6% of committed resources under PbR. With roughly 40% of Tanzania’s rural water points in need of repair, it wasn’t that this money wasn’t needed. So, what happened?
Reliable data and monitoring system essential for Payment-by-Results
The PbR programme requires a rigorous baseline against which to measure progress, and a reliable monitoring system which can record progress on outcomes.
From the beginning, PbR aimed to rely on government’s system for reporting, monitoring and measuring progress on outcomes to trigger disbursements. However, when the independent verification of baseline data was carried out in 2014 (at the beginning of the programme), it was found that the level of reporting and data reported by LGAs was insufficient and of poor quality to use for PbR. This was not what either DFID or MoWI had expected. Very few districts reported timely and reliably; and there were questions whether the national water point inventory was complete, correct and accurate to award PbR payments to LGAs.
A decision was to be made: either to close the programme or delay the roll-out of PbR to allow the government to establish a credible verifiable baseline on which to base the PbR scheme.
Complementing PbR with adaptive programming
Since there is limited evidence on PbR in the water sector, the scheme was designed to be adaptive and flexible to respond to a changing context with the aim to strengthen the programme’s ability to achieve results, and generate learning. MoWI was also given flexibility to draw in technical assistance based on their needs.
This allowed flexibility to adapt the programme during implementation as evidence from performance evaluation and data verification findings kick in. For example, to address the reporting and data quality challenge, a decision was made to delay roll-out of PbR for a while and technical assistance (Capacity Building Service Provider) was brought in to work with MoWI to establish a credible baseline. This culminated in the establishment of the Central Data Management Team (CDMT) at MoWI responsible to ensure effective management of data and reporting by local governments.
Concurrently, LGAs eligibility criteria for the PbR scheme were expanded to include specific deliverables relating to the improvement of reporting and data quality, since data accuracy was fundamental for PbR to proceed, and even for future performance measurements and the related disbursements. A Qualification Support Grant (QSG) was also introduced to those LGAs who fail to meet the reporting and data quality criteria to help them improve data and reporting to qualify for subsequent phases of PbR.
These changes led to improved levels of reporting and data quality to an acceptable threshold for proceeding with PbR in subsequent years. Notably, the monthly reporting rate of districts’ water point inventory status increased from 26% to 99% within the first year and has been consistently maintained at over 95% for the past two years. Significant progress has also been made to improve completeness of the inventory, recovering over 30,000 additional water points (across 181 LGAs) which were missing from the national inventory. By April 2016 (when PbR resumed), the 57 LGAs qualified for the first phase, followed by 129 LGAs in the second phase of PbR (2017). In fact, all LGAs which received QSG in the first round, qualified for the second phase of PbR.
The measurement challenge
One of the puzzles of implementing the past two rounds of PbR was on building consensus amongst stakeholders on how to measure and reward PbR incentives in a way which reflects the achievements of the recipients. Essentially, PbR needs to observe the level of water point functionality achieved in each recipient LGA. This is not a trivial task, because external verifications cannot feasibly check every water point.
Key issues to agree on were: what is success and how can it be measured? What qualifies as a functional water point? For instance, what if the flow is satisfactory, but taste is not? And what is a water point, anyway? If two taps come out of one pipe, are they one water point, or two? And what if it takes a water point 30 minutes to fill a 5 litre jerry can with water? Without consensus, each stakeholder may have their own answers to these questions, be it DFID, districts, the central government, or the firm sent to independently verify water point functionality. Also, the water sector faces specific measurement challenges: with over 120,000 rural water points in mainland Tanzania, reporting can be cumbersome and verifying their functionality yearly is a complex task, requiring expensive sampling methods.
Complementing adaptive programming with adaptive evaluation and technical assistance
The adaptive nature of the water PbR has been heavily supported by a partnership with the World Bank’s Development Impact Evaluation (DIME) unit from the beginning of the programme, as the research and evaluation have been embedded throughout the implementation.
For example, DIME’s process evaluation of PbR in year 1 found that while there is a strong awareness among District Water Engineers (DWEs) of PbR’s high-level objective of promoting rural water maintenance, most of them could not clearly explain the PbR eligibility criteria, incentive structure, and verification results.
To address these shortcomings, the MoWI with the support of the Capacity Building Service Provider (CBSP) developed a communication strategy on PbR eligibility rules and incentives for stakeholders, and provided feedback on verification results to each LGA through targeted training to regional and district teams.
In addition, DIME and CBSP worked in collaboration with DFID and MoWI to review and simplify the PbR incentives and payment structure thereby making it easier to anticipate for recipients. The verification methodology has also been strengthened and streamlined to align with local government data management and reporting procedures. These improvements will be applied starting in year 3 of PbR (2018) and are expected to have a positive knock-on effect to data improvement, qualification of LGAs and amount of funds (bonuses) that LGAs, regions and MoWI will receive.
Learning: PbR and adaptive programming as lever to resolving fundamental challenges
It is still too early to claim that PbR is improving maintenance of rural water points in Tanzania. However, it is safe to say that PbR has created a strong incentive for government to increase the level of political will to systemically address the rural water sustainability challenge, with the intention to maximise its bonuses for the subsequent phases of the project. For instance, with only 6% of total disbursements, PbR provided a strong vehicle to improve data systems through review of various processes and systems within the Ministry of Water and Irrigation, and training district water teams around a cohesive set of definitions for water points and functionality, and to report on them monthly at an exceptionally large scale. These step-wise changes are fundamental to developing an operationally efficient service delivery system, critical to addressing long term sustainability of rural water services in Tanzania.
Besides, progress recorded to date could not have been achieved if PbR and adaptive programming were implemented in isolation. In fact, PbR has provided the operational freedom and the political space for the adaptive approach to work in the best possible way; and the adaptive programming has provided the feedback necessary to learn and adapt the performance measurement and disbursement system to best promote positive change within the sector. The two elements form the core innovation of the programme that is rare to find in a large scale development programme.
Credit: Daniel Rogger (DIME) and Chris Brown (CBSP) for providing data and information for this blog.
Disclaimer: Lukas Kwezi currently works for the UK Department for International Development (DFID) as Water and Sanitation Adviser, based in Dar es Salaam. He writes blog posts in his spare time. Though he may talk about the work he does in the sector, this is neither a corporate nor a political blog and the opinions and ideas expressed here are solely his own, not those of his employers.
At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.