Comparing of country scores after assessments of WASH systems can be both tricky and attractive.
Published on: 29/04/2019
The paper ‘WASH in districts through IRC’s systems lens’ by Seleen Suidman and René van Lieshout gives a first insight into the results of the assessments of the WASH systems done in Ethiopia, Ghana and Uganda, using the traffic-light scoring method. The paper aims to provide context-specific explanations for the scores.
Since 2017, IRC has started developing and using a framework for monitoring changes in the WASH systems in its focus countries. These regular assessments of the water, sanitation and hygiene sectors are used for learning and programming. Scorecards are used to rate the relative strengths of the systems approaches and building blocks of the system. An advantage of this method is that the results of the assessments can be easily visualised and communicated to larger audiences. At the same time, the traffic-light type visualisations carry the risk that sector stakeholders such as a District Health Officer or the Water Resources Ministry will feel judged and will disagree with the assessments and even more with the fact that they are published.
It is therefore important to ensure that the assessments are owned by the sector before they are made public. Ideally, the assessments are carried out as part of the sector planning and learning processes in a participatory manner. For example, as part of the master planning process in our partner districts. Another way of creating ownership of the assessments is by asking the sector to validate the results of the assessments. In addition, the credibility of the scores is improved by providing narrative justifications of the scores, the sources of information and methods of scoring and validation.
The paper states clearly that the purpose of the scoring is not to compare between countries, because the assessments are done by stakeholders from their specific perspective and influenced by the focus of the programme: they are subjective and context specific. At the same time, when the results of assessments for different countries are presented together, like is done in the paper, this automatically triggers questions that compare the scores. The IRC website will soon be sharing these assessments and will therefore not show results across countries on one page. The above shows how these traffic-light visualisations can be both attractive and tricky at the same time.
At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.
