Published on: 15/03/2013
Reflections by Dr Christine Sijbesma and Mahjabeen Ahmed on the QIS monitoring system.
This is the very first blog in my life. Since many younger colleagues are blogging away, I have been inspired to give it a try. The subject I'd like to bring up is gender mainstreaming in the set-up of the Qualitative Information System used for monitoring.
Over the last 12 months IRC and BRAC have developed, and BRAC has implemented, a performance monitoring system to measure behavioural change of a large WASH programme. Large because it aims at reaching 37 million people. The behavioural change concerns sanitation and hygiene at the household level, but also in programme management (village WASH committees) and in the set-up of sanitation enterprises.
The crux of the QIS monitoring system lies in the development of a series of what we call "mini-scenario scales" which involve 3 workshops and 2 pilots before going to scale. In each of the mini-scenarios, level 0 at the bottom represents non-performance (e.g. a village water committee does not hold meetings) and the top, level 4 is the ideal performance, and the levels in between describe the steps required to go through till the ideal scenario is reached.
Village WASH Committee INDICATOR 2: PERFORMANCE OF VWC
IDEAL: (1) Committee (male and female members) meets every 2 months + (2) maintains list of decisions and meeting minutes + (3) identifies gaps and takes action+ (4) mobilizes ADP funds for hard core poor
(1) Committee (male and female members) meets every 2 months + (2) maintains list of decisions and meeting minutes + (3) identifies gaps and takes action
BENCHMARK: (1) Committee (male and female members) meets every 2 months + (2) maintains list of decisions and meeting minutes
(1) Committee (male and female members) meets every 2 months
No full VWC OR VWC exists but does not meet
Reason(s) why score is high/not high:
In the scoring system, gender is mainstreamed in several ways. First, some scales are gender-specific themselves. The ideal scenario for latrine use, for example, is that the latrine is used by all family members, female and male, adult, adolescent, young, old, including by those with special needs and that excreta of babies and infants also end up in it. In many cultures women and girls are the first to adopt the use of latrines and it takes some time before children, adult men and adolescent boys also adopt this behaviour. Thus, climbing the toilet use mini-scenario ladder generally takes time and is gender sensitive. This is also an outcome of the QIS when discussing who in the families use the toilet consistently. Since this is the first annual monitoring there is still a substantial percentage of households where latrine use by children, and even more by men and adolescent boys is a problem.
Since this is the first annual monitoring there is still a substantial percentage of households where latrine use by children, and even more by men and adolescent boys is a problem.
It will be interesting to see if this "hard to reach" change in the second monitoring round, especially now that the programme also organizes hygiene promotion for men. The scores allocated to each of the scales are based on evidence, that is, there has to be some kind of proof for giving a specific score. This can come from structured observation of a toilet, from the minutes of the last three village committee meetings or by asking further questions in such a way that it is easier to disclose more sensitive behaviour, a technique also known as probing. Both the scales and the scores have been formulated together with those who know the programme inside out and were tested and adjusted in field trials before taken to scale.
There is also a stand-alone gender scale (or mini-scenario) on women and men. This scale is about the functioning of the Village WASH Committees. In my long years in the WASH sector I have often experienced that at these meetings women, if present, sit silently at the back or do the catering and the men talk and decide. Well, that is better than not being there at all, because then they are at least informed. But in a truly integrated committee all members can and will speak up and decisions are made by the majority or through consensus irrespective of sex, age and class.
We agreed that if we wanted reliable data for the QIS mini-scenarios, the scoring needed to be done in separate sub-groups for the male and female committee members and that each group had to be facilitated by a monitoring team member of the same sex. Both sub-groups then met to compare scores and if different, discuss what this meant and what to do about that. The interesting thing in this monitoring round is that for virtually all VWCs in the sample the male and female members gave the same scores. This does not mean that there are no different scores on other parameters of gender equality in VWCs, but views of women and men hardly differed concerning gender aspects of the village WASH committees.
When we discussed these results, one hypothesis was that women and men give the same scores is an effect of the intensive and equal training, in pairs of 2, for all committee members, and the intensive support thereafter. Another possible explanation is that the scoring process has not been gender-sensitive enough. While scoring was done in two sub-groups, the groups may not always have sat far enough from each other to speak freely and this may have influenced the sex-specific scores. Perhaps more will come out of the analysis of the reasons that the VWCs have given for their low or high scores on gender – we still need to analyze these. But there is also a simple way to check this in the next round: the teams can send a photo with the setting of each sub-group along with the scores. This is easy to do, since the teams use smartphones for monitoring and a picture of the settings can be uploaded along with the scores.
The smart phones themselves are the last item that is part of the programme's gender mainstreaming. BRAC WASH Programme decided to give each monitoring team of 1 man and 1 woman one single phone. This caused some of us to fear that in practice especially the men would handle the phones. I checked this concern with a number of the women team members and according to them a simple rule helped prevent this: each team had to fill in a hard copy and a virtual copy with the scale scores and compare notes before sending the data through the internet. These tasks had to be alternated between the man and the woman in each team. This rule not only ensured that both sexes used the phone equitably, but also increased the reliability of the data, such as by taking out errors from pressing the wrong digit in the phone. A simple rule of double entry with double benefit!
We are now at the first workshop where we are reviewing the outcomes of the 2012 monitoring round.
We are now at the first workshop where we are reviewing the outcomes of the 2012 monitoring round. We are also discussing how we shall use these further to make the necessary programmatic changes. We are also learning about the QIS methodology itself and where and how it can be further developed. One thing is clear, though: QIS is a valuable addition to our Monitoring Information System (MIS) because it adds quantitative data on outcomes to our quantitative data on inputs and outputs.
Dr. Christine Sijbesma is an IRC Associate. Mahjabeen Ahmed is a Senior Sector Specialist Water, Sanitation and Hygiene of the BRAC WASH II Programme.
At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.