This document discusses monitoring handwashing behavior. It begins by explaining the importance of shifting from solely monitoring infrastructure and outputs to also measuring behavior change, as behavior is necessary for health impacts. It then describes several indicators used to measure handwashing behavior, including proxy indicators, self-reported behavior, and direct observation, along with their strengths and limitations. Specifically, it discusses challenges with directly observing behavior at large scales but provides an example of how this was done through partnerships in Nepal. The document emphasizes the value of monitoring behavior to fully understand program impacts.
1. Monitoring
Handwashing
Behavior
Thursday, April 11th, 2013
Monitoring Sustainable WASH
Service Delivery Symposium,
Addis Ababa, Ethiopia
Jelena Vujcic
Dept. of Social and Preventive Medicine
University at Buffalo, Buffalo, NY, USA
jelenavu@buffalo.edu
2. Overview
• Why monitor behavior? Shifting paradigm for
handwashing programming and monitoring
• Current indicators used to measure
handwashing behavior
– Utility, strengths, limitations
• Measuring behavior at scale
3. Handwashing promotion at a glance…
Ben Nygren Pavani K. Ram http://www.unicef.org/wash/index_45948.html
5. A shifting paradigm in handwashing
programming/implementation
Supply-driven Demand-driven
Infrastructure focused Behavior-focused
Government and support Room for private sector to respond to
agents as main drivers of household demands
change
Government and support agents
facilitate communities’ change process
van der Voorden, Keynote paper
…and a shift in monitoring methods
and actors
6. Input Core human and financial ex. Number of health workers
resources required to develop the hired
program for implementation
Process Activities and efforts ex. Health worker visits to
Monitoring
implemented to achieve program homes
goals
Output Direct results of the efforts at the ex. Number of soap bars
program level distributed
Outcome Effects of program outputs at the ex. Increased proportion of
population level mothers who wash hands
with soap at critical times
Impact Effects of program at the ex. Reduced risk of diarrheal
population level with adequate disease
account of other factors
7. Why monitoring handwashing
behavior is important/relevant
Evidence from research
Knowledge only
Necessary but not sufficient for practice
Hardware only
• Old habits are hard to break, new habits are hard to make
Role of behavior in prevention, in health, in self
empowerment, in equity is clear
• Without behavior do we have a complete picture of what the
program achieved at the population level?
9. Indicators
PROGRAM
GOAL COMPONENT INDICATORS DATA COLLECTION METHOD
Advocacy Outputs (A1) Number of handwashing promotion advertisements distributed/broadcasted Program records/Media tracking
(A2) Number of handwashing promotion events Program records/Monitoring
(A3) Number of participants at handwashing promotion event(s) Program records/Monitoring
(A4) Number of stakeholders introduced to benefits of handwashing with soap Program records
Outcomes (A5) Recall of the event/advertisement Survey
(A6) Recall of the main message(s) from an event/advertisement Survey
Impact (A7) Progress toward commitments Program records
(A8) Number of commitments (funding, sponsorship, participation) Program records
Education Outputs (E1) Number of education related events Program records
Outcomes (E2) Knowledge of the benefits of handwashing with soap Survey
(E3) Knowledge the critical times for handwashing Survey
(E4) Soap use during a handwashing demonstration (also a proxy indicator of Behavior Change) Rapid observation
Impact (B2-6) Behavior change as measured by indicators listed below (see below)
Behavior Outputs (B1) Number of behavior change communication events Program records
Change (B2) Number of participants at behavior change communication events Program records
Outcomes (Proxy indicators)
(B3) Soap and water present together at a handwashing place Rapid observation
(B4) Soap present in the household Rapid observation
(B5) Hand cleanliness score (visual inspection of hand cleanliness) 3-pt. hand inspection
(Self-reported behavior)
(B6) Self-reported handwashing with soap at any critical event/at specific critical event
Self-report
(Direct observation of behavior)
(B7) Observed handwashing with soap and water at any critical event/at a specific critical event Structured observation
Impact Prevalence of illness during the 72 hours preceding interview (e.g. diarrhea, or respiratory illness) Morbidity survey
10. Indicators
PROGRAM DATA COLLECTION
GOAL COMPONENT INDICATORS METHOD
Behavior Outcomes (Proxy indicators)
Change Soap and water present together at a handwashing place Rapid observation
Soap present in the household Rapid observation
Used in (Self-reported behavior)
MICS/DHS Self-reported handwashing with soap at any critical event/at Self-report
specific critical event (questionnaire)
(Direct observation of behavior) Structured observation
Observed handwashing with soap and water at any critical
event/at a specific critical event
11. Limitations and Strengths
Indicator Limitations Strengths
(Proxy indicators) • As a proxy, cannot tell us • Relatively quick and simple
Soap and water present together at how often hands are to collect
a handwashing place washed, if soap or other • Easily incorporated in
materials are used to wash surveys
Soap present in the household hands, when hands are • Validated against direct
washed observation of behavior
• Scalable, used in MICS/DHS
(Self-reported behavior) • Over reports handwashing • Relatively quick and simple
Self-reported handwashing with behavior to collect
soap at any critical event/at specific
critical event
(Direct observation of behavior) • Resource and time intensive • Direct observation
Observed handwashing with soap • Hard to do at scale • Rich detail regarding
and water at any critical event/at a • Reactivity behavior
specific critical event
12. Is monitoring behavior feasible at
scale?
• Using proxy indicators at scale is feasible
– MICS and DHS indicators will give us key insights
to behavior at the global and regional levels
• Structured observations are difficult to scale,
however….
– Using representative sub-sets of the target
population to supplement proxy indicators
– Using a partnership approach can provide support
for larger scale (Nepal case study)
13. Direct observation of behavior at
large scale -Nepal Case Study
• National handwashing promotion program
– Supported by Nepal’s Private-Public Partnership for Handwashing
– Government-lead implementation
– Technical, resource and funding support from partners
– Capacity building, mass media, door-to-door visits at community level,
school- based promotion
• Clear behavioral objectives
– MICS/DHS handwashing indicators needed to be supplemented by
structured observation
• Robust monitoring and evaluation plan (UNICEF, UB)
– Structured observation in approximately 1,100 households across
geographical zones
14. Today’s discussion
• Value of monitoring behavior
• Examples from implemented programs
• Challenges
• Ways forward
• Gaps we still need to address
15. Resources
• UNICEF M&E module for programs the promote
handwashing
• Practical guidance for measuring handwashing
behavior (https://www.wsp.org/sites/wsp.org/files/publications/WSP-
Practical-Guidance-Measuring-Handwashing-Behavior-2013-Update.pdf)
• Global Public-Private Partnership for
Handwashing (http://www.globalhandwashing.org/handwashing-resources)
Editor's Notes
Good afternoon, everyone. Thanks for joining us in this discussion. My name is Jelena Vujcic, I’m researcher from the University at Buffalo in Buffalo, New York. I work as a part of a research team lead by Dr. Pavani Ram, who is one of the leaders in research on this topic I’m also here on behalf of the Global Private Public Partnerships for Handwashing which is a coalition of stakeholders that works explicitly to promote handwashing with soap and recognizes that hygiene, sanitation, and water are pillars of development. This is an introductory presentation for this session to start off the discussion around a topic that I feel is both buzzing around but is also causing some long sighs. *rephrase
The context handwashing promotion is diverse, to say the least. Typically, programs focus on household-level promotion (picture on left), school-based promotion (middle), and/or communal practice (right). It is common for handwashing promotion to be nested within programs that have larger overarching goals, for example WASH programs, CLTS, nutrition programs, disease prevention programs, etc. This also present an important opportunity to gather monitoring data on handwashing behavior and behavior change within these various contexts. And it’s important to have feasible methods to do so.
Each program may also have different goals. Some focus on advocacy, others on education, or a combination of the two, most aim to change or improve handwashing with soap, but the ultimate goal is usually to impact health and livelihood of communities.
[Talk through paradigm shift]. Now measuring behavior has become more relevant to understand whether or not programs are meeting their objectives. Outputs were not enough.
So when you break it down by program components, typical approach was to monitor inputs, processes and outputs, and leave outcomes and impact to evaluation. Where as now we are trying to shift here (play animation).
There will always be a behavioral component to many of the programs or health or equity issues everyone here is involved in. Even if we had vaccines for diarrheal diseases or respiratory illnesses which these programs address, WASH will always have a role in dignity and equity and behavior is at the core of that. What’s unsettling is how variable or messy understanding, changing and measure human behavior seems. Appreciating that, I still wondering to what extent we can keep circumventing behavior? And by doing so, will we have a complete picture of what our programs are actually doing and how well they are serving the people they are intended to serve? I’m speaking about population level effects as opposed to program level effects. Some of thing points we’re making today is that with the direction of the shift we discussed earlier for sanitation and hygiene programs, it won’t make much sense to do. But there are challenges and realities to measuring behavior and we’ll talk more about that and what we can or can’t do with what we have now.
The bad news is human behavior is generally difficult to measure. We see this with sanitation and measuring actual use, we see it in nutrition related behaviors, etc. The good news is there are indicators that have been employed in various capacities and have been tested and in some cases validated.
This is a comprehensive set of indicators that we put together for a monitoring and evaluation module for UNICEF programs that promote handwashing. It’s a lot to take in but I’d like to focus on the indicators that are relevant to behavior change.These are commonly used indicators for behavior change, plus there are other indicators to consider that measure components of behavior (such as knowledge, perceptions, attitudes) which are not included here but could certainly be useful for understand certain aspects of behavior, not necessarily actual practice.There are three general types of indicators: proxy indicators, self reported behavior, and observed behavior. Each come with a set of strengths and limitations. Proxy indicators or behavior give us clues about part related to that behavior. Such indicators cannot tell use how often a person washes hands, how often they use soap and at what times. However, They are quick and relatively simple to collect, and soap and water at a handwashing have been validated against observation of direct handwashing behavior. A few studies have found that those that have soap and water present at the place they most often wash hands are also more likely to be observe washing hands with soap during direct observation compared to those that do not have these materials or have just water at a handwashing place. These indicators have been instrumental in providing us with some indication of handwashing behavior at the global level. Both of the ones included here are used in MICS and/or DHS which make them relative comparable between years within the same region or country and between regions and countries. These indicators, albeit proxies, have enabled us to look at a handwashing indicators a much larger scales. I believe UNICEF and it’s partners will sharing global handwashing data soon.**supplementation using SOsSelf reported behavior is a common approach, it’s also relative simple to collect and take minimal training. However, there is good evidence that behavior tends to be over reported with this indicator. One study showed that only a small portion of those who said they washed their hands with soap at critical times (such as after using the toilet) where observed to do so in direct observation.Direct observation of behavior is the best indicator available at the moment. This is a direct observation of handwashing behavior where an observer is placed in the household or school for several hours and records whether hands were washed and what materials were use, along with other details, at specified critical times. It’s a very detail rich method. However, a couple studies have shown reactivity to the presence of the observer. This method is also resource intensive. It takes time and a well trained staff. Still, with this method that could give a somewhat inflated sense of handwashing behavior, we are seeing overall low rates of handwashing with soap.
This is a comprehensive set of indicators that we put together for a monitoring and evaluation module for UNICEF programs that promote handwashing. It’s a lot to take in but I’d like to focus on the indicators that are relevant to behavior change.These are commonly used indicators for behavior change, plus there are other indicators to consider that measure components of behavior (such as knowledge, perceptions, attitudes) which are not included here but could certainly be useful for understand certain aspects of behavior, not necessarily actual practice.There are three general types of indicators: proxy indicators, self reported behavior, and observed behavior. Each come with a set of strengths and limitations. Proxy indicators or behavior give us clues about part related to that behavior. Such indicators cannot tell use how often a person washes hands, how often they use soap and at what times. However, They are quick and relatively simple to collect, and soap and water at a handwashing have been validated against observation of direct handwashing behavior. A few studies have found that those that have soap and water present at the place they most often wash hands are also more likely to be observe washing hands with soap during direct observation compared to those that do not have these materials or have just water at a handwashing place. These indicators have been instrumental in providing us with some indication of handwashing behavior at the global level. Both of the ones included here are used in MICS and/or DHS which make them relative comparable between years within the same region or country and between regions and countries. These indicators, albeit proxies, have enabled us to look at a handwashing indicators a much larger scales. I believe UNICEF and it’s partners will sharing global handwashing data soon.**supplementation using SOsSelf reported behavior is a common approach, it’s also relative simple to collect and take minimal training. However, there is good evidence that behavior tends to be over reported with this indicator. One study showed that only a small portion of those who said they washed their hands with soap at critical times (such as after using the toilet) where observed to do so in direct observation.Direct observation of behavior is the best indicator available at the moment. This is a direct observation of handwashing behavior where an observer is placed in the household or school for several hours and records whether hands were washed and what materials were use, along with other details, at specified critical times. It’s a very detail rich method. However, a couple studies have shown reactivity to the presence of the observer. This method is also resource intensive. It takes time and a well trained staff. Still, with this method that could give a somewhat inflated sense of handwashing behavior, we are seeing overall low rates of handwashing with soap.
This is a comprehensive set of indicators that we put together for a monitoring and evaluation module for UNICEF programs that promote handwashing. It’s a lot to take in but I’d like to focus on the indicators that are relevant to behavior change.These are commonly used indicators for behavior change, plus there are other indicators to consider that measure components of behavior (such as knowledge, perceptions, attitudes) which are not included here but could certainly be useful for understand certain aspects of behavior, not necessarily actual practice.There are three general types of indicators: proxy indicators, self reported behavior, and observed behavior. Each come with a set of strengths and limitations. Proxy indicators or behavior give us clues about part related to that behavior. Such indicators cannot tell use how often a person washes hands, how often they use soap and at what times. However, They are quick and relatively simple to collect, and soap and water at a handwashing have been validated against observation of direct handwashing behavior. A few studies have found that those that have soap and water present at the place they most often wash hands are also more likely to be observe washing hands with soap during direct observation compared to those that do not have these materials or have just water at a handwashing place. These indicators have been instrumental in providing us with some indication of handwashing behavior at the global level. Both of the ones included here are used in MICS and/or DHS which make them relative comparable between years within the same region or country and between regions and countries. These indicators, albeit proxies, have enabled us to look at a handwashing indicators a much larger scales. I believe UNICEF and it’s partners will sharing global handwashing data soon.**supplementation using SOsSelf reported behavior is a common approach, it’s also relative simple to collect and take minimal training. However, there is good evidence that behavior tends to be over reported with this indicator. One study showed that only a small portion of those who said they washed their hands with soap at critical times (such as after using the toilet) where observed to do so in direct observation.Direct observation of behavior is the best indicator available at the moment. This is a direct observation of handwashing behavior where an observer is placed in the household or school for several hours and records whether hands were washed and what materials were use, along with other details, at specified critical times. It’s a very detail rich method. However, a couple studies have shown reactivity to the presence of the observer. This method is also resource intensive. It takes time and a well trained staff. Still, with this method that could give a somewhat inflated sense of handwashing behavior, we are seeing overall low rates of handwashing with soap.
These are commonly used indicators for behavior change, plus there are other indicators to consider that measure components of behavior (such as knowledge, perceptions, attitudes) which are not included here but would inform on certain aspects of behavior, not necessarily actual practice.There are three general types of indicators: proxy indicators, self reported behavior, and observed behavior. Each come with a set of strengths and limitations. Proxy indicators are quick and relatively simple to collect, but are still proxies of behavior. These indicators cannot tell use how often a person washes hands, how often they use soap and at what times. However, soap and water at a handwashing have been validated against observation of direct handwashing behavior. A few studies have found that those that have soap and water present at the place they most often wash hands are also more likely to be observe washing hands with soap during direct observation compared to those that do not have these materials or have just water at a handwashing place. Both of the ones included here are used in MICS and/or DHS which make them relative comparable between years within the same region or country and between regions and countries. These indicators, albeit proxies, have enabled us to look at a handwashing indicators a much larger scales. I believe UNICEF and it’s partners will sharing global handwashing data soon.Self reported behavior is a common approach, it’s also relative simple to collect and take minimal training. However, behavior tends to be over reported with this indicator. One study showed that only a small portion of those who said they washed their hands with soap at critical times (such as after using the toilet) where observed to do so in direct observation.Direct observation of behavior is the best indicator available at the moment. An observer is place in the household or school for several hours and records whether hands were washed and what materials were use, along with other details, at specified critical times. It’s a very detail rich method. However, a couple studies have shown reactivity to the presence of the observer. Still, with this method that could give a somewhat inflated sense of handwashing behavior, we are seeing overall low rates of handwashing with soap. This method is also resource intensive. It takes time and a well trained staff.
** show as example of supplementing with MICSThis Fall they plan to roll out a 5 year National handwashing promotion program in a phased manner. They have 3 main components to their program: 1) Capacity building 2) Social mobilization and media mobilization, 3) community and school level outreach. [ say objectives, and target population]This is a pretty comprehensive program. The monitoring of the program in it’s entirety including sustainability is very involved but today I’m focusing on monitoring improvements in handwashing behavior.