This document summarizes the interim findings of a case study conducted in Kenya to discuss accountability and project quality with Christian Aid's partner UCCS. Over the course of a week, the researcher held meetings with community groups and management committees across several locations. Participatory exercises were used to assess accountability mechanisms and their contribution to project quality. Preliminary findings indicated that information sharing and participation were strong, but feedback and complaints saw weaker understanding. Accountability was linked to higher trust, ownership and motivation. Issues identified for the research included adapting the methodology for development projects and ensuring facilitation supported participation.
Dear Kathleen Flowers,
Congratulations! The Hawaii International Conference on Education is pleased to inform you that your submission, “CONSORTIAL COLLABORATION AND THE CREATION
OF AN ASSESSMENT INSTRUMENT FOR COMMUNITY-BASED LEARNING”, has been
accepted for presentation at the 16th Annual Hawaii International Conference on Education to
be held from January 4 to January 7, 2018 in Honolulu, Hawaii. The decision to accept your submission was based on a peer review process.
Presentation at 2014 Canadian Knowledge Mobilization Forum, Saskatoon, Saskatchewan, June 9-10, 2014: The Next Generation: Students and Young Professionals in Knowledge Mobilization by
Elizabeth Shantz
Knowledge Mobilization Officer
Canadian Water Network
This presentation describes the process of a planit-waste workshop.Planit-waste models a county with 3 districts, all with different socio-economic, environmental and waste profiles. The 5 are judged on sustainability, recycling targets, landfill allowances, waste arisings and viability. Planit-WASTE has been used in 3 regions to bring partnerships together and to consider waste in sustainable terms.
Dear Kathleen Flowers,
Congratulations! The Hawaii International Conference on Education is pleased to inform you that your submission, “CONSORTIAL COLLABORATION AND THE CREATION
OF AN ASSESSMENT INSTRUMENT FOR COMMUNITY-BASED LEARNING”, has been
accepted for presentation at the 16th Annual Hawaii International Conference on Education to
be held from January 4 to January 7, 2018 in Honolulu, Hawaii. The decision to accept your submission was based on a peer review process.
Presentation at 2014 Canadian Knowledge Mobilization Forum, Saskatoon, Saskatchewan, June 9-10, 2014: The Next Generation: Students and Young Professionals in Knowledge Mobilization by
Elizabeth Shantz
Knowledge Mobilization Officer
Canadian Water Network
This presentation describes the process of a planit-waste workshop.Planit-waste models a county with 3 districts, all with different socio-economic, environmental and waste profiles. The 5 are judged on sustainability, recycling targets, landfill allowances, waste arisings and viability. Planit-WASTE has been used in 3 regions to bring partnerships together and to consider waste in sustainable terms.
Planning the future of openness in Scotland Niamh Webster
The Scottish Government have been working with the civil society network in Scotland to co-create the next Action Plan on open government. We wanted to hear what people across the country wanted to see in terms of having a government taht is more open, transparent, involves people and helps them to hold government to account. We hosted public discussion events and engagement over the summer - these slides were used at the events, and also include summary of all the key themes we heard from people. This will feed into the creation of the Scottish Open Government Partnership Action Plan for 2018-20, due to be published Autumn 2018.
Opportunities for local people to hold NGO’s to account for their actions have improved in recent years, but there has been little evidence to suggest that they can actually influence the quality and results of aid itself - until now.
This report provides concrete evidence of the way accountability mechanisms improve the value for money, effectiveness, relevance, and sustainability of humanitarian and development projects.
Evaluating a community media approach to promote nutrition in NigerJSI
A presentation from the SBCC Summit in Addis Ababa, Ethiopia, February 2016, presented by Peggy Koniz-Booher.
The collaboration was established between SPRING/Digital Green and 3 USAID/Niger programs - REGIS-ER (NCBA CLUSA), LAHIA (Save the Children) & Sawki (Mercy Corps)
Focused on the development and dissemination of 10 videos by community facilitators working with 4 distinct groups in each participating village.
The presentation covers the approach to community video and lessons learned from the activities.
ECEEE summer study 2011 presentation on using social media to promote energy efficiency research in New Zealand. Panel 8 - Dynamics of Consumption (which I co-led with Michael Ornetzeder)
Professor Elizabeth Waters, Coordinating Editor of the Cochrane Public Health Review Group & Melbourne School of Population Health, University of Melbourne
Knowledge Management Strengthens Decision Making: The Case of the “Knowledge ...CIAT
Under the REGATTA Initiative (Regional Portal for Technology Transfer and Action against Climate Change in Latin America and Caribbean) the project seeks to develop actions towards vulnerability and adaptation to climate change assessment in agriculture and water resources in the Andes of Colombia, Ecuador and Peru.
The implementation 'black box' and evaluation as a driver for change. Presentation by Katie Burke and Claire Hickey of the Centre for Effective Services.
Searching for outcomes in rural Tanzania: Harvesting directly from those infl...John Mauremootoo
The benefits and challenges of using Outcome Harvesting to evaluate a short-term intervention are explored using the example of an 18 month social change project supported by the UK Department for International Development in Tanzania. The project was that was highly ambitious: it sought to influence changes in gender attitudes and behaviour of the general public in Tanzania. Challenges included the lack of outcome indications in project document and the lack of knowledge of outcomes among project personnel. Outcome Harvesting was adapted to allow the harvesting of outcomes using focus groups of those the project sought to influence directly. The concept of ‘proto-outcome’ was used for suggestions of attitude changes that may lead ultimately to behaviour changes. Substantiation of outcomes involved not only third parties but direct observation. The resulting descriptions of outcomes and the evaluation findings proved valuable for learning in the organisation, Search for Common Ground.
There are many examples of evidence-informed decision making (EIDM) among public health professionals and organizations in Canada. However, there are limited mechanisms in place to facilitate the sharing of these stories within the public health community. The National Collaborating Centre for Methods and Tools (NCCMT) seeks to address this gap with an interactive, peer-led webinar series featuring a collection of EIDM success stories in public health.
These success stories will illustrate what EIDM in public health practice, programs and policy looks like across the country.
Join us to engage with public health practitioners across Canada as they share their success stories of using or implementing EIDM in the real world. Learn about the strategies and tools used by presenters to improve the use of evidence.
Building a cultural foundation for EIDM: An evaluative thinking communications campaign
Kristin Beaton, Huron County Health Unit
In an effort to build evaluation and evidence-informed decision making capacity, Huron County Health Unit has implemented several strategies to encourage evaluative thinking. Learn more about how this health unit built a learning organizational culture.
Testing integrated knowledge translation processes to improve the participation of children with disabilities in leisure activities in British Columbia
Dr. Ebele Mogo and Dr. Keiko Shikako-Thomas, McGill University
To improve policies on physical activity promotion for people with disabilities, this team undertook a project to bridge the evidence to policy gap. Learn more about how a community forum and policy dialogue were used to help bridge this gap and inform policymakers about evidence.
Planning the future of openness in Scotland Niamh Webster
The Scottish Government have been working with the civil society network in Scotland to co-create the next Action Plan on open government. We wanted to hear what people across the country wanted to see in terms of having a government taht is more open, transparent, involves people and helps them to hold government to account. We hosted public discussion events and engagement over the summer - these slides were used at the events, and also include summary of all the key themes we heard from people. This will feed into the creation of the Scottish Open Government Partnership Action Plan for 2018-20, due to be published Autumn 2018.
Opportunities for local people to hold NGO’s to account for their actions have improved in recent years, but there has been little evidence to suggest that they can actually influence the quality and results of aid itself - until now.
This report provides concrete evidence of the way accountability mechanisms improve the value for money, effectiveness, relevance, and sustainability of humanitarian and development projects.
Evaluating a community media approach to promote nutrition in NigerJSI
A presentation from the SBCC Summit in Addis Ababa, Ethiopia, February 2016, presented by Peggy Koniz-Booher.
The collaboration was established between SPRING/Digital Green and 3 USAID/Niger programs - REGIS-ER (NCBA CLUSA), LAHIA (Save the Children) & Sawki (Mercy Corps)
Focused on the development and dissemination of 10 videos by community facilitators working with 4 distinct groups in each participating village.
The presentation covers the approach to community video and lessons learned from the activities.
ECEEE summer study 2011 presentation on using social media to promote energy efficiency research in New Zealand. Panel 8 - Dynamics of Consumption (which I co-led with Michael Ornetzeder)
Professor Elizabeth Waters, Coordinating Editor of the Cochrane Public Health Review Group & Melbourne School of Population Health, University of Melbourne
Knowledge Management Strengthens Decision Making: The Case of the “Knowledge ...CIAT
Under the REGATTA Initiative (Regional Portal for Technology Transfer and Action against Climate Change in Latin America and Caribbean) the project seeks to develop actions towards vulnerability and adaptation to climate change assessment in agriculture and water resources in the Andes of Colombia, Ecuador and Peru.
The implementation 'black box' and evaluation as a driver for change. Presentation by Katie Burke and Claire Hickey of the Centre for Effective Services.
Searching for outcomes in rural Tanzania: Harvesting directly from those infl...John Mauremootoo
The benefits and challenges of using Outcome Harvesting to evaluate a short-term intervention are explored using the example of an 18 month social change project supported by the UK Department for International Development in Tanzania. The project was that was highly ambitious: it sought to influence changes in gender attitudes and behaviour of the general public in Tanzania. Challenges included the lack of outcome indications in project document and the lack of knowledge of outcomes among project personnel. Outcome Harvesting was adapted to allow the harvesting of outcomes using focus groups of those the project sought to influence directly. The concept of ‘proto-outcome’ was used for suggestions of attitude changes that may lead ultimately to behaviour changes. Substantiation of outcomes involved not only third parties but direct observation. The resulting descriptions of outcomes and the evaluation findings proved valuable for learning in the organisation, Search for Common Ground.
There are many examples of evidence-informed decision making (EIDM) among public health professionals and organizations in Canada. However, there are limited mechanisms in place to facilitate the sharing of these stories within the public health community. The National Collaborating Centre for Methods and Tools (NCCMT) seeks to address this gap with an interactive, peer-led webinar series featuring a collection of EIDM success stories in public health.
These success stories will illustrate what EIDM in public health practice, programs and policy looks like across the country.
Join us to engage with public health practitioners across Canada as they share their success stories of using or implementing EIDM in the real world. Learn about the strategies and tools used by presenters to improve the use of evidence.
Building a cultural foundation for EIDM: An evaluative thinking communications campaign
Kristin Beaton, Huron County Health Unit
In an effort to build evaluation and evidence-informed decision making capacity, Huron County Health Unit has implemented several strategies to encourage evaluative thinking. Learn more about how this health unit built a learning organizational culture.
Testing integrated knowledge translation processes to improve the participation of children with disabilities in leisure activities in British Columbia
Dr. Ebele Mogo and Dr. Keiko Shikako-Thomas, McGill University
To improve policies on physical activity promotion for people with disabilities, this team undertook a project to bridge the evidence to policy gap. Learn more about how a community forum and policy dialogue were used to help bridge this gap and inform policymakers about evidence.
It wouldn’t be KMb without KB - Insights into the role of knowledge brokers in supporting child and youth mental health and addictions communities of interest in Ontario
by: MaryAnn Notarianni and Angela Yip
9-10 June 2014
Canadian Knowledge Mobilization Forum
Saskatoon, SK
Accountability impact research - results from the kenya pilot study
1. Accountability Research
Kenya case study
Discussion on interim findings with Christian Aid in Kenya
Andy Featherstone, 25 February 2013
2. Itinerary
Day Activity
Sunday Contextualising methodology
induction with translator
Monday Meeting with UCCS
Trial exercise with community – Kyawango location
Tuesday Meeting with Prog Mgt. Com. – Itoleka location
Meeting with mixed group
Wednesday Meeting with PMC – Mutulu location
Meeting with mixed group
Thursday Meeting with PMC – Kalawani location
Meeting with mixed group
Friday Meeting with PMC – Kithungu location
Meeting with mixed group
Saturday Meeting with 4 groups – Ukanga location
Return to Nairobi
3. Context and partner profile
• Makueni County in Eastern
Kenya
• Communities practice small
scale rain fed agriculture
• Unreliable and erratic
rainfall associated with
water scarcity has led to
increased frequency of
droughts
• UCCS is CA‟s main partner in
the East and has been
operating in Kitui Machakos
and Garissa since 1987
• Action plans developed out of the PVCA process led to the prioritisaiton of
a project partnering the community to construct sand dams that will not
only serve the livestock and crops but also help in shortening the distance
that people take to access water for domestic use, particularly women.
4. The approach
A methodology was developed which mixed quantitative
and qualitative approaches and was used fairly
consistently throughout the research
Scorecards and opinion ranking exercises were used to
describe accountability mechanisms and as entry points
to wider discussions about their contribution to project
quality
All of the exercises were translated into Kikamba and
the discussions were facilitated by a translator
Each community meeting lasted between 2.5 – 3.5 hours
5. Participation
Total male
participants = 63
Total female
participants = 87
Discussions were held in mixed groups of 10-
15 people. Typically a meeting was held with
the Management Committee in the morning
and with a selection of youth, female or
mixed groups in the afternoon. Participation
was disaggregated by gender (although the
results of the exercises were not)
6. Lessons about the methodology
It took time to get the language right and to ensure that
the correct meaning was translated
Fast-tracking the methodology to allow for shorter
meetings significantly eroded the utility of the results
Strong facilitation (as opposed to translation alone) was
key to the success of the discussions
Using the exercises as a foundation for each of the
discussions ensured active engagement and strong
participation from all community members
The meetings were long but they were also fruitful and
community participation was excellent
7. Initial findings – accountability
„Information-sharing‟ was scored fairly strongly across the
groups – the control projects tended to know less (although
also over-exaggerated the extent of their knowledge).
„Participation‟ was routinely strong across all groups, built on
the foundation provided by the PVCA but maintained through
strong community support to implementation and
participation in monitoring
Knowledge and use of „feedback‟ and „complaints‟ was
weakest in the control where the management committee
tended to have a far better understanding than others. For
non-control groups, there was a good understanding of the
mechanism and confidence in it. Non-members affected by
the projects were not well served
9. Contribution to project quality
There was generally a good understanding of the causal link
between accountability mechanisms and project quality
although it was at times difficult to capture this
„Trust‟, „ownership‟ and „motivation‟ were all considered to
be far greater where there were well-functioning
accountability mechanisms
Examples were given to show how the perceived clarity about
and utility of UCCS‟ accountability mechanism lay the
foundation for good quality projects
In the broadest terms, the case study provided evidence for
the hypothesis that functioning accountability mechanisms
tend to produce good quality projects
10. Community voices
“Because we participated in choosing the project, it is ours
and so we will give more of our time and value the
intervention”
“Information sharing has helped people come together to
work in the project”
“[when we selected the project] we prioritised the dry areas
and so we made a choice about who was most in need. In
doing this we influenced a better outcome as we know the
community and the needs”
“Before UCCS came, other programmes have failed because
they lacked accountability and there was corruption.
Accountability is a key part of the success of the programme”
11. Issues for the research
The difficulties associated with designing a methodology relevant to
projects along the development continuum – the long-term nature
of UCCS‟ relationship with the community meant that groups tended
to score the partnership as a whole rather than judge it according to
a single „project‟
The need to focus on the „spirit‟ of HAP rather than the specifics for
development projects - The focus of UCCS on supporting their
groups (rather than those most in need) and directing their
accountability mechanisms accordingly is an example.
The importance of discussions to temper the results of the exercises
– at times the community was „apologetic‟ that they gave a low
score. At other times follow-up discussions suggested they
exaggerated the scores
The importance of facilitation in the success of the research – while
the use of the exercises ensured good participation, sound
explanation and facilitation was essential to the exercise.
The challenge of how to analyse and present the evidence
coherently