Monitoring and evaluation plays an important role in effective policymaking by providing useful data. However, decision-makers often overlook or undermine the importance of data. Data presented is also sometimes complex and difficult to understand. Collecting relevant data is paramount to understanding issues from the beneficiary perspective and designing impactful policies. An evidence-based policy process involves identifying needs through research and situational analysis, using evidence to draft policies, implementing policies while monitoring for gaps, and evaluating impact through ongoing data collection. However, capacity is often lacking, data is collected ad hoc without accountability, and the importance of data is not fully understood. For policies to be effective, quality data collection must be a priority throughout the entire process.
This policy brief summarizes policymakers’ perspectives on what constitutes barriers to evidence-informed policymaking. It also presents strategies for making research results more accessible to high-level policymakers at the country level, based on what they say they want as well as evidence about what information policymakers can and do use in policymaking. Finally, the brief includes examples of how PopPov-supported researchers addressed policy-relevant questions and applied some of the outreach strategies that policymakers suggest.
Solid evidence on the links between preventing adolescent childbearing and alleviating poverty can motivate policymakers and donors to invest in reproductive health and family planning programs for youth. Research that documents the clear cause-and-effect relationship between program interventions and outcomes, such as better health and delayed childbearing among teens, can guide decisions about investments in research or programs.
This report examines the evidence for investing in adolescent reproductive health and family planning programs from the perspective of making an evidence-based argument to guide the investment or spending decisions of public or private organizations. Key steps in developing such an argument—a business case—include:
1. The consequences of relevant trends.
2. Evidence on the potential of particular actions or interventions to change the status quo.
3. The costs associated with different actions.
A strategic approach to policy engagement for research organisationsJames Georgalakis
This is the presentation delivered as part of a two day workshop held in Nepal in 2014 aimed at communications professionals or the point person for communication within fifteen South Asian think tanks. Participants explored how they could adopt a systematic approach to planning research or knowledge outputs for policy engagement and influence. They explored the types of influencing outcomes they are focused on and their individual and institutional capacities to deliver strategic communication and policy engagement work. By the end of the workshop it was hoped that each participating institution would have identified a clear set of steps towards the development of a strategic approach to policy engagement and research communication at an institutional or programmatic level.
This workshop formed part of the IDRC funded Think Tanks Initiative South Asia programme. http://www.idrc.ca/EN/Programs/Social_and_Economic_Policy/Think_Tank_Initiative/Pages/About.aspx
Event designers are continuously developing new strategies that will optimize any competitive advantages that their event may have, while, at the same time, minimizing their operation’s vulnerability to external threats and emulation. Yet, limited research has been done to date with respect to visitor experience at events, in the Irish setting. With this in mind, the principal aim of this study was to establish how event design techniques and practices are influential to attendees’ experience and to determine main motivations for attendance and to review the range of satisfaction levels. Overall, a total of 100 visitor surveys were collected, with results showing that there was a significant difference among event visitors with respect to their reasons for attending the event, intensity of event design factors that influence their experiences and assessment of it when comparisons across key study variables were determined. These findings provide clear insights into the changing nature of visitor behaviors in the experience economy and the importance of responding to the diverse needs of discrete niche groups who might congregate at a particular event.
This policy brief summarizes policymakers’ perspectives on what constitutes barriers to evidence-informed policymaking. It also presents strategies for making research results more accessible to high-level policymakers at the country level, based on what they say they want as well as evidence about what information policymakers can and do use in policymaking. Finally, the brief includes examples of how PopPov-supported researchers addressed policy-relevant questions and applied some of the outreach strategies that policymakers suggest.
Solid evidence on the links between preventing adolescent childbearing and alleviating poverty can motivate policymakers and donors to invest in reproductive health and family planning programs for youth. Research that documents the clear cause-and-effect relationship between program interventions and outcomes, such as better health and delayed childbearing among teens, can guide decisions about investments in research or programs.
This report examines the evidence for investing in adolescent reproductive health and family planning programs from the perspective of making an evidence-based argument to guide the investment or spending decisions of public or private organizations. Key steps in developing such an argument—a business case—include:
1. The consequences of relevant trends.
2. Evidence on the potential of particular actions or interventions to change the status quo.
3. The costs associated with different actions.
A strategic approach to policy engagement for research organisationsJames Georgalakis
This is the presentation delivered as part of a two day workshop held in Nepal in 2014 aimed at communications professionals or the point person for communication within fifteen South Asian think tanks. Participants explored how they could adopt a systematic approach to planning research or knowledge outputs for policy engagement and influence. They explored the types of influencing outcomes they are focused on and their individual and institutional capacities to deliver strategic communication and policy engagement work. By the end of the workshop it was hoped that each participating institution would have identified a clear set of steps towards the development of a strategic approach to policy engagement and research communication at an institutional or programmatic level.
This workshop formed part of the IDRC funded Think Tanks Initiative South Asia programme. http://www.idrc.ca/EN/Programs/Social_and_Economic_Policy/Think_Tank_Initiative/Pages/About.aspx
Event designers are continuously developing new strategies that will optimize any competitive advantages that their event may have, while, at the same time, minimizing their operation’s vulnerability to external threats and emulation. Yet, limited research has been done to date with respect to visitor experience at events, in the Irish setting. With this in mind, the principal aim of this study was to establish how event design techniques and practices are influential to attendees’ experience and to determine main motivations for attendance and to review the range of satisfaction levels. Overall, a total of 100 visitor surveys were collected, with results showing that there was a significant difference among event visitors with respect to their reasons for attending the event, intensity of event design factors that influence their experiences and assessment of it when comparisons across key study variables were determined. These findings provide clear insights into the changing nature of visitor behaviors in the experience economy and the importance of responding to the diverse needs of discrete niche groups who might congregate at a particular event.
A presentation by Nalini Takeshwar as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
This presentation shares and reflects on the practical implications of the design choices made around standards of rigor, inclusiveness and feasibility in the impact evaluation of the IFAD-funded Root & Tuber Improvement and Marketing Program (RTIMP) in Ghana. The approach used in this evaluation was developed with support from IFAD and the BMGF to assess and explain the impact of program/project investments on rural poverty in a collaborative and participatory manner.
The following resource was developed by RESYST for a research uptake workshop held in Kilifi, Kenya.
In this resource:
- Understand the importance of strategic planning for research uptake
- Familiarise key aspects of a research uptake strategy
- Develop research uptake objectives for your research group, project, hub or an event
- Identify key stakeholders using stakeholder analysis techniques
- Review communications channels, outputs and activities
- Explore indicators and tools for monitoring and evaluation
- Key questions to consider in a research uptake strategy
Find more: http://resyst.lshtm.ac.uk/resources/resource-bank-research-uptake
“State of capacities and needs for greater evidence-based policy making in developing Asian countries” presented by Suresh Babu, IFPRI at the ReSAKSS-Asia Conference, Nov 14-16, 2011, in Kathmandu, Nepal.
Seven Steps to EnGendering Evaluations of Public Health ProgramsMEASURE Evaluation
Because international development increasingly focuses on gender, evaluators need a better understanding of how to measure and incorporate gender—including its economic, social, and health dimensions—in their evaluations. This interactive training, consisting of this presentation and a tool, will help participants learn to better evaluate programs with gender components. Access the tool at https://www.measureevaluation.org/resources/publications/tl-19-40
Analyzing financial behavior of a person based on financial literacyShantanu Deshpande
6. Analysed consumer behaviour and relationship using Financial literacy dataset. Identified patterns and predictor variables using logistic regression.
Methodologies & Tools: IBM SPSS, RapidMiner, PowerBI
What Makes a Good Performance Management Plan? A new tool for managersMEASURE Evaluation
Led by Tory M. Taylor, a monitoring and evaluation specialist with MEASURE Evaluation from Tulane University.
The webinar introduced a tool to assist project managers in conducting effective Performance Management Plan (PMP) reviews. The tool provides feedback to implementing partners and is a brief, comprehensive checklist that covers the essential elements of a comprehensive PMP.
Policy entrepreneurs and development entrepreneurs: a discussion with The Asi...Arnaldo Pellini
Had a very interesting meeting and discussion with about 20 staff of the Asia Foundation in Manila. We talked about policy entrepreneurship and development entrepreneurship and discussed the differences and similarities between the two concepts. The slides refer to the policy entrepreneur concept developed by ODI's RAPID programme
A presentation by Rachel Hinton as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
A presentation by Nalini Takeshwar as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
This presentation shares and reflects on the practical implications of the design choices made around standards of rigor, inclusiveness and feasibility in the impact evaluation of the IFAD-funded Root & Tuber Improvement and Marketing Program (RTIMP) in Ghana. The approach used in this evaluation was developed with support from IFAD and the BMGF to assess and explain the impact of program/project investments on rural poverty in a collaborative and participatory manner.
The following resource was developed by RESYST for a research uptake workshop held in Kilifi, Kenya.
In this resource:
- Understand the importance of strategic planning for research uptake
- Familiarise key aspects of a research uptake strategy
- Develop research uptake objectives for your research group, project, hub or an event
- Identify key stakeholders using stakeholder analysis techniques
- Review communications channels, outputs and activities
- Explore indicators and tools for monitoring and evaluation
- Key questions to consider in a research uptake strategy
Find more: http://resyst.lshtm.ac.uk/resources/resource-bank-research-uptake
“State of capacities and needs for greater evidence-based policy making in developing Asian countries” presented by Suresh Babu, IFPRI at the ReSAKSS-Asia Conference, Nov 14-16, 2011, in Kathmandu, Nepal.
Seven Steps to EnGendering Evaluations of Public Health ProgramsMEASURE Evaluation
Because international development increasingly focuses on gender, evaluators need a better understanding of how to measure and incorporate gender—including its economic, social, and health dimensions—in their evaluations. This interactive training, consisting of this presentation and a tool, will help participants learn to better evaluate programs with gender components. Access the tool at https://www.measureevaluation.org/resources/publications/tl-19-40
Analyzing financial behavior of a person based on financial literacyShantanu Deshpande
6. Analysed consumer behaviour and relationship using Financial literacy dataset. Identified patterns and predictor variables using logistic regression.
Methodologies & Tools: IBM SPSS, RapidMiner, PowerBI
What Makes a Good Performance Management Plan? A new tool for managersMEASURE Evaluation
Led by Tory M. Taylor, a monitoring and evaluation specialist with MEASURE Evaluation from Tulane University.
The webinar introduced a tool to assist project managers in conducting effective Performance Management Plan (PMP) reviews. The tool provides feedback to implementing partners and is a brief, comprehensive checklist that covers the essential elements of a comprehensive PMP.
Policy entrepreneurs and development entrepreneurs: a discussion with The Asi...Arnaldo Pellini
Had a very interesting meeting and discussion with about 20 staff of the Asia Foundation in Manila. We talked about policy entrepreneurship and development entrepreneurship and discussed the differences and similarities between the two concepts. The slides refer to the policy entrepreneur concept developed by ODI's RAPID programme
A presentation by Rachel Hinton as part of the Cohort Research for Programme and Policy panel discussion at the International Symposium on Cohort and Longitudinal Studies in Developing Contexts, UNICEF Office of Research - Innocenti, Florence, Italy 13-15 October 2014
Bridgingg the research policy gap influencing policy change-nairobiThe Scinnovent Centre
Presentation by Dr. Maurice Bolo, during the Scinnovent Centre' training on The Art of Influencing policy Change: tools and strategies for researchers, held on 12th -14th February 2013 at The African Academy of Sciences Campus Nairobi
Preparation for Policy Formulation, Data on Educational Personnel_20231223_11...DrHafizKosar
What Is Policy?
Policy refers to a purposeful set of principles designed to steer decision making and attain logical results. It serves as a declaration of intentions and is put into action as a method or set of guidelines. Typically endorsed by a governing body within an organization, policies provide a framework for consistent and informed choices, helping to align actions with desired objectives.
Educational Policy
Educational policy refers to a set of principles, guidelines, laws, and strategies formulated by governments or educational authorities to regulate and guide the development and functioning of the education system within a specific jurisdiction.
Policy Formulation
The policy formulation process is a crucial aspect of governance that involves the systematic development and establishment of principles, guidelines, and actions to address specific issues or achieve particular objectives. The Importance of Policy Formulation
Policy formulation plays a critical role in shaping society and addressing important issues. Policies guide decisionmaking and action, and they can have a significant impact on individuals and communities. Effective policies can promote economic growth, protect public health and safety, and advance social justice and equality.Factors Affecting Educational Policies
Social Factors: Changing demographics drive the need for inclusive education systems. Social pressure prompts policymakers to create policies ensuring equal educational opportunities. Policies aim to promote diversity, multiculturalism, and inclusivity in schools.
Political Factors: Education policy is shaped by the ideologies and priorities of the ruling government. Governments recognize the importance of an educated citizenry for power and dominance. Specific policies depend on political beliefs; conservative governments may streamline curriculum, while liberal ones emphasize social justice and inclusion.
Economic Factors: Economic development, resource availability, and funding priorities determine financial investment in education. Limited financial backing results in a lack of resources and support for students. Affluent societies prioritize student needs and provide adequate funding for services and resources.
The failure of researchers to link evidence to policy and practice produces evidence that no one uses, impedes innovation, and leads to mediocre or even detrimental development policies. To help improve the definition, design, and implementation of policy research, researchers should adopt a strategic outcome-oriented approach.
The Stakeholder Engagement tool helps ensure that the appropriate stakeholders in decision processes have been identified and involved.
Tool: https://www.cpc.unc.edu/measure/publications/ms-11-46-e
Webinar Recording: http://universityofnc.adobeconnect.com/p99y8bhnosx/
Conducting High Impact Research: Building data ownership and improving data use
Data and MandE
1. The contribution of Monitoring and Evaluation to effective policy making through
useful data
Quality data has often been overlooked by decision makers as something trivial, too tedious and time consuming for
their necessary attention. Often, the notion of data and its usefulness is undermined, resulting in an ad hoc inefficient
use of resources as well as poorly implemented social development interventions and policies. Albeit, often data that
is presented is not simple to understand and is presented in a complex manner contributing to the lack in enthusiasm
in interpreting and using it. This has given rise to the increase usage of infographics which depict accurate, simplified
data that is easy to remember, yet due to the jadedness of data and its meaning, importance and relevance, insufficient
attention has been given to mandate the collection of useful data.
Decision-makers across the world need to base their decisions on information from reliable sources. This is imperative
to evidenced based decision making through which relevant, implementable and impactful policies and developmental
interventions can arise. They need to learn from the best evidence based knowledge and experience available and
they need to know what kinds of research and what type of data could help them make the right choices. Successfully
implemented and impactful polices and developmental interventions hinge on appropriate and well-designed
monitoring and evaluation frameworks which depend heavily on useful quality data. However, practical work in the
development field across sectors has often demonstrated that either too much data is collected with little regard for
the quality, or that data is missing, or that useless data is collected because monitoring and evaluation frameworks
that clearly articulate what type of data is necessary and relevant is missing.
Why is data missing and what is the importance of collecting relevant data?
The missing link between research, practice and policy is data that is accurate, valid and reliable. In Africa, there is a
large gap between the producers and consumers of knowledge, and research could have a greater impact on
development policy than it has had to date. Researchers as “knowledge makers” struggle to understand the resistance
to policy change despite clear and convincing evidence whilst policymakers as “knowledge consumers” lament the
inability of many researchers to make their findings accessible and digestible in time for policy decisions (Jones, 2011:
7).”1
Furthermore, the politics surrounding access to information and permission to collect useful data negatively
impacts the generation of useful research and subsequently current data. In other instances, slow bureaucratic
processes and political bickering can further negatively impact the publication of data contributing to the absence of
data to the general public. Additionally, fieldwork experience has highlighted the ignorance of monitoring and
evaluation officers in understanding the importance of collecting, collating, analysing, interpreting and presenting user
friendly data. Not only are records unkempt, but the nonchalant attitudes towards the value of data is disturbing.
Collecting relevant data is of paramount importance to sourcing the correct information regarding a socio-economic
challenge from the beneficiary point of view. Developmental interventions are still heavily carried out with a top down
approach with the target beneficiaries often complaining that what is given is not what is required to make the
necessary changes the policy seeks to address. Before or during the policy design phase, it would make sense to carry
out a situational analysis to understand the key target groups needs and priorities, the demographic factors and the
capacity challenges of government, NGOs (non-governmental organisations) and CBOs (community based
organisations) staff. There is sufficient evidence from my working experience in the development field that capacity is
severely lacking in implementing the required policies hence, the collection of timely, useful and relevant data is
frequently compromised. The diagram below illustrates an evidenced based policy pathway.2
1
Jones, B. 2011. Linking Research to Policy: The African Development Bank as Knowledge Broker, Series N° 131, African
Development Bank, Tunis, Tunisia.
2
Bowen, S., & Zwi, A.B. 2005. Pathways to “evidence-informed” policy and practice: A framework for action. PLoS Med, 2(7):
e166.
2. Source: Adapted from Bowen and Zwi, (2005)
The above pathway to evidence-based policy and practice involves five phases which are:
(1) policy idea,
(2) sourcing the evidence,
(3) implementing the evidence,
(4) identifying implementation gaps and
(5) monitoring and evaluation
The policymaking context is exceedingly political and rapidly changing and depends on a variety of factors, inputs, and
relationships. It is of paramount importance to ensure that monitoring and evaluation is part of the agenda setting of
the policy to guarantee that useful, relevant and current data is collected. This will enable the assessment of the overall
impact of the policy on the beneficiaries. Problem identification of the policy setting phase needs to be informed
through research and a situational analysis- the situational analysis can enrich or negate the data of existing research
and can necessitate the need of a national survey to update old or missing data. This process can further highlight the
correct target group for the policy intervention and areas that are an urgent priority. Often a policy is drafted and only
at the evaluation stage does it come to light that there is no useful data that can tell us what impact the policy had. It
is only at this late stage that decision makers realise the need for accurate data.
Once the evidence is used to draft a policy discussion paper to make a case for the policy, there needs to be a discussion
around what the data says and to ensure that the implementing partners of the policy have the capacity to implement
the policy. In developing countries, often it is discovered that great polices are drafted however, there is limited
capacity to implement it. Thus, in some countries, more structures are created wasting more financial resources rather
than addressing the capacity constraints. From my work in the development field, a significant capacity constraint is
work ethic coupled with the ignorance in understanding the job description of monitoring and evaluation officers when
it comes to data collection, collation, analysis and presentation. In addition, the absence of understanding the
importance of data by key management staff results in data collected in an ad hoc manner with no one being held
accountable for the failure of missing data.
Once the policy is in place and is being implemented, monitoring needs to occur to collect data for ongoing activities
and outputs and to inform management when the implementation diverges from the objectives of the policy.
Monitoring further supports management when implementation of the policy itself has become futile enabling
sufficient response time to address these issues bringing implementation back on track. An evaluation has to be
•Government
directive to design
and implement
policy addresssing a
particular need
Policy Idea
•Research
•Situational analysis
to understand
context on the
ground
•Identfication of
needs and priorities
•Collection of data
Sourcing the
evidence
•Data collation,
analysis,
interpretation and
presentation to
inform policy
Using the
evidence
• Identification of
capacity gaps of
implementing staff and
government partners to
ensure that the policy is
implementable
Identifying
implementation
gaps
• Continuing the data
collection through
monitoring and
assessing the policy
through evaluation to
understand how the
policy worked and what
the impact was on the
lives of the beneficiaries
Monitoring and
evaluation
EVIDENCE BASED POLICY PATHWAY
3. conducted to assess what impact the policy had on the target beneficiaries and what areas worked well and what did
not and what could be done to improve the policy design.
Important to note is that neither policy nor monitoring and evaluation can occur without useful quality data. It is data
that presents a case for a policy or development intervention and it is data that describes how, when and why the
policy or intervention worked or did not work and provides insights as to how to improve for the future. The value of
data cannot be underestimated and the mandate to ensure accountability for quality and timely collection of data
needs to be championed by managers. After all, without data we do not know what the reality is on the ground is and
how to address the issues that will result in impactful and meaningful change in the lives of the beneficiaries of the
policy or intervention.