Conducting High Impact ResearchIUSSP September 26 & 27, 2009Tara Nutley & Scott Moreland

Workshop ObjectivesProvide a framework for linking research to program and policy needs & decision making processProvide guidance, tips and job aides to apply to the research process that will facilitate the ultimate use of research results Provide hands on experience applying guidance and practicing job aides

IntroductionsIn 5 minutes…Introduce yourselves to your colleagues at your tableProvide your name, country of current residence, affiliated institution and research interests

Background: Why are we here?Global/local health contextHIV epidemicResurgence of TBContinued prevalence of malariaPockets of stalled fertility declinePopulation burdenShortage of health care workers

ContextPressing need to develop health policies, strategies and interventions

ContextIncreased financial investments for service deliveryIncreased accountability requirementsImproved national HMISIncreased demand for evaluation and other research

Level of Dissatisfaction that Policy is Based on Scientific EvidencePercent dissatisfiedOverseas Development Institute, Jones et al., 2008.

Evidence-based Decision Making ProcessStakeholders

Culture Divide: Researchers vs. Decision-makersDecision makers think that health researchers:Lack responsiveness to prioritiesTake too long to answer questionsFavor jargon to transparent communicationDon’t commit to a yes or a no answerRely mainly on written reports rather than face-to-face conversation

Culture Divide:Researchers vs. Decision-makersHealth researchers think that decision-makers:Impose unrealistic timelinesDon’t understand what is a researchable questionDon’t distinguish good from bad researchValue “political” considerations over evidenceAre unprepared to measure or evaluate the consequences of their decisions

Impose unrealistic timelines

Don’t understand what is a researchable question

Don’t distinguish good from bad research

Value “political” considerations over evidence

Are unprepared to measure or evaluate the consequences of their decisions

Breakdown in Decision-making CycleFailure to link the program and policy process to the research processStakeholders not involved throughout the research cycleData not fully relevant to decision-making needsLow understanding of data, thus data ownership

Breakdown in Decision-making CycleData not availableData not accessible in user friendly formatsExternal factors: political, cultural or religious ideology

Strengthening Evidence-based Decision Making Apply improved research paradigmStart with the decision to be informed with new dataConsider the program/policy continuum in planning phaseInvolve varied stakeholders throughout the research processDevelop action oriented recommendationsMake data, results and recommendations available & accessible in varied formats for data usersFollow-up

Placing Research in the Policy-Program Continuum

Continuum: What & Why?Organizes how research supports both policy and programsPlacing research in the continuum assists to:Determine how results of research will be usedDevelop action oriented research questionsInclude appropriate stakeholders in the research process

The Policy - Program ContinuumAdvocacy & Policy formulationProgram formulationProgram monitoringProgram evaluation

Advocacy & Policy Formulation

Program Formulation

Program Monitoring

Program Evaluation

PRACTICUM 1: Mapping Research Questions to the Program – Policy Continuum

Report Back – PRACTICUM 1One individual will volunteer from each table to report on the decision the group made and how they came to that decision.

Role of Stakeholders in the Research Process

What is a Stakeholder?Any person or group with a particular interest or ‘stake’ in your researchProviders / ImplementersPolicymakersProgram managersFunding agenciesBeneficiariesProfessional associations

Context of Decision-MakingInformation/dataStake-holdersDecisions

Implications for ResearchDifferent stakeholdersHave different perspectivesNeed/want different informationNeed information at different levels of complexityHave different intensities of interestStakeholders can facilitate and/or impede the eventual use of study results

Involving Stakeholders Throughout the Research ProcessIncreases understanding of the research process Increased knowledge of the quality of dataIncreased opportunity to discuss and identify key programmatic questions and concerns Jointly analyze and interpret data to answer programmatic questionsDirect link to users of the data

Involving Stakeholders Throughout the Research Process Relevance of data Ownership of data Appropriate dissemination of data Use of data

When & How: Involving StakeholdersStudy question developmentProtocol developmentData collectionData interpretation & recommendation developmentResults disseminationData use plan development

Stakeholder AnalysisClarifies who has interest in a topic/program and what that interest isIdentifies who can help a program and how, and who can hinder itPrioritizes stakeholders and clarifies roles

Stakeholder Engagement Matrix

Stakeholder Analysis Matrix

Formulating Meaningful Research Questions

Identifying, Assessing, Refining, Prioritizing QuestionsRefined QuestionInitial QuestionFinal QuestionImportantPriorityActionable Data gapNot already underwayAnswerable Method available Reasonable time period

Important

Priority

Actionable

Data gap

Not already underway

Answerable

Method available

Reasonable time period

Criteria for Meaningful and High Impact Research QuestionsImportance: Could the answer to the question lead to a policy or program change that would have a large effect on the population in question?Priority: Does the question address a current and pertinent issueActionable: Can the results of the research be used to identify clear policy or program recommendations?Answerable: Are data available, or can data be collected, to address the question?

Let’s look at some examples (1)Research topic: Physical activity and cardiovascular diseaseResearch problem: Is there a link between lack of physical activity and hypertension?Research question: Will 30 minutes of moderate activity daily reduce hypertension in office workers?Hypothesis: 30minutes of moderate activity daily in sedentary workers will reduce the median systolic blood pressure by 10mm mercury.

Let’s look at some examples (2)Research topic: Parity and contraceptive useResearch problem: Do low parity women have a lower demand for family planning?Research question: How does contraceptive use vary by parity among married women?Hypothesis: Low parity married women will have lower contraceptive prevalence rates?

PRACTICUM 2:Developing High Impact Research Questions & Identifying Stakeholders

Report Back – PRACTICUM 2Spend 15 minutes with your neighboring table and discuss what you found challenging about the exercise and seek feedback from the visiting groupSwitch roles

Developing Recommendations Based on Research FindingsProcess begins with the formulation of the initial research questions and the initial involvement of stakeholders

Developing Actionable RecommendationsHow did we get to actionable findings?Ensured research questions were relevant and of priority to the local contextLinked them to specific programmatic and policy actionsHow do we get to actionable recommendations?Interpret results with stakeholders

Considerations When Developing RecommendationsImpact - What will be the impact on health programs both long-term and short-term? Resources - Are additional resources needed?Supporting factors - Are policy, operational or management changes needed?Acceptability - Politically and culturally acceptable?Timeliness – Can implementation be timely?

Data Use Action Plans

Data Use Action Plans

Communicating Study Results & Program / Policy RecommendationsProcess begins in the study planning phase with the initial identification of stakeholders

Consider Varied Information NeedsDifferent stakeholders make different types of decisions Tailor key messages and communication channels to different audiencesPrioritize key recommendations that are actionable & practical

Illustrative Information Needsby Stakeholder Group

Assessing the Effect of Research

Why Assess Effect? Validates the conduct of the studyProvides additional evidence that research can improve decision making Increases demand for future researchHelps identify best practices in research communication & utilization

Validates the conduct of the study

Provides additional evidence that research can improve decision making

Increases demand for future research

Helps identify best practices in research communication & utilization

How Can You Assess Effect?Did all identified and important stakeholders see and understand research results?Were research results taken into consideration in decision making?Were recommendations acted upon?Has there been an evaluation of the impacts of such action?

PRACTICUM 3:Developing a Data Use Action Plan

Report Back – PRACTICUM 3One individual will volunteer from each table to discuss what they found useful about the data use action plan and how they can apply it in their own work.

Strengthening Evidence-based Decision Making Apply improved research paradigmStart with the decision to be informed with new dataConsider the program/policy continuum in planning phaseInvolve varied stakeholders throughout the research processDevelop action oriented recommendationsMake data, results and recommendations available & accessible in varied formats for data usersFollow-up

Thank YouGive us feedback Tara Nutleytnutley@futuresgroup.com Scott Moreland smoreland@futuresgroup.com

Join Data Use NetSend an email to listserv@unc.edu. Leave the subject field blank and in the body of the message type ‘subscribe DataUseNet.’ For example: To: listserv@unc.eduFrom: youremail@youremail.comSubject:Subscribe Data Use Net

MEASURE Evaluation is funded by the U.S. Agency for International Development and is implemented by theCarolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures GroupInternational, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. The views expressed in this presentation do not necessarily reflectthe views of USAID or the United States Government.