This document provides an overview of a workshop on data demand and use. The workshop objectives are to develop a framework for linking data with action, create an action plan for overcoming barriers to data use and improving information flow, and establish three commitments to improve data use in participants' jobs. The workshop covers various monitoring and evaluation concepts like results chains, indicators, baselines, and targets. It also discusses data demand and use, the context of decision making, barriers to data use, and descriptive data analysis techniques like service delivery analysis and unit cost analysis. Participants will learn a seven-step process for using information to make data-informed decisions.

1. Data Demand and Use
Workshop
Isabel Brodsky and Verne Kemerer
MEASURE Evaluation
October 24–27, 2017
Freetown, Sierra Leone

2. Knowledge learning objectives
 Describe the role of data demand and use (DDU)
in a functional M&E system
 Define DDU
 Understand the context of data use
 Identify barriers to data use and develop
recommendations to overcome them
 Identify locally relevant programmatic questions
and link those questions to available data
 Understand types of descriptive data analysis
Skills learning objectives
 Use tools and strategies to facilitate data use during
decision making
Workshop Objectives

3.  Framework for linking data
with action
 Action plan for overcoming barriers to
data use and improving information
flow
 Three commitments to improve data
use in your job
Workshop Outputs

4. Session 1: Monitoring and Evaluation
Concepts

5. Session 1: Learning Objectives
Knowledge learning objective:
 Describe the role of data demand and use
in a functional monitoring and evaluation
(M&E) system

6. DDU’s Role in a Functional M&E System

7. Strategic Information
 Are we making progress towards or are we
achieving our goals, objectives, and targets?
 How does service delivery vary across facilities?
 What are we doing well and how are we
under-performing?
 Can we improve service coverage with
existing resources?
 How should we allocate resources?
 Do we need to advocate for additional resources or
for changes to policies
and programmes?

8. Review of Basic M&E Concepts
 Goals
 Objectives need to be smart
• Specific
• Measureable
• Achievable
• Realistic
• Time bound

9. Results Chain
Inputs Activities Outputs Outcomes Impacts
Processes
Results
Objectives Goal(s)

10. Source: UNAIDS RST-ESA, 2010
Results Chain

11. Results Framework
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
 Outputs are the products, like the number of people enrolled on
antiretroviral therapy (ART) during the past 12 months.
 Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load and service
coverage.
 Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a valid and
reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current
condition against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.

12. Indicator
 Measures an aspect of a program’s
performance
 Measures changes over a period of time
• # of new family planning users
• # of clients currently on ART
 Expressed as a number or percentage

13. Baselines
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
 Outputs are the products, like the number of people enrolled
on ART during the past 12 months.
 Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load
and service coverage.
 Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a
valid and reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current condition
against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.

14.  What are the sources of data?
 What are the data collection
methods?
 Who collects the data?
 How often are data collected?
 Who analyses these data?
 Who reports these data?
 Who uses these data?
Establishing a Baseline

15. Targets
Result
The outputs, outcomes, impacts identified in the results chain, that is, what the
programme plans to achieve at different result levels.
Result
level
 Outputs are the products, like the number of people enrolled on
ART during the past 12 months.
 Outcomes reflect the programme’s objectives to change knowledge,
attitudes, behaviour, and biomedical markers like viral load
and service coverage.
 Impacts align with the goal(s) of the programme and describe changes in
morbidity and mortality, typically at the national level.
Indicator
An indicator is a quantitative or qualitative variable that provides a
valid and reliable way to measure achievement, assess performance,
or reflect changes connected to an intervention. Single indicators are limited in
their utility for understanding programme effects and must be collected and
interpreted as part of a set of indicators.
Baseline
The first measurement of an indicator that measures the current
condition against which we can track future changes.
Target
Targets state the desired level of performance the program wants to achieve
within a specified period. Each indicator needs a target. Setting targets
understanding what can be realistically achieved with available resources.

16. Calculating Targets
Things to consider
 Existing capacity of staff and facilities
 Staff turn-over
 Budget allocations through the
Ministry of Health (MOH)
 Other funding streams like bilateral
and multilateral donors

17. Learning Activity #1
Setting targets

18. Learning Activity #2
Session 1 review

19. Session 2: DDU Concepts

20. Knowledge learning objective:
 Define what data demand and use is
Session 2: Learning Objectives

21. Data use
 Create or revise
a program or
strategic plan
 Develop or revise
a policy
 Advocate for a policy
or program
 Allocate resources
 Monitor a program
Data Demand and Use Defined
Data demand
 Decision-makers
specify what kind of
information they
need to inform the
decision-making
process and then
seek it out

22. Data-Informed Decision-Making Cycle

23. Why Improve DDU?

24. Learning Activity 3:
Session 2 review

25. Session 3: Context of Decision Making

26. Knowledge learning objective:
 Understand the context of data use
Session 3: Learning Objective

27. Data-Informed Decision Making
Not an event, but a process
Data
x
Stakeholders
x
Decisions

28. “We are always giving patient forms and
data to our M&E Unit, who then gives
data to donors and the government. I am
the head doctor and I never have the
chance to look through the data before
they go up. We just keep giving data up
and up, and we never hear back about
it…”
Head of ART facility, Nigeria

29.  Census
 Vital events data
 Surveillance data
 Household surveys
 Facility-level service statistics
 Financial and management information
 Modeling, estimates, and projections
 Health research
Data and Information

30.  Paves path between data collectors
and users
 Leads to greater appreciation
of data
 Important element of management
and supervision
Working toward a Culture of Data Use

31. Data
x
Stakeholders
x
Decisions
Data-Informed Decision Making
Not an event, but a process

32.  View activities from different perspectives
 Have different degrees of understanding
 Need/want different information
 Need information at different levels
of complexity
 Have different intensities of interest
 Have different roles in the decision-making
process
Knowing Your Stakeholders
Why is it important?

33. Learning Activity 4
Stakeholder analysis matrix
Stakeholder
Description
Role
Knowledgelevel
Commitment
Resources
Constraints
Government Sector
Political Sector
Commercial Sector
Source: MEASURE Evaluation, 2011

34. Involving Stakeholders in the
Data Use Process
Relevance of data
Ownership of data
Appropriate dissemination
of data
Use of data

35. Data-Informed Decision Making
Not an event, but a process
Data
x
Stakeholders
x
Decisions

36.  Program design and evaluation
 Program management
and improvement
 Strategic planning
 Advocacy and policy development
Decision Areas

37. Learning Activity 5
Session 3 review

38. Session 4: 7 Steps for Using Information

39. Knowledge learning objectives:
 Define data demand and use
 Identify barriers to data use and develop
recommendations to overcome them
 Identify locally relevant programmatic
questions and link those questions to
available data
Session 4 Learning Objective

40. No Need to Fish
Photo by Bread for the World

41. 1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
7 Steps to Use Information

42.  Should we expand care and treatment
programs in Njombe province?
 Am I meeting the HIV prevention needs of
my key populations?
 Are men and women seeking HIV
care equally?
 What is the quality of my HIV prevention
service?
Step 1: Identify Questions
Examples

43. Should we expand care and treatment
programs in Njombe province?
 What is the burden of HIV in the province over
time?
 How many facilities are offering counseling
and testing (C&T) in the region?
 How many people are being seen
at each facility?
 Are there key facilities that have capacity to
expand C&T to reach gaps?
Step 2: Prioritize Questions

44.  Demographic and health surveys (DHS),
AIDS indicator surveys (AIS), etc.
 Routine health information system (RHIS)
 External data sources: behavioral surveillance systems,
mapping exercises, surveys
 Projections
Step 3: Identify Data Sources

45. Step 4: Transform Data into Information
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page 9

46. Step 5: Interpret Information
and Draw Conclusions
Relevance
of finding
Reasons
for finding
Consider
other data
Conduct
further
research

47.  Gather additional information
 Prioritize Njombe in national expansion strategy
 Expand voluntary counseling and testing (VCT) and
C&T in high priority facilities
 Current VCT facilities do outreach to hotspots
 Create referral protocol to ensure those tested are
entering into C&T
Step 6: Craft Solutions and Take Action

48.  Monitor implementation of action plan
 Consider frequency and duration of monitoring
 Develop tool for monitoring
Step 7: Continue to Monitor Key Indicators

49. Learning Activity 6
Session 4 review

50. Session 5: Linking Data to Action

51. Knowledge learning objective
 Identify locally relevant programmatic
questions and link those questions to
available data
Skills learning objective
 Use tools and strategies to facilitate data
use during decision making
Session 5: Learning Objectives

52. 1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators
7 Steps to Use Information

53. Questions
 Are we implementing the
program as planned?
 Are we fully meeting the
needs of our clients?
 What additional resources are
needed to meet
the needs of our clients?
 Are our interventions making
a difference?
Questions Lead to Decisions
Decisions
 Allocation of resources across
districts/community-based
organizations (CBOs)
 Revising program approaches
to emphasize results
 Addition of outreach
education programs to most-
at-risk populations
 Advocacy for new program
areas

54.  Brainstorming about what different staff
are interested in knowing
 Participatory discussion of indicators/data
— desire to know more
 Gathering feedback from clients
 Preparing for upcoming decisions
that have to be made or
planning exercises
 External factors — donors questions
How Do I Identify Questions of Interest?

55. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and
assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators

56.  Relevance
 Answerable
 Actionable
 Timeliness
Criteria for Prioritizing Questions

58. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and
assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators

59.  Creates a time-bound plan for
data-informed decision making
 Encourages greater use of existing
information/generated demand
for data
 Monitors the use of information
in decision making
Framework for Linking Data with Action

60. Learning Activity 7:
Framework for linking data with action
Question Data
source/
indicator
Person
responsible
(analysis/
research)
Timeline Communication
channel
Possible
decision/
action

61. Learning Activity 8:
Session 5 review

62. Session 6: Transforming Data into Information

63. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and assess
findings
6. Craft solutions and take action
7. Continue to monitor key indicators

64. Knowledge learning objective
 Understand types of descriptive
data analysis
Session 6: Learning Objective

65. Descriptive Analysis
 Turns raw data into useful information
 Provides answers to questions
 Describes the sample/target population
(demographic and clinic characteristics)
 Does not define causality – tells you what, not
why

66.  Question: Is my program meeting its
objectives?
 Analysis: Compare program targets and
actual program performance
to learn how far you are from target
 Interpretation: Why you have, or have
not, achieved the target and what
this means for your program
Descriptive Analysis Answers
Programmatic questions

67.  Service delivery descriptive analysis
 Service coverage analysis
 Unit cost analysis
 Trend analysis
Types of Descriptive Analyses
Using Routine Data

68. Service Delivery Descriptive Analysis
Selection of ARV Indicators
Private
sector
Public
health
system
Civil
society
# of clients who have refilled their prescriptions on-time at a
fee-for-service pharmacy
√
# of clients on ARVs who have been admitted into a
care facility for end-of-life support in the past six months √ √ √
# of clients on ARVs who have been counselled on ARV
adherence in the past three months
√ √ √
# of clients who have had CD4 and VL tested in the past six
months
√ √ √
# of clients enrolled in ARV programme √ √
# of client eligible for ARVs √ √ √
# of clients who have refilled their prescriptions on time at a
subsidized pharmacy
√ √
# of CD4 test kits procured through bilateral development
partners and distributed by NGOs
and INGOs to public-sector ARV clinics
√ √

69. Service Coverage Analysis
 Supply of services data
 Need for HIV services
 Calculate:
Service coverage =
Supply of services
Demand for services

70. Service Coverage Analysis
Service
coverage
=
# of individuals who have
received the services
# of individuals in need
of the service
Service coverage =
467 individuals have received
HIV testing and counselling (HTC) services
1,000 individuals in need of HTC services
= 47%

71. Unit Cost Analysis
What is the unit cost of delivering antenatal care (ANC) services
through an existing mobile health unit that provides maternal and
child health (MCH) services?
Item Amount
Fixed costs per month
Overhead SLL 7.8M
Personnel SLL 30.5M
Variable costs per month
Commodities SLL 7.8M
Other SLL 15M
# MCH clients/month 100
# MCH clients/month receiving ANC services 23
Cost data adapted from 10.1371/journal.pone.0119236

72.  Observe the overall pattern of change
 Reveal patterns due to seasonal variations
 Identify outliers and high/low performance in
certain time periods
 Compare the effect before or after
an event
 Produce projections to help with programme
planning and target setting
Descriptive Statistics
Trend analysis

73. Trend Analysis
Comparing time periods and sites
Source: 2016 Sierra Leone Malaria Indicator Survey
Full Report, page 51

74.  Display trends, relationships, and comparisons
 Simple and self-explanatory
Charts and Graphs

75. Column Charts
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page
8

76. Stacked Bar Chart
Source: Key findings of the 2013 Sierra Leone Demographic & Health Survey,
page 10

77. Line Graph
Source: 2016 Sierra Leone Malaria Indicator Survey Full Report, page 33

78. Pie Chart
Source: Key Findings of the 2013 Sierra Leone Demographic & Health Survey, page 11

79. Learning Activity 9:
Trend analysis

80. Session 6 Review: Interpreting Data
Relevance of
finding
Reasons for
finding
Consider
other data
Conduct
research
 Have we met our target?
 How does performance compare
to other time periods? Facilities?
 What could explain the observation?
 Talk to beneficiaries and community
leaders
 Ask program managers
 Analyze other data sources
• Do they confirm or reject
your finding?
o Why?
 If there are no data, need for research
 Methodology depends on question
and available resources

81. Learning Activity 10:
Session 6 review

82. Session 7: Craft Solutions, Take Action, and
Monitor Progress

83. 7 Steps to Use Information
1. Identify questions of interest
2. Prioritize key questions of interest
3. Identify data needs and potential
sources
4. Transform data into information
5. Interpret information and assess findings
6. Craft solutions and take action
7. Continue to monitor key indicators

84. Knowledge learning objective
 Identify barriers to data use
and develop recommendations
to overcome them
Skills learning objective
 Use tools and strategies to facilitate
data use during decision making
Learning Objectives

85. What Determines DDU?
* Based on PRISM analytical framework (LaFond, Fields et al. (2005). The PRISM: An analytical framework for
understanding performance of health information systems in developing countries. MEASURE Evaluation).
Organizational Technical
Behavioral

86. Organizational Technical
Behavioral
Politics
Culture
Society
What Determines DDU?

87. Technical
 Have you ever had an experience while making a policy or
program-related decision when you were concerned about the
quality of the information being used?
Behavioral
 What specific challenges have you experienced among your
staff when it comes to using data?
Organizational
 How does your organization support having the necessary
information to make decisions?
Learning Activity 10:
Identifying barriers to data use: discussion questions

88. Learning Activity 11:
Developing an action plan to address
barriers to data use
Barrier Action Steps
Responsible
person
Timeline Indicators
1. 1a.
1b.
1c
1a.
1b.
1c.
1a.
1b.
1c.
1a.
1b.
1c.
2. 2a.
2b.
2c.
2a.
2b.
2c.
2a.
2b.
2c.
2a.
2b.
2c.

89.  Plan, plan, plan!!!
 Engage key stakeholders to fully
understand:
• Decisions they make
• Information they need
• Appropriate dissemination
Building Data Use into Your Work

90.  Institute regular meetings
 Document and
communicate data
use successes
Practical Steps to Integrate DDU
 Institutionalize tools (ex.,
framework for linking
data with action)

91.  Identify a group leader
 Identify 2–3 things your group can do
to facilitate data use within the first two
months of returning back to work, based
on your action plan
 Small group discussion: 20 mins
 Group presentations: 15 mins
Make Commitments

92. This presentation was produced with the support of the United
States Agency for International Development (USAID) under the
terms of MEASURE Evaluation cooperative agreement AID-OAA-L-
14-00004. MEASURE Evaluation is implemented by the Carolina
Population Center, University of North Carolina at Chapel Hill in
partnership with ICF International; John Snow, Inc.; Management
Sciences for Health; Palladium; and Tulane University. Views
expressed are not necessarily those of USAID or the United States
government.
www.measureevaluation.org