How are evaluation findings actually used? This presentation examines what are the different types of evaluation use and what influences use. Originally presented at the Swiss National Health Promotion Conference, 19 January 2017

1. Use of evaluation findings:
Types and influences
National Health Promotion Conference
Neuchâtel Switzerland
19 January 2017
Dr Glenn O’Neil
oneil@owlre.com
www.owlre.com

2. 2
Types of use
Use
Instrumental
ConceptualSymbolic
Process
Misuse
Non-use
= most common types of use found

3. 3
What influences use?
Quality
Influences
Credibility
Relevance
Comm.
quality
Timeliness
Findings
Decision
- making
Info.
needs
Political
climate
Competing
Info.
Receptiveness
Personal
characteristics
ContextStakeholder
involvement
Internal context
Organisational structure, culture and strategy
= stronger influences on use

4. 4
“Engagement, interaction, and
communication between evaluation
clients and evaluators is critical to the
meaningful use of evaluations”
Johnson et al (2009)
What influences use?

5. 5
– Assumption: what is used from an evaluation are
the recommendations
What is used from an evaluation?
– Yet in a study by this author, out of 28 instances of
use identified in two evaluations, 18 were not from
recommendations
– For example: Staff improve evaluation skills
Modify programme planning
Input into broad policy

6. 6
How does evaluation use occur?
Individual Interpersonal Collective
Informal change
Formal change
Four
Years

7. 7
How does evaluation use occur?
Use occurred: Unintended and direct Use occurred : Unintended and indirect
Use occurred : Intended and direct
Use occurred: Intended and indirect
Use did not occur: Intended and indirect Use did not occur: Intended and direct
More
Less

8. 8
1. Use may not be instrumental and direct as expected
2. Stakeholder involvement is critical to use
3. Organisations can influence evaluation use
4. Use can be unpredictable, opportunistic & unexpected
Conclusions

9. 9
Contact details
oneil@owlre.com
glennoneil
www.owlre.com
@glenn_oneil
www.intelligentmeasurement.com
Contacts:

10. Main references
• Ciarlo, J. A. (Ed.). (1981). Utilizing evaluation: Concepts and measurement techniques. Beverly Hills, CA: Sage
Publications.
• Contandriopoulos, D., & Brousselle, A. (2012). Evaluation models and evaluation use. Evaluation, 18(1), 61-77.
• Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research in evaluation utilization. Review of
Educational Research, (536), 331-364.
• Højlund, S. (2014a). Evaluation use in the organizational context–changing focus to improve theory. Evaluation,
20(1), 26-43.
• Højlund, S. (2014b). Evaluation use in evaluation systems–the case of the European Commission. Evaluation,
20(4), 428-446.
• Johnson, K., Greenseid, L. O., Toal, S. A., King, J. A., Lawrenz, F., & Volkov, B. (2009). Research on Evaluation
Use A Review of the Empirical Literature From 1986 to 2005. American Journal of Evaluation, 30(3), 377-410.
• Loud, M. L., & Mayne, J. (Eds.). (2013). Enhancing evaluation use: insights from internal evaluation units. Beverly
Hills, CA: Sage Publications.
• Mark, M. M., & Henry, G. T. (2004). The mechanisms and outcomes of evaluation influence. Evaluation, 10(1), 35-
57.
• O’Neil, G. (2015) Communication evaluation in international organisations: methodology, influence and use. PhD
thesis, The London School of Economics and Political Science (LSE).
10