Integrating Communications in
Evaluation
How can communications be used to optimally
support the evaluation process?
Geneva Evaluation Network
ILO Evaluation Office
30 January 2020
Dr Glenn O’Neil
oneil@owlre.com
www.owlre.com
“Evaluation without communications would not
be possible”
Alkin, Christie & Rose (2006) Communicating evaluation
“Commissioners of evaluations complain that
the messages from evaluations are not useful,
while evaluators complain that the messages
are not used.” Cronbach (1980) Toward reform of program evaluation
“Communication overload is a common fault;
many an evaluation is reported with self-
defeating thoroughness” Cronbach (1980)
2
What is communications?
“The process through which people
share thoughts, ideas and feelings
with each other in commonly
understood ways”
Hamilton (2013) Communicating for Results
3
Communication aims to achieve
change…to knowledge, attitudes,
behavior and practices
Five Myths
Let us look at five myths of communications
and evaluations – myths - or assumptions
that I often come across where evaluation
does not take into account communication
theory and practice
4
5
Myth #1: communication is one way
ReceiverSenderu
We often see communications in evaluation based on the “silver bullet”
theory of communications – a sender communicates and the receiver
understands! This dates from some 100 years ago and has been proven
to not work in most situations
6
Myth #1: communication is one way
ReceiverSenderu
Noise
Competing
messages
Context
Ability to
receive
Ability to
send
Channel
used
Frequency
The “silver bullet” model was challenged by the “minimal effects” model
that identified many factors that can influence the ability to both send
and receive communications – from an evaluation perspective we
experience many of these in our communications
7
Myth #1: communication is one way
ActorsActors Mutual
understanding
The last model developed was Grunig’s* two way symmetrical model –
it found that sender/receiver was obsolete – we have two parties that
are communicating – and should be on equal grounds – what is known
as a dialogue to develop mutual understanding
*Grunig, J., Grunig, L. (1992) Models of public relations and communication.
8
Myth #1: communication is one way
Grunig’s research found two key points:
1) That many organisations have different models of communications
functioning – from the one-way model to a fully interactive two-way model
2) The most effective model was the two-way model
For an evaluation, we will communicate in both one- and two-way manners
- but a dialogue will likely increase the success of our communications,
e.g. use of the evaluation findings
9
Myth #2: We are already
communicating
“The single biggest problem in communication is the
illusion that it has taken place” George Bernard Shaw
An evaluation is constantly communicating to ensure it
progresses … but communication activities are those that
are planned and carried out systematically to support
identified needs
As early as possible in an evaluation we should carry out an analysis
of communication needs:
Situation
analysis
Communication
objectives
Stakeholders /
audiences
Communication
activities
10
Myth #3: An evaluation only
communicates findings
AfterDuringBefore
 Signal evaluation approach
 Create awareness
 Develop ownership
 Build relationships
 Manage expectations
 Communicate progress
 Maintain relationships
 Present initial findings
 Test findings
 Feedback from stakeholders
 Present findings
 Dialogue on findings
 Promote usage
Communication has a support role to play in all three stages of an
evaluation:
Possible communication needs per stage:
11
Myth #4: Stakeholders will receive and
understand what is communicated
Channel
Format
Interest
The
messenger
Stakeholders
These factors will influence
the ability of stakeholders to
receive and understand
communications
Context
ContextContext
Context
12
Myth #5: If the evaluation is
known it will bring about change
Potential evaluation
users
Evaluation
Report
Evaluation
Process
Learn
Use
Participation
#§¬¢{ }!
Many people will
learn from the
process
People will learn
from different formats
of evaluation findings
Evaluation findings are
absorbed with other
inputs, context/culture,
feelings and interests
This is greatest
predictor of use!
Thus need for
2-way
communications
Evaluation use
is often indirect,
opportunistic
and
unpredictable
13
Using communications optimally for
evaluation
1. Assess the communication needs of an evaluation early on
2. Design communication activities to support the evaluation
process and not only the dissemination of findings
3. Use different channels and formats to communicate
4. Encourage communications as a dialogue to support the
evaluation process and use of findings
5. Consider that most people will learn from the evaluation
through the process – and of findings verbally or visually –
not through the written report
6. The ability to communicate will depend on the
organisation’s evaluation culture, policies and approaches
14
Further resources
oneil@owlre.com
glennoneil
www.owlre.com
@glenn_oneil
www.intelligentmeasurement.net
Contacts:
Resources on communication and evaluation:
https://intelligentmeasurement.net/category/communicating-evaluation-results//
Download the guide:

Integrating communications in evaluation

  • 1.
    Integrating Communications in Evaluation Howcan communications be used to optimally support the evaluation process? Geneva Evaluation Network ILO Evaluation Office 30 January 2020 Dr Glenn O’Neil oneil@owlre.com www.owlre.com
  • 2.
    “Evaluation without communicationswould not be possible” Alkin, Christie & Rose (2006) Communicating evaluation “Commissioners of evaluations complain that the messages from evaluations are not useful, while evaluators complain that the messages are not used.” Cronbach (1980) Toward reform of program evaluation “Communication overload is a common fault; many an evaluation is reported with self- defeating thoroughness” Cronbach (1980) 2
  • 3.
    What is communications? “Theprocess through which people share thoughts, ideas and feelings with each other in commonly understood ways” Hamilton (2013) Communicating for Results 3 Communication aims to achieve change…to knowledge, attitudes, behavior and practices
  • 4.
    Five Myths Let uslook at five myths of communications and evaluations – myths - or assumptions that I often come across where evaluation does not take into account communication theory and practice 4
  • 5.
    5 Myth #1: communicationis one way ReceiverSenderu We often see communications in evaluation based on the “silver bullet” theory of communications – a sender communicates and the receiver understands! This dates from some 100 years ago and has been proven to not work in most situations
  • 6.
    6 Myth #1: communicationis one way ReceiverSenderu Noise Competing messages Context Ability to receive Ability to send Channel used Frequency The “silver bullet” model was challenged by the “minimal effects” model that identified many factors that can influence the ability to both send and receive communications – from an evaluation perspective we experience many of these in our communications
  • 7.
    7 Myth #1: communicationis one way ActorsActors Mutual understanding The last model developed was Grunig’s* two way symmetrical model – it found that sender/receiver was obsolete – we have two parties that are communicating – and should be on equal grounds – what is known as a dialogue to develop mutual understanding *Grunig, J., Grunig, L. (1992) Models of public relations and communication.
  • 8.
    8 Myth #1: communicationis one way Grunig’s research found two key points: 1) That many organisations have different models of communications functioning – from the one-way model to a fully interactive two-way model 2) The most effective model was the two-way model For an evaluation, we will communicate in both one- and two-way manners - but a dialogue will likely increase the success of our communications, e.g. use of the evaluation findings
  • 9.
    9 Myth #2: Weare already communicating “The single biggest problem in communication is the illusion that it has taken place” George Bernard Shaw An evaluation is constantly communicating to ensure it progresses … but communication activities are those that are planned and carried out systematically to support identified needs As early as possible in an evaluation we should carry out an analysis of communication needs: Situation analysis Communication objectives Stakeholders / audiences Communication activities
  • 10.
    10 Myth #3: Anevaluation only communicates findings AfterDuringBefore  Signal evaluation approach  Create awareness  Develop ownership  Build relationships  Manage expectations  Communicate progress  Maintain relationships  Present initial findings  Test findings  Feedback from stakeholders  Present findings  Dialogue on findings  Promote usage Communication has a support role to play in all three stages of an evaluation: Possible communication needs per stage:
  • 11.
    11 Myth #4: Stakeholderswill receive and understand what is communicated Channel Format Interest The messenger Stakeholders These factors will influence the ability of stakeholders to receive and understand communications Context ContextContext Context
  • 12.
    12 Myth #5: Ifthe evaluation is known it will bring about change Potential evaluation users Evaluation Report Evaluation Process Learn Use Participation #§¬¢{ }! Many people will learn from the process People will learn from different formats of evaluation findings Evaluation findings are absorbed with other inputs, context/culture, feelings and interests This is greatest predictor of use! Thus need for 2-way communications Evaluation use is often indirect, opportunistic and unpredictable
  • 13.
    13 Using communications optimallyfor evaluation 1. Assess the communication needs of an evaluation early on 2. Design communication activities to support the evaluation process and not only the dissemination of findings 3. Use different channels and formats to communicate 4. Encourage communications as a dialogue to support the evaluation process and use of findings 5. Consider that most people will learn from the evaluation through the process – and of findings verbally or visually – not through the written report 6. The ability to communicate will depend on the organisation’s evaluation culture, policies and approaches
  • 14.
    14 Further resources oneil@owlre.com glennoneil www.owlre.com @glenn_oneil www.intelligentmeasurement.net Contacts: Resources oncommunication and evaluation: https://intelligentmeasurement.net/category/communicating-evaluation-results// Download the guide: