2. 2
Preamble
This is nothing personal
It’s not about you or your career or your
organization nor mine
It’s not about your opinion or experience nor
mine
It’s not about being optimistic or pessimistic
It is about objectively assessing the quality and
quantity of evidence for employee engagement
It is about thinking critically about what this
means for practice
2
3. 3
Outline
What is an evidence-based approach to
anything?
Why isn’t HR and management evidence-
based?
Five key evidence-based challenges for
employee engagement
Why isn’t employee engagement more
evidence-based?
3
4. 4
What’s the problem evidence-based
practice aims to fix?
Practitioners in all fields use evidence (or
information) in their decision-making
BUT the evidence used is often limited
– Not enough of it
– Not from/of multiple and diverse sources/types
– Not critically appraised for its quality or relevance
– Not used in a systematic way (e.g., not focusing
on the best available evidence, not weighting, not
aggregating)
4
5. 5
Why is this a problem? The irrefutable
logic of evidence-based practice
Using limited quantities, sources, and types
of evidence in an uncritical and unsystematic
way will increase the chances of poor
decision processes and unintended outcomes
Using greater quantities, sources and types
of evidence in a critical and systematic way
will increase the chances of better decision
processes and intended outcomes
5
6. 6
Why is it a particular problem in
management and HR?
It may not be but management has come
very late to evidence-based practice
The concept is not well-known or understood
by organizations, managers or management
schools
It sounds weird in a management context
but is common in other professions and
disciplines
6
17. 17
What is it?
Evidence-based practice is about making decisions
through the conscientious, explicit and judicious
use of the best available evidence from multiple
sources to increase the likelihood of a favourable
outcome.
17
19. 19
Why isn’t HR and management more
evidence-based?
The training and skills of HR and managers
Organizations driven more by power and politics
and short-termism than evidence
The power of management fads: The ‘solution
in search of a problem’ problem
More effort spent on problem solving rather
than problem identification
The business models of management
consultancies
Management/business schools prioritize other
things in their teaching and research
19
20. 20
Key challenges for engagement
1. Defining engagement
2. Measuring engagement
3. Engagement is nothing new or different
4. There is almost no good quality evidence
with which to answer the most important
questions about engagement
5. Over-claiming and mis-claiming the
importance and role of engagement.
20
21. 21
1. Defining engagement
No agreed definition
Definitions highly inconsistent (behaviour, attitudes, feelings, work
conditions, various combinations, causal combinations)
Creates serious practical problems as we literally do not know what
we are talking about
– contributes to a deep misconception of the complexities around the
concept
– engagement just becomes an umbrella term for whatever one wants it
to be
– makes the state of knowledge of employee engagement difficult to
determine
– adding this term to our vocabulary…has done more to confuse than to
clarify
If engagement is important why don’t we take definition and
meaning seriously
21
22. 22
2. Measuring engagement
If definitions are poor and confused measures are likely to
be the same
Correlate very highly with traditional attitudes (e.g., Gallup
Q12 correlates .9 with job satisfaction and .8 with
commitment)
Analysis shows many engagement items identical to items
in traditional attitude measures
Only one published study found engagement scores to be
associated with (not cause) performance over and above
traditional attitudes
No studies showing engagement has predictive validity
If engagement is important why don’t we take
measurement seriously? (Don’t get me started on
engaged/neutral/disengaged categorization)
22
23. 23
3. Engagement is nothing new or
different
If it is not new it adds no value
– The…concept does not constitute new content
– does not necessarily add conceptual or
phenomenological clarity
– nothing new with respect to how attitudes and
performance are related...puts old wine in new
bottles
– Failure to make distinctions and define and measure
engagement in terms of older constructs is likely to
muddy the engagement water even more
There appears to be nothing new about
engagement so it therefore adds no value
23
24. 24
3. Engagement is nothing new or
different
Only two possibilities
– Engagement is not a new and different idea: If
so the term and idea should be abandoned because
using a new term to describe existing concepts is
confusing and unhelpful.
– Engagement is a new and different idea: If so
then we need to define and measure engagement in
ways that are demonstrably different from existing
concepts
If engagement is important, why don’t we take
working out if it’s something new seriously?
24
25. 25
4. Almost no good quality evidence
There are only two fundamental quesitons
– Do increases in engagement cause increases in
performance? Does engagement do anything?
– Do engagement interventions cause increases
levels of engagement and subsequent increases
in performance? Can you do anything about
engagement?
If we cannot answer these questions using a
reasonable quantity of good quality evidence
then claims about engagement are
speculative at best or just plain wrong
25
26. 26
4. Almost no good quality evidence
The fundamental questions are causal
– X > Y, covariation of X and Y, no other plausible
explanations
How many published studies of engagement
can establish cause and effect?
What else do we mean by quality of
evidence?
26
28. 28
4. Almost no good quality evidence
28
None
None (3 x-sectional)
None
None
Around 10
Quite a lot
Lots and lots
29. 29
5. Over-claiming and mis-claiming the
importance and role of engagement
Huge number of claims made which
– Exaggerate given the quality of evidence
– Mis-claim given nature of evidence
Claims made by Engage for Success, some
academics, practitioners, consultants
These claims just cannot be trusted and
should not be believed unless better quality
evidence is provided to support them
29
30. 30
5. Over-claiming and mis-claiming the
importance and role of engagement
Does engagement cause performance?
– The relationships…have not been rigorously conceptualized,
much less studied
– Without empirical research…practitioners are especially
vulnerable to positive-sounding repackaging of workplace issues
– we know little about engagement’s uniqueness as a predictor of
job performance
But does satisfaction cause performance?
– The search for a relationship between job satisfaction and job
performance has been referred to as the 'Holy Grail‘…study after
study failed to produce the expected strong relationship
– the satisfaction–performance relationship is largely spurious
– results consistently showed low or no correlation between the
two…correlation only because performing well made employees
more satisfied 30
31. 31
Why isn’t employee engagement more
evidence-based?
Training and incentives for HR
HR loves a fad
If things are viewed as ‘doing good’ they are
not subject to usual scrutiny (e.g., charities,
recently convicted TV celebrities)
Too much good intentions (road to hell, etc)
Lack of focus on the problem – what is the
specific problem engagement can fix?
31
32. 32
My suggestion
Whenever you hear a claim about
engagement ask
– What does engagement mean precisely?
– What is the claim really suggesting?
– What specific evidence is being presented to
support that claim
– How much evidence is there?
– What is the quality of the evidence?
– How much should I trust the claim?
32
