Talent Management – What’s the Evidence? With Rob Briner & Eric Barends 28th April 2016, 3:00PM-5:00PM, IMI Conference Centre, Dublin 16 In this session, Rob Briner, Prof. of Organisational Psychology, University of Bath and Eric Barends, MD, Centre for Evidence-Based Management, put talent management under the microscope. They challenge some of the traditional thinking behind talent management shaped almost 20 years ago by The War for Talent – including the idea that potential can readily be identified in complex roles and that the best organisations have the best people.

1. Talent Management
What’s The Evidence?
IMI Talent Forum, 28th April 2016, Dublin

2. 30 min Talent Management: What is it what does it do?
20 min What the problem and solution? What’s your
evidence?
20 min An alternative model: EBP
15 min Case
30 min Discussion / questions
Outline

3. Dangers of believing things that aren’t true
 It ain't what you don't know that gets
you into trouble. It's what you know for
sure that just ain't so. (Mark Twain)
 The greatest enemy of knowledge is
not ignorance, it is the illusion of
knowledge. (Stephen Hawking)

4. Some examples
What’s happened to average job tenure in
the UK and US over the past 10 years?
 Up?
 Down?
 Stayed the same?

7. Some examples: Job satisfaction
What’s happened to job satisfaction in the
US over the past 10 years?
 Up?
 Down?
 Stayed same?

9. Some examples: VUCA
Is the world moving ever faster – more
VUCA (volatility, uncertainty, complexity and ambiguity)?
 Faster?
 Slower?
 Same?

11.  The idea that time is speeding up is clearly popular. It is
also plausible. There is just one problem. It is very hard to
prove that it is actually happening.
 Hard evidence of a great acceleration is hard to come
by. The Economist has considered a variety of measures
by which the speed of business in America can be
quantified. A few do show some acceleration. But a lot do
not.

13. UK Patents

14. Some examples: Generational differences
Are there generational differences in work
attitudes?
 Big differences?
 Small differences?
 No differences?

16. So what?
 There are many taken-for-granted assumptions
and ‘truths’ in HR that turn out to be wrong or at
least overstated
 It’s important to examine these assumptions
because if they are wrong will lead to poor
decisions
 Possibly also in the case of Talent Management

17. 20 min Talent Management: What is it what does it do?
20 min What the problem and solution? What’s your
evidence?
30 min An alternative model: EBP
20 min Case
30 min Discussion / questions
Outline

18. All big consulting firms claim it’s important …

21. You use Talent Management: Right?
But, What Is Talent Management?
0Discuss with your neighbour (1 min)
Discussion

25. nice pictures ...

26. … and a plethora of models

27. Model 1

28. Model 2

29. Model 3

30. Model 4

31. Model 5

32. etc etc

33.  Version 1: More or less exactly the same as
strategic HRM – it’s most things HR does
 Version 2: One specific thing HR does (e.g.,
succession)
 Version 3: Full-on, hardcore War for Talent – it’s
talented individuals that determine
organizational success – our focus now
‘Talent Management’: 3 versions
Ariss et al. Talent management: Current theories and future research directions. Journal of
World Business, 2014, vol. 49, issue 2, pages 173-179

34. Talent Management:
Where does it come from?

35. McKinsey (1997 paper / 2001 book)

36. “If you pick the right people and give them the
opportunity to spread their wings and put
compensation as a carrier behind it you don't
have to manage them.”
How is it assumed
to work?
Jack Welch
(Straight from the gut)

37. McKinsey: case study
?

38. McKinsey: case study

39. War on Talent

40. Talent Management:
The scientific literature

44. Discuss with your neighbour (1 min):
What Is (a) Talent?
0

47. H E A L T H W E A L T H C A R E E R
NOVEMBER 23 2015
Iulia Alina Cioca
Lorenzo Gallì
Milano
A S S E S S M E N T O F
H I G H - P O T E N T I A L
R E S E A R C H R E S U L T S

48. © MERCER 2015 48
C O N C L U S I O N
D E F I N I T I O N O F H I G H P O T E N T I A L
Most definitions come from unreliable sources of evidence, and are not valid. Some
examples:
“The employee who leaves tracks in the sand”
“An individual with something special hidden in
their lives that, under the right conditions, could
emerge and be leveraged for greater success.”
”

49. © MERCER 2015 49
R E S U L T S
T H E C H A R A C T E R I S T I C S O F H I G H P O T E N T I A L S
A number of 41 characteristics were mentioned in the literature
Leadership skills (34%)
Learning ability (26%)
Motivation (24%)
Competencies specific to the job or
career path they will follow (23%)
Risk taking (20%)
General cognitive ability (18%)
Ambition (to advance, to excel
(18%)
Adaptability (15%)
Fit with the values of the
organization (15%)
Teamwork (13%)
Results orientation (13%)
Communication skills (11%)
Action-oriented (11%)
Curiosity (11%)
Generic competencies (that allow
performance in various roles (11%)
Broad perspective on the whole
business (11%)
High performance record (11%)
Interpersonal skills (10%)
Trustworthiness (10%)
Intercultural competencies (8%)
Commitment (8%)
Creativity (7%)
Alternative, different and challenging
way of thinking (7%)
Seeks/uses feedback (7%)
Is available for mobility (7%)
Competencies specific to the
organization as a whole (7%)
Dealing with ambiguity (5%)
Self-confidence (5%)
Integrity (5%)
Strategic thinking (3%)
Using information (to take decisions,
to convince, etc. (3%)
Assertiveness (3%)
Emotional intelligence (3%)
Conscientiousness (3%)
Proactivity (3%)
High education level (3%)
Customer orientation (3%)
Analytical thinking (2%)
Problem solving (2%)
Resilience (2%)
Tolerance (2%)

50. Some evidence-based insights …
1. Talent management is an ill-defined
construct. It lacks a precise terminology
as well as a reliable and valid approach
regarding the measurement of its
effect.
And if you can’t measure it …

51. Some evidence-based insights …
2. The idea that talent (or potential) can
be readily identified in relatively
complex jobs is plain wrong.
A senior executive may have a splendid track record but how much
of their apparent success can be attributed to their abilities?
A junior manager may score brilliantly on every appraisal thrown
at them, but how good is the evidence that those scores predict
future performance? It’s hard to define or assess talent.

52. 3. The idea that the best organisations have
the best people just doesn’t stand up to
scrutiny..
Success depends on a whole range of factors, not least having
good systems, processes and co-ordination.
Having talented individuals may help a little bit in some roles, but
other things are likely to be much more important. In other words,
we over-rate our ‘stars’
Some evidence-based insights …

53. 4. There is even evidence that selecting and
promoting people at random may work just
as well
Some evidence-based insights …
Pluchino, A., Rapisarda, A. and Garofalo, C., “The Peter Principle Revisited: A
Computational Study,”, Physica A, vol. 389, no. 3, February 2010, pp. 467-72.
Haslam, S. A, C. McGarty, R. A. Eggins, B. E. Morrison, & K. J. Reynolds,
“Inspecting the Emperor’s Clothes: Evidence that Randomly Selected Leaders
can Enhance Group Performance”, Group Dynamics: Theory, Process and
Research 2 (1998): 168-18

54. … and some critical questions
1. Why do we need to recruit the most
talented person? Isn’t a good enough
person, well, good enough?
In almost all performance domains, having more ability only makes
a difference to performance up to a point. So why spend time
recruiting or developing the most talented people?

55. … and some critical questions
2. Given the fact that ‘talent’ is hard to
measure / identify: have you considered
the consequences of false positives and
false negatives?

56. … and some critical questions
3. What might be the unintended negative
sides-effects of identifying talent on those
who are not identified as talent? All HR
practices that have any effect may also
have negative side-effect.

57. Great! …….now what?

58. 30 min Talent Management: What is it what does it do?
20 min What the problem and solution? What’s your
evidence?
20 min An alternative model: EBP
20 min Case
30 min Discussion / questions
Outline

59.  Returning to your own organization and what YOU mean
by talent management
 At some point your team made a decision about some
problem that needed to be fixed and a TM solution that
would fix it
 We are going to consider (discussing in pairs) your
evidence for
 The problem(s) you identified which you TM
activities are aiming to fix
 The solutions (your TM activities) you chose to fix
those problems
Exercise: What’s your evidence?

60. What’s the evidence for the problem(s) you identified?
(ask each other these questions)
 What exactly was the problem? Be as specific and precise
as possible (probe!)
 What types of evidence or information did you have about
the problem (internal data, external, professional
judgement and experience, scientific, stakeholders)
 How much evidence?
 How trustworthy or reliable was that evidence?
 Overall how confident were you that you identified a real,
specific, important and pressing problem?
Exercise: What’s your evidence?

61. What’s the evidence for the TM solution(s) you identified
(ask each other these questions)
 What exactly was the solution? Be as specific and precise as
possible (probe!)
 What types of evidence or information told you that the
solution was likely to work (internal data, external,
professional judgement and experience, scientific,
stakeholders)
 How much evidence?
 How trustworthy or reliable was that evidence?
 Overall how confident were you that you identified an
effective and relevant solution?
Exercise: What’s your evidence?

62. 30 min Talent Management: What is it what does it do?
20 min What the problem and solution? What’s your
evidence?
20 min An alternative model: EBP
15 min Case
30 min Discussion / questions
Outline

63. Evidence based management:
What is it?

65. Evidence-based management
Central Premise:
Decisions should be based on
the ‘best available evidence‘.

66. Evidence?
information, facts or data
supporting (or contradicting)
a claim, assumption or hypothesis

67. Evidence?
outcome of scientific research,
organizational facts & figures,
benchmarking, best practices,
personal experience

68. All managers and leaders base
their decisions on ‘evidence’

69. But…many managers pay
little or no attention to
the quality of the evidence
they base their decisions on
and use
too few sources of evidence

70. Trust me, 20 years of
management experience

71. Multiple sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

72. Evidence based practice:
Where does it come from?

73. Evidence-Based Practice
1991 Medicine
1998 Education
2000 Social care, public policy
Nursing, Criminal justice,
Policing, Architecture, Conservation
2010 Management

76. Evidence-Based Practice

77. Evidence-Based Practice

78. Evidence-based … whatever
=
the use of evidence from multiple
sources to increase the likelihood of a
favourable outcome
Focus on the decision making process
Think in terms of probability

79. In general: managers don’t seem to like
EBMgt

80.  Undermines formal authority (JFDI, HiPPO)
 They feel it constrains freedom to make
managerial decisions
 Speed valued and rewarded more than accuracy
 Managers not necessarily rewarded for doing
what works (organizations rarely evaluate)
 SEEM TO LOVE FADS & QUICK FIXES
Why don’t managers like EBMgt?

82. 82

83. Evidence-Based Decision-Making
Why do we need it?

84. Advice: lie babies down to sleep on their belly
(unanimous support through to the 1990s)
Example: medicine

85. Nr of cot deaths (Holland)

86. Collateralized Debt Obligations > AAA
p = 0.12 (about 1 chance in 850) default in 5 years
Example: finance

87. Forecasted Actual
Forecasted and actual 5-year default rates for
AAA-rated CDO tranches
0.12%
28%

88. Scared straight
Example: policy / prevention

90. Example: HR management

91. 1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
Likely or unlikely?

92.  959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)
Likely or unlikely?

93. Outcome: not better than random chance

94. Relying on only 1 source: bad idea!
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

95. Discuss with your neighbor (1 min)
Over a 5 year period,
why is an orthopedic surgeon's
experience, as a rule, more trustworthy
than an change manager’s experience?
60595857565453525150494847464544434241403938373635343332313029282726252423222120191817161514131211109876543210

96. Developing expertise
1. A sufficiently regular, predictable environment
2. Numerous opportunities to practice
3. Receive accurate (objective) feedback
The management domain is not highly
favorable to expertise!

98. Bounded rationality

99. How your brain works
System 1
 Fast
 Intuitive, associative
 heuristics & biases
 emotional
System 2
 Lazy
 Slow
 Deliberate
 Rational

101. System 1: short cuts

102. System 1 or system 2?
10 seconds

103. System 1 or system 2?
 A bat and a ball cost $1.10 in total.
 The bat costs $1 more than the ball
 How much does the ball cost?
0

104. System 1: necessary to survive
95%

105.  Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

106.  Pattern recognition
 Overconfidence bias
 Halo effect
 False consensus effect
 Group think
 Self serving attribution bias
 Sunk cost fallacy
 Cognitive dissonance reduction
System 1: cognitive errors
 Confirmation bias
 Authority bias
 Small numbers fallacy
 In-group bias
 Recall bias
 Anchoring bias
 Availability bias

107. “I’ve been studying judgment for 45 years, and I’m no better
than when I started. I make extreme predictions. I’m over-
confident. I fall for every one of the biases.”

108. Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess
Four sources of evidence (not only 1)

109. 30 min Talent Management: What is it what does it do?
20 min What the problem and solution? What’s your
evidence?
20 min An alternative model: EBP
15 min Case
30 min Discussion / questions
Outline

110. The performance of
knowledge workers
A Practical Example

111. Discuss with your neighbour
0
What are the strongest predictors of
future performance?

113. Predictors (r & R2)
1. Work sample test .54 29%
2. General mental ability .51 26%
3. Structured interviews .51 26%
4. Peer ratings .49 24%
5. Job knowledge tests .48 23%
6. Job try out .44 19%
7. Integrity test .41 17%

114. Predictors (r & R2)
8. Employment interview .38 14%
9. Assessment centers .37 14%
10. Reference check .26 6%
11. Job experience .18 3%
12. Years of education .10 1%
13. Interests .10 1%
14. Age -.01 0%

115. Predictors (r & R2)
GMA & work sample test .63 40%
GMA & conscientiousness .60 36%

116. Knowledge workers
Whether nurses, lawyers, engineers, managers, or staff members,
nowadays most workers in organizations are highly dependent on
information and communication technology and are involved in
work that involves a high level of cognitive activity.

117. Discuss with your neighbour
0
“Which of the factors that are related to the productivity of
knowledge workers are most widely studied and what is
known of their effect?”

118. I Don’t Know
(but I know how to find out)
The 3 hardest words in management

119. Step 2: ACQUIRE
Search for the best available scientific evidence

120.  ABI, BSP, PsycINFO
 Scholarly journals, peer reviewed
 1980 – 2013
 English
 performance, productivity, knowledge work*
ACQUIRE

122. step 3: APPRAISE & AGGREGATE

123. Effect size?

124. Largest effect
1. Social cohesion .5 / .7
2. Perceived supervisory support .5
3. Information sharing / TM .5
4. Vision / goal clarity .5
5. Trust .3 / .6

125. step 3b: CROSS VALIDATE

126. Step 4: APPLY

127. Three examples
social cohesion supervisory
support
information
sharing

128. Social cohesion

129. Social cohesion
… a shared liking or team attraction
that includes bonds of friendship, caring,
closeness, and enjoyment of each
other’s company.

130. Social cohesion

131. Measuring social cohesion

132. Perceived supervisory support

133. …how employees feel the supervisor
helps them in times of need, praises
them for a job well done or recognizes
them for extra effort.
Perceived supervisory support

134. Perceived supervisory support

135. Measuring perc. sup. support

136. Information sharing

137. Information sharing?
…refers to how teams pool and access their
knowledge and expertise – which positively
affects decision making and team processes.
This has led to the idea of a team ‘Transactive
Memory System’ (TMS), which can be thought of
as a collective memory in a collective mind -
enabling a team to think and act together

138. Information sharing

139. Measuring information sharing

140. The departments with the lowest performance
scored under average on most factors
Step 5: ASSESS

141. Reactions
Who
knew?

142. 30 min Talent Management: What is it what does it do?
20 min What the problem and solution? What’s your
evidence?
20 min An alternative model: EBP
15 min Case
30 min Discussion / questions
Outline

144. Postgraduate Course

145. Postgraduate Course

146. > 80 Fellows

147. CEBMa: what we do
 Promote (seminars, papers, blogs, tweets)
 Educate (universities & business schools)
 Train & coach (companies > projects)
 Support / REAs (companies)
 Support / 2nd opinion (BS detector)