How does the personality of ratees relate to their multi-rater feedback outcomes?

1. © 2017 Peter Berry Consultancy Pty Ltd.
Daniel Yee
July 2017
Ratee Personality and
Multi-rater Feedback
How does the personality of ratees relate to
their multi-rater feedback outcomes?

2. © 2017 Peter Berry Consultancy Pty Ltd.
BACKGROUND

3. © 2017 Peter Berry Consultancy Pty Ltd.
Personality and Performance
▪ Significant foundation of research supporting the importance of personality characteristics
in employee performance and general employability (Schmidt & Hunter, 1998; Hogan & Holland, 2003; Hogan,
Chamorro-Premuzic, & Kaiser, 2013)
▪ For employees, the key characteristics revolve around being:
▪ Rewarding, Able and Willing
▪ Day-to-day personality traits like conscientiousness and openness typically predict overall
job performance (Strang & Kuhnert, 2009)
▪ Different aspects of behaviour and performance are predicted by different personality-
based characteristics (e.g., Getting ahead with social confidence) (Hogan & Holland, 2003)

4. © 2017 Peter Berry Consultancy Pty Ltd.
Personality and Leadership
▪ Individual differences like personality and self-efficacy are associated with leader
effectiveness (Ng, Ang, & Chan, 2008)
▪ Leaders with greater emotional stability and conscientiousness tend to receive better
effectiveness ratings (Ng, Ang, & Chan, 2008)
▪ Big Five personality traits (particularly Extraversion and Openness) are related to
leadership performance, emergence, and perceptions of change leadership (Strang & Kuhnert,
2009; Bergman, Lornudd, Sjoberg, & Schwarz, 2014)
▪ Agreeableness and conscientiousness are particularly important for ethical and
employee-oriented (transformational) leadership (Bergman et al., 2014; Judge & Bono, 2000)
▪ Dark side personality traits (e.g., scepticism) are negatively associated with leader
performance (Gaddis & Foster, 2013)

5. © 2017 Peter Berry Consultancy Pty Ltd.
Personality and Multi-rater Feedback
▪ Ratees that are more responsible and open to new experiences are more likely to welcome
and act on feedback from a multi-rater process, while those who are more emotionally
stable are more likely to be motivated to use the results (Smither, London, & Richmond, 2005)
▪ Ratees that are more agreeable are more likely to care about what others think of them,
while those who are more socially confident are not (Smither, London, & Richmond, 2005)
▪ Achievement-orientation positively associated with feedback responsiveness (Nowack & Mashihi,
2012)
▪ Typical to explain between 0 to 27% of variance in multi-rater outcomes (Bergman et al., 2014)
▪ FFM traits predict both task performance (e.g., Results) ratings and contextual performance
(e.g., Coaching) ratings (Oh & Berry, 2009)

6. © 2017 Peter Berry Consultancy Pty Ltd.
Limitations of Previous Research
▪ Defined personality from a normal lens only (Five Factor Model)
▪ Lack of robust statistical methodologies used to assess the relationship between ratee
characteristics and multi-rater outcomes – e.g. Hierarchical regression
▪ Few studies have looked at the relative explanatory variance of different personality
constructs (see Bergman et al. 2014 for an example)
▪ Typically poor/absence of theoretical frameworks to drive research objectives and
understand findings – e.g. Socioanalytic theory
▪ Need for more research using multi-rater feedback tools with good psychometric
properties

7. © 2017 Peter Berry Consultancy Pty Ltd.
PRESENT STUDY

8. © 2017 Peter Berry Consultancy Pty Ltd.
Framework
▪ Personality assessment
• Motives, Values, Preferences Inventory
(MVPI)
• 10 motivational theory-driven values
• Focuses on understanding how individuals
draw motivation from:
• Getting along with others
• Getting ahead of others
• Finding meaning(Hogan & Hogan, 2010)

9. © 2017 Peter Berry Consultancy Pty Ltd.
Framework
▪ Personality assessment
• Hogan Personality Inventory (HPI)
• 7 day-to-day personality-based factors
• Based on FFM
• Provides an indication of an individual’s likely
reputation with others
(Hogan & Hogan, 2013)

10. © 2017 Peter Berry Consultancy Pty Ltd.
Framework
▪ Personality assessment
• Hogan Development Survey (HDS)
• 11 derailment risks
• Predicts counter-productive work behaviour
and reputation
• Uncovers risks that are likely to emerge when
individuals are not at their best / not self-
monitoring as effectively as usual
(Hogan & Hogan, 2009)

11. © 2017 Peter Berry Consultancy Pty Ltd.
Framework
▪ Multi-rater feedback (PBC, 2015)
Hogan 360
• Hogan 360
• Gathers feedback from multiple rater groups
• 50 rate-on-scale items (7-pt Likert type),
ranked items, open comments
• Internally reliable, construct and criterion-valid

12. © 2017 Peter Berry Consultancy Pty Ltd.
Framework
▪ Multi-rater feedback (PBC, 2015)

13. © 2017 Peter Berry Consultancy Pty Ltd.
Methodology
▪ Global sample of 1,084 leaders collected between 2011 and 2015
▪ Leaders completed personality assessment and participated in a multi-rater feedback process
▪ Private, public, not-for-profit sectors represented
▪ Cross-industry
▪ Hierarchical linear regression analyses

14. © 2017 Peter Berry Consultancy Pty Ltd.
FINDINGS

15. © 2017 Peter Berry Consultancy Pty Ltd.
Findings
▪ Day-to-day personality accounted for 15.3%*** of the variance
in ratings
▪ Derailment risk +2.7%**, core values +1.9%*
▪ Ratees who were more emotionally stable and agreeable
received higher ratings
▪ Ratees at greater risk of becoming overly cautious received
higher ratings, while those at greater risk of becoming
emotionally volatile received lower ratings
▪ Ratees motivated by certainty and managing risk received
higher ratings, while those motivated by pleasure and
stimulation received lower ratings
Self Management
(p < .001***, p < .01**, p < .05*)

16. © 2017 Peter Berry Consultancy Pty Ltd.
Findings
▪ Day-to-day personality accounted for 10.2%*** of the variance
in ratings
▪ Derailment risk +2.3%*, core values non-sig
▪ Ratees who were more agreeable received higher ratings,
while those who were more curious and creative received
lower ratings
▪ Ratees at greater risk of becoming attention-seeking and
dramatic received higher ratings
Relationship Management
(p < .001***, p < .01**, p < .05*)

17. © 2017 Peter Berry Consultancy Pty Ltd.
Findings
▪ Day-to-day personality accounted for 6.6%*** of the variance
in ratings
▪ Derailment risk +3.5%***, core values +1.7%*
▪ Ratees who were more proactive with and open to learning
received higher ratings
▪ Ratees at greater risk of becoming overly perfectionistic
received higher ratings, while those at greater risk of
becoming stubborn and privately uncooperative received
lower ratings
▪ Ratees motivated by achievement and influence received
higher ratings, while those motivated by social interaction as
well as technical and analytical interests received lower
ratings
Working In the Business
(p < .001***, p < .01**, p < .05*)

18. © 2017 Peter Berry Consultancy Pty Ltd.
Findings
▪ Day-to-day personality accounted for 6.6%*** of the variance
in ratings
▪ Derailment risk +4.0%***, core values +1.9%*
▪ Ratees who were more confident and assertive received
marginally higher ratings (p = .06)
▪ Ratees at greater risk of becoming attention-seeking and
dramatic as well as unconventional and imaginative received
higher ratings, while those at greater risk of becoming
stubborn and privately uncooperative received lower ratings
▪ Ratees motivated by social interaction as well as technical
and analytical interests received lower ratings
Working On the Business
(p < .001***, p < .01**, p < .05*)

19. © 2017 Peter Berry Consultancy Pty Ltd.
IMPLICATIONS

20. © 2017 Peter Berry Consultancy Pty Ltd.
Theoretical Implications
▪ Majority of research has examined the role of personality using a narrow lens /
perspective – these findings may suggest that a lot of the associations may be explained
by additional or different personality-based characteristics
▪ Contrary to some research evidence, day-to-day personality characteristics are
associated with some not all multi-rater outcomes
▪ Personality-based derailment characteristics may not always be detrimental to multi-rater
feedback outcomes
▪ Values have long been understood to be important for both task and contextual
performance – these findings suggest they may play a role in multi-rater feedback
▪ Further study is needed to disentangle these results further (e.g., rater group differences)

21. © 2017 Peter Berry Consultancy Pty Ltd.
Practical Implications
▪ Certain personality characteristics of ratees may facilitate greater social desirability and
subsequent multi-rater feedback outcomes
▪ To understand individuals’ impact in the workplace, it is important to examine individual-
level differences as holistically as possible
▪ For individuals, these findings provide insights into perceptions of what makes an
effective leader
▪ For organisations, these findings provide a foundation for looking at what intervention
strategies might be more successful when targeting specific competency gaps across
individuals and cohorts

22. © 2017 Peter Berry Consultancy Pty Ltd.
Certain personality-based characteristics and values of
ratees appear to be tied to the multi-rater feedback
outcomes they receive. We all have a responsibility to
continue to understand how we might be involved in the
phenomena presented, whether it be as ratees or as
raters.
Conclusion

23. © 2017 Peter Berry Consultancy Pty Ltd.
QUESTIONS