Georgiou, K. & Nikolaou, I. (2017). Gamification in recruitment and selection. In I. Nikolaou (2017): European Network of Selection Researchers (ENESER) Symposium; Recruitment in the Digital Era. 18th congress of the European Association of Work and Organizational Psychology (EAWOP), Dublin Ireland.

1. Ioannis Nikolaou
Konstantina Georgiou
EAWOP 2017
Athens University of Economics and Business
Athens, Greece

2. Gamification - a
top trend in the
field of
recruitment &
selection!
Game elements
are applied to
non-game
contexts to evoke
game like
experiences &
behaviors
“You can
discover more
about a person
in an hour of
play than in a
year of
conversation”
Plato
Effectiveness in
the selection
process?
EAWOP 2017

3.  Improve recruitment process
 applicant pool, organizational image, positive behavioral
intentions (Chow & Chapman, 2013)
 Efficient in candidates skills assessment
 difficult for test-takers to fake, elicit job relevant behaviors,
prediction of job performance (Armstrong, Landers & Collmus,
2016)
EAWOP 2017

4.  Address calls for research on the main psychometric properties of
serious games development (Armstrong et al., 2016)
 Identify new avenues of selection methods by developing a gamified
assessment method
 An empirical analysis of the efficacy of a gamified selection method
based on a Situational Judgment Test (SJT)
 Construct validity
EAWOP 2017

5.  SJTs: a popular personnel selection method (Weekley & Ployhart, 2006)
 assess job-related skills, predict future job performance
 low adverse impact, positive applicant reactions
 video technology successfully applied
 SJT development to assess candidates’ soft skills: resilience, adaptability,
flexibility and decision-making
 Guidelines of Motowidlo, Dunnette, & Carter (1990)
EAWOP 2017

6.  A) 8 Subject Matter Experts:
 Cohen’s Kappa (over .75)
 7 Scenarios per skill, Scoring key
 B) 130 test takers:
 Hit ratio analysis
 Scores per scenario from .60 to .85
(appropriate levels of congruence)
Content Validity
EAWOP 2017

7. Sample:
N=321 business school students/graduates
 Gender: 60.7% female, 39.3% male
 Average Age: 26.5 years (SD = 5.41)
 Education: 15.3% high school, 42.4% bachelor, 41.6%
master, 0.6% PhD
Process:
Completion of SJT & well-established measures:
 Resilience (Wagnild & Young, 1993): α= .89
 Adaptability (Martin, Nejad, Colmar & Liem, 2012): α= .89
 Flexibility (Lee & Ashton, 2004): α= .74
 Decision-making (Mincemoyer & Perkins, 2003): α= .77
Construct
Validity
EAWOP 2017

8. SJT’s Construct Validity:
 Convergent validity
 Regression coefficients for SJT’s dimensions: resilience (β=.350), adaptability
(β=.166), decision-making (β=.389), flexibility (β=.366) are significant (p<.001) and
moderately correlated to well-established measures
 Discriminant validity
 Inter-correlations among SJT’s dimensions at a very low level (e.g. resilience is
related to decision-making (β=.104) and flexibility (β=.-140), p<.001)
EAWOP 2017

9. EAWOP 2017
The SJT was
converted into an
online adventure
gamified assessment
adapting the
scenarios into
fictional dilemmas
 4 Islands: 4 skills
https://www.owiwi.co.uk/

10. EAWOP 2017

11. EAWOP 2017

12. Construct validity
 Convergent validity (N=97)
 Regressing with well-established measures (from β= .447 to β= .570, p<.001)
 Discriminant validity
 CFA: Satorra‐Bentler Scaled χ2 (N=410) = 306.94, p=.05; CFI= .81, NNFI= .79; IFI =
.80; RMSEA= .046, 90% interval (.037, .085)
 Paths among game’s dimensions are low to medium (from .35 to .50)
 indication of discrimination between the facets (Bagozzi, Yi, & Phillips, 1991)
Congruence between the game & SJT (N=97): Pearson r from .541 to .595, p<.001
EAWOP 2017

13.  Preliminary support of construct validity of a gamified selection method based on
a SJT
 Game elements can be applied to SJTs to effectively assess candidates’ soft skills
 Extend selection methods and gamification’s literature
 exploring a major psychometric property of a gamified selection method
 emphasizing the use of serious games that focus on behavior, an important criterion in
employee selection
 Further explore whether gamified assessments methods are better able to elicit
behaviors than traditional selection methods
EAWOP 2017

14.  Reliability: Test – retest
 Predictive Validity
 Incremental Validity
EAWOP 2017

15.  Organizations might improve their selection process
 Benefits of gamified selection methods (ease of administration, test a large
group at once/on various locations, automatically record answers)
 Applicants perceive the multimedia tests as more valid and enjoyable and are
more satisfied with the selection process
 Increase organizational attractiveness and positive behavioral intentions (e.g.,
accepting a job offer)
 Recruiters might minimize the “cost” of bad hires
 Obtain higher quality candidates: serious games are more difficult to fake and
better able to elicit behaviors than traditional selection methods
EAWOP 2017

16. Thank you
Ioannis Nikolaou
E-mail: inikol@aueb.gr
EAWOP 2017

17.  Armstrong, M. B., Landers, R. N., & Collmus, A. B. (2016). Gamifying Recruitment, Selection, Training, and Performance Management.
 Bagozzi, R. P., Yi, Y., & Phillips, L. W. (1991). Assessing Construct Validity in Organizational Research. Administrative Science Quarterly, 36, 421-458.
 Chow, S., & Chapman, D. (2013). Gamifying the employee recruitment process. Paper presented at the Proceedings of the First International Conference
on Gameful Design, Research, and Applications.
 Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
 Lee, K., & Ashton, M. C. (2004). Psychometric properties of the HEXACO personality inventory. Multivariate Behavioral Research, 39(2), 329-358.
 Martin, A. J., Nejad, H., Colmar, S., & Liem, G. A. D. (2012). Adaptability: Conceptual and empirical perspectives on responses to change, novelty and
uncertainty. Australian Journal of Guidance and Counselling, 22(01), 58-81.
 Mincemoyer, C. C., & Perkins, D. F. (2003). Assessing decision-making skills of youth. Paper presented at the The Forum for Family and Consumer
Issues.
 Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology,
75(6), 640.
 Wagnild, G. M., & Young, H. (1993). Development and Psychometric Evaluation of the Resilience Scale. Journal of nursing measurement, 1(2).
 Weekley, J. A., & Ployhart, R. E. (2006). An introduction to situational judgement testing. Situational judgment tests: Theory, measurement, and
application, 1-12.
EAWOP 2017