TRACK 3(3) | QUALITY IMPROVEMENT | DAY 1 DAY 1 - 31 OCT 2016 Jim Lumsden, PhD Student | University of Bristol (UK) Games for Health Europe 2016

1. The effects of game mechanics on cognitive test
performance and participant enjoyment
Prepared for TRACK 3: Quality Improvement
Games for Health Europe, Utrecht, 2016
Jim Lumsden
jim.lumsden@bristol.ac.uk
School of Experimental Psychology,
University of Bristol,
BS8 1TU,
Bristol, UK

2. Abstract
Computerised cognitive assessments are a vital tool in the behavioural sciences, but participants often view them as effortful and unengaging. One
potential solution is to add gamelike elements to these tasks in order to make them more intrinsically enjoyable.
In this talk I will present two studies into the effects of individual game mechanics on the data and enjoyment ratings from cognitive tasks designed to
measure inhibitory control. In both studies, we tested three variants of the tasks: one in which participants were rewarded with points for performing
optimally, one where the task was given an overall theme and graphical upgrade, and a third version which was a non-gamified comparator. I will
discuss our findings, and how they might guide the development of future gamified cognitive assessments.
Lumsden J, Skinner A, Woods AT, Lawrence NS, Munafò M. (2016)
The effects of gamelike features and test location on cognitive test performance and participant enjoyment, PeerJ

3. Cognitive Tasks
• Cognitive tasks are a staple tool in psychological research
• Must collect pure measures of cognition and therefore are simplistic and
involve lots of repetition
• Cannot produce valid measures if the participant is bored
• They need to be engaging and gamification may be the answer

4. Research Questions
1. Do different game mechanics effect subjective ratings of engagement?
2. Do different game mechanics effect the cognitive data collected?
3. Do different game mechanics influence attrition rates?
B(Study 2 only)
We ran two studies to look at the effects of game mechanics on
the data collected by two cognitive tasks

5. Study 1 Overview
(the cowboy study)
• Three variants of a Go-No-Go task
(non-game, points and theme)
• 287 participants (lab: 84, online: 203)
Signup
through
MTurk
Signup in
person in
Bristol
Randomly
assigned to
variant
Complete task
(~ 10min)
Subjective
questionnaire
delivered

6. Study 2 Overview
(the attrition study)
• Three variants of a Stop Signal Task
(non-game, points and theme)
• Longitudinal (10 days long)

7. Participants and Procedure
(the attrition study)
• 291 participants total (97 in each variant)
• Reimbursed between £4 and £7
Signup through
Prolific
Academic
Randomly
assigned to
variant
4 consecutive days of test sessions
Reimbursed £4 after all completed
Day 1 Day 2 Day 3 Day 4
6 optional days of test sessions
Reimbursed £0.50 for each completed
Day 5 Day 6 Day 7 Day 8 Day 9
Day
10
Study ends

8. Cognitive Task Results
(the cowboy study)
Go RT:
points vs non-game (BF = .157)
No-Go Accuracy:
points vs non-game (BF = .253)
Go Accuracy:
points vs non-game (BF = .459)

9. Cognitive Task Results
(the attrition study)

10. Questionnaire Results
(the cowboy study)

11. Questionnaire Results
(the attrition study)

12. Participant Attrition
(the attrition study)

13. Key findings
• Points did not disrupt validity and increased participant enjoyment
• Mixed effects of Theme variant on data and engagement
• Study 2: No difference in participant attrition between task variants

14. Conclusions
• Don’t waste time making graphics or complex games: just pointify
• Gamifying cognitive tests without invalidating the data is possible and can be
effective for increasing engagement
• Further research needed to translate improved subjective ratings into
objective engagement
• Further research needed to understand the effects of themed graphics

15. Acknowledgements
Tobacco and Alcohol Research Group:
Angela Attwood Postdoc
Emily Crowe PhD Student
Kayleigh Easey PhD Student
Meg Fluharty PhD Student
Therese Freuler Research Assistant
Suzi Gage Postdoc
Meryem Grabski PhD Student
Gemma Hammerton Postdoc
Eleanor Kennedy PhD Student
Jasmine Khouja PhD Student
Glenda Lassi Postdoc
Rebecca Lawn PhD Student
Jim Lumsden PhD Student
Olivia Maynard Postdoc
Andy Skinner Postdoc
Alex Board Administrator
Amy Taylor Postdoc
Gemma Taylor Postdoc
Chris Stone Research Assistant
David Troy PhD Student
Miriam Cohen PhD Student
Andy Eastwood PhD Student
My supervisors:
Dr Jenny Barnett
Dr David Coyle
Dr Charlotte Housden
Prof Natalia Lawrence
Prof Marcus Munafò
jim.lumsden@bristol.ac.uk
@jl9937

16. 16
23 November 2016
Any questions?