Desperately Seeking Theory: Gamification, Theory, and the Promise of a Data/AI-Driven New Science of Design

1. DESPERATELY SEEKING THEORY Gamification, Theory, and the Promise of a Data/AI-Driven New Science of Design #GamifIR 2016 Sebastian Deterding @dingstweets

2. <1> the challenge

3. we are stuck in a groundhog day

4. “does gamification work?”

6. http://www.ﬂickr.com/photos/ewanrayment/1250158647

7. “does medicine work?”

8. some better questions What active substance in what dosage affects what condition under what circumstances? How? Do 500 mg of orally administered acetylsalicylic acid affect migraine in adult chronic migraine patients? How? Do publicly displayed achievements linked to publicly known as hard-to-attain goals support goal- setting in achievement-oriented individuals? How?

9. To ask such questions, we need theory

10. what is theory? (a quick refresh) A theory is a set of statements expressing a nomological network of constructs and their relations. It is tied to observable phenomena through operationalisations. Whitney, Kite & Adams, 2013; Cronbah & Meehl, 1955

11. why theory? (a quick refresh) Whitney, Kite & Adams, 2013 • Allows to describe, explain, predict, and control reality • Allows to systematically organise and extend knowledge

12. without theory, no science

13. how do cognitive states affect behaviour affectdesign features ?

14. This is a grandquestion inviting a new science of design

15. the current basic mediation model design element 2 magic motivation! behaviour design element 1 design element 3 e.g. Hamari, Koivisto & Sarsa, 2014

16. the state of gamification theory

17. our biblical texts

18. So where is the issue?

19. design element 2 magic motivation! behaviour design element 1 design element 3 the issues with design elements

20. the state of gamification theory “… a certain Chinese encyclopaedia entitled ‘Celestial Empire of benevolent Knowledge’. In its remote pages it is written that the animals are divided into: (a) belonging to the emperor, (b) embalmed, (c) tame, (d) sucking pigs (e) sirens, (f) fabulous, (g) stray dogs, (h) included in the present classification, (i) frenzied, (j) innumerable, (k) drawn with a very fine camelhair brush, (l) et cetera, (m) having just broken the water pitcher, (n) that from a long way off look like flies.” issue #1 we have chinese encyclopaedias

21. badges points leader boards levels feedbackcleargoals reward progress challenge story/ theme

22. when is “a badge” a badge? issue #2

23. all exegesis, no discovery of new species issue #3

24. we’re stuck on low-level design features issue #3a Deterding et al., 2011 where the action is

25. we need a linné

26. an empirically grounded, well-operationalised, well-formed taxonomy of design elements

27. this is a massive undertaking

28. design element 2 magic motivation! behaviour design element 1 design element 3 the issues with motivational processes

29. issue #1 motives are multiple

30. johnmarshall reeve »Motivation’s new paradigm is one in which behavior is energized and directed not by a single grand cause but, instead, by a multitude of multilevel and coacting influences.« understanding motivation & emotion (2009: 45)

31. gaming motives Competence Autonomy Relatedness Meaning Curiosity Absorption (»flow«) Control beliefs Goal-setting Recognition, belonging, power Identity/Self Emotions Habituation …

32. design element 1 design element 2 design element 3 behaviour competence autonomy relatedness meaning curiosity ...

33. Aparicio et al., 2012 Sailer et al., 2013 Zhang, 2008 Zichermann& Cunningham,2011 none of this is empirically tested issue #2

34. (Another massive undertaking)

35. Elements are multi-functional Antin & Churchill 2011 goal-settinginstruction group identification status reputation issue #3

36. design element 1 design element 2 design element 3 behaviour competence autonomy relatedness meaning curiosity ...

37. feedback appraised as controlling thwarts autonomy motivation appraised as informing supports competence + – motivational function is an appraisal issue #4 Deci & Ryan 2001

38. design element 1 design element 2 design element 3 behaviour competence autonomy relatedness meaning curiosity ... appraisal

39. motivation and appraisal are contextual issue #5

40. Reeve, 2006

41. license to reconfigure & leave Deterding, 2016

42. minimised consequence Deterding, 2016

43. situation element 1-n design element 1 design element 2 behaviour competence autonomy relatedness meaning curiosity ... appraisal situation

44. different people, different motives issue #6

45. trait level Croson & Gneezy, 2000

46. state level Bowman & Tamborini, 2013

47. situation element 1-n design element 1 design element 2 behaviour competence autonomy relatedness meaning curiosity ... appraisal situation person

48. http://www.flickr.com/photos/8147452@N05/2913356030/sizes/o/ motivational function is systemic-dynamic issue #7

49. AestheticsMechanics Dynamics Hunicke, LeBlanc & Zubek mda: a formal approach to game design (2004)

50. Monopoly aesthetic Frustrating end game mechanic dynamic Slow poverty gap +$ !+ -$ !-

51. Difficulty Skill/Time frustration boredom function is a dynamic person-situation relation »flow«

52. points badges leaderboards Tracking, Feedback Goals, surprise Competition we focus on features … when we should look at how features in relation to dispositions afford motivational functions

53. Difficulty Skill/Time frustration boredom »flow« e.g. balanced challenge Csikszentmihalyi, 1990

54. http://www.ﬂickr.com/photos/amrufm/2593920251/sizes/z/in/photostream/ e.g. curiosity Silva, 2006

55. http://www.ﬂickr.com/photos/amrufm/2593920251/sizes/z/in/photostream/ e.g. Meaningfulness Wong, 2012

56. motivational affordance The complex of necessary and sufficient relations of actor dispositions and environment features that render an action or event functionally significant for a specific motive Deterding, under review

57. competence autonomy relatedness meaning curiosity ... appraisal p/s relation affordance 1 p/s relation affordance 2 p/s relation affordance 3 behaviour

58. <2> the opportunity

59. Kramer,Guillory&Hancock,2015

60. not that opportunity

61. jad abumrad »Facebook has created a laboratory of human behavior the likes of which we’ve never seen.« the trust engineers (2015)

62. At any moment, software companies are running millions of a/B tests to optimise user engagement, each of which is an experiment in waiting

63. the scale for a massive undertaking

64. design is being automated

65. But to know what to test & how to automate, you need … theory

66. »Online hypothesis testing can accelerate both applied and basic Interaction Design Science by making it fast and easy to obtain ecologically- valid measures of the effects of designs on user behavior. Online controlled experiments can help build practical, generalizable, and scientifically- validated theories of how and why designs affect human interactions.« derek Lomas optimizing motivation and learning with large- scale game design experiments (2014: 7)

67. derek Lomas optimizing motivation and learning with large- scale game design experiments (2014: 7)

68. Data, suggestions Designs/tweaks Theories, formalisations, indicators Data, indicators researcher designeruser Adapted content & interface Experience & behaviour data data/ai

69. an example

70. »flow« Difficulty Skill/time frustration boredom flow (1990) Mihaly Csikszentmihalyi basic theory

71. Difiiculty Skill/time frustration boredom what’s the optimal curve? Alexander, Sear & Oikonomou, 2013; Sampayo-Vargas et al., 2013; Brazil & Blau, 2014

72. crowdsourcing design application

73. Difficulty Skill/time crowdsourcing tasks are predetermined ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

74. Difficulty Skill/time ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Difficulty is unknown; identification may obsolete crowdsourcing crowdsourcing tasks are predetermined

75. Difficulty Skill/time ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? … hence tasks are often randomly served crowdsourcing tasks are predetermined

76. %Playerretained Time/levels * idealised tutorial actual tasks * hence retention is very poor

77. Difficulty Skill/time How to order tasks w/o solving? ? ? ? ? ? ? ? ? the question ? ? ?

78. Difficulty Skill/time ? ? ? ? ? ? ? ? ? ? ? … which requires skill concepts & indicators player x Pattern recognition: 7/10 Spatial rotation: 2/10

79. Difficulty Skill/time ? ? ? ? ? ? ? ? ? ? ? player x Pattern recognition: 7/10 Spatial rotation: 2/10 … which requires difficulty concepts & indicators task y Pattern recognition: 7/10 Spatial rotation: 2/10

80. Performance data Improvement suggestions Task length design Task descriptions Initial balancing theories Initial indicators of skill Optimal difficulty curve Indicators for skill, difficulty researcher designeruser Task sequence & task support & feedback Engagement & performance data data/ai

81. <3> summary

82. to get out of groundhog day …

83. we need to ask better questions …

84. to systematically build knowledge …

85. how do cognitive states affect behaviour affectdesign features ? … for a grand new science of design.

86. To ask such questions, we need theory

87. and we must abandon scholasticism.

88. we need better, empirical taxonomies …

89. competence autonomy relatedness meaning curiosity ... appraisal p/s relation affordance 1 p/s relation affordance 2 p/s relation affordance 3 behaviour … and better, empirical mechanistic models.

90. this is a massive undertaking …

91. with a matching massive opportunity.

92. we can be at the forefront of design evolution …

93. … if we loop research into the world’s largest design lab that is running every day. Data, suggestions Designs/tweaks Theories, formalisations, indicators Data, indicators researcher designeruser Adapted content & interface Experience & behaviour data data/ai

94. sebastian@codingconduct.cc @dingstweets codingconduct.cc Thank you.