![LEVEL 6: Nature of Responses
Quality of responses
“
Imagine you are on
death row…
”
“Scallops with black pudding and
Qualitative cream, rib eye steak with chips and a dolce
latte cream sauce, stinking bishop (cheese)
difference with 1960 port (year of my birth). Wine
would have to be Chateau Lafite 1st Cru
Pauillac 2000. I would skip pudding [of]
course, I would not want indigestion!”
Data Open-endeds,
processing text analysis](https://image.slidesharecdn.com/samra2012gamification2012-07-05-120827041227-phpapp01/75/gamification-in-market-research-22-2048.jpg?cb=1665161733)
Gamification in Market Research
•32 likes•12,792 views
Download to read offline
Report
This deck collects mini-case studies of gamification approaches applied specifically to market research
Recommended
More Related Content
More from Socialphysicist
Beneath the Surface: The Hidden World of Individual Buying DynamicsSocialphysicist
1.4K views•41 slides
More from Socialphysicist (11)
Recently uploaded
Recently uploaded(20)
Gamification in Market Research
- 3. LEVEL 1: Introduction Definition “ mechanics that make The integration of the games fun and absorbing into non-game platforms and experiences in order to improve engagement and participation ~ The Authors ”
- 4. LEVEL 1: Introduction Contents Enjoyment Completion Return rates Engagement rates Response Nature of Consistency Data quality length response Setup considerations
- 6. LEVEL 2: Enjoyment Consumption diary Ave. happiness (out of 10) 8.0 3.5 UK USA (non-gamified) (gamified) Source: Lumi Mobile
- 7. LEVEL 2: Enjoyment Online surveys Enjoyment +26% Interactive elements Interesting e.g. Drag-and-drop brand selection +27% Rotating attribute lists Easier to answer +13% Source: InTouch
- 8. “ LEVEL 2: Enjoyment Spill-over effects Really liked the chat, where people discussed the show “ Noticed more details in the show than normally ” “ It was like being a part of a community ” “ Fun to be a part of this new kind of test ” “ Found it funny to rate the TV- show and see the results ” ” Source: TNS Image: http://www.joyandfood.com/wp-content/uploads/2012/05/Beer-beer.jpg
- 9. LEVEL 2: Enjoyment Pitfalls “ …slightly facile… ” Source: Puleston & Sleep, 2011
- 11. LEVEL 3: Completion Rates Consumption diary Completion rate UK (non-gamified) USA (gamified) 30% 99% Completed Completed Source: Lumi Mobile
- 12. LEVEL 3: Completion Rates Re-wording “ How would you describe yourself? “ In exactly seven words, how would your friends describe ” you? ” 82% 98% Source: Puleston & Sleep, 2011
- 13. LEVEL 3: Completion Rates Increased drop-outs Source: Downes-Le Guin, et al, 2011
- 15. LEVEL 4 Return Rates Source: TNS
- 16. LEVEL 4: Return Rates Coming back for more Returning & asking for more questions and to continue chatting Source: TNS
- 17. LEVEL 4: Return Rates Conjoint analysis Stage 1 100% Regular 50% Stage 2 90% Stage 3 87% ““I loved it, the whole thing was fun to do (though my diet was nearly ruined as it made me hungry!). It was challenging at times but really made me think and that is never a bad thing. Thank you!” Source: Puleston & Sleep, 2011 Image: http://www.fanpop.com/spots/pizza/images/30424281/title/pizza-photo
- 19. Framing as a challenge Framing for creativity “ “ Imagine you are on death row… We challenge you… ” ” Ads recalled X3 LEVEL 5: Response Lengths Re-wording questions Source: Puleston & Sleep, 2011
- 20. Responded faster Results comparable Reduced data variation when respondents could see next attribute LEVEL 5: Response Lengths Reduced time Source: InTouch
- 21. LEVEL 6 Nature of Responses
- 22. LEVEL 6: Nature of Responses Quality of responses “ Imagine you are on death row… ” “Scallops with black pudding and Qualitative cream, rib eye steak with chips and a dolce latte cream sauce, stinking bishop (cheese) difference with 1960 port (year of my birth). Wine would have to be Chateau Lafite 1st Cru Pauillac 2000. I would skip pudding [of] course, I would not want indigestion!” Data Open-endeds, processing text analysis
- 23. LEVEL 6: Nature of Responses Quality of responses “ Imagine that you are a DJ creating a playlist. Which artists would you include? ” ½ neutral ½ uncertain Source: Puleston & Sleep, 2011 Image: http://thediggersunion.com/wp-content/uploads/2011/09/Turntable.jpg
- 24. LEVEL 6: Nature of Responses Quality of responses An independent group of respondents consistently rated the ideas generated by the gamified group as better
- 25. Level 6: Nature of Responses Consistency between approaches 15% deviation in the nature of responses between ‘yes’ and ‘no’ questions
- 26. Level 6: Nature of Responses Consistency between “ Can you guess…? approaches ” +10 seconds Which one is deliberation time best? Implicit vs. rational?
- 27. FINAL BOSS BATTLE: Setup Considerations
- 28. FINAL BOSS BATTLE: Setup considerations
- 30. FINAL BOSS BATTLE: Is it a fad? “ “In some ways it is a fad - adding points and badges in tacky ways, looking at ‘gamification’ as an easy way to make boring things seem interesting - that is a fad. “ “In three years, we will talk about what is at the core of it - design for motivation - not about the one strategy to get there: getting inspiration from games.” However, the idea of designing business processes ” ~ Sebastian Deterding, researcher so that those who engage in them find them more intrinsically rewarding - that is a long term trend”. ~ Jesse Schell , CEO Schell Games ”
- 31. FINAL BOSS BATTLE: Gamification can benefit our industry… “ “ A double shift in focus and framing: (1) from usability(reducing friction) to motivation (increasing drive), (2) from extrinsic motivation (incentives) to intrinsic motivation (competence, autonomy, relatedness needs). At best, it is a set of lenses and design patterns to improve intrinsic motivation.” ~ Sebastian Deterding , researcher ”
- 33. CONGRATS!!! You defeated the Final Boss
Editor's Notes
- Lumi Mobile created two versions of a consumption diary, a gamified version in the USA with bright colours, progress bars and varied reminder times based on respondent activities and a UK version using traditional consumption diary techniques (Findlay & Alberts, 2011). Respondents were asked to rate their average happiness with the approach on a scale of 1 to 10. UK respondents (non-gamified version) gave the research experience an average rating of 3.5 out of 10, while gamified USA respondents rated it 8.0 out of 10 – a massive difference in enjoyment between the two approaches.
- As a more subtle example, InTouch routinely asks respondents to rate the various surveys that they create. InTouch has found that by simply making traditional surveys more visual through the use of interactive mechanisms such as drag-and-drop brand selection and animations between attribute statements (without introducing any fundamental changes to the wording of the actual question), they are able to improve respondent satisfaction. Presenting respondents with an animated, rotating list of attributes rather than with a single attribute per screen, for example, improved respondents’ enjoyment of the survey by 26%. In addition, they rated the survey as 27% more interesting and 13% easier to answer.
- Dramatically improved engagement levels can result in unintended spill-over effects. In the case of a TNS Nielsen study for the Eurovision Song Contest, conducted in partnership with Lumi Mobile, respondents were tasked with “playing” along with the show on their mobile phones. As the show progressed, respondents were asked questions such as what they thought about the presenters’ outfits, who they thought would win, etc. According to Bo Nielsen of TNS (Findlay & Alberts, 2011), some respondents were so engaged in the experience that they turned the research into a drinking game.
- This is not to say that all gamified approaches improve respondent enjoyment across the board. Puleston and Sleep (2011) detail a study where the survey was created to look and behave like a version of the classic arcade game, Space Invaders, with respondents shooting the multiple choice option they most agree with. They found that some respondents thought that the experience was “slightly facile”. This points to the fine line that researchers walk when incorporating explicit game elements into a survey.
- GMI Interactive reworded a question from, “How would you describe yourself” to a more game-like, “In exactly seven words how would your friends describe you?” (Puleston & Sleep, 2011). Doing so improved response rates from 82% to 98%.
- GMI Interactive reworded a question from, “How would you describe yourself” to a more game-like, “In exactly seven words how would your friends describe you?” (Puleston & Sleep, 2011). Doing so improved response rates from 82% to 98%.
- Downes-Le Guin, et al (2011) created four versions of a survey, each with increasing levels of game elements (starting with a standard text survey, and then adding two layers of increasing graphical elements and finally ending off with a total redesign of the survey as a role-playing game in which respondents advance by completing questions). They found that simply adding increasing levels of graphical content improved completion rates. However, turning the entire survey into a role-playing game actually saw an increase in drop-out rates. They posited that this happened for a few reasons: increased load times for the survey, increased software incompatibilities with the new survey and increased complexity which saw respondents lose interest. Downes-Le Guin, et al’s findings again point to the fine line that researchers need to walk between creating compelling and off-putting surveys. In addition, their findings point to the value of piloting studies that include new elements and the need for tracking statistics relating to respondents’ survey behaviour in order to evaluate these new elements.
- GMI Interactive (Puleston & Sleep, 2011) re-imagined a conjoint analysis for a pizza brand as a game where respondents designed their own combinations of pizza toppings. They found that 90% returned to the second stage survey and 87% completed all three survey stages, as opposed to only 50% in their regular survey. In addition, respondents had very positive things to say about the experience, including statements such as:“I loved it, the whole thing was fun to do (though my diet was nearly ruined as it made me hungry!). It was challenging at times but really made me think and that is never a bad thing. Thank you!”
- GMI Interactive (Puleston & Sleep, 2011) found that prefixing a question with the phrase “we challenge you” increased the number of ads recalled three-fold. Similarly, challenging respondents to complete a question within a two minute time resulted in ten times as much feedback. They also found that framing a question in a way that forces respondents to be creative in their answers can improve the quality and length of responses. For example, by asking respondents to imagine that they are on death row and what their last meal would be rather than simply asking them what their favourite meal was increased the responses from a few words (e.g. “steak and chips”) to several paragraphs, quadrupling the word count with more focused answers.
- InTouch have found that by presenting respondents with question mechanics that are intuitive to answer (such as a rotating list of questions), they were able to reduce the amount of time that respondents took to answer. However, this can be a double-edged sword as they also found that in some circumstances, respondents answered the questions faster but there was also less variation between responses. Specifically, they found that people answered the questions faster when presented with a rotating list of attributes rather than a single attribute per page. The results were comparable in terms of the nature and variation in responses between a single attribute per page and the rotating list when respondents were only able to see one attribute at a time. However, when the rotating list indicated to respondents what the next attribute would be before it was presented to them; they found that the variation in the data was actually reduced. This points to the importance of clearly thinking through and testing the mechanics one employs to ensure that their primary focus is to aid of the question at hand.
- Returning to the death row example, the nature of a response can differ qualitatively depending on how the question is framed. For example, asking respondents what their favourite meal is and presenting them with a list of predefined responses differs substantially from asking them to imagine themselves on death row, which can result in answers such as: “Scallops with black pudding and cream, rib eye steak with chips and a dolce latte cream sauce, stinking bishop (cheese) with 1960 port (year of my birth). Wine would have to be Chateau Lafite 1st Cru Pauillac 2000. I would skip pudding [of] course, I would not want indigestion!” With a substantial difference in the length and quality of the response also come considerations around how one processes the response on the backend. Deciding to present respondents with an open-ended response option as in the death row example has implications for data processing. A predefined list of responses is generally easier to deal with than an open-ended response, which requires more time and resources to process (although advances in text analytics continue to make these differences smaller).
- Puleston & Sleep (2011) offer up another example of how the nature of responses might differ based on the way in which the question is framed and/or presented to the respondent. They turned a simple question about music artists which was originally worded along the lines of “How much do you like each artist?” into a type of “quest” for the respondent to complete by tasking them with “Imagine that you are a DJ creating a playlist. Which artists would you include?”. Doing so halved the incidence of neutral or uncertain answers, creating more insightful and varied responses.
- Puleston & Sleep (2011) offer up another example of how the nature of responses might differ based on the way in which the question is framed and/or presented to the respondent. They turned a simple question about music artists which was originally worded along the lines of “How much do you like each artist?” into a type of “quest” for the respondent to complete by tasking them with “Imagine that you are a DJ creating a playlist. Which artists would you include?”. Doing so halved the incidence of neutral or uncertain answers, creating more insightful and varied responses.
- Again, it is very important to pilot studies and to carefully consider their design up front. For example, in GMI Interactive’s multiple choice Space Invaders game (see Figure 2) where respondents shot their preferred answer, the responses were consistent with traditional survey versions. However, another game had respondents skiing down a hill with limited time to pass through gates labelled with their preferred responses. The ski mechanic required timing and skill and it resulted in a 15% deviation in the nature of responses between ‘yes’ and ‘no’ questions from the traditionally measured approach, implying that there is some limit to the degree to which it is possible to turn something into a real game without changing the nature of responses (Puleston & Sleep, 2011).
- As already mentioned, while some scenarios result in differences in response, it is worth asking which responses are more valid since respondents are almost always more engaged and enjoy answering the gamified versions more. Puleston & Sleep (2011) found that prefacing questions with “Can you guess…?” increased respondents’ deliberation time from ten seconds to as much as two minutes. Indeed, consideration times seem to increase when it comes to games. This seems to imply that such questions tap into more considered insights through deeper deliberation while at the same time reducing implicit responses in favour of explicit ones. This effect can be seen in either a positive or negative light, depending on the nature of the responses required and so should be carefully considered up front (for example, sometimes an implicit, gut response might be more accurate than a carefully considered one).