This document discusses the potential benefits and limitations of gamification in healthcare research surveys. It summarizes the results of an experiment comparing a traditional survey to one with some gamified elements. While the gamified survey was found to be more enjoyable and visually engaging for respondents, it did not necessarily lead to more in-depth responses or more accurate results. Gamification techniques may help with respondent experience and time efficiency but have challenges to overcome in healthcare contexts compared to consumer settings. More research is still needed to understand what gamification approaches are most effective for different contexts and respondent types.
This is a presentation and workshop I shared during the launch of Intel Capital's EdTech Accelerator.
It covers two core topics:
1) Developing and prioritizing strong hypothesis statements
2) Testing hypotheses via qualitative and quantitative tools
My thinking draws draws on Steve Blank's customer development process and Javelin's Experiment Board and enhances it with the startup marketing process I've written about on my blog (http://www.cezary.co).
The workshop is relevant to both technology companies as well as any startups looking to validate their business.
It is possible for a product to pass quality assurance tests and acceptance testing without being user-friendly. It is also too easy for those of us who build digital products to make assumptions about what our users need. As a design thinker, I strive to bring the authentic voices of complex audiences into the product lifecycle through pragmatic research.
A sound design research process not only shapes digital products to be more usable, it also adds value to drive engagement.
ProductCamp Boston is the world's largest and most exciting
crowd-sourced one-day event for product people. It's
organized by and for product managers, product marketers and
entrepreneurs, so attendees get the most out of the day.
Attendees learn about and discuss topics in product
management and product marketing, product discovery,
product development & design, go-to-market, product strategy
and lifecycle management, and product management 101,
startups, and career development.
www.ProductCampBoston.org
This is a presentation and workshop I shared during the launch of Intel Capital's EdTech Accelerator.
It covers two core topics:
1) Developing and prioritizing strong hypothesis statements
2) Testing hypotheses via qualitative and quantitative tools
My thinking draws draws on Steve Blank's customer development process and Javelin's Experiment Board and enhances it with the startup marketing process I've written about on my blog (http://www.cezary.co).
The workshop is relevant to both technology companies as well as any startups looking to validate their business.
It is possible for a product to pass quality assurance tests and acceptance testing without being user-friendly. It is also too easy for those of us who build digital products to make assumptions about what our users need. As a design thinker, I strive to bring the authentic voices of complex audiences into the product lifecycle through pragmatic research.
A sound design research process not only shapes digital products to be more usable, it also adds value to drive engagement.
ProductCamp Boston is the world's largest and most exciting
crowd-sourced one-day event for product people. It's
organized by and for product managers, product marketers and
entrepreneurs, so attendees get the most out of the day.
Attendees learn about and discuss topics in product
management and product marketing, product discovery,
product development & design, go-to-market, product strategy
and lifecycle management, and product management 101,
startups, and career development.
www.ProductCampBoston.org
Desirability Testing: Analyzing Emotional Response to a DesignMegan Grocki
In the design process we follow, once we have defined the conceptual direction and content strategy for a given design and refined our approach through user research and iterative usability testing, we start applying visual design. Generally, we take a key screen whose structure and functionality we have finalized—for example, a layout for a home page or a dashboard page—and explore three alternatives for visual style. These three alternative visual designs, or comps, include the same content, but reflect different choices for color palette and imagery. The idea is to present business owners and stakeholders with different visual design options from which they can choose. Sometimes there is a clear favorite among stakeholders or an option that makes the most sense from a brand perspective. However, there can often be disagreements among the members of a project team on which direction to choose. If we’ve done our job right, there are rationales for our various design decisions in the different comps, but even so, there may be disagreement about which rationale is most appropriate for the situation.
As practitioners of user-centered design, it is natural for us to turn to user research to help inform and guide the process of choosing a visual design. But traditional usability testing and related methods don’t seem particularly well suited for assessing visual design for two reasons:
1. When we reach out to users for feedback on visual design options, stakeholders are generally looking for large sample sizes—larger than are typical for a qualitative usability study.
2. The response we are looking for from users is more emotional—that is, less about users’ ability to accomplish tasks and more about their affective response to a given design.
With this in mind, we were very interested in articles we saw on Desirability Testing. In one article, the author posits desirability testing as a mix of quantitative and qualitative methods that allow you to assess users’ attitudes toward aesthetics and visual appeal. Inspired by his overview, we researched desirability studies a bit further and tried a modified version of the techniques on one of our projects. This presentation reviews the variants of desirability testing that we considered and the lessons we learned from a desirability study on visual design options for one of our projects. Interestingly, we found that while desirability testing did help us better understand participant’s self reported emotional response to a visual design, it also helped us identify other key areas of the experience that could be improved.
The Art & Science of Facial Imaging in Market Research - Part 1 The BasicsAlastair Gordon
Facial Imaging (Facial Coding/Recognition) is one of Market Research's most interesting new technologies, providing many of the benefits of lab-based neuro-science, but in a much more scalable and cost-effective manner. In this presentation (part 1 of 2) Gordon & McCallum outline the basics of how the method works, what outputs look like and implementation requirements, Part 2, a separate presentation also on Slideshare, discusses more advanced applications and the potential of the technology to build new solutions in 6 key aspects of interest to marketers.
Pt 2: The Art & Science of Facial Imaging for Market Research - Advanced Alastair Gordon
This presentation, part two of a two part introduction, Gordon & McCallum looks at the value of using facial imaging (facial coiding/recognition) for more than "single stimuli" ad-test type applications. We provide examples, and suggest 6 key ways facial imaging can make a huge contribution to solving key marketing problems.
A bit of fun, showing part of the process gone through to create the Gamification User Type Hexad that we know and love now.
Ideas take time to mature - this one took a few years!
THE NEED FOR A SALES PROCESS
PROBLEM
The results of a recent study conducted by The Sales Board confirmed what we have known for some time. Prospects are speaking up about how they feel about salespeople who are less than professional. We thought that you would like to see these statistics as they reinforce the need for a sales process and challenge you to improve your qualifying efforts.
DIAGNOSIS
The study showed the following startling facts. Can you relate to them as a salesperson or as a buyer?
Fact: 82% of salespeople fail to differentiate
Result: They lose the business, fail to sell value
Fact: 86% of salespeople ask the wrong questions.
Result: They miss selling opportunities and end up wasting time while appearing unprofessional.
Fact: Only 18% of salespeople close without discounting price.
Result: Discounting becomes a habit and profit margins are eroded.
Fact: 95% of customers say salespeople talk too much.
Result: Customers are bored and feel salespeople don’t care about understanding their problems.
Fact: 62% of salespeople do not earn the right to ask questions.
Result: They fail to position the sale properly and don’t gain commitment.
Fact: 85% of salespeople use a selling process that is extremely ineffective, compared to the buyer’s system.
Result: They close less than 50% of the business that they should close, with disastrous effects on their companies’ sales and their personal incomes. You may be ‘winging it’ if you find yourself relating to any of the following: (a) chasing prospects who don’t return calls; (b) hearing ‘think it over’ all too often when you ask for the business; (c) cutting price in an effort to obtain or keep business; and (d) spending most of your time in front of people who are not decision makers.
SOLUTION
1. Stop assuming that your prospect needs what you’re selling.
2. Learn how to ask more questions to see if the prospect has any serious ‘pain’ issues that your product or service can resolve.
3. Learn a sales process to help you stay in control of the sales interview.
The Bright Future of Market Research Smartees WorkshopInSites on Stage
This is the full slidedeck of our Smartees Workshop on 'the Bright Future of Market Research' (11 February, 2014). The main focus is on how both traditional quantitative and qualitative research can be better, fresher and more contemporary by approaching participants and internal stakeholders differently.
Exceptional customer service is the greatest assurance for client retention and for developing a successful "word of mouth" marketing campaign. Many venues have a solid foundation for treating customers well, but many do so without a clear philosophy and/or specific standards in place. In this session, designed for upper-level management, attendees will be introduced to a step-by-step process for implementing a universal customer service philosophy that is structured and able to be evaluated for its success. Make exceptional service a standard and an expectation in your operation.
Desirability Testing: Analyzing Emotional Response to a DesignMegan Grocki
In the design process we follow, once we have defined the conceptual direction and content strategy for a given design and refined our approach through user research and iterative usability testing, we start applying visual design. Generally, we take a key screen whose structure and functionality we have finalized—for example, a layout for a home page or a dashboard page—and explore three alternatives for visual style. These three alternative visual designs, or comps, include the same content, but reflect different choices for color palette and imagery. The idea is to present business owners and stakeholders with different visual design options from which they can choose. Sometimes there is a clear favorite among stakeholders or an option that makes the most sense from a brand perspective. However, there can often be disagreements among the members of a project team on which direction to choose. If we’ve done our job right, there are rationales for our various design decisions in the different comps, but even so, there may be disagreement about which rationale is most appropriate for the situation.
As practitioners of user-centered design, it is natural for us to turn to user research to help inform and guide the process of choosing a visual design. But traditional usability testing and related methods don’t seem particularly well suited for assessing visual design for two reasons:
1. When we reach out to users for feedback on visual design options, stakeholders are generally looking for large sample sizes—larger than are typical for a qualitative usability study.
2. The response we are looking for from users is more emotional—that is, less about users’ ability to accomplish tasks and more about their affective response to a given design.
With this in mind, we were very interested in articles we saw on Desirability Testing. In one article, the author posits desirability testing as a mix of quantitative and qualitative methods that allow you to assess users’ attitudes toward aesthetics and visual appeal. Inspired by his overview, we researched desirability studies a bit further and tried a modified version of the techniques on one of our projects. This presentation reviews the variants of desirability testing that we considered and the lessons we learned from a desirability study on visual design options for one of our projects. Interestingly, we found that while desirability testing did help us better understand participant’s self reported emotional response to a visual design, it also helped us identify other key areas of the experience that could be improved.
The Art & Science of Facial Imaging in Market Research - Part 1 The BasicsAlastair Gordon
Facial Imaging (Facial Coding/Recognition) is one of Market Research's most interesting new technologies, providing many of the benefits of lab-based neuro-science, but in a much more scalable and cost-effective manner. In this presentation (part 1 of 2) Gordon & McCallum outline the basics of how the method works, what outputs look like and implementation requirements, Part 2, a separate presentation also on Slideshare, discusses more advanced applications and the potential of the technology to build new solutions in 6 key aspects of interest to marketers.
Pt 2: The Art & Science of Facial Imaging for Market Research - Advanced Alastair Gordon
This presentation, part two of a two part introduction, Gordon & McCallum looks at the value of using facial imaging (facial coiding/recognition) for more than "single stimuli" ad-test type applications. We provide examples, and suggest 6 key ways facial imaging can make a huge contribution to solving key marketing problems.
A bit of fun, showing part of the process gone through to create the Gamification User Type Hexad that we know and love now.
Ideas take time to mature - this one took a few years!
THE NEED FOR A SALES PROCESS
PROBLEM
The results of a recent study conducted by The Sales Board confirmed what we have known for some time. Prospects are speaking up about how they feel about salespeople who are less than professional. We thought that you would like to see these statistics as they reinforce the need for a sales process and challenge you to improve your qualifying efforts.
DIAGNOSIS
The study showed the following startling facts. Can you relate to them as a salesperson or as a buyer?
Fact: 82% of salespeople fail to differentiate
Result: They lose the business, fail to sell value
Fact: 86% of salespeople ask the wrong questions.
Result: They miss selling opportunities and end up wasting time while appearing unprofessional.
Fact: Only 18% of salespeople close without discounting price.
Result: Discounting becomes a habit and profit margins are eroded.
Fact: 95% of customers say salespeople talk too much.
Result: Customers are bored and feel salespeople don’t care about understanding their problems.
Fact: 62% of salespeople do not earn the right to ask questions.
Result: They fail to position the sale properly and don’t gain commitment.
Fact: 85% of salespeople use a selling process that is extremely ineffective, compared to the buyer’s system.
Result: They close less than 50% of the business that they should close, with disastrous effects on their companies’ sales and their personal incomes. You may be ‘winging it’ if you find yourself relating to any of the following: (a) chasing prospects who don’t return calls; (b) hearing ‘think it over’ all too often when you ask for the business; (c) cutting price in an effort to obtain or keep business; and (d) spending most of your time in front of people who are not decision makers.
SOLUTION
1. Stop assuming that your prospect needs what you’re selling.
2. Learn how to ask more questions to see if the prospect has any serious ‘pain’ issues that your product or service can resolve.
3. Learn a sales process to help you stay in control of the sales interview.
The Bright Future of Market Research Smartees WorkshopInSites on Stage
This is the full slidedeck of our Smartees Workshop on 'the Bright Future of Market Research' (11 February, 2014). The main focus is on how both traditional quantitative and qualitative research can be better, fresher and more contemporary by approaching participants and internal stakeholders differently.
Exceptional customer service is the greatest assurance for client retention and for developing a successful "word of mouth" marketing campaign. Many venues have a solid foundation for treating customers well, but many do so without a clear philosophy and/or specific standards in place. In this session, designed for upper-level management, attendees will be introduced to a step-by-step process for implementing a universal customer service philosophy that is structured and able to be evaluated for its success. Make exceptional service a standard and an expectation in your operation.
Transparency: what it means to your customers and its impact to your businessTransparency Marketing
What you'll learn:
1. Why transparency is important to your sales and marketing.
2. What is transparency.
3. The key benefits of transparency to your company.
4. How you can create transparency in your sales and marketing communications.
In this presentation, part of Multiplicity's ongoing speaker series about Launching and Growing: Your First 100 Customers, AskingCanadians Vice President Raj Manocha focuses on how insights can help companies better understand their customers and what’s important to them. The presentation highlights the role insights can play in innovation, cost effective ways for start-ups to collect insights and how research can make you a thought leader.
Virginia Franchise Forum - Franchisee Roundtable Nov 11, 2009InfoTrends
Presentation given to Franchise Business Owners In Richmond Virginia which was sponsored by the Greater Richmond Chamber of Commerce. Title: Moving From Survival To Growth
Now that industry leaders have raised the bar for all,
how do customers feel about their experiences with the
companies they patronize? What kind of experience
do customers really expect? And how often do unhappy
customers share their stories with the world?
Similar to Rachel Medcalf, Jo Thompson & Jon Puleston (20)
MRS Company Partners have access to an exclusive group, the Operations Network. This Network holds free quarterly meetings to discuss a variety of topics to help your organisation. It is a great opportunity to network and learn new things.
6. • Somewhat misleading phrase – prefer to think of a
process of art direction, copy writing and critical editing
• Thinking of a survey like an advertising agency thinks
about an advert
• Treating it as a piece of entertainment
GAMIFICATION
COPY WRITING
& ART DIRECTION
15. Research objectives
“Traditional” research arm
N=80
Questions framed around
peer review
N=64
“Gamified” research arm
N=125
Questions framed around
a junior colleague
N=61
Physicians
Mix of closed
and open
ended
questions
20 minute
online survey
17. How enjoyable
did you find the
survey?
Very - 9% Very - 17%
How visually
engaging did you
find the survey?
Very - 11%
Very - 47%
Significant
difference
(95%)
Significant
difference
(99%)
VS.
TRADITIONAL GAMIFIED
18. “Much better than
most I have done. I like
the visual element”
What did you like
about it?
Appearance/visuals - 39%
Easy to complete - 13%
Different - 12%
“Novelty was wearing
off towards the end”
“Was a bit silly”
“Stimulating - it
keeps your attention
going for longer.”
GAMIFIED
19. Short - 31%
Simple - 23%
Thought provoking - 18%
“Easy and simple
format”
What did you like
about it?
“Some ratings can
get tedious.”
“It was short and
to the point”
TRADITIONAL
22. Visual questions are quicker to answer but yield the
same results
14% 15%
23%
22%
18% 18%
15% 16%
1% 1%
6%
7%
10%
9%
13% 11%
Traditional Gamified
Brand I
Brand H
Brand G
Brand F
Brand E
Brand D
Brand C
Brand B
Brand A
% of patients currently receiving each treatment
Average:
87seconds
Average:
53seconds
24. Can a simple game mechanic lead to greater feedback?
1 question:
If you only had 40
words to describe
Product X to this
patient/patient’s carer
as a treatment option,
how would you sum
up its benefits vs.
drawbacks?
2 questions:
What are the
advantages of Product
X in a patient like this?
What are the
disadvantages of
Product X in a patient
like this?
VS.
TRADITIONAL GAMIFIED
25. Yes!
Average 19 words
“From clinical evidence and my own
experience, it has been extremely efficacious for
patients and has improved patients symptoms,
kept them out of inpatient settings and helped
them get back a quality of life without too
many side effects.”
Average 14 words
(both questions combined)
Advantage: “Improved compliance”
Disadvantage: “He might not like
taking a regular injection” VS.TRADITIONAL
GAMIFIED
26. But decoration doesn’t always lead to greater depth
The question itself needs to
engage respondents to think
differently – just looking pretty
doesn’t encourage people to
write more
27. Critical peer review leads to revelations around barriers
Q: What do you think
physicians think about
when they are choosing a
Tx?
Q: What are the factors
they may not admit to?
Led to the most useful insights!
More in-depth nuggets of information
GAMIFIED
28. 2
3
4
5
6
7
8
9
Attribute 1 Attribute 2 Attribute 3 Attribute 4 Attribute 5 Attribute 6
Meanscore(1-10)
Brand X Brand A Brand B Brand C Brand D
2
3
4
5
6
7
8
9
Attribute 1 Attribute 2 Attribute 3 Attribute 4 Attribute 5 Attribute 6
Meanscore(1-10)
Product ratings show
clearer differentiation
in the gamified version
TRADITIONAL
GAMIFIED
31. Improvements to accuracy is much harder to prove!
• Physicians we spoke to post survey did say that thinking
in the context of others reflected their own opinions
• Changes to how we asked the rating and ranking scales
demonstrated a clear differentiation
• But does this actually reflect reality
TRADITIONAL GAMIFIED
34. We face challenges in healthcare that do not exist
in consumer
Limited universe of
respondents
Slow processes
Ethics &
compliance
considerations
35. We also pay our respondents
Financial motivation to
complete surveys – we are not
relying on good will
36. Encouraging signs
Worth pursuing – we have learned a lot from
one experiment but have also raised more
questions – more myths to be busted
Some techniques work
better than others?
Gamification is
particularly beneficial in
longer surveys?
Different respondent
types respond
differently to
different gamification
elements?
Creates better
engagement in other
applications i.e.
qualitative research,
workshops?
37. Thank you
Jon Puleston
VP Innovation
Rachel Medcalf
Managing Director
Joanna Thompson
Research Director