SlideShare a Scribd company logo
Six crucial
survey concepts
that UX professionals need to know
Workshop at UXPA 2014 led by Caroline Jarrett
Today’s
concepts
Better survey questions
Warm up: the SUS
1: Ask questions that people can answer
2: Satisfaction is a slippery topic
Better survey processes
3: Assess the total survey error
4: Understand who responds
5. Your survey goals drive your analysis
6. Test everything
2
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
3
Introductions
(I’m Caroline Jarrett - @cjforms)
Work with your neighbour
• Your name and role
• A random thing about yourself
Image credit: http://www.visitlondon.com/things-to-do/event/28528358-tour-de-france-grand-depart-2014-third-stage.
4
The SUS is the most-used questionnaire
for post-task satisfaction
• John Brooke created the
System Usability Scale
(SUS) in 1986
• Used after a usability test
• It has been shown to be
valid and reliable
• You get a score between
0 and 100
• You can compare your SUS
score with other systems
Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In
Usability Evaluation in Industry.
P. W. Jordan, B. Thomas, B. A. Weerdmeester and A. L.
McClelland. London:, Taylor and Francis.
5
We’ll try cognitive interviewing
• In a cognitive interview, you ask your respondent to
explain the mental processes in answering the question
Cognitive
interviewing
focuses on
the
questions
Usability
testing
focuses on
interaction
Willis, G. B. (2005). “Cognitive interviewing : a tool for improving questionnaire design”. Thousand Oaks, Calif., Sage Publications.
6
• Silent two minute task:
– Find an explanation of the difference between the European
Commission and the European Union
– Use this site: http://ec.europa.eu
• Decide whether you had a good or bad experience
7
• I’ll assign the roles of ‘interviewer’ and ‘respondent’
• Interviewer:
– Ask your respondent to fill out the first three questions of the
SUS with the EU task in mind
– Take notes about any challenges in the questions on the SUS
• Respondent:
– Please fill out the first three questions of the SUS, explaining
your thought processes for each question
8
Relating our findings from the cognitive
interview to today’s concepts
Concepts What we learned from SUS
Ask questions that people can answer
Satisfaction is a slippery topic
Assess the total survey error
Understand who responds
Your survey goals drive your analysis
Test everything
9
Today’s
concepts
Better survey questions
Warm up: the SUS
1: Ask questions that people can answer
2: Satisfaction is a slippery topic
Better survey processes
3: Assess the total survey error
4: Understand who responds
5. Your survey goals drive your analysis
6. Test everything
10
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
11
There are four steps to answer a
question
Step
1. Read and understand
2. Find an answer
3. Judge the answer
4. Place the answer
Adapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000)
“The psychology of survey response”
There are four steps to answer a
question
Step A good question …
1. Read and understand is legible and makes sense
2. Find an answer asks for answers that we know
3. Judge the answer asks for answers we’re happy to reveal
4. Place the answer offers appropriate spaces for the answers
Adapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000)
“The psychology of survey response”
Use the four steps
• Swap roles and try the remaining SUS questions
• Think about the four steps of answering a question
1. Read and understand
2. Find the answer
3. Judge the answer
4. Place the answer
• Identify any problems in the questions according to the
four steps
14
Four step examples:
1: read and understand
15
Four step examples:
1: read and understand
Hermann grid illusion16
Four step examples:
2: find the answer
In your last five days at work, what
percentage of your work time do you
estimate that you spend using publicly-
available online services (not including
email, instant messaging and search) to
do your work using a work computer or
other device?
17
The approximate curve of forgetting
Four step examples:
3: judge the answer
19
Four step examples:
4: place the answer
20
21
22
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
23
My journey into user experience
started a long time ago, with usability
The extent to which a product
can be used by specified users
to achieve specified goals with
effectiveness, efficiency and
satisfaction
in a specified context of use
(ISO 9241:11 1998)
24
In government, we were interested in
effectiveness and efficiency
The extent to which a product
can be used by specified users
to achieve specified goals with
effectiveness, efficiency and
satisfaction
in a specified context of use
(ISO 9241:11 1998)
25
But what about user experience?
What about satisfaction?
The extent to which a product
can be used by specified users
to achieve specified goals with
effectiveness, efficiency and
satisfaction
in a specified context of use
(ISO 9241:11 1998)
Picture credit: Flickr jek in the box
26
27
Satisfaction is a complex matter
Compared experience to what? Resulting thoughts
(nothing) Indifference
Expectations Better / worse / different
Needs Met / not met / mixture
Excellence (the ideal product) Good / poor quality (or ‘good enough’)
Fairness Treated equitably / inequitably
Events that might have been Vindication / regret
Adapted from Oliver, R. L. (1996) and (2010)
“Satisfaction: A Behavioral Perspective on the Consumer”28
Example: bronze medal winners tend
to be happier than silver medal winners
Nathan Twaddle, Olympic Bronze Medal Winner in Beijing
Matsumoto D, & Willingham B (2006). The thrill of victory and the
agony of defeat: spontaneous expressions of medal winners of the
2004 Athens Olympic Games.
Photo credit: peter.cipollone, Flickr
29
• The first question was about rating satisfaction
• What were they asking us to rate?
– Just a guess from what you recall
30
31
The challenge of UX and surveys:
which bit to measure?
The extent to which a product
can be used by specified users
to achieve specified goals with
effectiveness, efficiency and
satisfaction
in a specified context of use
(ISO 9241:11 1998)
???
Some ideas about what we could
measure (as well as satisfaction)
In the definition GoDaddy customer
support
GoDaddy as a provider of
domain names
Product This contact with help desk Overall experience of moving
a domain to GoDaddy
Users What proportion of
customers contact support
Demographics (example: type
of job)
Goals Reason for contacting help Reason for looking at
GoDaddy
Effectiveness Whether support fixed the
problem
Whether GoDaddy offers the
right products
Efficiency Whether it took a reasonable
time
Whether the product is priced
correctly
Satisfaction Helpfulness of support
person
Likely to purchase again /
recommend
Context of use Home/office; alone/helped Business / personal
33
Tip
Find out about users’ goals
34
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
35
36
Survey lifecycle for total survey error
The answers you use
The answers you get
The questions
you ask
What you want
to find out about
What
The sample you use
The ones who answer
The sample you ask
The list you use to
sample from
Who you want
to find out from
Who
The number
37
38
Choices compared to the lifecycle
The answers you use
The answers you get
The questions
you ask
What you want
to find out about
What
The sample you use
The ones who answer
The sample you ask
The list you use to
sample from
Who you want
to find out from
Who
The number
Services so good that
people prefer to use
them
Satisfaction
Overall, how satisfied
were you with renewing
your car tax today?
Scale of 5 points from
very satisfied to very
dissatisfied
Ratings weighted from
100% for 'very satisfied' to
0% for 'very dissatisfied'
UK population
People who renew car tax
online
Everyone who finishes the
renewal
People who answer the
question
94%
(Any exclusions for repeated
answers or test answers?)
Everyone who finishes the
renewal
39
And now in “survey methodologist”
Edited response
Response
Measurement
Construct
Measurement
Post-survey
adjustments
Respondents
Sample
Sampling frame
Target population
Representation
Survey statistic
40
Total Survey Error diagram as presented in
Groves, R. M., F. J. Fowler, M. P. Couper, J. M.
Lepkowski, E. Singer and R. Tourangeau (2009).
Survey methodology. Hoboken, N.J., Wiley.41
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
42
Total Survey Error diagram as presented in
Groves, R. M., F. J. Fowler, M. P. Couper, J. M.
Lepkowski, E. Singer and R. Tourangeau (2009).
Survey methodology. Hoboken, N.J., Wiley.43
Ineke Stoop did
a deep dive into
nonresponse error
http://www.scp.nl/english/Publications/Publications_by_year/Publications_2005/The_Hunt_for_the_Last_Respondent
.pdf available free: http://bit.ly/1kKTc0U (small charge for a printed copy)44
Nonresponse can happen for
unexpected reasons
“ There is anecdotal evidence of interviewers not pushing
the doorbell of sample addresses with aspidistras on the
windowsill because this indicated elderly people and
possibly very long interviews.”
Stoop, I. A. L. (2005). The Hunt for the Last Respondent
Image credit: Flickr, Sunchild5745
The ‘zone of indifference’ is a common
cause of nonresponse error
46
Total Survey Error diagram as presented in
Groves, R. M., F. J. Fowler, M. P. Couper, J. M.
Lepkowski, E. Singer and R. Tourangeau (2009).
Survey methodology. Hoboken, N.J., Wiley.47
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
48
Survey methodologists don’t tell us
much about how to deal with raw data
49
Total Survey Error diagram as presented in
Groves, R. M., F. J. Fowler, M. P. Couper, J. M.
Lepkowski, E. Singer and R. Tourangeau (2009).
Survey methodology. Hoboken, N.J., Wiley.50
Steve Krug and I did
an exploratory survey
http://www.slideshare.net/cjforms/why-do-usability-problems-go-unfixed51
To deal with open questions:
read and think
Read all
the answers
Ignore them:
focus on your key
findings instead
Tiny number of
irrelevancies
Quote them:
your stakeholders
should see them
A few illuminating
comments
Code them:
summarize into
categories
Lots of themes,
similarities, repetition
52
A code is a symbolic phrase
“A code in qualitative inquiry is most often a word or short
phrase that symbolically assigns a summative, salient,
essence-capturing, and/or evocative attribute for a portion
of language-based or visual data”
Saldaña (2013)
Saldaña, J. (2013).
The coding manual for qualitative researchers.
Los Angeles, [Calif.] ; London, SAGE.
Image credit: txmx 2 on Flickr
53
Some coding methods to consider
Method What you do
“In vivo” coding Choose small chunks of the respondent’s own words
that are representative of the response
Adjectival coding Use an adjective that sums up the response
Descriptive coding Summarise in a word or short phrase (most often as a
noun) the basic topic
Provisional coding Establishes a predetermined start list of codes prior to
the survey
(at least 20 others)
Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, [Calif.] ; London, SAGE.54
Try two coding methods
• Two teams
• One will try: descriptive coding
– Summarise in a word or short phrase (most often as a noun)
the basic topic in each answer
– Try to group the codes you find into topics
• One will try: provisional coding
– I have given you a set of four codes
– Try to assign each response to a code
55
Choose your coding method according
to the goals of your survey
If you want to… Consider….
Help your team to understand the way users think
about a topic and the words they choose to
express those thoughts
“In vivo” coding
Find out whether users are in favour of or against
a proposed idea
Adjectival coding
Explore new ideas; be open to fresh suggestions Descriptive coding
Support or disprove an existing idea or
suggestion
Provisional coding
Something else (plenty more to choose
from)
Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, [Calif.] ; London, SAGE.56
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
57
Total Survey Error diagram as presented in
Groves, R. M., F. J. Fowler, M. P. Couper, J. M.
Lepkowski, E. Singer and R. Tourangeau (2009).
Survey methodology. Hoboken, N.J., Wiley.58
Tip
Test everything
59
Which one works?
• How often to show a progress indicator?
– Always
– Intermittently (after about eight pages)
– On demand (if the user asks for it)
• What does the progress indicator tell you?
– Fast to slow (progression is quicker at the start of the survey)
– Slow to fast (progression is slower at the start of the survey)
– Constant speed (exactly the correct amount of progress)
60
Removing a progress indicator:
no difference in completion rates
https://designnotes.blog.gov.uk/
2014/07/07/do-less-problems-
as-shared-spaces/61
Mick Couper has devoted years
to testing everything for you
• Buy the book
• If you have access to an
academic library, look for vast
numbers of Couper papers
Couper, M. (2008). Designing effective
Web surveys. Cambridge, Cambridge
University Press.
62
Today’s
challenge
Better survey questions
Warm up: the SUS
1: Tourangeau, Rips, Rasinski (2000)
2: Oliver (2010)
Better survey processes
3: Groves (many references)
4: Stoop (2005)
5. Saldaña (2013)
6: Couper (2008)
63
Today’s
concepts
Better survey questions
Warm up: the SUS
1: Ask questions that people can answer
2: Satisfaction is a slippery topic
Better survey processes
3: Assess the total survey error
4: Understand who responds
5. Your survey goals drive your analysis
6. Test everything
64
Caroline Jarrett
Twitter @cjforms
http://www.effortmark.co.uk
carolinej@effortmark.co.uk
65
More resources on
http://www.slideshare.net/cjforms
66

More Related Content

What's hot

Better UX Surveys at UCD2012 by @cjforms
Better UX Surveys at UCD2012 by @cjformsBetter UX Surveys at UCD2012 by @cjforms
Better UX Surveys at UCD2012 by @cjforms
Caroline Jarrett
 
Labels on forms for STC Dallas, 2010
Labels on forms  for STC Dallas, 2010Labels on forms  for STC Dallas, 2010
Labels on forms for STC Dallas, 2010
Caroline Jarrett
 
Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010
Caroline Jarrett
 
How to look at a form - Industry Conf 2016
How to look at a form -  Industry Conf 2016How to look at a form -  Industry Conf 2016
How to look at a form - Industry Conf 2016
Caroline Jarrett
 
Design tips for surveys UIE 2012
Design tips for surveys UIE 2012Design tips for surveys UIE 2012
Design tips for surveys UIE 2012
Caroline Jarrett
 
Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...
Caroline Jarrett
 
10 tips for a better survey at STC2011
10 tips for a better survey at STC201110 tips for a better survey at STC2011
10 tips for a better survey at STC2011
Caroline Jarrett
 
How to design a form: Discussing a curriculum
How to design a form: Discussing a curriculumHow to design a form: Discussing a curriculum
How to design a form: Discussing a curriculum
Caroline Jarrett
 
How to avoid research debt
How to avoid research debtHow to avoid research debt
How to avoid research debt
Caroline Jarrett
 
10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol
Caroline Jarrett
 
Buttons on forms and surveys: a look at some research 2012
Buttons on forms and surveys: a look at some research 2012Buttons on forms and surveys: a look at some research 2012
Buttons on forms and surveys: a look at some research 2012
Caroline Jarrett
 
How to look at the content in a form
How to look at the content in a formHow to look at the content in a form
How to look at the content in a form
Caroline Jarrett
 
How to find out about the usability of your web site using a survey by @cjforms
How to find out about the usability of your web site using a survey by @cjformsHow to find out about the usability of your web site using a survey by @cjforms
How to find out about the usability of your web site using a survey by @cjforms
Caroline Jarrett
 
Web Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & TestingWeb Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & Testing
Jennifer Romano Bergstrom
 
Forms workshop for ConCon Manchester 2016 by @cjforms
Forms workshop for ConCon Manchester 2016 by @cjformsForms workshop for ConCon Manchester 2016 by @cjforms
Forms workshop for ConCon Manchester 2016 by @cjforms
Caroline Jarrett
 
Discussing a Topic Map for How to Design a Better Form 2019 March
Discussing a Topic Map for How to Design a Better Form 2019 MarchDiscussing a Topic Map for How to Design a Better Form 2019 March
Discussing a Topic Map for How to Design a Better Form 2019 March
Caroline Jarrett
 
The Tester's Three C’s: Communication, Criticism, Confidence
The Tester's Three C’s: Communication, Criticism, ConfidenceThe Tester's Three C’s: Communication, Criticism, Confidence
The Tester's Three C’s: Communication, Criticism, Confidence
TechWell
 
2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)
Kath Straub
 
Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!
UXPA Boston
 
UX Lesson 2: User Research
UX Lesson 2: User ResearchUX Lesson 2: User Research
UX Lesson 2: User Research
Joan Lumanauw
 

What's hot (20)

Better UX Surveys at UCD2012 by @cjforms
Better UX Surveys at UCD2012 by @cjformsBetter UX Surveys at UCD2012 by @cjforms
Better UX Surveys at UCD2012 by @cjforms
 
Labels on forms for STC Dallas, 2010
Labels on forms  for STC Dallas, 2010Labels on forms  for STC Dallas, 2010
Labels on forms for STC Dallas, 2010
 
Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010Label placement on forms, STC Seattle 2010
Label placement on forms, STC Seattle 2010
 
How to look at a form - Industry Conf 2016
How to look at a form -  Industry Conf 2016How to look at a form -  Industry Conf 2016
How to look at a form - Industry Conf 2016
 
Design tips for surveys UIE 2012
Design tips for surveys UIE 2012Design tips for surveys UIE 2012
Design tips for surveys UIE 2012
 
Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...Inwards and outwards research: choosing your research methods according to th...
Inwards and outwards research: choosing your research methods according to th...
 
10 tips for a better survey at STC2011
10 tips for a better survey at STC201110 tips for a better survey at STC2011
10 tips for a better survey at STC2011
 
How to design a form: Discussing a curriculum
How to design a form: Discussing a curriculumHow to design a form: Discussing a curriculum
How to design a form: Discussing a curriculum
 
How to avoid research debt
How to avoid research debtHow to avoid research debt
How to avoid research debt
 
10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol10 tips for a better survey at UX Bristol
10 tips for a better survey at UX Bristol
 
Buttons on forms and surveys: a look at some research 2012
Buttons on forms and surveys: a look at some research 2012Buttons on forms and surveys: a look at some research 2012
Buttons on forms and surveys: a look at some research 2012
 
How to look at the content in a form
How to look at the content in a formHow to look at the content in a form
How to look at the content in a form
 
How to find out about the usability of your web site using a survey by @cjforms
How to find out about the usability of your web site using a survey by @cjformsHow to find out about the usability of your web site using a survey by @cjforms
How to find out about the usability of your web site using a survey by @cjforms
 
Web Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & TestingWeb Survey and Forms Usability Design & Testing
Web Survey and Forms Usability Design & Testing
 
Forms workshop for ConCon Manchester 2016 by @cjforms
Forms workshop for ConCon Manchester 2016 by @cjformsForms workshop for ConCon Manchester 2016 by @cjforms
Forms workshop for ConCon Manchester 2016 by @cjforms
 
Discussing a Topic Map for How to Design a Better Form 2019 March
Discussing a Topic Map for How to Design a Better Form 2019 MarchDiscussing a Topic Map for How to Design a Better Form 2019 March
Discussing a Topic Map for How to Design a Better Form 2019 March
 
The Tester's Three C’s: Communication, Criticism, Confidence
The Tester's Three C’s: Communication, Criticism, ConfidenceThe Tester's Three C’s: Communication, Criticism, Confidence
The Tester's Three C’s: Communication, Criticism, Confidence
 
2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)2 Studies UX types should know about (Straub UXPA unconference13)
2 Studies UX types should know about (Straub UXPA unconference13)
 
Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!Predictive Analytics with UX Research Data: Yes We Can!
Predictive Analytics with UX Research Data: Yes We Can!
 
UX Lesson 2: User Research
UX Lesson 2: User ResearchUX Lesson 2: User Research
UX Lesson 2: User Research
 

Similar to Six crucial survey concepts that UX professionals need to know

A deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLxA deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLx
Caroline Jarrett
 
Real-Time Community Question Answering: Exploring Content Recommendation and ...
Real-Time Community Question Answering: Exploring Content Recommendation and ...Real-Time Community Question Answering: Exploring Content Recommendation and ...
Real-Time Community Question Answering: Exploring Content Recommendation and ...
Jinho Choi
 
Understanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent AssistantsUnderstanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent Assistants
Julia Kiseleva
 
Understanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent AssistantsUnderstanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent Assistants
PromoVeTue
 
CSAL UX Sessions in Atlanta - Part 1
CSAL UX Sessions in Atlanta - Part 1CSAL UX Sessions in Atlanta - Part 1
CSAL UX Sessions in Atlanta - Part 1
Patrick Hays
 
Understanding User Satisfaction with Intelligent Assistants
Understanding User Satisfaction with Intelligent AssistantsUnderstanding User Satisfaction with Intelligent Assistants
Understanding User Satisfaction with Intelligent Assistants
Julia Kiseleva
 
Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2
Caroline Jarrett
 
SOC2002 Lecture 5
SOC2002 Lecture 5SOC2002 Lecture 5
SOC2002 Lecture 5
Bonnie Green
 
ENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdj
ENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdjENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdj
ENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdj
ADManiego
 
Bj research session 8 gathering quantitative data
Bj research session 8 gathering quantitative dataBj research session 8 gathering quantitative data
Bj research session 8 gathering quantitative data
Ian Cammack
 
John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...
Sax Institute
 
factor survey_new Data Analytics Marketing.pptx
factor survey_new Data Analytics Marketing.pptxfactor survey_new Data Analytics Marketing.pptx
factor survey_new Data Analytics Marketing.pptx
bitesysiims
 
LearningInCrowdWorkplace
LearningInCrowdWorkplaceLearningInCrowdWorkplace
LearningInCrowdWorkplace
Anoush Margaryan
 
Questionnair Designing lec3 RESEARCH METHODOLOGY
Questionnair Designing lec3 RESEARCH METHODOLOGYQuestionnair Designing lec3 RESEARCH METHODOLOGY
Questionnair Designing lec3 RESEARCH METHODOLOGY
SaadSohail54
 
UI/UX Foundations - Research
UI/UX Foundations - ResearchUI/UX Foundations - Research
UI/UX Foundations - Research
Meg Kurdziolek
 
10 tips for a better UX survey
10 tips for a better UX survey10 tips for a better UX survey
10 tips for a better UX survey
Caroline Jarrett
 
Rss Oct 2011 Mixed Modes Pres2
Rss Oct 2011 Mixed Modes Pres2Rss Oct 2011 Mixed Modes Pres2
Rss Oct 2011 Mixed Modes Pres2
GerryNicolaas
 
Format Effect in Looping Questions
Format Effect in Looping QuestionsFormat Effect in Looping Questions
Format Effect in Looping Questions
Stephanie Eckman
 
Engaging with Users on Public Social Media
Engaging with Users on Public Social MediaEngaging with Users on Public Social Media
Engaging with Users on Public Social Media
Jeffrey Nichols
 
BRS SA 2.0 (2021) - Part 3 of 3.pptx
BRS SA 2.0 (2021) - Part 3 of 3.pptxBRS SA 2.0 (2021) - Part 3 of 3.pptx
BRS SA 2.0 (2021) - Part 3 of 3.pptx
HajiRock
 

Similar to Six crucial survey concepts that UX professionals need to know (20)

A deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLxA deep dive into questions by @cjforms at UxLx
A deep dive into questions by @cjforms at UxLx
 
Real-Time Community Question Answering: Exploring Content Recommendation and ...
Real-Time Community Question Answering: Exploring Content Recommendation and ...Real-Time Community Question Answering: Exploring Content Recommendation and ...
Real-Time Community Question Answering: Exploring Content Recommendation and ...
 
Understanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent AssistantsUnderstanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent Assistants
 
Understanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent AssistantsUnderstanding and Predicting User Satisfaction with Intelligent Assistants
Understanding and Predicting User Satisfaction with Intelligent Assistants
 
CSAL UX Sessions in Atlanta - Part 1
CSAL UX Sessions in Atlanta - Part 1CSAL UX Sessions in Atlanta - Part 1
CSAL UX Sessions in Atlanta - Part 1
 
Understanding User Satisfaction with Intelligent Assistants
Understanding User Satisfaction with Intelligent AssistantsUnderstanding User Satisfaction with Intelligent Assistants
Understanding User Satisfaction with Intelligent Assistants
 
Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2Surveys that work: training course for Rosenfeld media, day 2
Surveys that work: training course for Rosenfeld media, day 2
 
SOC2002 Lecture 5
SOC2002 Lecture 5SOC2002 Lecture 5
SOC2002 Lecture 5
 
ENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdj
ENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdjENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdj
ENGLISH REPORT.pptxjsisisjdjdjsjsidijdjdjdj
 
Bj research session 8 gathering quantitative data
Bj research session 8 gathering quantitative dataBj research session 8 gathering quantitative data
Bj research session 8 gathering quantitative data
 
John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...John Lavis | Making research work for decision makers: international perspect...
John Lavis | Making research work for decision makers: international perspect...
 
factor survey_new Data Analytics Marketing.pptx
factor survey_new Data Analytics Marketing.pptxfactor survey_new Data Analytics Marketing.pptx
factor survey_new Data Analytics Marketing.pptx
 
LearningInCrowdWorkplace
LearningInCrowdWorkplaceLearningInCrowdWorkplace
LearningInCrowdWorkplace
 
Questionnair Designing lec3 RESEARCH METHODOLOGY
Questionnair Designing lec3 RESEARCH METHODOLOGYQuestionnair Designing lec3 RESEARCH METHODOLOGY
Questionnair Designing lec3 RESEARCH METHODOLOGY
 
UI/UX Foundations - Research
UI/UX Foundations - ResearchUI/UX Foundations - Research
UI/UX Foundations - Research
 
10 tips for a better UX survey
10 tips for a better UX survey10 tips for a better UX survey
10 tips for a better UX survey
 
Rss Oct 2011 Mixed Modes Pres2
Rss Oct 2011 Mixed Modes Pres2Rss Oct 2011 Mixed Modes Pres2
Rss Oct 2011 Mixed Modes Pres2
 
Format Effect in Looping Questions
Format Effect in Looping QuestionsFormat Effect in Looping Questions
Format Effect in Looping Questions
 
Engaging with Users on Public Social Media
Engaging with Users on Public Social MediaEngaging with Users on Public Social Media
Engaging with Users on Public Social Media
 
BRS SA 2.0 (2021) - Part 3 of 3.pptx
BRS SA 2.0 (2021) - Part 3 of 3.pptxBRS SA 2.0 (2021) - Part 3 of 3.pptx
BRS SA 2.0 (2021) - Part 3 of 3.pptx
 

More from Caroline Jarrett

Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024
Caroline Jarrett
 
Using surveys to uncover audience insights
Using surveys to uncover audience insightsUsing surveys to uncover audience insights
Using surveys to uncover audience insights
Caroline Jarrett
 
2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx
Caroline Jarrett
 
The Phylogenetic Tree in forms design - making forms work for complex academ...
The Phylogenetic Tree in forms design - making forms work for complex  academ...The Phylogenetic Tree in forms design - making forms work for complex  academ...
The Phylogenetic Tree in forms design - making forms work for complex academ...
Caroline Jarrett
 
Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023
Caroline Jarrett
 
Content design in practice May 2023.pptx
Content design in practice May 2023.pptxContent design in practice May 2023.pptx
Content design in practice May 2023.pptx
Caroline Jarrett
 
Two ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptxTwo ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptx
Caroline Jarrett
 
How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023
Caroline Jarrett
 
Two ways to improve your surveys: the Most Crucial Question and the Burning I...
Two ways to improve your surveys: the Most Crucial Question and the Burning I...Two ways to improve your surveys: the Most Crucial Question and the Burning I...
Two ways to improve your surveys: the Most Crucial Question and the Burning I...
Caroline Jarrett
 
How to look at a form Boye 22.pptx
How to look at a form Boye 22.pptxHow to look at a form Boye 22.pptx
How to look at a form Boye 22.pptx
Caroline Jarrett
 
Prune tune postpone explain.pptx
Prune tune postpone explain.pptxPrune tune postpone explain.pptx
Prune tune postpone explain.pptx
Caroline Jarrett
 
What is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptxWhat is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptx
Caroline Jarrett
 
Helping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptxHelping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptx
Caroline Jarrett
 
Some thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference callSome thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference call
Caroline Jarrett
 
Plain language to improve your survey houston 2022
Plain language to improve your survey houston 2022Plain language to improve your survey houston 2022
Plain language to improve your survey houston 2022
Caroline Jarrett
 
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorSurveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
Caroline Jarrett
 
The inevitable survey 2021
The inevitable survey 2021The inevitable survey 2021
The inevitable survey 2021
Caroline Jarrett
 
Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021
Caroline Jarrett
 
Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021
Caroline Jarrett
 
A chat about forms xgov 2020 cjforms
A chat about forms xgov 2020 cjformsA chat about forms xgov 2020 cjforms
A chat about forms xgov 2020 cjforms
Caroline Jarrett
 

More from Caroline Jarrett (20)

Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024Creating truly accessible forms, SDinGov 2024
Creating truly accessible forms, SDinGov 2024
 
Using surveys to uncover audience insights
Using surveys to uncover audience insightsUsing surveys to uncover audience insights
Using surveys to uncover audience insights
 
2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx2023 Surveys that work Olin.pptx
2023 Surveys that work Olin.pptx
 
The Phylogenetic Tree in forms design - making forms work for complex academ...
The Phylogenetic Tree in forms design - making forms work for complex  academ...The Phylogenetic Tree in forms design - making forms work for complex  academ...
The Phylogenetic Tree in forms design - making forms work for complex academ...
 
Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023Four ways to make a better survey WebExpo2023
Four ways to make a better survey WebExpo2023
 
Content design in practice May 2023.pptx
Content design in practice May 2023.pptxContent design in practice May 2023.pptx
Content design in practice May 2023.pptx
 
Two ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptxTwo ways to improve your survey, webinar for Delib 2023.pptx
Two ways to improve your survey, webinar for Delib 2023.pptx
 
How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023How to save money by fixing forms, WebExpo2023
How to save money by fixing forms, WebExpo2023
 
Two ways to improve your surveys: the Most Crucial Question and the Burning I...
Two ways to improve your surveys: the Most Crucial Question and the Burning I...Two ways to improve your surveys: the Most Crucial Question and the Burning I...
Two ways to improve your surveys: the Most Crucial Question and the Burning I...
 
How to look at a form Boye 22.pptx
How to look at a form Boye 22.pptxHow to look at a form Boye 22.pptx
How to look at a form Boye 22.pptx
 
Prune tune postpone explain.pptx
Prune tune postpone explain.pptxPrune tune postpone explain.pptx
Prune tune postpone explain.pptx
 
What is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptxWhat is a service designer SDinGOV 22 with all stickies.pptx
What is a service designer SDinGOV 22 with all stickies.pptx
 
Helping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptxHelping teenage boys to become responsible adults.pptx
Helping teenage boys to become responsible adults.pptx
 
Some thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference callSome thoughts on surveys: Boye and Company member conference call
Some thoughts on surveys: Boye and Company member conference call
 
Plain language to improve your survey houston 2022
Plain language to improve your survey houston 2022Plain language to improve your survey houston 2022
Plain language to improve your survey houston 2022
 
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey ErrorSurveys that work: an introduction to the Survey Octopus and Total Survey Error
Surveys that work: an introduction to the Survey Octopus and Total Survey Error
 
The inevitable survey 2021
The inevitable survey 2021The inevitable survey 2021
The inevitable survey 2021
 
Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021Surveys that work: a webinar for FocusVision 2021
Surveys that work: a webinar for FocusVision 2021
 
Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021Surveys that work: a webinar for UXPA 2021
Surveys that work: a webinar for UXPA 2021
 
A chat about forms xgov 2020 cjforms
A chat about forms xgov 2020 cjformsA chat about forms xgov 2020 cjforms
A chat about forms xgov 2020 cjforms
 

Recently uploaded

PDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in NoidaPDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in Noida
PoojaSaini954651
 
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptxUNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
GOWSIKRAJA PALANISAMY
 
Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1
Decomart Studio
 
UNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptx
UNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptxUNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptx
UNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptx
GOWSIKRAJA PALANISAMY
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
 
Technoblade The Legacy of a Minecraft Legend.
Technoblade The Legacy of a Minecraft Legend.Technoblade The Legacy of a Minecraft Legend.
Technoblade The Legacy of a Minecraft Legend.
Techno Merch
 
Revolutionizing the Digital Landscape: Web Development Companies in India
Revolutionizing the Digital Landscape: Web Development Companies in IndiaRevolutionizing the Digital Landscape: Web Development Companies in India
Revolutionizing the Digital Landscape: Web Development Companies in India
amrsoftec1
 
ARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdfARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdf
Knight Moves
 
Practical eLearning Makeovers for Everyone
Practical eLearning Makeovers for EveryonePractical eLearning Makeovers for Everyone
Practical eLearning Makeovers for Everyone
Bianca Woods
 
一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理
一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理
一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理
k7nm6tk
 
定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样
qo1as76n
 
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
pmgdscunsri
 
Graphic Design Tools and Software .pptx
Graphic Design Tools and Software   .pptxGraphic Design Tools and Software   .pptx
Graphic Design Tools and Software .pptx
Virtual Real Design
 
Impact of Fonts: in Web and Apps Design
Impact of Fonts:  in Web and Apps DesignImpact of Fonts:  in Web and Apps Design
Impact of Fonts: in Web and Apps Design
contactproperweb2014
 
Connect Conference 2022: Passive House - Economic and Environmental Solution...
Connect Conference 2022: Passive House -  Economic and Environmental Solution...Connect Conference 2022: Passive House -  Economic and Environmental Solution...
Connect Conference 2022: Passive House - Economic and Environmental Solution...
TE Studio
 
一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理
peuce
 
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdfAHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
talaatahm
 
International Upcycling Research Network advisory board meeting 4
International Upcycling Research Network advisory board meeting 4International Upcycling Research Network advisory board meeting 4
International Upcycling Research Network advisory board meeting 4
Kyungeun Sung
 
Heuristics Evaluation - How to Guide.pdf
Heuristics Evaluation - How to Guide.pdfHeuristics Evaluation - How to Guide.pdf
Heuristics Evaluation - How to Guide.pdf
Jaime Brown
 
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With DesignsGame Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
184804
 

Recently uploaded (20)

PDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in NoidaPDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in Noida
 
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptxUNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
 
Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1
 
UNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptx
UNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptxUNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptx
UNIT V ACTIONS AND COMMANDS, FORMS AND CONTROLS.pptx
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
 
Technoblade The Legacy of a Minecraft Legend.
Technoblade The Legacy of a Minecraft Legend.Technoblade The Legacy of a Minecraft Legend.
Technoblade The Legacy of a Minecraft Legend.
 
Revolutionizing the Digital Landscape: Web Development Companies in India
Revolutionizing the Digital Landscape: Web Development Companies in IndiaRevolutionizing the Digital Landscape: Web Development Companies in India
Revolutionizing the Digital Landscape: Web Development Companies in India
 
ARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdfARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdf
 
Practical eLearning Makeovers for Everyone
Practical eLearning Makeovers for EveryonePractical eLearning Makeovers for Everyone
Practical eLearning Makeovers for Everyone
 
一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理
一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理
一比一原版(LSBU毕业证书)伦敦南岸大学毕业证如何办理
 
定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样
 
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
 
Graphic Design Tools and Software .pptx
Graphic Design Tools and Software   .pptxGraphic Design Tools and Software   .pptx
Graphic Design Tools and Software .pptx
 
Impact of Fonts: in Web and Apps Design
Impact of Fonts:  in Web and Apps DesignImpact of Fonts:  in Web and Apps Design
Impact of Fonts: in Web and Apps Design
 
Connect Conference 2022: Passive House - Economic and Environmental Solution...
Connect Conference 2022: Passive House -  Economic and Environmental Solution...Connect Conference 2022: Passive House -  Economic and Environmental Solution...
Connect Conference 2022: Passive House - Economic and Environmental Solution...
 
一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理
 
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdfAHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
 
International Upcycling Research Network advisory board meeting 4
International Upcycling Research Network advisory board meeting 4International Upcycling Research Network advisory board meeting 4
International Upcycling Research Network advisory board meeting 4
 
Heuristics Evaluation - How to Guide.pdf
Heuristics Evaluation - How to Guide.pdfHeuristics Evaluation - How to Guide.pdf
Heuristics Evaluation - How to Guide.pdf
 
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With DesignsGame Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
 

Six crucial survey concepts that UX professionals need to know

  • 1. Six crucial survey concepts that UX professionals need to know Workshop at UXPA 2014 led by Caroline Jarrett
  • 2. Today’s concepts Better survey questions Warm up: the SUS 1: Ask questions that people can answer 2: Satisfaction is a slippery topic Better survey processes 3: Assess the total survey error 4: Understand who responds 5. Your survey goals drive your analysis 6. Test everything 2
  • 3. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 3
  • 4. Introductions (I’m Caroline Jarrett - @cjforms) Work with your neighbour • Your name and role • A random thing about yourself Image credit: http://www.visitlondon.com/things-to-do/event/28528358-tour-de-france-grand-depart-2014-third-stage. 4
  • 5. The SUS is the most-used questionnaire for post-task satisfaction • John Brooke created the System Usability Scale (SUS) in 1986 • Used after a usability test • It has been shown to be valid and reliable • You get a score between 0 and 100 • You can compare your SUS score with other systems Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In Usability Evaluation in Industry. P. W. Jordan, B. Thomas, B. A. Weerdmeester and A. L. McClelland. London:, Taylor and Francis. 5
  • 6. We’ll try cognitive interviewing • In a cognitive interview, you ask your respondent to explain the mental processes in answering the question Cognitive interviewing focuses on the questions Usability testing focuses on interaction Willis, G. B. (2005). “Cognitive interviewing : a tool for improving questionnaire design”. Thousand Oaks, Calif., Sage Publications. 6
  • 7. • Silent two minute task: – Find an explanation of the difference between the European Commission and the European Union – Use this site: http://ec.europa.eu • Decide whether you had a good or bad experience 7
  • 8. • I’ll assign the roles of ‘interviewer’ and ‘respondent’ • Interviewer: – Ask your respondent to fill out the first three questions of the SUS with the EU task in mind – Take notes about any challenges in the questions on the SUS • Respondent: – Please fill out the first three questions of the SUS, explaining your thought processes for each question 8
  • 9. Relating our findings from the cognitive interview to today’s concepts Concepts What we learned from SUS Ask questions that people can answer Satisfaction is a slippery topic Assess the total survey error Understand who responds Your survey goals drive your analysis Test everything 9
  • 10. Today’s concepts Better survey questions Warm up: the SUS 1: Ask questions that people can answer 2: Satisfaction is a slippery topic Better survey processes 3: Assess the total survey error 4: Understand who responds 5. Your survey goals drive your analysis 6. Test everything 10
  • 11. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 11
  • 12. There are four steps to answer a question Step 1. Read and understand 2. Find an answer 3. Judge the answer 4. Place the answer Adapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000) “The psychology of survey response”
  • 13. There are four steps to answer a question Step A good question … 1. Read and understand is legible and makes sense 2. Find an answer asks for answers that we know 3. Judge the answer asks for answers we’re happy to reveal 4. Place the answer offers appropriate spaces for the answers Adapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000) “The psychology of survey response”
  • 14. Use the four steps • Swap roles and try the remaining SUS questions • Think about the four steps of answering a question 1. Read and understand 2. Find the answer 3. Judge the answer 4. Place the answer • Identify any problems in the questions according to the four steps 14
  • 15. Four step examples: 1: read and understand 15
  • 16. Four step examples: 1: read and understand Hermann grid illusion16
  • 17. Four step examples: 2: find the answer In your last five days at work, what percentage of your work time do you estimate that you spend using publicly- available online services (not including email, instant messaging and search) to do your work using a work computer or other device? 17
  • 18. The approximate curve of forgetting
  • 19. Four step examples: 3: judge the answer 19
  • 20. Four step examples: 4: place the answer 20
  • 21. 21
  • 22. 22
  • 23. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 23
  • 24. My journey into user experience started a long time ago, with usability The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) 24
  • 25. In government, we were interested in effectiveness and efficiency The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) 25
  • 26. But what about user experience? What about satisfaction? The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) Picture credit: Flickr jek in the box 26
  • 27. 27
  • 28. Satisfaction is a complex matter Compared experience to what? Resulting thoughts (nothing) Indifference Expectations Better / worse / different Needs Met / not met / mixture Excellence (the ideal product) Good / poor quality (or ‘good enough’) Fairness Treated equitably / inequitably Events that might have been Vindication / regret Adapted from Oliver, R. L. (1996) and (2010) “Satisfaction: A Behavioral Perspective on the Consumer”28
  • 29. Example: bronze medal winners tend to be happier than silver medal winners Nathan Twaddle, Olympic Bronze Medal Winner in Beijing Matsumoto D, & Willingham B (2006). The thrill of victory and the agony of defeat: spontaneous expressions of medal winners of the 2004 Athens Olympic Games. Photo credit: peter.cipollone, Flickr 29
  • 30. • The first question was about rating satisfaction • What were they asking us to rate? – Just a guess from what you recall 30
  • 31. 31
  • 32. The challenge of UX and surveys: which bit to measure? The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241:11 1998) ???
  • 33. Some ideas about what we could measure (as well as satisfaction) In the definition GoDaddy customer support GoDaddy as a provider of domain names Product This contact with help desk Overall experience of moving a domain to GoDaddy Users What proportion of customers contact support Demographics (example: type of job) Goals Reason for contacting help Reason for looking at GoDaddy Effectiveness Whether support fixed the problem Whether GoDaddy offers the right products Efficiency Whether it took a reasonable time Whether the product is priced correctly Satisfaction Helpfulness of support person Likely to purchase again / recommend Context of use Home/office; alone/helped Business / personal 33
  • 34. Tip Find out about users’ goals 34
  • 35. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 35
  • 36. 36
  • 37. Survey lifecycle for total survey error The answers you use The answers you get The questions you ask What you want to find out about What The sample you use The ones who answer The sample you ask The list you use to sample from Who you want to find out from Who The number 37
  • 38. 38
  • 39. Choices compared to the lifecycle The answers you use The answers you get The questions you ask What you want to find out about What The sample you use The ones who answer The sample you ask The list you use to sample from Who you want to find out from Who The number Services so good that people prefer to use them Satisfaction Overall, how satisfied were you with renewing your car tax today? Scale of 5 points from very satisfied to very dissatisfied Ratings weighted from 100% for 'very satisfied' to 0% for 'very dissatisfied' UK population People who renew car tax online Everyone who finishes the renewal People who answer the question 94% (Any exclusions for repeated answers or test answers?) Everyone who finishes the renewal 39
  • 40. And now in “survey methodologist” Edited response Response Measurement Construct Measurement Post-survey adjustments Respondents Sample Sampling frame Target population Representation Survey statistic 40
  • 41. Total Survey Error diagram as presented in Groves, R. M., F. J. Fowler, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau (2009). Survey methodology. Hoboken, N.J., Wiley.41
  • 42. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 42
  • 43. Total Survey Error diagram as presented in Groves, R. M., F. J. Fowler, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau (2009). Survey methodology. Hoboken, N.J., Wiley.43
  • 44. Ineke Stoop did a deep dive into nonresponse error http://www.scp.nl/english/Publications/Publications_by_year/Publications_2005/The_Hunt_for_the_Last_Respondent .pdf available free: http://bit.ly/1kKTc0U (small charge for a printed copy)44
  • 45. Nonresponse can happen for unexpected reasons “ There is anecdotal evidence of interviewers not pushing the doorbell of sample addresses with aspidistras on the windowsill because this indicated elderly people and possibly very long interviews.” Stoop, I. A. L. (2005). The Hunt for the Last Respondent Image credit: Flickr, Sunchild5745
  • 46. The ‘zone of indifference’ is a common cause of nonresponse error 46
  • 47. Total Survey Error diagram as presented in Groves, R. M., F. J. Fowler, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau (2009). Survey methodology. Hoboken, N.J., Wiley.47
  • 48. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 48
  • 49. Survey methodologists don’t tell us much about how to deal with raw data 49
  • 50. Total Survey Error diagram as presented in Groves, R. M., F. J. Fowler, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau (2009). Survey methodology. Hoboken, N.J., Wiley.50
  • 51. Steve Krug and I did an exploratory survey http://www.slideshare.net/cjforms/why-do-usability-problems-go-unfixed51
  • 52. To deal with open questions: read and think Read all the answers Ignore them: focus on your key findings instead Tiny number of irrelevancies Quote them: your stakeholders should see them A few illuminating comments Code them: summarize into categories Lots of themes, similarities, repetition 52
  • 53. A code is a symbolic phrase “A code in qualitative inquiry is most often a word or short phrase that symbolically assigns a summative, salient, essence-capturing, and/or evocative attribute for a portion of language-based or visual data” Saldaña (2013) Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, [Calif.] ; London, SAGE. Image credit: txmx 2 on Flickr 53
  • 54. Some coding methods to consider Method What you do “In vivo” coding Choose small chunks of the respondent’s own words that are representative of the response Adjectival coding Use an adjective that sums up the response Descriptive coding Summarise in a word or short phrase (most often as a noun) the basic topic Provisional coding Establishes a predetermined start list of codes prior to the survey (at least 20 others) Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, [Calif.] ; London, SAGE.54
  • 55. Try two coding methods • Two teams • One will try: descriptive coding – Summarise in a word or short phrase (most often as a noun) the basic topic in each answer – Try to group the codes you find into topics • One will try: provisional coding – I have given you a set of four codes – Try to assign each response to a code 55
  • 56. Choose your coding method according to the goals of your survey If you want to… Consider…. Help your team to understand the way users think about a topic and the words they choose to express those thoughts “In vivo” coding Find out whether users are in favour of or against a proposed idea Adjectival coding Explore new ideas; be open to fresh suggestions Descriptive coding Support or disprove an existing idea or suggestion Provisional coding Something else (plenty more to choose from) Saldaña, J. (2013). The coding manual for qualitative researchers. Los Angeles, [Calif.] ; London, SAGE.56
  • 57. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 57
  • 58. Total Survey Error diagram as presented in Groves, R. M., F. J. Fowler, M. P. Couper, J. M. Lepkowski, E. Singer and R. Tourangeau (2009). Survey methodology. Hoboken, N.J., Wiley.58
  • 60. Which one works? • How often to show a progress indicator? – Always – Intermittently (after about eight pages) – On demand (if the user asks for it) • What does the progress indicator tell you? – Fast to slow (progression is quicker at the start of the survey) – Slow to fast (progression is slower at the start of the survey) – Constant speed (exactly the correct amount of progress) 60
  • 61. Removing a progress indicator: no difference in completion rates https://designnotes.blog.gov.uk/ 2014/07/07/do-less-problems- as-shared-spaces/61
  • 62. Mick Couper has devoted years to testing everything for you • Buy the book • If you have access to an academic library, look for vast numbers of Couper papers Couper, M. (2008). Designing effective Web surveys. Cambridge, Cambridge University Press. 62
  • 63. Today’s challenge Better survey questions Warm up: the SUS 1: Tourangeau, Rips, Rasinski (2000) 2: Oliver (2010) Better survey processes 3: Groves (many references) 4: Stoop (2005) 5. Saldaña (2013) 6: Couper (2008) 63
  • 64. Today’s concepts Better survey questions Warm up: the SUS 1: Ask questions that people can answer 2: Satisfaction is a slippery topic Better survey processes 3: Assess the total survey error 4: Understand who responds 5. Your survey goals drive your analysis 6. Test everything 64