How to (and not to!) hopelessly bias a survey.

1. Eva Kaniasty | @UXorDie
Using surveys
to get the answers you want.
Evel Knievel aka Eva Kaniasty
UX Consultant
UX for Evil

2. Eva Kaniasty | @UXorDie
2
The Challenge
Big Motorcycle, Inc.
is convinced that
motorcycles are the future
of transportation.
They need a survey to
prove it.
Big Motorcycle
is EVIL
#uxpa2018|#ux4evil

3. Eva Kaniasty | @UXorDie
1. Don’t…use distinct categories
2. Don’t…speak the users’ language
3. Don’t…set neutral defaults
4. Don’t…randomize equivalent
answer options
5. Don’t…leave room for uncertainty
3
6. Do…lead with sensitive
questions
7. Do…use leading questions
8. Do…use loaded questions
9. Do…double barrel
10.Do…mix up logical sequences.
UX For Evil
#uxpa2018|#ux4evil
EVIL DON’Ts

4. Eva Kaniasty | @UXorDie
4
1. Don’t…use disEnct categories
If you add them up,
you’ll get an
inflated esEmate.
#uxpa2018|#ux4evil

5. Eva Kaniasty | @UXorDie
5
2. Don’t…speak the users’ language
They’ll probably
pick the first, or
largest, opEon.
#uxpa2018|#ux4evil

6. Eva Kaniasty | @UXorDie
6
3. Don’t…set neutral defaults
Pro Tip:
You can point to
the right answer!
#uxpa2018|#ux4evil

7. Eva Kaniasty | @UXorDie
7
4. Don’t…randomize equivalent answer opEons
They’ll pick one of
the first choices.
#uxpa2018|#ux4evil

8. Eva Kaniasty | @UXorDie
8
5. Don’t… leave room for uncertainty.
Pro Tip:
Assume they know.
#uxpa2018|#ux4evil

9. Eva Kaniasty | @UXorDie
1. Don’t…use distinct categories
2. Don’t…speak the users’ language
3. Don’t…set neutral defaults
4. Don’t…randomize equivalent
answer options
5. Don’t…leave room for uncertainty
9
6. Do…lead with sensitive questions
7. Do…use leading questions
8. Do…use loaded questions
9. Do…double barrel.
10.Do…mix up logical sequences.
Bonus! Don’t…Pre-test your survey!
UX For Evil
#uxpa2018|#ux4evil
EVIL DOs

10. Eva Kaniasty | @UXorDie
10
6. Do…lead with sensiEve quesEons
#uxpa2018|#ux4evil
Make them
feel insecure
right away.

11. Eva Kaniasty | @UXorDie
11
7. Do…use leading quesEons
Oh HAI, I’m
the right
answer!
#uxpa2018|#ux4evil

12. Eva Kaniasty | @UXorDie
12
8. Do…use loaded quesEons.
Who wouldn’t
support
alternaEves?
#uxpa2018|#ux4evil

13. Eva Kaniasty | @UXorDie
13
9. Do…double barrel!
Of course we want to
save the environment…
and why not… we’ll do
it by driving a two-
wheeled vehicle.
#uxpa2018|#ux4evil

14. Eva Kaniasty | @UXorDie
14
10. Do…mix up logical sequences.
This looks like
the middle
frequency.
#uxpa2018|#ux4evil

15. Eva Kaniasty | @UXorDie
15
Bonus… don’t pre-test your survey
#uxpa2018|#ux4evil
If your survey has problems,
but nobody knows about it,
is it really a problem?

16. Eva Kaniasty | @UXorDie
1. Do…use distinct categories
2. Do…speak the users’ language
3. Do…set neutral defaults
4. Do…randomize equivalent
answer options
5. Do…leave room for uncertainty
16
1. Don’t…lead with sensitive
questions
2. Don’t…use leading questions
3. Don’t…use loaded questions
4. Don’t…double barrel.
5. Don’t…mix up logical sequences.
Bonus! Pre-test your survey!
UX For Good #uxpa2018|#ux4good
Do good

17. Eva Kaniasty | @UXorDie
1. Do…use distinct categories
2. Do…speak the users’ language
3. Do…set neutral defaults
4. Do…randomize equivalent
answer options
5. Do…leave room for uncertainty
17
1. Don’t…lead with sensitive
questions
2. Don’t…use leading questions
3. Don’t…use loaded questions
4. Don’t…double barrel
5. Don’t…mix up logical sequences.
Bonus! Pre-test your survey!
UX For Good #uxpa2018|#ux4good
Do no
wrong