Insights on Research Design
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Insights on Research Design

on

  • 424 views

Colleen Roller's presentation from the UXPA Boston 2014 conference.

Colleen Roller's presentation from the UXPA Boston 2014 conference.

Statistics

Views

Total Views
424
Views on SlideShare
420
Embed Views
4

Actions

Likes
3
Downloads
18
Comments
0

1 Embed 4

https://twitter.com 4

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Insights on Research Design Presentation Transcript

  • 1. Research Design Gaining deeper insight Colleen Roller Author & Practitioner
  • 2. Areas of research  Behavioral decision theory  The study of how people judge, decide and behave  Cognitive psychology  The study of how people perceive, think, remember and learn  Social psychology  The study of how people are influenced and affected by other people
  • 3. No neutral design  Preferences, judgments, and decisions are remarkably sensitive to the context in which they are made.
  • 4. No neutral design
  • 5. No neutral design
  • 6. No neutral design
  • 7. This presentation  Uses research findings to inform research design
  • 8. Clever Hans
  • 9. The power of the subconscious Rosenthal/Jacobsen 1968
  • 10. ‘When we expect certain behaviors of others, we are likely to act in ways that make the expected behavior more likely to occur.’ - Rosenthal and Babad
  • 11. Confirmation bias  The tendency to seek out and assign more weight to evidence that confirms one’s hypothesis, and ignore or underweight evidence that could disconfirm it.
  • 12. So what?  We may be influencing participants – and research findings – in ways we’re not even aware of.
  • 13. FRAMING
  • 14. What is framing?
  • 15. Framing ‘I’ve got great news. You managed to avoid a salary decrease.’
  • 16. Frame the study  The purpose of this study is to test the design, not you.
  • 17. Child custody Parent A Average income Average health Average working hours Reasonable rapport with the child Relatively stable social life Shafir, Simonson, Tversky, 1993  Which parent should get sole custody? Award – 64% Parent B Above-average income Minor health problems Lots of work-related travel Very close relationship with the child Extremely active social life
  • 18. Child custody Parent A Average income Average health Average working hours Reasonable rapport with the child Relatively stable social life Shafir, Simonson, Tversky, 1993  Which parent should not get sole custody? Award – 64% Deny – 55% Parent B Above-average income Minor health problems Lots of work-related travel Very close relationship with the child Extremely active social life
  • 19. Positive/negative  What did you like/dislike…  How useful, or not,…  How confident, or not…  Why would you continue/stop using…
  • 20. How we ask the question  Positive/negative  Breadth/depth – broad/narrow
  • 21.  Did you notice anything different about this process…  How does this process compare to how you currently do this?
  • 22.  What information do you need in order to feel confident about doing x…  What would make you feel confident about doing x…
  • 23.  How often do you use the x website…  Tell me about how and when you use the x website…
  • 24. Counterintuitive frame  What do you think of the brand?  What does the brand think of you?
  • 25. ‘A small question can never hold a large answer.’ - Gerald Zaltman (How Customers Think)
  • 26. Assumptions  Be careful that your assumptions aren’t built into the question  Turn your assumptions into questions
  • 27. Rating scales  How successful have you been in life, so far? 0 1 2 3 4 5 6 7 8 9 10 Not satisfied Very satisfied -5 -4 -3 -2 -1 0 1 2 3 4 5 Not satisfied Very satisfied 34% 13% Schwartz et al, 1991
  • 28. Scholarships  Two scholarships are available.  You prefer a scholarship with less waiting time and more money.  Which one would you pick? Sun, Li, Bonini 2010
  • 29. Graphical scale  Figure 1: wait time salient – preferred A  Figure 2: $ amount salient – preferred B Sun, Li, Bonini 2010
  • 30. Graphical comparison 0 10 20 30 40 Toothpaste B Toothpaste A People (per 5000) with serious gum disease E.R. Stone et al. 2003
  • 31. Graphical comparison Toothpaste A 30 out of 5000 Toothpaste B 15 out of 5000 E.R. Stone et al. 2003
  • 32. So what?  How does the frame itself influence the meaning that people take away?
  • 33. ORDERING OF QUESTIONS
  • 34. Survey design  Please rate your overall satisfaction.  How many times have you had repairs?
  • 35. German students  How happy are you these days?  How many dates did you have last month? Strack et al, 1988
  • 36. Reverse order  How happy are you these days?  How many dates did you have last month?  How many dates did you have last month?  How happy are you these days? Strack et al, 1988
  • 37. ‘The sequential arrangement of information often creates the very meaning of that information. The order of presentation determines the reaction.’ - Dr. Frank Luntz (Words That Work)
  • 38. So what?  In what ways are you creating (unintended) meaning?  What are you measuring?
  • 39. THE POWER OF WORDS
  • 40. Different meanings  Gambling v. gaming  Liberal v. progressive  Liquor v. spirits  Used v. pre-owned  Etc.
  • 41. An auto accident  At what speed did car A contact car B?  At what speed did car A smash into car B? Loftus/Palmer 1974 31.8 mph 40.5 mph Actual: 12 mph
  • 42. Power of words  Make contact  Smash into  Crash into  Bump into  Run into  Collide with
  • 43. So what?  How are your words impacting  user perception – and consequently, research findings?  how your business partners perceive the findings?
  • 44. Delivering findings  Users were…  Aggravated…  Annoyed…  Confused…  Disappointed…  Frustrated…  Irritated…  Surprised…
  • 45. Neutral  Users…  Said…  Expected…  Thought…  Anticipated…
  • 46. IN SUMMARY
  • 47. Research design ‘The function of a research design is to ensure that the evidence obtained enables us to answer the initial question as unambiguously as possible.’
  • 48. The design matters There is no neutral design
  • 49. ‘All research methods involve compromises with reality.’ - Gerald Zaltman (How Customers Think)
  • 50. The solution  Be aware  Control what we can  How we ask questions  Ordering of questions  Choice of words  Assumptions and their effect  Use of ‘scale’ – charts, graphs, rating scales  Comparison – graphs, ratings
  • 51.  UXmatters.com  UX magazine (uxmag.com) Colleen Roller