Fast, Cheap, and   Actionable Creating an Affordable User     Research Program
Michael PowersDirector of Web Services
On the Web• museyroom.com• twitter.com/mjpowers• www.slideshare.net/michaelpowers• www.linkedin.com/in/mjpowers
What kinds of user researchhave you done?
It all started with a    usability test
Today• Getting Permission for Research• A Process for User Research• Inventory of Actionable User Research You  Can Do Fas...
User Research Study          ↓User Research Program
0      Getting    Permission
Don’t Ask
“We don’t want“a research“study.”
“A research study?“Look, just make a red box on the home page, and put this text in there.“I just need to make sure studen...
Nobody wants aresearch study.
Nobody wants a website.
What We Really Want• Applicants• Successful Students• Academic Research• Donations
Create a ResearchCulture• Do a usability test and invite  stakeholders to watch• Make research part of every project• Appl...
Research Process1. Frame a Good        5. Run Study and   Question               Analyze Results2. Know What You       6. ...
1         Frame a    Good Question
Good questions are related      to your goals
“Students can’t find stuff on our website.”
Focusing a Question How can we   • Can students find decrease       information about attrition?     support services?    ...
“The admissions website needs work.”
Focusing a Question How can we get   1. Can prospectives find                    our list of majors? more/more          Ou...
Goal-related questions lead to faster, cheaper, more actionable user research
2   Know What You      Already Know
What Do You Already Know?• Best Practices    • Server Logs• Published Studies • Analytics• Data about Your   • Past Studie...
Get OrganizedQuantitative Data   Qualitative DataKPI Spreadsheet     User Personas
KeyKPI = Performance      Indicator
KPIs measure progress  toward your goals.
KPI Spreadsheet•   Focus on the KPIs youve   •   Include anything    chosen                         numeric: survey result...
KPI SpreadsheetAvailable on museyroom.com
User Personas
Include Scenarios                         Site Features Motivation Scenario     Used          Behaviors Find the Checks to...
User Personas•   Fiction—but a useful        •   Helps bring new team    fiction                         members up to spe...
Use these two tools to track       your progress         over time.
3   Choose Study            Type
If you’ve never done one, do     a usability test first.
FocusGroups
Some Types of User Research• Usability Test   • Navigation Test• Surveys          • Layout Test• Pop-up Survey    • A/B Te...
Some Types of User Research• Usability Test   • Card Sort• Surveys          • Navigation Test• Pop-up Survey    • Layout T...
Usability Test          Photo source: http://www.flickr.com/photos/l-i-n-k/3654390818/
Analytics
Survey
Pop-Up Survey
Pop-up Survey      What You Do: Ask site visitors questions about their                  experience on a particular page o...
Pop-up Survey  Tools Available: •4Q Survey                •iPerceptions                •CrowdScience                •Roll ...
4Q Survey Questions1. Based on todays visit, how would you rate your site   experience overall?2. Which of the following b...
“I am always able to   “I can never easilyfind what I am          find what Imlooking for. Thewebsite is really      looking...
Coding Comments• Visual Design        • Navigation Positive  Positive                       • Navigation Negative• Visual ...
637,809:-%&-*(##%+,;+<                   NF,)/+O)-./"               (:L8")/2"M*:/58"                  J+8I)4"K:8+,/""     ...
6757"3#45&"89":&*;&"/#00*12"                               <#&2"%#&()*=>+,*-.()*=>"?1@"                   SK1.40T.234#    ...
Action• Changed our yearly goals to spend more time  on these issues
Card Sort      Photo Source: http://www.flickr.com/photos/rosenfeldmedia/3343501403/
Card Sort      What You Do: Give participants cards with website                   topics on them, then ask them to put   ...
Card Sort   Tools Available: •Index cards                 •OptimalSort (remote)                 •WebSort (remote)         ...
Open                        Closed•   Users stack cards       • Give users header    then                      cards•   Gi...
Where do we put  counseling?
Action• Complete restructuring of the Student Life  section of our website• Move counseling websites out of “Health” and  ...
One Subsite,Before and After               Change between      Compared to              Comparison Periods    Whole SitePa...
KJ Session        Photo Source: http://www.flickr.com/photos/drewm/370231088/
Navigation Test
Navigation Test         What You Do: Give participants a proposed site                      structure/navigation and let u...
Navigation Test   Tools Available: •Treejack           Cost: $150–$700
Action• Rearranged Admissions Site navigation to • Combine repeated material • Put “Costs and Tuition” higher in the site ...
Layout Test
Layout Test       What You Do: Participants respond to a static image                    of your site          Good For: B...
Layout Test   Tools Available: •Paper                    •PowerPoint                    •Five-Second Usability Test       ...
Layout TestPositive              Negative•Get reactions to real •No interactivity designs without       •Users can be buil...
A/B Test
4           Find    Participants
The Right Number ofParticipants• Different methods have different requirements• Plan to recruit more than you need• But do...
The Right Kind of Participants• Steve Krug: “Recruit loosely and grade on a  curve”• Good advice, but when possible try to...
Finding Participants• People you know        • On your website (link                           on homepage or pop-• Studen...
Photo source: http://en.wikipedia.org/wiki/File:Ethnio_recruit_ad.png
Offer a prize
5    Run Study and    Analyze Results
Keep track of results in KPIspreadsheet and personas
6   Take Action
Share results and proposedsolutions with stakeholders
7   Measure and         Repeat
Share results
Challenge:Do some sort of userresearch each month
Research Process1. Frame a Good        5. Run Study and   Question               Analyze Results2. Know What You       6. ...
More Information museyroom.com twitter.com/mjpowers www.slideshare.net/michaelpowers
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Fast, Cheap, and Actionable: Creating an Affordable User Research Program
Upcoming SlideShare
Loading in...5
×

Fast, Cheap, and Actionable: Creating an Affordable User Research Program

1,223

Published on

Done a usability study? Ready for the next step? Today we have an abundance of fast, affordable website user research methods, many of which can be done remotely with real users. Learn about available user research options and how IUP runs successful research projects that lead to actionable insights.

Published in: Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,223
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Fast, Cheap, and Actionable: Creating an Affordable User Research Program

  1. 1. Fast, Cheap, and Actionable Creating an Affordable User Research Program
  2. 2. Michael PowersDirector of Web Services
  3. 3. On the Web• museyroom.com• twitter.com/mjpowers• www.slideshare.net/michaelpowers• www.linkedin.com/in/mjpowers
  4. 4. What kinds of user researchhave you done?
  5. 5. It all started with a usability test
  6. 6. Today• Getting Permission for Research• A Process for User Research• Inventory of Actionable User Research You Can Do Fast and Cheap
  7. 7. User Research Study ↓User Research Program
  8. 8. 0 Getting Permission
  9. 9. Don’t Ask
  10. 10. “We don’t want“a research“study.”
  11. 11. “A research study?“Look, just make a red box on the home page, and put this text in there.“I just need to make sure students don’t miss it.”
  12. 12. Nobody wants aresearch study.
  13. 13. Nobody wants a website.
  14. 14. What We Really Want• Applicants• Successful Students• Academic Research• Donations
  15. 15. Create a ResearchCulture• Do a usability test and invite stakeholders to watch• Make research part of every project• Apply previous research to new projects• Show results early and often
  16. 16. Research Process1. Frame a Good 5. Run Study and Question Analyze Results2. Know What You 6. Take Action Already Know 7. Measure and Repeat3. Choose Study Type4. Find Participants
  17. 17. 1 Frame a Good Question
  18. 18. Good questions are related to your goals
  19. 19. “Students can’t find stuff on our website.”
  20. 20. Focusing a Question How can we • Can students find decrease information about attrition? support services? • Can students find out what’s happening on campus?
  21. 21. “The admissions website needs work.”
  22. 22. Focusing a Question How can we get 1. Can prospectives find our list of majors? more/more Our tuition qualified information? applicants? 2. Can prospectives register for a visit? 3. Can prospectives successfully apply on line?
  23. 23. Goal-related questions lead to faster, cheaper, more actionable user research
  24. 24. 2 Know What You Already Know
  25. 25. What Do You Already Know?• Best Practices • Server Logs• Published Studies • Analytics• Data about Your • Past Studies Institution and Students
  26. 26. Get OrganizedQuantitative Data Qualitative DataKPI Spreadsheet User Personas
  27. 27. KeyKPI = Performance Indicator
  28. 28. KPIs measure progress toward your goals.
  29. 29. KPI Spreadsheet• Focus on the KPIs youve • Include anything chosen numeric: survey results, 4Q survey, analytics,• Give specific reports to server logs, etc. stakeholders who request them • Create specific analyses for specific projects• Track over time without running multiple tools
  30. 30. KPI SpreadsheetAvailable on museyroom.com
  31. 31. User Personas
  32. 32. Include Scenarios Site Features Motivation Scenario Used Behaviors Find the Checks to see •Site Search •Looks for school if her major “Pre- with the is available •Undergrad Law” right pre- Admissions major, Law Website does not program find it. for me •Academics Link
  33. 33. User Personas• Fiction—but a useful • Helps bring new team fiction members up to speed• Puts what youve learned into a format you can share with your team and clients• If it is wrong, team or clients will let you know
  34. 34. Use these two tools to track your progress over time.
  35. 35. 3 Choose Study Type
  36. 36. If you’ve never done one, do a usability test first.
  37. 37. FocusGroups
  38. 38. Some Types of User Research• Usability Test • Navigation Test• Surveys • Layout Test• Pop-up Survey • A/B Test• KJ Session• Card Sort
  39. 39. Some Types of User Research• Usability Test • Card Sort• Surveys • Navigation Test• Pop-up Survey • Layout Test• KJ Session • A/B Test
  40. 40. Usability Test Photo source: http://www.flickr.com/photos/l-i-n-k/3654390818/
  41. 41. Analytics
  42. 42. Survey
  43. 43. Pop-Up Survey
  44. 44. Pop-up Survey What You Do: Ask site visitors questions about their experience on a particular page or the whole website. Good For: Capturing user intent; settling arguments about what belongs where When To Use: When you have a question to answer, or all the timeParticipants Needed: Calculate your sample size: http://www.surveysystem.com/ sscalc.htm
  45. 45. Pop-up Survey Tools Available: •4Q Survey •iPerceptions •CrowdScience •Roll your own Cost: •$0–$10,000+
  46. 46. 4Q Survey Questions1. Based on todays visit, how would you rate your site experience overall?2. Which of the following best describes the primary purpose of your visit?3. Were you able to complete the purpose of your visit today?4. Comment • If yes: What do you value most about the website? • If no: Please tell us why you were not able to fully complete the purpose of your visit today
  47. 47. “I am always able to “I can never easilyfind what I am find what Imlooking for. Thewebsite is really looking for, if I caneasy to use.” find it at all.”
  48. 48. Coding Comments• Visual Design • Navigation Positive Positive • Navigation Negative• Visual Design Negative • Content Quality Positive• Usability Positive • Content Quality• Usability Negative Negative
  49. 49. 637,809:-%&-*(##%+,;+< NF,)/+O)-./" (:L8")/2"M*:/58" J+8I)4"K:8+,/"" @8)34:" 1+/2"A:.=4:"2(405- 9./5:/5"HI)4+56" (:,)-*:" <+5:"<:)F>G" A.8+-*:" ;@A"B:38+5:C"/./D9E<" <=:>+?>"=),:08+5:" 7)8"9./5:/5";"(::2" ()*+,)-./01+/2)3+4+56" !" #!" $!" %!" &!" !!" #!" $!" %!" &!" !"#$%&()*(##%+,-.%+/(+0+1230-2(405
  50. 50. 6757"3#45&"89":&*;&"/#00*12" <#&2"%#&()*=>+,*-.()*=>"?1@" SK1.40T.234# -?Q=#.47#R/?4:=# O0=N.9#P?=014## E=.89?# 6047#F?3B9?#3#45&" >34:?4:#MN.90:;# A0:?#A?.KCL# -?1.2/?# @EF#G?8=0:?H#434I>JA# F3=02/?# AB?C0DC#B.1?5=0:?# <.=#>34:?4:#@#-??7# -./01.23456047.8090:;# !"# $!"# %!"# &!"# !"# (!"# )!"# *!"# +!"# ,!"# $!!"# !"#$"%#&()*+,*-.()*"/#00*12&"
  51. 51. Action• Changed our yearly goals to spend more time on these issues
  52. 52. Card Sort Photo Source: http://www.flickr.com/photos/rosenfeldmedia/3343501403/
  53. 53. Card Sort What You Do: Give participants cards with website topics on them, then ask them to put them into piles that make sense to them Good For: Understanding the mental models of your users When To Use: When creating or improving your information architecture/navigation Participants Needed: 15 face to face 50 online
  54. 54. Card Sort Tools Available: •Index cards •OptimalSort (remote) •WebSort (remote) •xSort Cost: $40-$600
  55. 55. Open Closed• Users stack cards • Give users header then cards• Give each pile a name • They assign cards to the header cards
  56. 56. Where do we put counseling?
  57. 57. Action• Complete restructuring of the Student Life section of our website• Move counseling websites out of “Health” and into “Support”
  58. 58. One Subsite,Before and After Change between Compared to Comparison Periods Whole SitePageviews 76% 140%Unique 81% 142%PageviewsBounce Rate -21% 27%
  59. 59. KJ Session Photo Source: http://www.flickr.com/photos/drewm/370231088/
  60. 60. Navigation Test
  61. 61. Navigation Test What You Do: Give participants a proposed site structure/navigation and let users drill down to find different things. Good For: Verifying an IA in a more realistic way (not everything is visible); finding problem areas When To Use: To verify a new information architecture Participants Needed: 50
  62. 62. Navigation Test Tools Available: •Treejack Cost: $150–$700
  63. 63. Action• Rearranged Admissions Site navigation to • Combine repeated material • Put “Costs and Tuition” higher in the site tree
  64. 64. Layout Test
  65. 65. Layout Test What You Do: Participants respond to a static image of your site Good For: Behavior, Remote, Automated, Quant. When To Use: Before building out a new design Participants Needed: About 50 per test
  66. 66. Layout Test Tools Available: •Paper •PowerPoint •Five-Second Usability Test •Chalkmark •Usabila Cost: $40–$600
  67. 67. Layout TestPositive Negative•Get reactions to real •No interactivity designs without •Users can be building out a confused when whole site doing this remotely
  68. 68. A/B Test
  69. 69. 4 Find Participants
  70. 70. The Right Number ofParticipants• Different methods have different requirements• Plan to recruit more than you need• But don’t test more than you need
  71. 71. The Right Kind of Participants• Steve Krug: “Recruit loosely and grade on a curve”• Good advice, but when possible try to recruit participants that are more like your users• For instance: faculty and students see your website very differently
  72. 72. Finding Participants• People you know • On your website (link on homepage or pop-• Students, faculty, and up) staff at your school• Local high schools • Recruitment tools like Ethnio (talk to a guidance counselor)
  73. 73. Photo source: http://en.wikipedia.org/wiki/File:Ethnio_recruit_ad.png
  74. 74. Offer a prize
  75. 75. 5 Run Study and Analyze Results
  76. 76. Keep track of results in KPIspreadsheet and personas
  77. 77. 6 Take Action
  78. 78. Share results and proposedsolutions with stakeholders
  79. 79. 7 Measure and Repeat
  80. 80. Share results
  81. 81. Challenge:Do some sort of userresearch each month
  82. 82. Research Process1. Frame a Good 5. Run Study and Question Analyze Results2. Know What You 6. Take Action Already Know 7. Measure and Repeat3. Choose Study Type4. Find Participants
  83. 83. More Information museyroom.com twitter.com/mjpowers www.slideshare.net/michaelpowers
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×