Fast, Cheap, and Actionable: Creating an Affordable User Research Program

Fast, Cheap, and Actionable Creating an Aﬀordable User Research Program

Michael PowersDirector of Web Services

On the Web• museyroom.com• twitter.com/mjpowers• www.slideshare.net/michaelpowers• www.linkedin.com/in/mjpowers

What kinds of user researchhave you done?

It all started with a usability test

Today• Getting Permission for Research• A Process for User Research• Inventory of Actionable User Research You Can Do Fast and Cheap

User Research Study ↓User Research Program

0 Getting Permission

Don’t Ask

“We don’t want“a research“study.”

“A research study?“Look, just make a red box on the home page, and put this text in there.“I just need to make sure students don’t miss it.”

Nobody wants aresearch study.

Nobody wants a website.

What We Really Want• Applicants• Successful Students• Academic Research• Donations

Create a ResearchCulture• Do a usability test and invite stakeholders to watch• Make research part of every project• Apply previous research to new projects• Show results early and often

Research Process1. Frame a Good 5. Run Study and Question Analyze Results2. Know What You 6. Take Action Already Know 7. Measure and Repeat3. Choose Study Type4. Find Participants

1 Frame a Good Question

Good questions are related to your goals

“Students can’t find stuff on our website.”

Focusing a Question How can we • Can students find decrease information about attrition? support services? • Can students find out what’s happening on campus?

“The admissions website needs work.”

Focusing a Question How can we get 1. Can prospectives find our list of majors? more/more Our tuition qualified information? applicants? 2. Can prospectives register for a visit? 3. Can prospectives successfully apply on line?

Goal-related questions lead to faster, cheaper, more actionable user research

2 Know What You Already Know

What Do You Already Know?• Best Practices • Server Logs• Published Studies • Analytics• Data about Your • Past Studies Institution and Students

Get OrganizedQuantitative Data Qualitative DataKPI Spreadsheet User Personas

KeyKPI = Performance Indicator

KPIs measure progress toward your goals.

KPI Spreadsheet• Focus on the KPIs youve • Include anything chosen numeric: survey results, 4Q survey, analytics,• Give specific reports to server logs, etc. stakeholders who request them • Create specific analyses for specific projects• Track over time without running multiple tools

KPI SpreadsheetAvailable on museyroom.com

User Personas

Include Scenarios Site Features Motivation Scenario Used Behaviors Find the Checks to see •Site Search •Looks for school if her major “Pre- with the is available •Undergrad Law” right pre- Admissions major, Law Website does not program find it. for me •Academics Link

User Personas• Fiction—but a useful • Helps bring new team fiction members up to speed• Puts what youve learned into a format you can share with your team and clients• If it is wrong, team or clients will let you know

Use these two tools to track your progress over time.

3 Choose Study Type

If you’ve never done one, do a usability test ﬁrst.

FocusGroups

Some Types of User Research• Usability Test • Navigation Test• Surveys • Layout Test• Pop-up Survey • A/B Test• KJ Session• Card Sort

Some Types of User Research• Usability Test • Card Sort• Surveys • Navigation Test• Pop-up Survey • Layout Test• KJ Session • A/B Test

Usability Test Photo source: http://www.flickr.com/photos/l-i-n-k/3654390818/

Analytics

Survey

Pop-Up Survey

Pop-up Survey What You Do: Ask site visitors questions about their experience on a particular page or the whole website. Good For: Capturing user intent; settling arguments about what belongs where When To Use: When you have a question to answer, or all the timeParticipants Needed: Calculate your sample size: http://www.surveysystem.com/ sscalc.htm

Pop-up Survey Tools Available: •4Q Survey •iPerceptions •CrowdScience •Roll your own Cost: •$0–$10,000+

4Q Survey Questions1. Based on todays visit, how would you rate your site experience overall?2. Which of the following best describes the primary purpose of your visit?3. Were you able to complete the purpose of your visit today?4. Comment • If yes: What do you value most about the website? • If no: Please tell us why you were not able to fully complete the purpose of your visit today

“I am always able to “I can never easilyfind what I am find what Imlooking for. Thewebsite is really looking for, if I caneasy to use.” find it at all.”

Coding Comments• Visual Design • Navigation Positive Positive • Navigation Negative• Visual Design Negative • Content Quality Positive• Usability Positive • Content Quality• Usability Negative Negative

637,809:-%&-*(##%+,;+< NF,)/+O)-./" (:L8")/2"M*:/58" J+8I)4"K:8+,/"" @8)34:" 1+/2"A:.=4:"2(405- 9./5:/5"HI)4+56" (:,)-*:" <+5:"<:)F>G" A.8+-*:" ;@A"B:38+5:C"/./D9E<" <=:>+?>"=),:08+5:" 7)8"9./5:/5";"(::2" ()*+,)-./01+/2)3+4+56" !" #!" $!" %!" &!" !!" #!" $!" %!" &!" !"#$%&()*(##%+,-.%+/(+0+1230-2(405

6757"3#45&"89":&*;&"/#00*12" <#&2"%#&()*=>+,*-.()*=>"?1@" SK1.40T.234# -?Q=#.47#R/?4:=# O0=N.9#P?=014## E=.89?# 6047#F?3B9?#3#45&" >34:?4:#MN.90:;# A0:?#A?.KCL# -?1.2/?# @EF#G?8=0:?H#434I>JA# F3=02/?# AB?C0DC#B.1?5=0:?# <.=#>34:?4:#@#-??7# -./01.23456047.8090:;# !"# $!"# %!"# &!"# !"# (!"# )!"# *!"# +!"# ,!"# $!!"# !"#$"%#&()*+,*-.()*"/#00*12&"

Action• Changed our yearly goals to spend more time on these issues

Card Sort Photo Source: http://www.flickr.com/photos/rosenfeldmedia/3343501403/

Card Sort What You Do: Give participants cards with website topics on them, then ask them to put them into piles that make sense to them Good For: Understanding the mental models of your users When To Use: When creating or improving your information architecture/navigation Participants Needed: 15 face to face 50 online

Card Sort Tools Available: •Index cards •OptimalSort (remote) •WebSort (remote) •xSort Cost: $40-$600

Open Closed• Users stack cards • Give users header then cards• Give each pile a name • They assign cards to the header cards

Where do we put counseling?

Action• Complete restructuring of the Student Life section of our website• Move counseling websites out of “Health” and into “Support”

One Subsite,Before and After Change between Compared to Comparison Periods Whole SitePageviews 76% 140%Unique 81% 142%PageviewsBounce Rate -21% 27%

KJ Session Photo Source: http://www.flickr.com/photos/drewm/370231088/

Navigation Test

Navigation Test What You Do: Give participants a proposed site structure/navigation and let users drill down to find different things. Good For: Verifying an IA in a more realistic way (not everything is visible); finding problem areas When To Use: To verify a new information architecture Participants Needed: 50

Navigation Test Tools Available: •Treejack Cost: $150–$700

Action• Rearranged Admissions Site navigation to • Combine repeated material • Put “Costs and Tuition” higher in the site tree

Layout Test

Layout Test What You Do: Participants respond to a static image of your site Good For: Behavior, Remote, Automated, Quant. When To Use: Before building out a new design Participants Needed: About 50 per test

Layout Test Tools Available: •Paper •PowerPoint •Five-Second Usability Test •Chalkmark •Usabila Cost: $40–$600

Layout TestPositive Negative•Get reactions to real •No interactivity designs without •Users can be building out a confused when whole site doing this remotely

A/B Test

4 Find Participants

The Right Number ofParticipants• Different methods have different requirements• Plan to recruit more than you need• But don’t test more than you need

The Right Kind of Participants• Steve Krug: “Recruit loosely and grade on a curve”• Good advice, but when possible try to recruit participants that are more like your users• For instance: faculty and students see your website very differently

Finding Participants• People you know • On your website (link on homepage or pop-• Students, faculty, and up) staff at your school• Local high schools • Recruitment tools like Ethnio (talk to a guidance counselor)

Photo source: http://en.wikipedia.org/wiki/File:Ethnio_recruit_ad.png

Oﬀer a prize

5 Run Study and Analyze Results

Keep track of results in KPIspreadsheet and personas

6 Take Action

Share results and proposedsolutions with stakeholders

7 Measure and Repeat

Share results

Challenge:Do some sort of userresearch each month

Research Process1. Frame a Good 5. Run Study and Question Analyze Results2. Know What You 6. Take Action Already Know 7. Measure and Repeat3. Choose Study Type4. Find Participants

More Information museyroom.com twitter.com/mjpowers www.slideshare.net/michaelpowers

Fast, Cheap, and Actionable: Creating an Affordable User Research Program

by Michael Powers

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Fast, Cheap, and Actionable: Creating an Affordable User Research Program Presentation Transcript