Getting Buy-In for UX Research

GETTING BUY-IN FOR
UX RESEARCH
CHRISTINE PERFETTI & EZRA GILDESGAME
ACQUIA
THE LOGISTICS
Ask questions as they occur to you
The schedule
You’ll be working in groups of 2
Lean UX NYC Perfetti & Gildesgame, © 2014 1

EXERCISE
CONDUCT A
USABILITY TEST
USABILITY TEST
A usability test
involves putting your
users in front of the
product and observing
what they do

One person is the user
‣ Think aloud as you work
One person is the observer
‣ Observe silently and take notes
After we perform the task, we’ll ask you to switch roles and do
another
EXPLORATION: USABILITY TEST
‣ We’re evaluating the design, not you
‣ The answer is somewhere on the site
‣ If you experience problems, it’s not your fault
‣ You’re helping the observers learn more about the design problems
AS THE USER

Take notes as the user works
‣ Was the user successful?
‣ How did the user go about performing the task?
‣ What obstacles did the user encounter?
‣ What confusion did they experience?
‣ Note things that worked well
When you complete a task, say, “Got it!”
AS THE OBSERVER
‣ How many users found the answer?
‣ What helped users succeed?
‣ What obstacles prevented them from ﬁnding what they wanted?
‣ What do you expect will happen?
DISCUSSION

Planning your research program
‣ The ﬁrst 90 days
‣ How to choose your ﬁrst project
‣ Identifying your allies and champions
Involving stakeholders in planning
‣ Planning and running usability tests
‣ Quick testing techniques for buy-in
WHAT WE’LL COVER
Analyzing the data
‣ Rolling list of observations
‣ KJ technique
‣ Journey maps
Communicating results effectively
‣ Weekly research days
‣ Formal and informal reporting strategies
‣ Effective communication techniques
DISCUSSION
What prevents you from
getting buy-in for your research?

THE FIRST 90 DAYS

Text
THE FIRST 90 DAYS
Meet with all stakeholders
Know your audience
Write a study plan
Conduct your ﬁrst usability test

‣ Meet with all of your product stakeholders to understand their priorities
‣ Treat these meetings just like any other research project
‣ Who are your users?
‣ What are their main goals?
‣ What challenges do they face today?
‣ What do they currently know about their users?
‣ What do they know about the product? What are the areas of risk?
KNOW YOUR AUDIENCE
What are all of the components of your product?

PRODUCT COMPONENTS
Who is on the product team?

‣ Interview anyone who has knowledge of the product or the business
‣ The product team includes:
‣ Engineering
‣ Product Management
‣ UX
‣ Marketing
‣ Sales
‣ Support
‣ Legal
THE PRODUCT TEAM
‣ What is your role with respect to the product?
‣ What did you do before this?
‣ Who is the product for?
‣ How would you describe the client onboarding process?
‣ Is there anything that concerns you about the product?
‣ What should the user research accomplish for the business?
‣ How will you deﬁne success for the research?
‣ How would you like to be involved in the research?
GENERAL QUESTIONS

QUESTIONS: SPECIFIC TO AUDIENCE
Sales
‣ Why do customers buy the product?
‣ Why do you win/lose sales?
‣ Who are the users today?
‣ How should they be different in 5 years?
‣ What do customers complain about most?Who
Support
‣ Who typically contacts you?
‣ Why do they contact?
‣ What are the top problems for customers?
‣ What areas of the product cause problems?
‣ How can we best reduce calls?
The research team is at your service

SELLING THE VALUE: FRAME BENEFITS IN THEIR TERMS
Sales
‣ Increase win rates
‣ Make selling easier
Product
‣ Research can inform debate about product strategy
‣ Come armed with great data & insights
Support
‣ Decrease # of tickets
‣ Decrease ticket response time
Finance and leadership
‣ Increase renewals
‣ Improve win/growth rate
‣ Improve margins
Identify your Executive Champion
‣ Who drives the product and business decisions?
‣ What leader most values and understands user research and design?
Conduct a research project to:
‣ Answer the Executive Champion’s questions
‣ Focus on their risks and concerns
‣ Accomplish a quick win
CHOOSE YOUR FIRST PROJECT

CREATE A
STUDY PLAN
THE STUDY PLAN
Serves as a blueprint for the test
Provides a systematic approach to testing
Facilitates buy-in from stakeholders

‣ Study goals
‣ Research questions and issues to explore
‣ User characteristics
‣ Schedule
‣ Task list
‣ Data to collect
‣ Reporting strategy
THE TOPICS
THE PLANNING MEETING
At the beginning of the project, we hold
a planning meeting with all product
team members and stakeholders to
discuss:
‣ An overview of the testing process
‣ Project scope
‣ Issues (risks and concerns) with the design
‣ Number of users to test
‣ Testing schedule
GOAL: No surprises! Avoid the game of telephone

MEETING ATTENDEES
‣ Product and Design
‣ Marketing and Engineering
‣ Anyone involved in design decisions
PROJECT SCOPE
What are you testing?
‣ Complete working prototype
‣ Prototype with only some screens
‣ Paper mockup
‣ Just the navigation

‣ What do you want to learn?
‣ Most studies aren’t purely exploratory
‣ The issues and research questions will drive the tasks and recruitment
‣ Work with the team to outline the research questions
‣ Vague statement: Will this work for users?
‣ Ask increasingly speciﬁc questions
RISKS AND CONCERNS
‣ What are you hoping to learn from the study?
‣ What areas of the design are you most concerned about?
‣ Why are you concerned?
‣ Are you worried about a speciﬁc group of users?
‣ What are the greatest risks for the organization?
QUESTIONS TO ASK

‣ What do you want to learn?
‣ What areas are you most concerned about?
‣ Is our mortgage application too long?
‣ Will users know all ﬁelds are required?
‣ Will they use account lookup? (It will be hard to build.)
‣ Do our users understand what “conforming” means?
EXAMPLE: MORTGAGE APPLICATION
TECHNIQUES FOR BUY-IN

THE EXCUSES
Not enough time
No resources and staff
No money No buy-in from management
Unsure where to start
Testing is too scientiﬁc
We already know the product
TECHNIQUES FOR STAKEHOLDER BUY-IN
First Click Test
5 Second Test
Comprehension Test
Ethnographic Interviews

Quick and dirty technique for measuring content pages
Takes less than 10 minutes to run
Measures if content pages quickly convey their purpose
5 SECOND PAGE TESTS
You want to post pictures online from your last vacation
You are concerned that the upload process will be difﬁcult
How conﬁdent are you that you can upload photos quickly and easily?
SHARING PICTURES

SHARING PICTURES

Useful method to assess where users first click on your site’s home or entry page
Provide users with a specific task to complete when they arrive at the site
By observing where users first click, it’s a clear indicator whether they’ll succeed
FIRST CLICK TEST

Pages containing complex content
Such as policies or procedures
How your product works
User comprehension is imperative to their success
Questions determine if users understand content
COMPREHENSION TEST

Excellent technique for understanding users’ goals
ETHNOGRAPHIC INTERVIEWS
Context of use
Pain points
Workarounds
Motivations

With a field study is, we learn about:
‣ How technology ﬁts into users’ life
‣ Tasks that users haven’t talked about
‣ Pain points and user frustration
‣ Evidence of technology that isn’t working for the user
“If I had asked people what they wanted, they would have said faster horses.”
- Henry Ford (maybe)
FIELD STUDIES
VISIT USERS IN THEIR NATURAL SETTING

‣ Typically takes at least two hours at the users’ site
‣ If it’s not possible to visit users onsite, we use a remote meeting tool, such as GoToMeeting or
WebEx
‣ Equipment
‣ Digital recorder or audio recorder
‣ When approaching the interview, take the mindset of an apprentice
‣ The user is the expert
INTERVIEW LOGISTICS
‣ Purpose of interview or site visit
‣ Demographics and background
‣ Overview of work process
‣ What is your typical workday?
‣ Can you give an overview of your work?
‣ Who do you interact with?
‣ What are the most important and
frequent tasks?
RECOMMENDED TOPICS
‣ Current behavior relevant to project
without focusing on technology
‣ Overview of technology and software user
works with
‣ How does technology support your work?
‣ What are you trying to accomplish
by performing the tasks? (Goals)
‣ Follow up on key user feedback
‣ Closing

Avoid leading questions
‣ Users like to respond, “yes”
Avoid:
‣ Do you like this feature?
‣ Are you confused by this content?
BEWARE: LEADING QUESTIONS
‣ What activities waste your time or
drive you crazy?
‣ How did you learn about the
product?
‣ What kind of training did you
receive?
‣ What are you most important
activities?
‣ Who do you provide information to?
What information do you collect?
SAMPLE QUESTIONS
‣ What are your most important and
frequent activities with the product?
‣ What are the two things you like best
about the product?
‣ What are the two things you would like
to see improved?
‣ Are there any activities you currently
perform that you’d like to see
automated?
‣ What shortcuts help you?

‣ Understanding of work environment
‣ Tasks that users haven’t talked about
‣ “Cheat sheets”
‣ Evidence of technology that isn’t working for the user
WHAT TO OBSERVE
‣ Product pain points that were previously unknown
‣ Customers successfully using product in unexpected ways
‣ Customer needs that were previously unknown
‣ Examples that help tell the story (supplementing data)
‣ Identify clients who want to continue to partner
WHAT STAKEHOLDERS LEARN

ANALYZING DATA WITH
STAKEHOLDERS
ANALYZING DATA
The KJ Method
Rolling list of observations
Journey mapping

‣ Gather all observations
‣ Identify problems
‣ Prioritize problems based on importance and frequency
‣ Iterate on the design
ANALYZE DATA
ROLLING LIST: OBSERVATIONS

‣ Named after Kawakita Jiro
‣ Consensus method for grouping and prioritizing usability
ﬁndings
‣ Quickly determines most important observations
‣ Captures everyone’s perspective and observation
THE KJ: METHOD FOR PRIORITIZING DATA
‣ Call together the group
‣ Only invite members of the team
who observed a test session
‣ KJ takes one hour
STEP 1: ORGANIZE GROUP

‣ Every KJ has a focus question that drives the prioritization
exercise:
‣ What are the biggest usability problems we observed in the
test?
‣ What needs to be ﬁxed in the product to improve the user
experience?
STEP 2: DETERMINE FOCUS QUESTION
KJ EXERCISE
What are the biggest problems
that need to be fixed with the airline
experience?

STEP 3: LIST USABILITY PROBLEMS OBSERVED
STEP 4: PUT OBSERVATIONS ON WALL

STEP 5: GROUP NOTES
STEP 6: NAME EACH GROUP

STEP 7: CHOOSE AND RANK GROUPS
STEP 8: VOTING

STEP 9: ORDER GROUPS AND DISCUSS
‣ We order the groups based on
number of votes, highest
numbers at the top
‣ Participants combine groups and
discuss identified priorities
‣ Everyone is involved
‣ Writing down observations lets all team members contribute
‣ Prevents too much influence from select team members
‣ Identifies top priorities
METHOD ADVANTAGES

‣ Up-front study planning
‣ Focus question: What are your research questions?
‣ Brainstorming sessions with stakeholders?
‣ Focus question: What are the biggest user problems we should address?
‣ Usability test debriefs
‣ Focus question: What are the biggest problems we observed in the tests?
‣ Immediately following ﬁeld studies
‣ Focus question: What are the biggest pain points you observed? What solutions could help?
WHEN TO USE THE KJ METHOD
JOURNEY MAPPING

‣ Excellent technique for identifying pain points in the user journey
‣ Clearly communicates the opportunities for improvements and innovation
‣ Prioritizes the areas of the product that need the most attention
‣ Go from abstract to actionable
“World-class product” “Delight users” “Leading user experience”
JOURNEY MAPPING

`

Carbonite.com Journey Map
Mapping out the current experience

Task proﬁle: Well performing areas
Task proﬁle: Areas to improve

Q1 Update: 11 Critical Tasks Tested
Q1 2013 compared to Q4 2012

CREATIVE BRIEF
Beneﬁts
‣ Helps people to fully articulate their ideas
‣ Shared understanding of goals and risks (and even disagreement)
‣ Help establish a shared understanding before starting a project
Components:
‣ Product statement
‣ Business problem
‣ Success criteria
‣ Risks
‣ Personas
‣ Deadlines and milestones
‣ Inspiration
COMMUNICATING
RESEARCH

COMMUNICATING RESULTS
Formal methods
Informal methods
Involve stakeholders
‣ Communicate the results quickly
‣ Debrief often
‣ Report to stakeholders and project team when they need it
‣ Don’t report too much information
‣ Determine best delivery method considering the needs of your different
audiences
‣ No surprises!
SUCCESS FACTORS

Informal
‣ Emails or internet dashboard with short memos of ﬁndings and
recommendations
‣ Working meetings with project teams
‣ Integration with existing problem/bug reporting strategy
Formal
‣ Presentations and workshops
‣ Usability reports
METHODS: COMMUNICATING RESULTS
‣ Effective communication tool for sharing test results immediately
‣ After each day of tests, send a short email to design team with major
observations from the day
EMAIL / INTERNET DASHBOARD

INVOLVE STAKEHOLDERS
`
`
`

‣ When quick changes are
required after a round of tests,
schedule a one-hour meeting
to share and prioritize the test
results
‣ Review the usability test
observations
‣ Determine next steps and
short-term ﬁxes
WORKING MEETING
‣ Most organizations already
have a method for
collecting problem or bug
reports
‣ Merge usability problems
with other bugs
‣ Prevents having two
tracking systems
‣ Helps determine relative
importance of a problem
INTEGRATE WITH BUG TRACKING

‣ 1-2 hour presentation with design team, engineers, marketing, and key
stakeholders
‣ Share most important usability findings based on prioritization of
problems
‣ Effective communication technique
‣ People often ignore written communications
‣ Active involvement with design team and stakeholders
PRESENTATIONS/ WORKSHOPS
EXECUTING ON UX IMPROVEMENTS: MORE THAN BUY-IN
‣ Nobody says, “Let’s have a poor UX”
‣ “Experience Rot” as a risk
‣ It’s about the ideas, not the people
‣ Encourage open discussion of data
‣ Avoid sacrificing long-term quality for short-term goals
‣ Document commitment to UX and get buy-in for when things deviate
‣ Eat vegetables, floss daily, follow through on UX goals

INVOLVE
STAKEHOLDERS
URTH AND SURF
WEEKLY RESEARCH

QUESTIONS?

Getting Buy-In for UX Research

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Getting Buy-In for UX Research

Similar to Getting Buy-In for UX Research (20)

More from Perfetti Media

More from Perfetti Media (6)

Recently uploaded

Recently uploaded (20)

Getting Buy-In for UX Research