The document outlines tips for agencies to conduct more effective user research. It recommends: (1) making testing a rule by including it in all proposals; (2) setting clear expectations for clients about the research process; and (3) developing a variety of research methods to suit different projects. It also suggests agencies learn to moderate their own studies to reduce costs, include clients to give them visibility, and proactively manage results by focusing on trends rather than isolated findings.

1. #UTwebinar
@UserTesting
Aiden Bordner
Co-founder and Principal Designer
Parade
The agency’s guide to
effective user research

2. The Agency’s Guide to Effective User Research #UTWebinar
Who is this for?
•Large Agencies
•Small Agencies
•Anyone who works with Stakeholders

3. The Agency’s Guide to Effective User Research #UTWebinar
Why do agencies struggle?
• Large Agencies:
• Traditionally, testing was to be avoided
• Hierarchical nature, focus on client relationship
• Perception of high cost
• Fear of “bad results”
Issues largely to do with Culture and Business Model

4. The Agency’s Guide to Effective User Research #UTWebinar
Why do agencies struggle?
• Small Agencies:
• No staff on hand to focus on research
• Project-basis and of fixed-fee scopes
• No experience doing research before
Issues largely to do with Culture and Business Model

5. The Agency’s Guide to Effective User Research #UTWebinar
How do we improve?
1. Make Testing a Rule
2. Set Expectations Early, Often
3. Develop a Broad Toolkit to Right-Size Research
4. Reduce Costs by Learning to Moderate Yourself
5. Include the Client in Testing
6. Proactively Manage Results
Six tips for doing more effective user research and overcoming cultural and business impediments.

6. The Agency’s Guide to Effective User Research #UTWebinar
1. Make Testing a Rule
• Best way to ensure testing is to change culture
• Don’t let yourself take it out of proposals
• Don’t let the client take it out either
• Cite monetary or performance sales examples

7. The Agency’s Guide to Effective User Research #UTWebinar
1. Make Testing a Rule
• How to schedule
• Plan on testing for every ~3-4 weeks of design.
• Testing should take 1-2 weeks depending on
methodology
• You can keep designing while you’re testing
• Include time for revisions

8. The Agency’s Guide to Effective User Research #UTWebinar
2. Set Expectations Early, Often
• Work to diffuse feeling that testing is a “Big Deal”
• Not market research
• Qual vs. quant
• Formalize expectations in a deliverable
• (e.g. Research Plan)

9. The Agency’s Guide to Effective User Research #UTWebinar
Benefit Cosmetics • Brows Digital Experience • Outside Test Research Plan
Page /1 4
BENEFIT BROWS DIGITAL EXPERIENCE
Outside Research Plan
v1.0
December 8, 2015
Research Plan
• 1-3 page document
• Sections:
• Overview
• Methodology
• Stimulus
• Area of Focus
• Areas of Non-Focus
• Known Limitations
• Outcomes/Reporting
• Schedule
• Share at beginning of each
research effort
Solidifies Plan & Sets Expectations

10. The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Learn to develop a broad kit of tools

11. The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Recruiting:
• Services
• Live Recruiting
• Friends & Family
• Building Occupants
• Client Employees
• Intercepts
• Craigslist
• Pro-Tip: Look to Client for Incentives
Develop a Broad Kit of Tools

12. The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Usability Testing:
• DIY vs. Full-Service
• In-Person, Remote
• Moderated, Unmoderated
• Free Exploration, Task Completion, Natural Task
• “Hallway” Testing
Develop a Broad Kit of Tools

13. The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• User Research:
• Great way to focus before designing
• Simple Observation
• Interviews
• Card Sorts, Tree Testing
Develop a Broad Kit of Tools

14. The Agency’s Guide to Effective User Research #UTWebinar
3. Right-Size the Research for the Assignment
• Learn to develop a broad kit of tools
• Discuss options, recommendations with client
• Keep the recruit small
• Downsize vs. eliminating

15. The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Moderating internally saves time and money
• Two key outputs:
• Discussion Guide
• Moderation

16. The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Discussion Guide:
• Script For Moderation
• Sections:
• Orientation/Ease-Setting
• Recording Permission
• “Think Aloud” Conditioning
• Self-Directed Exploration or Natural Task Repeat
• Task Completion
• Open-Ended Questions
• Thank You/Incentive Information
Introduction (2 minutes)
TOPIC
MODERATOR
Introduction So, thanks again for participating. My name is _________, I’m going to beguiding you though this experiment today. This is ________, who’ll just betaking notes.
Before we get started I would like to tell you a little bit about what we aredoing today. We’re collecting feedback on some work in progress designs fora future Benefit website. This website will be about eyebrows—both ingeneral and about specific products for eyebrows, some of which you maybe familiar with and some of which may be new.
We’ll be testing just a [mobile or desktop] version of the website today, so ina few minutes I’m going to [hand you this mobile device or ask you to usethis laptop] which is set up to record your actions and where you tap on thescreen so we can review it later.
Makes sense?
Orientation /
Relaxation
Your job is really easy, you just have to be yourself and act as you naturallywould. Our goal today is to get honest feedback, so feel free to saywhatever’s on your mind—you won’t offend us.
As you interact with the site, you should know this is just a prototype—awork in progress—so some things may not be fully functioning or complete.If something doesn’t work, just tell me what you expected to happeninstead.
[Mobile only: You should also know there’s one point where we may need tochange apps for the test. If that’s necessary, I’ll let you know when, and youcan just hand to phone to me and I’ll make the switch before we continue.We’ll have to switch back when we’re done, but it will only take a second.]
Okay? Great!
Start Recording So I’m just going to start the recording on this device and we’ll be almostready to go.
[Hand device to participant or position participant at laptop]
Okay, and just for the record, could you please look at the screen and sayyour name for us.
…
Okay great!
Brows Digital Experience • Hallway Test Moderator Guide
Page /2 6

17. The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Moderating:
• Stop Being the Designer
• Above All: Avoid Leading Questions
• Answer Questions with More Questions
• Give the Subject Time; Never Interrupt
• When in Doubt, Don’t Talk
• Don’t Get Frustrated
• Be Encouraging and Thankful

18. The Agency’s Guide to Effective User Research #UTWebinar
4. Learn to Moderate Yourself
• Moderating internally saves time and money
• Two key outputs:
• Discussion Guide
• Moderation
• Don’t worry: trust your team

19. The Agency’s Guide to Effective User Research #UTWebinar
5. Include the Client
• Important to give clients real-time visibility
• Use technology to create a virtual lab
• Place someone with the client, if possible
• Schedule discussion breaks
• Provide recordings
• Do a Tech Rehearsal

20. The Agency’s Guide to Effective User Research #UTWebinar
6. Manage Results Proactively
• Proactively filter the “noise”
• Focus only on the trends
• Explain behavior vs. comprehension
• Make design recommendations proactively
• Dispel non-issues proactively, too
• Keep it lean
• Don’t bludgeon your client with research

21. The Agency’s Guide to Effective User Research #UTWebinar
Recap
1. Make Testing a Rule
2. Set Expectations Early, Often
3. Develop a Broad Toolkit to Right-Size Research
4. Reduce Costs by Learning to Moderate Yourself
5. Include the Client in Testing
6. Proactively Manage Results
Six tips for doing more effective user research and overcoming cultural and business impediments.

22. The Agency’s Guide to Effective User Research #UTWebinar
Our Approach
1. Plan to do a study for every 3-4 weeks of design
2. Do upfront user research for any project involving unknown user
types, special interactions
3. Build research into the schedule at time of proposal
4. Three Deliverables:
• Research Plan
• Discussion Guide (if necessary)
• Report
Parade’s process for research scheduling

23. The Agency’s Guide to Effective User Research #UTWebinar
Our Approach
5. Should be able to accomplish a moderated, one-day study in 7
working days:
• Day 1: Review, adjust and approve research plan
• Day 2: Review moderation Guide
• Day 3: Finalize guide
• Day 5: Testing
• Day 7: Present findings
Parade’s process for research scheduling

24. The Agency’s Guide to Effective User Research #UTWebinar
Questions?
• Learn more about Parade and Aiden’s team at
• parade-sf.com
• Questions about UserTesting?
• Email adam@usertesting.com