Remote Usability + Tools
Steve Schang
Midwood Usability

“I suppose it is tempting, if the only
tool you have is a hammer, to treat
everything as if it were a nail.”

Why do usability?
Watch real users and hear what they have to say
Quickly learn what works and what doesn’t
Innovate by discovering unmet needs
Validate strategy and requirements
Reduce risk of project failure and boost ROI
But user research is more than just usability….

Poll

User Research Across Product Design
• Develop empathy for users
• Understand user context
• Identify and articulate unmet needs
Strategic – Assess the real world environment of users
• Quickly gather user feedback
• User data accelerates and focuses idea generation
• Early validation of design direction
Innovation – Ensure user perspective is include in idea generation
• Simulate real world tasks
• Observe what works and does not work
• Use data to fix UI problems before development
• User data informs marketing and operational support groups
Tactical – Make sure the designs work with users Traditional focus of
usability research.
This underutilizes the
broad skillset of
experienced researchers.

Sample Research Methods
Strategic
Ethnography
Field research
Competitive Usability
Innovation
Participatory Design
Innovation Games
Rapid Prototyping
RITE
Tactical
Usability testing
Remote Usability
Eye Tracking
Card Sorting
These can all be done remote.

https://www.nngroup.com/articles/which-ux-research-methods/
Research Methods Landscape
What people DO
What people SAY
How many &
How much
Why &
How to Fix
Tools in this
talk enable
these methods.

Poll

Moderated vs Unmoderated
Moderated
• Facilitator skills are critical
• Can deep dive on issues
• Can redirect at dead ends
• Can clarify scenarios, tasks
• Longer sessions
• More methods available
• Richer insights
• With new online recruiting and testing
tools becoming easier to setup and run
Unmoderated
• Easy and accessible
• Great for short sessions, well defined
issues
• Ability to do quant
• Built in panels
• Quality of participants
• Everybody is a researcher
• Limited by prototype
• Can’t redirect
• Can’t explore an issue
Story time - a tale of unmoderated vs. moderated

Story time - a tale of unmoderated vs. moderated
Let’s build a loan
application.
MVP. Launch
fast and iterate.
Usability is
important. We need
to test.
All we have is our
unmoderated remote tool.
Don’t worry I got this. It’s all I
need.
Usability looks good.
Everyone likes it and
can complete the app.
We’re getting ready to
launch. Let’s do a
moderated study to
validate.

50,000-foot view of the usability process
• Goal of test, Scope
• Methodology
• Goals/hypothesis
• Tasks
• Participants
• Team
• Schedule
• Deliver and document
findings
Isn’t all that overkill?
I just want to know if
that button works.
Can’t my designer just set
up and run usability test?

Poll

Who should do the testing?
Bring someone in…
• Expertise and experience.
• Good for small teams that don’t
have dedicated researchers
• Outside eyes are objective
• Scale up research as needed
Use your own team…
• Knows the product
• This is a double edge sword
• Knows the team
• Available
• Better to test than not to test
(but be aware of biases,
limitations)
Bias in test design and facilitation is a big problem.
Garbage in = Garbage out

Questions? Comments?

Tools in The Process (remote moderated)
Planning
• Google doc
• Word
• Dovetail
• Miro
Recruiting &
Scheduling
• Respondent
• UserInterviews
• Calendly
• UserZoom Go
• UserTesting
Conduct sessions
• UserZoom
• UserZoom Go
• UserTesting
• Lookback
• Zoom, Webex,
Teams
Analyze &
Reporting
• Dovetail
• UserZoom Go
• Miro
• Word
• PowerPoint
• Google Sheets
• SharePoint
Discussion
Guide
Facilitation
How to
write tasks
PrototypesTalks for another day….
Tools

Recruiting & Scheduling

What to look for in a Recruiting service…
• Panel attributes (i.e. age, gender, location, technology)
• Custom screeners
• Double screen or review profile before scheduling
• Automatic Scheduling
• Outlook and Google calendar integration
• Payment options
• Multiple, Amazon, Paypal, Venmo
• NDAs
• Informed consent
The goal is to find
actual or
representative
users.
Real people not
professional test
takers.

Online Panels
Respondent.io
+ Good for recruiting
professionals
+ LinkedIn profile
+ Hand pick participants from
pool of eligible
+ PayPal incentive
- BYOP create account and pay
5% fee.
UserInterviews.com
+ Easy scheduling
+ Demographic filters
+ Amazon gift card or use your
own payment
+ Bring your own participants
do not need to create an
account
Integrated Panels
+ Convenience
+ Screeners
+ General consumers
- Session length
- Professional testers
- Participant incentive
Don’t forget local market research firms.
Bootstrapping? Use your own customer list with Calendly and Zoom.

Remote Testing Platforms

Remote testing platform
UserZoom
+ Quant and Qual platform
- Expensive
- Started off as unmoderated
tool
- Scheduling
UserZoom Go
+ Purpose built for moderated
+ Doesn’t require plugin
+ One link participation
+ Can chat with participant
+ Integrated panel
+ Tagging
Usertesting.com
+ Tasks
+ Internal panel
+ General consumers
- Purpose built for
unmoderated remote
- Expensive

Remote testing platform (cont.)
Lookback
+ Easy to setup
+ Single link
+ Inexpensive
- Can’t chat with participants
Online meeting platforms
+ Everyone has them
+ Everyone uses them
+ Inexpensive
+ Can chat
+ Can remote screenshare
- Participants can see
everyone
- Software install

What to look for
• Simple setup for participants (software, plugins, browser)
• Participant screenshare, video and audio
• Screenshare on mobile device (using app)
• Text with participant
• Hide observers from participant
• Text with observers
• Note taking, tagging
• Tasks
• Single URLs for testers and observers
• Transcription

Questions? Comments?

Analysis and Reporting

Miro
• Everything in one place
• Collaborative
• Visual – maintain context
• Written notes

Dovetail
• Analysis tool
• Research repository
• Tagging feature for analysis
• Use data from any platform
• Videos
• Transcripts
• Feedback forms
• Surveys
There are other qual analysis tools.
Don’t forget paper, Word docs, and spreadsheets

Questions? Comments?

Midwood Usability
• User research and UX design
• Est. 2011
• Based in Charlotte, NC
• We work anywhere.
• Expertise
• Financial services
• Qualitative user research
• Group facilitation
• IA & UX design
• Web accessibility
We customize our approach based on how
your team works.

About Us
Steve Schang
20+ years, experience in
usability, IA, and UX
design.
Expertise in remote
usability testing and
rapid prototyping.
Christina Lall
13 years experience in
UX design, IA, and
usability.
Advanced training in
WCAG accessibility
standards. CPACC.
Tayler Green
3 years experience in
usability, UX design, IA,
and content strategy.
Recently MA from
Clemson University, with
a focus on usability and
design.

Thank You
steve.schang@midwoodusability.com