This document discusses options for conducting moderated remote user research on mobile devices in real-time. It presents six options: 1) using an IT support tool like Bomgar; 2) using a document camera; 3) having participants turn their laptop around; 4) using Lookback's Unicorns.io; 5) using AirServer with AirPlay; and 6) using Reflector. Each option is evaluated based on criteria like whether participants' faces and interactions can be seen, device platforms supported, image quality, and costs. Lessons learned are provided, such as scheduling tech checks in advance and planning for attrition due to technical issues.

1. 1BellaVia Research ©2015
Julie Francis, UX Researcher & Founder / BellaVia Research
Alex Berg, Director of Strategy & Analytics / Fell Swoop (formerly
VP Product / Ritani.com)
Rebecca West, Global Vice President, Research / Civicom®
June 29, 2015 (Abridged)
Moderated Remote Research
on Mobile Devices
(aka, Stay off planes & save client money!)
UXPA 2015

2. 2BellaVia Research ©2015
Who we are
Julie / Founder, BellaVia Research
• 20 years in research / Focus on UX Research since 2000
• Launched BellaVia in 2001, post Creative Good
• Santa Cruz, CA / julie@BellaViaResearch.com, 831.454.8217, @BellaVia
Alex / Director, Strategy & Analytics, Fell Swoop
• Former VP Product Ritani.com
• 20 years in Product Mgmt. Leadership roles with NBA.com, Blue Nile, Expedia.
• Seattle, WA / alex.berg@fellswoop.com, 206.310.9959, @Fell_Swoop
Rebecca / Global Vice President, Research, Civicom®
• 25+ years in research, technology support and product/service development
• Investor in Civicom and Member, Board of Directors
• New York/Greenwich CT / rebecca.west@civi.com 203.413.2423, @civicom

3. 3BellaVia Research ©2015
So many UX research platforms!
Today’s focus:
Moderated
•Not unmoderated, and
Remote
•Not in-person, and
Real-time
•Not after-the-fact recordings,
and
 Visibility to mobile
devices
•Not just computers

4. 4BellaVia Research ©2015
Our viewpoints
Alex
Client
perspective:
What Ritani.com
needed & Why8
20 minutes
Julie
Researcher
perspective:
Options, Pros & Cons,
Tips & Tricks
30 minutes
Rebecca
Technical
perspective
Demo
10 minutes

5. 5BellaVia Research ©2015
Why was research so essential to Ritani.com?
Buying a ring is a very
unique purchase.
They are expensive.
There are a lot of options
in stores and online. It’s
not like buying a t-shirt!
Understanding the
dynamics at play were
critical – particularly with
Ritani’s unique offering.

6. 6BellaVia Research ©2015
Why was research so essential?
It’s an emotional experience.
For everyone.

7. 7BellaVia Research ©2015
Why was research so essential?
There’s lots to do. And a lot of advice. And that takes time.
GENTLEMEN'S ENGAGEMENT RING BUYING GUIDE
Now that you're ready to get engaged, it's important to ensure that you're
going to get the most value for your money. Doing research, looking
online, talking to friends to see how much they spent on rings or how
much diamonds cost and talking to a jeweler you trust are all great
starts towards becoming an informed buyer.
When we're talking about buying wise, we also recommend considering
all the components that go into a ring purchase. For example, as part of
setting a budget, you might want to set separate budgets for the
diamond, for the engagement ring setting, for your intended's wedding
band, and potentially for your own future wedding band. Visit a reputable
jewelry store, and have the associate guide you through each step.
Have them explain the difference in the service, have them explain the
difference between platinum and gold, have them talk you through
every question that you have. Buying wise means working with a jeweler
you trust, who is going to stand behind your purchase(s) for the lifetime
of the piece so that you know you have the very best quality that your
money can buy.
Koerbers Fine Jewelry

8. 8BellaVia Research ©2015
Why was research so essential?
The experience occurs over time & across channels, & it
is very hard to measure. There’s no tidy purchase funnel.
Ritani.com wanted to better understand the role of mobile
devices for ring research and purchasing.

9. 9BellaVia Research ©2015
Why not in-person research?
Money.
 Hotels and airplanes are
expensive.
 Longitudinal nature of ring-
buying required multiple
interviews (multiple trips).
 Multi-city for regional
sensibilities, in-store visit
partners (long trips).
 Start-up budget

10. 10BellaVia Research ©2015
Why not in-person research?
Observation.
 Remote research is easy to
share. “Hidden” research
offers less value to the team.
 Each team member got two
users for the entire journey
• 3 interviews over a few months,
4.5 hours per participant
 Active participation was key.

11. 11BellaVia Research ©2015
Why not unmoderated remote research?
 It’s a longitudinal purchase.
People are forgetful.
 It’s emotional. Not all guys
readily share their feelings.
 It’s not linear. Many factors
involved. We needed the ability
to easily go off script and go
deep.
How do I pick a
diamond? Where
do I start? Who
can I trust? What
will she like?
Depth of insights.

12. 12BellaVia Research ©2015
Ritani.com needed:
Remote, webcam
interviews
• To include geographically
distributed participants
(nationwide)
Real-time
visibility to
mobile devices
• Computer + tablet +
smartphone
Moderated,
in-depth
exploration
• Not unmoderated,
not task-based
research, not self-
reported diary
?
But
how?

13. 13BellaVia Research ©2015
The problem
* Moderated and real-time

14. 14BellaVia Research ©2015
The problem
We need to observe it remotely & real-time
and talk about it

15. 15BellaVia Research ©2015
What we did
3 rounds of research
(longitudinal purchase),
•>4 hours of interviewing each
14 ring shoppers,
early in funnel
•Wide geographic mix
And we found an approach for
remote moderated research
with visibility to mobile devices!

16. 16BellaVia Research ©2015
Landscape: Remote Research
Computer
online
Mobile
devices
moderated
Un-
moderated
If they can do it on
mobile, they can do it
on desktop, too

17. 17BellaVia Research ©2015
Considerations when choosing a solution
Interactions?
• How important is
it to see fingers on
screen?
Non-verbals?
• Do you need to
see the user’s
face?
Stimuli?
• Mobile web?
• Apps?
Platform?
• Android?
• iOS?
Who are your
users?
• Tech savvy? Not?

18. 18BellaVia Research ©2015
Option 1: IT Support Tool: e.g., Bomgar
How it works: Participant installs
Bomgar app. Enters session number
provided by researcher. Uses Bomgar
browser to access site or Android app.
Very clear view of device
Affordable if regularly used: single-
user license: >$1100 per year
Cannot test iOS apps: only Android
apps (and mobile web)
Slow screen refresh makes it hard to
see interactions
Not designed for research
(odd verbiage for participant)
Similar option: TeamViewer
Researcher view
Thanks to Nyle Feist of UX Guys for the demo!

19. 19BellaVia Research ©2015
Option 1: IT Support Tool: e.g., Bomgar
Cannot see finger
interaction
Cannot see user’s face
Bomgar browser
eats screen real estate
(address bar at top, action
buttons at bottom)
Ritani.com
via Bomgar app
Participant view Ritani.com

20. 20BellaVia Research ©2015
Option 2: document camera, e.g., Ipevo
How it works: Participant plugs in external USB-
Document camera to their computer. Starts screen-
sharing tool.
Great if you want to see the interaction (fingers
touching screen)
Very easy to switch between devices
BUT…you can’t see the user’s face. Can’t use
a document camera and webcam simultaneously
 Note: According to UXPA 2015 session attendees, you can see the
user’s face if the document camera is used in conjunction with a
webcam and Morae Observer. A Morae license is $2,000 per year.
Focus issues
Logistics / fulfillment hassle: Need to send
each participant a document camera (budget:
$100 per device with shipping). Vendors like
UserTesting.com can help & have a panel.
Photo from ipevo.com
Thanks to session attendees for the info about document
cameras used in conjunction with Morae Observer!!

21. 21BellaVia Research ©2015
Option 3: laptop love!
How it works: Participant turns their laptop
around & “hugs” it. Use webcam & screen
sharing tool to show their mobile device.
Free! (if user has a webcam)
Can see the interaction (fingers touching
screen)
Very easy to switch between devices
BUT…you can’t see the person’s face when
they are snuggling their laptop
Challenging for participant to keep device in
webcam view (participant can’t see it, so
researcher has to remind them)
Image quality can be an issue (focus issues,
low quality webcams)

22. 22BellaVia Research ©2015
Option 4: Lookback’s Unicorns.io
How it works: Participant
installs Unicorns.io on Mac.
Signs in via Twitter
credentials. Connects device
via USB cord. Clicks
“stream.” Shares link with
researcher.
Free download!
(currently)
Very clear image
Streams the video &
audio
If used with screen sharing
tool, can also see the
participant’s face

23. 23BellaVia Research ©2015
Option 4: Lookback’s Unicorns.io
Only works for participants who have
Apple devices and a Mac running recent
software
•only iOS 8+ (80%)
•only Mac OSX 10.10+ (Yosemite) (60%)
Participant must have Twitter account
Cannot see finger interaction
- Note: A similar option was volunteered by
a UXPA 2015 session attendee. X-Mirage is
another iOs- only option that he feels works
well. He recommended Zoom for screen
sharing.*
Thanks to the session attendee who recommended x-mirage and zoom!

24. 24BellaVia Research ©2015
Option 5: AirServer + AirPlay
How it works: Participant installs &
launches AirServer on their computer.
Swipe up on iOS device to start AirPlay.
Device is wirelessly mirrored to computer
screen. Starts screen sharing tool and
webcam. Shares screen.
Affordable: $15 per download
Very clear image
Can see the participant’s face via
webcam (screen sharing tool)
User can easily toggle between PC,
iPhone, iPad

25. 25BellaVia Research ©2015
Option 5: AirServer + AirPlay
Only works for
iOS
Only works on
iPhone 4S+ and
iPad 2+
Users may need
a second phone
(some iPhones
cannot mirror & be
used as a phone
simultaneously)

26. 26BellaVia Research ©2015
Option 6: Reflector
How it works: Participant installs &
launches the Reflector app on their
computer.
Ensures computer and device are on
same WiFi network. Shares screen
via screen sharing. Starts webcam.
Android: launch Screencast
app. (older Android devices
require ChromeCast)
iOS: Swipes up to launch
AirPlay
Wirelessly mirrors device to computer
screen.

27. 27BellaVia Research ©2015
Option 6: Reflector
Very clear view of device
Free! 7 days free per user
Visibility to everything (apps, mobile
websites, native phone function)
Ability to see user’s face via webcam &
screen sharing
Cannot see finger interaction on screen
Users may need a second phone (some
phones cannot mirror & talk simultaneously)

28. 28BellaVia Research ©2015
Reflector: In the field
MindBody.com has been in the
field using this technique this
week. It’s going great!
They aren’t crazy about the
Reflector “watermark” – probably
because they are using the free
version?
Thanks to Marianne Connor and Amaya Becvar Weddle from Mind Body Online for this participant quote!
It was fine, I’m really, really
nervous with tech stuff, so
the fact that this worked is
impressive. That never
happens for me. Ever.
Participants say it’s easy!

29. 29BellaVia Research ©2015
Pros & Cons: Things are changing rapidly!
IT Support
tool
Document
camera
Laptop hug Unicorns.io AirServer Reflector
See face?


If used with
Morae observer
   
See interaction?      
Apps? 
Not on iOS
    
Android & iOS?
   
iOS only

iOS only

iOS + Android
Clarity Clear, but
slow refresh
Focus can be
an issue
Focus can be
an issue
Excellent Excellent Excellent
Cost $1100 single
license / year
<$100 per
camera
Free Free (for now) $15 per license
Free for user
for 7 days
What else? Tech support
tool
Can’t see full
screen
Logistics
(sending
cameras)
Morae is
expensive!
$2,000 license
Awkward! Only works with
recent iOS &
Mac updates
Requires twitter
account
iPhone users
may need
second phone
for voice
iPhone users
may need
second phone
for voice

30. 30BellaVia Research ©2015
Lessons Learned & Tips for Success
Schedule a “tech check” before the interview
• Before the interview, help the participant get the technology set up. Do it
before the interview, so you don’t waste the time of client observers.
• Consider bringing in tech help.
Make time for tech checks
• Our projects move at Mach speed, but add 2-3 days before interviewing to
make sure all your participants are ready on the technology front.
• If all goes well, each tech check will take 10-20 minutes.
Incentivize for tech check
• What would you pay for a 30-minute interview? Add that to the incentive.
Plan for attrition
• You may lose more than normal due to tech issues / frustrations.
• Over-recruit! e.g., For 10 interviews, recruit 12-14

31. 31BellaVia Research ©2015
Lessons Learned & Tips for Success
Ask users to have a second phone available
• You’ll know from the tech check if the user can talk & mirror simultaneously.
Don’t be caught off guard!
Not a suitable methodology for all participants
• Works great for tech savvy participants
• Didn’t work as well for a general population study about smartphone app
abandonment. Less tech-savvy participants were frustrated and we lost
several during the tech checks. Solutions are getting easier, but...still may
not be ideal for less tech savvy participants.
Use it as part of a hybrid, mixed-method approach
• Be careful that this methodology doesn’t skew your sample towards tech
savvy participants.
• Consider using as part of a hybrid methodology: some remote interviews
for geographic mix + in-person interviews to ensure tech savvy mix.

32. 32BellaVia Research ©2015
Technical Perspective: Reflector
During the Recruiting Process:
 Confirm Mobile Device Meets Specs:
IOS 7 or higher
Android, 4.4.2 or higher
 Confirm Computer Meets Specs:
Apple: OS X 10.7.5 or Later
PC: Windows Vista, Windows 7 or Windows 8 (Windows 8 RT is not supported)
 Confirm Webcam:
If it is built in or external, and if one needs to be sent
 Confirm Network Connection:
Wireless 802.11n or Ethernet highly recommended
Both computer and mobile device must be on the same connection
No use of free WiFi systems
Beware of office firewalls
Disconnect from VPNs (eats up too much bandwidth)

33. 33BellaVia Research ©2015
Technical Perspective: Reflector
Conduct the Tech Check:
• Send email to respondent with link tech check room
• Schedule tech check session
• Technician walks respondent through steps involved in sharing mobile device on computer screen
(turn on webcam, start screen sharing, download Reflector, enable Reflector, connect mobile device)
• Respondents test should occur on the actual equipment they will use for the interview computer with
webcam (detachable or built-in) plus their mobile device/s
• Make sure respondent understands they will need to replicate equipment and set up at actual interview
• Confirm interview date and time
• If recruit is accurate, tech check is 10 to 20 minutes per respondent
Tech Check Tips :
• Variations in mobile device drives ability to connect to Reflector:
iOS users use Airplay to connect to Reflector
Android 5.0 Lollipop device users cast screens to Reflector
Devices running Android 4.4.2 use the Chromecast App to connect to Reflector
• Recruiter must be tech savvy and thorough in recruiting respondents that meet the tech specs or attrition
rate will be high during the tech check and respondents will have to be re-recruited.
• Unforeseen circumstances that require rescheduling include sudden loss of internet connection, computer
crash, weather, stolen or lost device, etc.

34. 34BellaVia Research ©2015
Methodology Benefits
• Deep learnings without breaking the bank with travel
Affordable
• Savvy population was not afraid of technology and trying new things. They were
curious about the approach and invested in making it work.
• Ring shopping is (often) secret! Easier to hide phone interview than in-home visit +
shopalong.
Good fit for affluent ring-buyers
• Wide audience at Ritani.com could observe sessions real-time. Built empathy for
customer among a diverse set of employees.
• Empowered the team with actionable information, created appetite for findings and
recommendations.
• Seeing is believing - direct exposure to *how* customers use devices on our site and
competitors surfaced lots of insights for the team.
Research observation helped with buy-in

35. 35BellaVia Research ©2015
Video Demo! Reflector Mirroring iPhone
If you’d like to see the video demo, please contact Rebecca at Civicom

36. 36BellaVia Research ©2015
Questions?
Alex
Ritani.com client
perspective
alex.berg@fellswoop.com
Julie
Researcher
perspective
julie@
BellaViaResearch.com
Rebecca
Technical
perspective
rebecca.west@civi.com