Mobile UX Research Methods and Tools

Like this? Share it with your network

Share

Mobile UX Research Methods and Tools

  • 3,393 views
Uploaded on

In this webinar, Nate Colker, UX Researcher at UserZoom, talks about different research methods and tools for testing mobile websites and apps. As a case study, Nate goes over a setup and results......

In this webinar, Nate Colker, UX Researcher at UserZoom, talks about different research methods and tools for testing mobile websites and apps. As a case study, Nate goes over a setup and results of a recent mobile benchmark study that we conducted using UserZoom's brand new Mobile Remote Usability Testing solution.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,393
On Slideshare
2,927
From Embeds
466
Number of Embeds
2

Actions

Shares
Downloads
110
Comments
0
Likes
12

Embeds 466

http://info.userzoom.com 430
https://twitter.com 36

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • ----- Meeting Notes (4/30/13 09:13) -----We are recording the webinar. In 24 hrs it should be available on our blog together with the slides.There will be a complete version with more details available on our blog due to time constraints we have selected the most imp.At the end of the webinar, we will announce the winner of the free userzoom usability study.
  • Alfonso speak.
  • Nate Speak methods and tools.
  • Add the slide on the VOC
  • Add link to this slide.
  • Pros:Add the benefit of having remote moderated usability testing.
  • Add the hugging one. Remote moderated. Huggy one.
  • Nate: This is something customers were demanding. As far as we know the only tool in the market.
  • ----- Meeting Notes (4/30/13 10:01) -----Task 2 transition slide
  • Nate.

Transcript

  • 1. Agenda WEBINAR: Mobile UX Research Methods and Tools www.userzoom.com
  • 2. Agenda for today 1. Introductions 2. Mobile UX Research Methods and Tools 3. Case Study: Fashion Industry Mobile/Tablet Benchmark Study – Details & Design – Study Results 4. Q&A www.userzoom.com
  • 3. Speakers Alfonso de la Nuez Nate Colker Sneha Kanneganti Co-Founder & Co-CEO UX Researcher UX Researcher Moderator Presenter Twitter hashtag: #uzwebinar www.userzoom.com
  • 4. About UserZoom  All-in-one Enterprise software solution that helps Businesses cost-effectively test, measure and improve UX over websites & mobile apps.  Offer online or remote user research & testing solutions, saving UXers time, money, effort, and a lot of actionable insights  UX Consultants since ‟01, SaaS since „09  In Sunnyvale (CA) Manchester (UK), Munich (DE) and Barcelona (Spain)  90% renewal rate, 50% revenue growth rate in the last 3 years www.userzoom.com
  • 5. Mobile UX Research Methods and Tools www.userzoom.com
  • 6. Mobile UX Research Methods Mobile UX research method depends on the goals of the study. • To validate and refine (testing). • To gather user perspectives. • To create user profiles & personas. • To understand behavior. There are several methods to test mobile applications. It depends on your: • Goal • Time • Budget www.userzoom.com
  • 7. Mobile UX Research – Moderated Studies www.userzoom.com
  • 8. Mobile UX Research Methods - Moderated Goal: To understand how users your mobile application, website or services. Method: Usability Testing in Lab. Users: 5-15 Project Timeline: 2+ Weeks (execution & analysis) Cost: $15,000 - $30,000 Pros Cons Moderated Lab Usability Testing • Probing by moderator • Artificial environment • Real-time viewing of interactions • Comparatively more cost and time consuming www.userzoom.com
  • 9. Mobile UX Research Tools - Moderated Mr Tappy  A kit for filming mobile phones and tablets from the user‟s point of view.  Price: $295 (including shipment) Pros • Simple to set up • Allows to test any mobile device you want Cons Moderated Lab Testing Tool: Mr. Tappy • Gets in the way of the natural use of the phone • You need your own webcam to do the recordings • The finger movements of the participant might obscure the view Review of the product:http://www.actualinsights.com/2012/mrtappy-mobile-usability-testing-rig/ www.userzoom.com
  • 10. Mobile UX Research Tools - Moderated DIY Document Camera by Objective Digital You’ll need  GorillaPod Stand by Joby (www.joby.com)  Logitech Webcam Pro 9000  Recording software for Mac or PC Moderated Lab Testing Tool Source: http://blog.objectivedigital.com/2010/11/19/diy-document-camera-for-mobile-testing-record/ www.userzoom.com
  • 11. Mobile UX Research Tools - Moderated Mod 1000  A camera and a rig for recording the user experience with mobile phones and tablet screens  Cost: $350 Pros • Comes with its own webcam and software • Simple and lightweight design Cons • Gets in the way of the natural use of the phone • The finger movements of the participant Moderated Lab Testing Tool: Mod 1000 might obscure the view www.userzoom.com
  • 12. Mobile UX Research Tools – Moderated UX Recorder  An iOS app that records on-screen activity and gestures along with video from the front facing camera (and audio)  Works on mobile websites only, doesn‟t work on mobile apps  Supported on devices running iOS 5.0 and up  Price: single session - $1.99; unlimited sessions - $59.99 Pros • Records on-screen activity as well as video from the front facing camera • Records audio in addition to video • Allows to export recorded video session Cons • Inability to record iOS applications (iOS limitation) • No analytics, just a video recording Moderated Remote Testing Tool: UX Recorder to a desktop www.userzoom.com
  • 13. Mobile UX Research – Moderated Remote www.userzoom.com
  • 14. Mobile UX Research Methods – Remote Moderated Goal: To understand how users your mobile application, website or services Method: Remote Moderated Usability Testing Users: 5 - 15 Project Timeline: 2+ Weeks (execution & analysis) Cost: $5,000 - $7,000 Pros Cons • Not tied to a location. • Difficult to observe body language. • Distributed & diverse user pool. Moderated Remote Usability Testing • Technical difficulties. www.userzoom.com
  • 15. Mobile UX Research Methods – Remote Moderated Delight.io  Enables you to record a screen of your iOS application and automatically capture all gestures  Price: 30 hours for $300 Pros • Allows to record audio in addition to video • Records user‟s emotions using the Cons • You need to install a piece of code into your app iPhone‟s front camera • Has a nice dashboard with all the recorded sessions www.userzoom.com
  • 16. Mobile UX Research Methods – Remote Moderated Reflector app  An app that will broadcast the screen of the iPhone or iPad onto your computer  Price: Single user license - $12.99; 5 seat license - $54.99 Pros • You can project your user testing sessions for stakeholders to see Cons • You cannot see user‟s Moderated Remote Testing Tool: reflector App gestures • Very easy to set up Has a nice http://www.90percentofeverything.com/2013/01/14/combining-reflector-and-silverback-for-ios-usability-testing/ www.userzoom.com
  • 17. Mobile UX Research Methods – Remote Moderated Mac Hug  Idea developed by MailChimp  User‟s webcam and Skype to moderate the sessions  Sessions recorded at the researcher‟s end(silverback, WebEx recorder, etc.) Pros • Cost Effective • Doesn‟t require too much Cons • Users expressions & body language lost http://blog.mailchimp.com/remote-usability-testing-on-mobile-devices/ www.userzoom.com
  • 18. Mobile UX Research – Unmoderated Remote www.userzoom.com
  • 19. Mobile UX Research Methods – Remote Unmoderated Goal: To understand how users your mobile application, website or services Method: Remote Unmoderated Usability Testing Users: 200+ Project Timeline: 10 days (execution & analysis) Cost: Variable ($1,000 - $6,000). Pros Cons • Distributed & Diverse Users • Probing difficult • Time & cost effective • Body language & immediacy of qualitative • Enables to collect quant and qual Unmoderated Remote Usability Testing feedback lost data www.userzoom.com
  • 20. Mobile UX Research Methods – Remote Unmoderated Usabilla Mobile  Create visual surveys in iOS or Android browser  Allows to test screenshots of an app or a mobile site  Users can scroll through an image and tap anywhere to leave a point and type up a note Results Charts and Heatmaps Un-moderated Remote Testing Tool: Usabilla Mobile www.userzoom.com
  • 21. Mobile UX Research Methods – Remote Unmoderated UserTesting Mobile  Create series of tasks for users to perform  Send it out to users from UserTesting panel  Watch recorded videos of the user‟s screen  Downside – limited to their own panel Results • Video of user‟s screen and hand motions • audio Moderated Remote Testing Tool: UserTesting Mobile www.userzoom.com
  • 22. Mobile UX Research Methods – Unmoderated Remote UserZoom Mobile - New  Create a study with tasks for users to perform in their natural environment  Test any mobile website and/or a hosted prototype on both iOS and Android devices  Test your competitors‟ mobile site since no coding is needed Results Quantitative and qualitative data (success ratios, time on task, number of clicks, survey responses, clickstreams, NPS) Coming Soon: Session Replay & Unmoderated Remote Testing Tool: UserZoom Mobile Heatmaps www.userzoom.com
  • 23. Mobile UX Research Tools: Opinion Research UserZoom Mobile Surveys  Conduct various types of research projects, including: satisfaction surveys, exit surveys and persona definition  Works on iOS and Android devices www.userzoom.com
  • 24. Fashion Industry Mobile Website UX Benchmark Study (Remote Unmoderated) www.userzoom.com
  • 25. Study Details How do fashion retail mobile websites perform on routine purchase related tasks? 4 Brands 3 Tasks 1. Choose a black dress and go through the purchase process as if to buy it. 2. Look for this* necklace that goes with your dress and continue to checkout. Study Design • • • • • Tested Between April 15-21, 2013 Between Subjects Design 200 Users/50 users per brand 3 Tasks and Survey Questions Recruitment Criteria  Female  between 25-45 3. Look for information on how to return an item ONLINE and whether there is an associated cost. *Product name provided  Regularly make purchases on their tablet • Used a panel Vendor (SSI). www.userzoom.com
  • 26. Mobile Phone/Tablet Optimization Brands Tablet Mobile site iPad Android www.userzoom.com
  • 27. Mobile Phone/Tablet Optimization www.userzoom.com
  • 28. Study Through Participant‟s Perspective www.userzoom.com
  • 29. Study Results www.userzoom.com
  • 30. Results – Top 4 Findings 1. Based on user feedback on 3 tasks, White House Black Market fares the best in searching and finding products (a black dress & necklace). It also has the highest Net Promoter Score of +26. 2. Users found it difficult to find return policy information on these product pages. 3. Users do not like registering and disclosing personal information before making a purchase. 4. Material, length, weight and how it looks on a person are important product information users seek while making purchasing decisions. www.userzoom.com
  • 31. Task -1 www.userzoom.com
  • 32. Results – Task 1 Choose a black dress and go through the purchase process as if to buy it Reasons for failure Success “I was being forced to enter a lot of personal information before I got to the payment details page.” 100 90 80 70 60 “The web page would not load.” 50 40 30 20 10 0 52 42 42 53 Kenneth Cole Guess Bebe WHBM Success rate in percentage “It takes a very long time to enter payment information on a tablet without a full keyboard. It was also difficult to find a black dress I liked in a size 10/m.” “It‟s probably easier on a computer, but on a mobile device it was very slow, I had to hit the buttons several times, and my keyboard would pop in and out.” www.userzoom.com
  • 33. Results – Task 1 Choose a black dress and go through the purchase process as if to buy it. 52% Success Clear Payment Methods Mentioned. 42% Success Too much to type on a touch pad. www.userzoom.com
  • 34. Results – Task 1 Choose a black dress and go through the purchase process as if to buy it Efficiency Satisfaction 7 5 6 4 5 3 4 3 2 2 1 1 6 2.76 2.9 3.28 3.33 Kenneth Cole Guess Bebe WHBM 0 Average Time Taken in Minutes 5.7 Kenneth Cole Guess 6.5 6.2 0 Bebe WHBM Satisfaction on a Scale of 1-7 Guess: “I had to give registration info to check out. I don‟t always like doing that.” www.userzoom.com
  • 35. Task - 2 www.userzoom.com
  • 36. Results – Task 2 Look for a necklace that goes with your dress and continue to checkout. Success Reasons for failure “It says that the necklace is available in "one size" but didn‟t state the actual length.” “There was a lack of information on the materials and I would like a picture of the item on a neck to estimate size.” “Not enough information about what the products were made out of. Wax??” Success rate in percentage “I scrolled through the 8 pages but couldn‟t find it. I had to search for it.” www.userzoom.com
  • 37. Results – Task 2 Look for a necklace that goes with your dress and continue to checkout. 93% Success Product materials clearly listed. 51% Success Vague product description. www.userzoom.com
  • 38. Results – Task 2 Look for a necklace that goes with your dress and continue to checkout. Efficiency Satisfaction 7 5 6 4 5 3 4 3 2 2 1 0 1 1.96 2.26 2.46 Kenneth Cole Guess Bebe Average Time Taken in Minutes 2.56 WHBM 0 6.4 5.4 6.3 6.2 Kenneth Cole Guess Bebe WHBM Satisfaction on a Scale of 1-7 WHBM: “Love it. Liked the suggestions, reviews and guide to size” Kenneth Cole: “I like how there are other pieces of jewelry presented as suggestions as to what could go with this particular necklace.” www.userzoom.com
  • 39. Task - 3 www.userzoom.com
  • 40. Results – Task 3 Look for information on how to return an item ONLINE and whether there is an associated cost. Reasons for failure Success “Too much information!” “I had to go to the full site to find the info.” Success rate in percentage “Not very easy to find and probably wouldn‟t have even tried that hard had it not been for this survey.” “I couldn't FIND any information about online returns, only in store and by mail.” www.userzoom.com
  • 41. Results – Task 3 Look for information on how to return an item ONLINE and whether there is an associated cost. 82% Success Clear online purchase return information. 40% Success Too much information to parse through. www.userzoom.com
  • 42. Results – Task 3 Look for information on how to return an item ONLINE and whether there is an associated cost. Satisfaction Efficiency 7 5 6 4 5 3 4 3 2 2 1 1 0 2.1 2.16 2.4 1.53 Kenneth Cole Guess Bebe WHBM Average Time Taken in Minutes 5.6 4.5 6.3 6.4 Kenneth Cole Guess Bebe WHBM 0 Satisfaction on a Scale of 1-7 Kenneth Cole: “It was not clear whether the cost for shipping returns was free” Guess: “I Had To Go To Check Out To Get The Return Policy. I Need It More Accessible.” www.userzoom.com
  • 43. Net Promoter Score - NPS 26 22 22 2 • While all brands have a positive NPS, White House Black Market has the highest with 26. (NPS is the measure of how likely a user is to recommend a site to friends and family.) Net Promoter Score, Net Promoter and NPS are registered trademarks of Bain & Company, Satmetrix Systems and Fred Reichheld. © 1996-2013. Bain & Company www.userzoom.com
  • 44. Mobile Phone/Tablet Optimization Study simultaneously conducted in Spain & U.K. www.userzoom.com
  • 45. Take-aways & Recommendations There are many mobile usability research methods & tools in the market today. There is no excuse not to conduct user research anymore. • Based on user feedback on 3 tasks, White House Black Market fares the best in searching and finding products (a black dress & necklace). It also has the highest Net Promoter Score of +26. • Crisp product information, precise search results and easy to use filters are aspects users care about while making purchases on websites. • Research what your customers care about. www.userzoom.com
  • 46. Questions? Follow us for updates on future webinars! Twitter: @userzoom Linkedin: http://www.linkedin.com/company/userzoom Facebook: https://www.facebook.com/userzoom www.userzoom.com