Remote (Mobile) Usability Testing Lab
•Download as PPT, PDF•
4 likes•907 views
1. The document describes the development of a remote usability testing system that allows observers to remotely view, record, and analyze test participants using mobile applications on their own devices from any location. 2. The goals of the system were to make mobile usability testing more natural and less intrusive for participants by eliminating the need for specialized software or traveling to a lab, while still allowing observers to see the mobile screen, user's interactions, and voice. 3. The approach involved using a camera to capture the mobile screen, transmitting the video and audio streams to observers, and implementing screen capturing for some devices. This would extend their existing usability lab capabilities to remote mobile testing.
![Remote (Mobile) Usability Testing Lab Human.Interaction.Design.2010 Jan Moons [email_address] November 23rd, 2010](https://image.slidesharecdn.com/mobileuxlab-101216023613-phpapp01/85/Remote-Mobile-Usability-Testing-Lab-1-320.jpg)
![About ,[object Object],Disclaimer: Work in progress](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Remote (Mobile) Usability Testing Lab
Similar to Remote (Mobile) Usability Testing Lab (20)
Recently uploaded
Recently uploaded (20)
Remote (Mobile) Usability Testing Lab
- 1. Remote (Mobile) Usability Testing Lab Human.Interaction.Design.2010 Jan Moons [email_address] November 23rd, 2010
- 3. Usability Testing? In a nutshell
- 6. Usability Testing @ Bell Labs Problems and Vision
- 7. Typical usability test @ Bell Labs Participant Observer Logging software Participants are inevitably invited to the “lab”!
- 10. Architecture RTMP Audio/Video Streaming Screencast Streaming RTMP A/V JDBC Observer’s browser Tester’s browser Remote Objects APP APP Java Applet Flash Applet
- 11. Observe / Replay Participant’s webcam Participant’s screen Logging panel Current task Mouse pointer
- 12. Analyze Effectiveness Efficiency Data Analysis
- 13. Mobile Usability Testing Popular techniques
- 15. Document Camera http://www.easeofweb.com/mobile_usability_test.html Fixed position User has to be seated
- 16. Sled / Lamp http://www.easeofweb.com/mobile_usability_testing.html “ Mobile” but very intrusive
- 18. Screen-capturing Jailbroken iPhone Rooted Android
- 20. Mobile Usability Testing Natural Experience
- 23. Approach 1: Camera capturing mobile screen DIY Document Camera PC for transmission of streams to server Webcam Face Capturing Also ideal for testing Paper Prototypes!
- 24. Architectural Changes RTMP Audio/Video Streaming Screen Streaming RTMP A/V JDBC Observer’s browser Tester’s browser Remote Objects APP APP Camera 2 Camera 1
- 25. Approach 2: A/V Screen-Capturing Screen capturing Issues!!!
- 26. Observer
- 27. Architectural Changes RTMP Video Streaming Screencast Streaming RTMP A/V JDBC Observer’s browser Remote Objects Audio SIP ADB Screen capture Connected Computer Also support for VNC APP APP Java App Flash Applet Tester’s Mobile
Editor's Notes
- Usability Criteria = requirements: Efficiency: How much time and how many steps are required for people to complete basic tasks? Effectiveness: How many mistakes did people make? Satisfaction: How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend? Learnability, memorability, errors, flexibility, user in control, etc… .
- Improving usability Usability definition (ISO 9241): a product is usable if it enables users to efficiently and effectively achieve their goals in a satisfactory manner Know why you are testing! Know what you are testing! Real Users Usability testing is all about seeing the design through the eyes of the test participants. As they work their way through the design, you get to see and hear what works well and where it becomes frustrating to accomplish their goals. If you've recruited the wrong participant, you'll not learn what you want. Real Tasks Designing the right tasks => don’t guide users in a certain direction by using certain terminology Observe and record Observe : To see and hear what works and what doesn’t Record : Memory and communication tool to stakeholders
- Software installation needed: You need to have a PC ready for performing tests => you miss a lot of context You cannot recruit users while they are actually using the application, web site, etc…. . Not all major OS’s: I am a Mac user and need to install fusion or parallels in order to use the observation software Hard to test applications on other OS’s Remote testing: Network issues Have to create a local network for each usability test
- Spread of technologies: Java: x% Flash: x% RTMP Streaming server: Broadcasts screen and audio/video Records streams RTMP = Real-Time Messaging and Presence Protocol Adobe Open source
- Well we got some bad news today. It was a matter of time but it looks like Apple doesn’t like applications like ScreenSplitr. They now prohibits taking screenshots while in the background and crashes ScreenSplitr when it tries to do so. This isn’t a perfect solution: we couldn’t capture click events and the output from the iPhone was a bit choppy. Also, the video we captured didn’t show the users hands actually interacting with the phone. ScreenSplitr on the iPhone in combination with iDemo on the Mac Pros: ScreenSplittr transmits the iPhone signal via USB so there is less delay, resulting in smoother animations. Cons: Having the iPhone connected to the Mac with a USB cable is less natural for user testers. It also doesn’t show where the user touches the screen. Display Recorder Pros: Records where user touches the screen. Transfers live video right to a browser on a Mac, so no additional software is needed on the Mac side. It also records everything right on the iPhone leading to smooth animations in the final output. Cons: Requires iPhone 3GS. We weren’t able to test this out ourselves. We would need to take the recorded screencast from the iPhone and match it to the recording of the user.
- Works with all mobile devices No need to install software on mobile Interaction with mobile is visible
- Because we are using RTMP streams it was very easy to exchange the screen-capturing Java-Applet by an extra Flash camera
- Problem: Android (but also iPhone) devices prevent you from capturing screens from a separate background service We didn’t want to root or jailbrake devices, because you cannot do that with participants’ mobiles! Solution: Connect mobile to PC, running ADB (debug) server Java application, running on the PC, acts as a client to the debug server requesting frame buffers from the mobile device
- Sip = session initiation protocol audio / video streaming over the Internet VoIP RTMP = real-time messaging and presence protocol Audio / video streaming over the Internet