Remote (Mobile) Usability Testing Lab


Published on

Describes the system we developed for conducting remote usability tests. It also gives you insights on how we changed the system to perform mobile tests... .

Published in: Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Usability Criteria = requirements: Efficiency: How much time and how many steps are required for people to complete basic tasks? Effectiveness: How many mistakes did people make? Satisfaction: How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend? Learnability, memorability, errors, flexibility, user in control, etc… .
  • Improving usability Usability definition (ISO 9241): a product is usable if it enables users to efficiently and effectively achieve their goals in a satisfactory manner Know why you are testing! Know what you are testing! Real Users Usability testing is all about seeing the design through the eyes of the test participants. As they work their way through the design, you get to see and hear what works well and where it becomes frustrating to accomplish their goals. If you've recruited the wrong participant, you'll not learn what you want. Real Tasks Designing the right tasks => don’t guide users in a certain direction by using certain terminology Observe and record Observe : To see and hear what works and what doesn’t Record : Memory and communication tool to stakeholders
  • Software installation needed: You need to have a PC ready for performing tests => you miss a lot of context You cannot recruit users while they are actually using the application, web site, etc…. . Not all major OS’s: I am a Mac user and need to install fusion or parallels in order to use the observation software Hard to test applications on other OS’s Remote testing: Network issues Have to create a local network for each usability test
  • Spread of technologies: Java: x% Flash: x% RTMP Streaming server: Broadcasts screen and audio/video Records streams RTMP = Real-Time Messaging and Presence Protocol Adobe Open source
  • Well we got some bad news today. It was a matter of time but it looks like Apple doesn’t like applications like ScreenSplitr. They now prohibits taking screenshots while in the background and crashes ScreenSplitr when it tries to do so. This isn’t a perfect solution: we couldn’t capture click events and the output from the iPhone was a bit choppy. Also, the video we captured didn’t show the users hands actually interacting with the phone. ScreenSplitr on the iPhone in combination with iDemo on the Mac Pros: ScreenSplittr transmits the iPhone signal via USB so there is less delay, resulting in smoother animations. Cons: Having the iPhone connected to the Mac with a USB cable is less natural for user testers. It also doesn’t show where the user touches the screen. Display Recorder Pros: Records where user touches the screen. Transfers live video right to a browser on a Mac, so no additional software is needed on the Mac side. It also records everything right on the iPhone leading to smooth animations in the final output. Cons: Requires iPhone 3GS. We weren’t able to test this out ourselves. We would need to take the recorded screencast from the iPhone and match it to the recording of the user.
  • Works with all mobile devices No need to install software on mobile Interaction with mobile is visible
  • Because we are using RTMP streams it was very easy to exchange the screen-capturing Java-Applet by an extra Flash camera
  • Problem: Android (but also iPhone) devices prevent you from capturing screens from a separate background service We didn’t want to root or jailbrake devices, because you cannot do that with participants’ mobiles! Solution: Connect mobile to PC, running ADB (debug) server Java application, running on the PC, acts as a client to the debug server requesting frame buffers from the mobile device
  • Sip = session initiation protocol audio / video streaming over the Internet VoIP RTMP = real-time messaging and presence protocol Audio / video streaming over the Internet
  • ×