Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
How to build a failsafe 
mobile usability testing set-up 
Fabien Marry - @AlphabUX 
1
I started my career working in TV…
Such control rooms have a large number of devices 
working in a chain, any of which can fail at any moment. 
Yet the live ...
Later on in my career, I was in charge of supervising 
teams of qualified audiovisual professionals with much 
more experi...
Many times, these professionals let me (and my employer) down in a big way. 
One VIP speech wasn’t recorded because they f...
These experiences taught me it’s unlikely that absolutely everything will 
work as intended. So now, I always have a plan ...
+ + + 
Web host Internet Local Network Test device 
When conducting mobile web usability testing (with a standard setup) y...
+ + 
Your Mac Local Network Test device 
You can use your own laptop as the web server (via the OS built in 
servers or an...
Now, let’s think about another part of the set-up. How do we 
record the action? 
Some use a document camera, requiring th...
But this is not how people use tablet or mobile devices. They are meant to be 
hand held. 
Putting them on a table changes...
Test 
Device Reflector Your Mac 
Local 
Network 
Silverback 
Observation 
TV 
+ + + 
So here’s my initial setup: testing o...
Test 
Device Reflector 
Local 
Network 
Your Mac 
Silverback 
Observation 
TV 
+ 
+ + 
Hue HD 
camera 
When recording hand...
Test 
Device Reflector Your Mac 
Local 
Network 
Silverback 
Observation 
TV 
+ + + 
Skype 
Remote 
client 
When I can’t r...
So this is what the recordings look like (clockwise from top left): 
- A view of the participant’s hand gestures (via the ...
15 
But remember Sod’s law…
Local 
Network 
Test 
Device Your Mac Silverback 
Observation 
TV 
You are still relying on the local network. If it fails...
Another option: use your test iPhone’s 
4G connection and the personal 
hotspot feature to connect it to the 
laptop. 
You...
Test 
Device Reflector Your Mac 
Your own 
network 
Silverback 
Observation 
TV 
+ + + 
So here you have it in full! 
Refl...
Fabien Marry - Senior UX consultant 
! 
fabien.marry@cxpartners.co.uk
Credits 
! 
Photos 
Slide 2: http://tvmedia.hpage.co.in/course_37574641.html 
Slide 3: Jérémy Dupeu, http://www.camera-for...
Upcoming SlideShare
Loading in …5
×

How to build a failsafe mobile usability testing set up

2,473 views

Published on

When conducting mobile web usability testing (with a standard setup) you need your web host, internet, local network and test device to work as they should.

But technology fails, and people fail. So how do you build a set-up that won't fail? (For under £100!)

Published in: Design

How to build a failsafe mobile usability testing set up

  1. 1. How to build a failsafe mobile usability testing set-up Fabien Marry - @AlphabUX 1
  2. 2. I started my career working in TV…
  3. 3. Such control rooms have a large number of devices working in a chain, any of which can fail at any moment. Yet the live show must go on, so they have endless amounts of contingency plans to immediately adapt to most equipment failure. The lesson I took away: technology fails. This is the norm, not the exception. This is not an excuse for your service to be interrupted.
  4. 4. Later on in my career, I was in charge of supervising teams of qualified audiovisual professionals with much more experience than me at some high-level events involving heads of state.
  5. 5. Many times, these professionals let me (and my employer) down in a big way. One VIP speech wasn’t recorded because they forgot to turn the volume back up after turning in down to avoid hearing the noise of delegates settling in their chair before the session. Another key 20 min speech recording was missing a part because the tape had run out and they didn’t anticipate it. Just like technology, people will fail. You need to anticipate that too.
  6. 6. These experiences taught me it’s unlikely that absolutely everything will work as intended. So now, I always have a plan B (and often a plan C, and D).
  7. 7. + + + Web host Internet Local Network Test device When conducting mobile web usability testing (with a standard setup) you need all four of these elements to work as they should. Here two elements are completely out of my control: the web host and the location’s internet connection. If they go down, I can’t do anything about it, and it’s game over. This could leave me with confused participants, annoyed clients that wasted their time, an expensive testing facility bill, and a project plan to redo. We can do better!
  8. 8. + + Your Mac Local Network Test device You can use your own laptop as the web server (via the OS built in servers or an application like Mixture). By doing this, internet and web host issues can be fully prevented.
  9. 9. Now, let’s think about another part of the set-up. How do we record the action? Some use a document camera, requiring the devices to be kept flat on their back on a table.
  10. 10. But this is not how people use tablet or mobile devices. They are meant to be hand held. Putting them on a table changes how they are used: - The fingers used will be different, potentially hiding ergonomic issues - The device will be positioned further away from the eyes than it would normally be held, preventing you from testing legibility properly.
  11. 11. Test Device Reflector Your Mac Local Network Silverback Observation TV + + + So here’s my initial setup: testing on an iOS device, using the Airplay mirroring feature to send the screen content to a Mac running @ReflectorApp. The content of that Mac screen and the face of the participant (captured via a webcam), are then recorded using @SilverbackApp, and mirrored to the TV in the observation room via a cable.
  12. 12. Test Device Reflector Local Network Your Mac Silverback Observation TV + + + Hue HD camera When recording hand gestures is important, as it often is for apps, I add another webcam.* The picture from this is displayed on the Mac using Quicktime player (choose “File’ / ‘New Screen Recording’, but don’t actually press record). Silverback grabs everything, including the additional webcam, and displays it on the TV in the observation room. * I use a @HueHD camera as it’s flexible neck is easy to attach to the participant’s chair.
  13. 13. Test Device Reflector Your Mac Local Network Silverback Observation TV + + + Skype Remote client When I can’t run a cable to the observation room TV, I use Skype to share the screen and audio with a device in the observation room. You can then connect your device (I use my iPad) to the large observation TV / sound system. With a group chat (sadly unsupported if one of the devices is an iPad), this setup even allow one or several clients to observe the sessions remotely. However, this requires the internet, which we don’t control (see slide 7), so I always flag that this as a nice bonus rather that something to rely on.
  14. 14. So this is what the recordings look like (clockwise from top left): - A view of the participant’s hand gestures (via the over-the-shoulder webcam) - A perfect view of the screen content, unrestricted by the hand or the participant moving the device away from the camera’s view (via Airplay mirroring to Reflection.app) - The participant’s face (via the laptop’s built-in webcam) Extra tips: - Use a flat colour desktop with no visible icons. This will reduce the output video file size, and help the viewer focus on the action. - When recording a mobile phone screen, you don’t need the laptop full resolution, so I set my laptop’s resolution to something low like 800x600, greatly reducing the space taken by the recording files.
  15. 15. 15 But remember Sod’s law…
  16. 16. Local Network Test Device Your Mac Silverback Observation TV You are still relying on the local network. If it fails, a fall back option is to connect the test device straight to the observation room TV via a VGA or HDMI display adapter. Of course, this means that your recordings won’t have the screen mirroring, but you can somewhat compensate for this with an over-the-shoulder webcam.
  17. 17. Another option: use your test iPhone’s 4G connection and the personal hotspot feature to connect it to the laptop. You can even use a direct usb connection for stronger interference resilience.
  18. 18. Test Device Reflector Your Mac Your own network Silverback Observation TV + + + So here you have it in full! Reflector is £13, Silverback is £70. Assuming you already own a laptop and your test mobile device, this set-up is basically a sub £100 full mobile usability lab. And it fits in a backpack.
  19. 19. Fabien Marry - Senior UX consultant ! fabien.marry@cxpartners.co.uk
  20. 20. Credits ! Photos Slide 2: http://tvmedia.hpage.co.in/course_37574641.html Slide 3: Jérémy Dupeu, http://www.camera-forum.fr/index.php?/topic/8376-captation-multicamera/ Slide 4: The World Bank / Fabien Marry, http://web.worldbank.org/WBSITE/EXTERNAL/EXTABOUTUS/ORGANIZATION/ EXTPRESIDENT/EXTPASTPRESIDENTS/EXTOFFICEPRESIDENT/0,,contentMDK:20872180~menuPK:51408951~pagePK: 51174171~piPK:64258873~theSitePK:1014541,00.html Slide 5: http://lolsnaps.com/news/27654/0/ Slide 6 plan A Slide 9: Harry Brignull, http://www.90percentofeverything.com/2011/12/14/anatomy-of-a-hardware-usability-testing-rig/ Slide 10: huffingtonpost.com Slide 15: Readee, http://s287.photobucket.com/user/Kailexanra/media/SodsLaw.jpg.html ! Icons from the Noun Project Wireless & iPad by Edward Boatman Laptop by Olivier Guin Cloud by Dmitry Baranovskiy Television by Andy Fuchs Bluetooth by Muharrem Senyıl

×