Your SlideShare is downloading. ×
Usability Testing on Mobile Devices - No More Excuses!
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Usability Testing on Mobile Devices - No More Excuses!

5,292
views

Published on

Laura Bowden's presentation on testing with mobile devices from UXPA Boston 2013.

Laura Bowden's presentation on testing with mobile devices from UXPA Boston 2013.

Published in: Design, Technology, Art & Photos

1 Comment
11 Likes
Statistics
Notes
  • Great presentation.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
5,292
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
1
Likes
11
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Usability Testing on MobileDevicesNo More Excuses!Laura Bowden & Cindy McCrackenFidelity InvestmentsDemoLog in here while you wait!https://febdesign.webex.comUXPA Mobile Device TestingPassword: 123456
  • 2. Barriers to Mobile Device Testing• Geography• Observation limitations• Perceived Cost• Mixed testing & observation platforms2
  • 3. The Scenario:• Testing on Smartphone or iPad– Participants are in the lab– Observers are all over the place– You want everyone to view the mobile devicescreen– You want everyone to view the participants’ faces– You want to record whole shebang to video3
  • 4. Setup Options4Smartphone setup 1: slides 6 - 30Smartphone setup 2: slides 31 - 44iPad setup: slides 45 - 54
  • 5. Litmus test to determine Setup 1 or 2Setup 1 is required if:• You’re not using an iPhone• You’re testing behind a “serious” firewall.• (But you can also use this setup with an iPhone if you want to, which is whatwe’re doing today.)Setup 2 can be used only when:• You’re testing on an iPhone or iPad (althoughthere may be similar technology with Android OS)• You’re not behind a “serious” firewall5
  • 6. Pros & Cons of Smartphone Setup 1 versus 26Setup 1 Setup 2Pros ProsOnly dependent on 1 network Look mom, no wires!Advanced class: Finger dotswith TouchPoseCons ConsWires Jailbreak required forTouchPoseHands/fingers can get in theway of the screenCan only test iPhones withReflector App, not othersmartphones.
  • 7. Smartphone Setup 17
  • 8. Smartphone Setup 1Materials• 2 laptops, 1 with built in webcam (mixing Macs and PCs is ok)• 1 tiny camera (we like IPEVO for $69.99)• 1 smartphone holder “sled” (we had one custom made at a plastics company for$150 using a photo from the web, here’s a cooler one: http://www.mrtappy.com/index.html)• Webex w/ VOIP option (no, not GoToMeeting. Webex.)• Screen capture software (we like Camtasia for Mac, Morae for PC)• Any smartphone• An app, website, or prototype• Patience, fortitude and the spirit of adventure! (You knowhow technology breaks down during lab testing sometimes? Multiply that by afactor of precisely 5, representing the number of additional potential points oftechnology failure compared to a regular lab test on a computer.)8
  • 9. Schematic – Smartphone Setup 1Phone ImageCaptureLaptop(w/ built-in webcam)SmartphoneTinyCameracablesledVideoRecordingLaptop(w/ lots of free space)Running:-Tiny Camera app-WebEx (sharing)-VOIP for audioRunning:-WebEx (attending)-CamtasiaObserversLaptop(attendingWebEx)Phoneor VOIPLab9
  • 10. • Are one of your 2 laptops more powerful than the other? Let that one dothe screen recording, and the other one host the phone image.• Do you know the video format your business partners can use on theirmachines? For example, if you’re recording with Camtasia for Mac, youcan’t convert to .wmv without lengthy file conversion resulting in terriblequality. Pick the right recording software for the job.• Are all your observers able to log into WebEx? (Some company’s firewallsget crabby about it, requiring special IT interventions.)• Are you able to permanently turn off your screensavers on BOTHmachines? Neither of them will have any keyboard action for the wholesessions, and will eventually go to sleep. 60 minutes of screensaver videois no fun. You can always download the app called “Caffeine” which will dothat for you on a mac. Its free.Smartphone Setup 1Step 1 – Plan for the end result10
  • 11. Smartphone Setup 1Helpful Tips Learned from(horrible)Experience:Tip #1• To avoid a dozen hours of file transfers, alwaysrecord the video on the same machine on which itwill be processed and edited..Tip #2• Don’t even think about attempting to copy rawrecordings onto a network “between sessions.”Make sure you have enough space on your harddrive for the total amount of video you will end upwith, plus extra, by a lot.11
  • 12. Follow this ordera. (Put the screen recording computer aside for a minute.)b. Launch the tiny cam’s driver application on the Image Sharing Computer.c. Plug tiny cam into USB port of Image Sharing Computer and turn it on.d. Attach the tiny cam to the slede. Attach the smartphone to the sled. (Using a rubber band? No shame!)f. Fiddle with the cameras application settings so that your image is right-side up and focusedSmartphone Setup 1Step 2 – Start plugging things into each other12
  • 13. 13b) Launch the tiny cam’s driver application on the Image Sharing Computer.c) Plug tiny cam into USB port of Image Sharing Computer and turn it on.
  • 14. 14d) Attach the tiny cam to the sled.
  • 15. 15e) Attach the smartphone to the sled.Even with just a rubber band, its prettysturdy and can accommodate anyorientation.
  • 16. 16f) Fiddle with the tiny cam’s application settings so that your image is right-side up and focused.
  • 17. 17a) On the image sharing laptop, start your WebEx meeting and phonecall…VOIP is recommended.
  • 18. a) On the phone image sharing laptop, launch a browser & startyour WebEx meeting and phone call.b) On WebEx, turn on the video cam option next to your name(this will capture the participant’s face). Now you should see2 windows: the participant’s face (small window) and thesmartphone screen (larger window). Share your screen.c) Drag images where you want them. Optional: Minimize thesmaller window with the face image so the participantdoesn’t have to watch themselves. WebEx: Participant menu > AssignPrivileges > Participants Tab > Uncheck Participant List checkbox > Assign buttond) Use static electricity to stick a piece of printer paper onto thescreen of the participant laptop*Smartphone Setup 1Step 3 – Fire up WebEx18
  • 19. 19a) Put recording machine aside for a second…b) In WebEx, turn on the video cam option next to your name (this willcapture the participant’s face). Now you should see 2 windows – the face(small window) and the phone image (large window). Share.c) Drag images where you want them.
  • 20. 20c) Optional: Minimize the smaller window with the face image so the participant doesn’thave to watch themselves. (You need to stop sharing to get the top menu.) WebEx: Participantmenu > Assign Privileges > Participants Tab > Uncheck Participant List checkbox > Assign button
  • 21. 21c) The participants’ image, minimized. Click the triangle to open it up again.
  • 22. 22d) Use static electricity to stick a piece of printer paper onto the screen of the participantlaptop so they aren’t distracted by the image*
  • 23. • In theory you can use an external webcam instead andleave the laptop closed. But that’s just one more itemto carry, and one more potential break-down point.• And even if you do use an external webcam, you willfind yourself constantly opening and closing the laptopto make sure the meeting is running and thescreensaver is turned off and the participants’ face islined up in the view… whereas the paper is easily“peeked behind”, completely ignored by the user, andcauses less distraction overall.Smartphone Setup 1*Um…Really?23
  • 24. Smartphone Setup 1Helpful Tip Learned from(horrible) ExperienceTip #3• Sometimes you get the reverse of what youwant in each window. You may have to reversethe plugging-in and turning-on order to getthe right thing on the right screen. It’s amystery why it only happens sometimes. Andbelieve us, it will revert back to the originalway the next time to try it. Embrace themystery.24
  • 25. Once WebEx is running on the Image Sharing machine….a) Launch a browser on the video recording machine and jointhe meeting. Make sure everything looks how you want it tolook.b) Launch Camtasia (or your favorite screen capture/editingsoftware).c) Make sure Camtasia is capturing VOIP audiod) Start recording your sessionSmartphone Setup 1Step 4 – Fire up the machine recording screen capture25
  • 26. 26a) Launch a browser on the video recording laptop and join the meeting. Arrange images.b) Launch Camtasia (or any screen capture/editing software) on the video recording laptop.c) Start recording your session – LIVE TRIAL…
  • 27. 27View behind the paperWhat the participant is really looking at:Tip: Adjust the size of theparticipants face larger, andconsider defining a customregion in Camtasia for lessextraneous stuff displayed onthe recording. Also, clear yourdesktop so the backgroundisn’t cluttered.What Camtasia recordsView in front of participantAll Views
  • 28. Do you want to see if it really worked?28
  • 29. • WebEx allows for 2 video inputs simultaneously, GTM does not. Soif you’re dead-set on displaying the participants’ faces alongside thesmartphone image, you must use WebEx.• GTM– Pros: Very easy interface and automatic VOIP with every meeting. Wehave found it to be more stable than WebEx mid-meeting and if youuse their recording service it will convert to .mov on a mac and .wmvon a PC.– Cons: User experience is less consistent while joining a meeting, and itonly allows for single video input so you can’t display users’ faces• WebEx– Pros: Supports 2 simultaneous video inputs and provides VOIP.– Cons: Your company must purchase the VOIP option. User interface isovercomplicated. It will record to a proprietary format which needsconversion before loading into a video editor.Smartphone Setup 1Why WebEx and not GTM?29
  • 30. Phones are…meh. There are pros and cons.• Pros– Phones in general are pretty much a sure thing. (knock on wood)• Cons– A phone tap is one more thing to carry and one more potential failure inthe chain if you connect it wrong. For observers listening through aphone, a phone’s placement is rarely between the moderator and theparticipant – usually that’s where the computer is, and the phone is off toone side.VOIP uses the computer’s mic, which is usuallyplaced more advantageously bydefault, between the moderator and theSmartphone Setup 1A word about Phone Audio30
  • 31. Smartphone Setup 1Helpful tips learned from(horrible)experienceTip #4• Do a test recording to understand how loudyour volume will be.Tip #5To record without VOIP or a phone tap• Put the lab phone on speaker so that the observers can hearyou, and just allow the recording laptop to pick up the voices inthe room via its internal mic. Mac audio we recorded like thishas been better than either the VOIP or the phone taps.31
  • 32. Smartphone Setup 1Helpful tips learned from(horrible)experienceTip #6 – If you’re a hard core minimalist.If you only have 1 laptop, sled & smartphone• You could run the webex meeting, VOIP and Camtasia on thesame machine that the smartphone is plugged into. But yourisk grinding the most robust machines down to ahalt, requiring restart between participants.32
  • 33. Smartphone Setup 2Ask yourself: Are you testing behind a “serious”firewall? If so, move along - nuthin’ to see here.Go to slide 43.33
  • 34. Smartphone Setup 2Materials• 2 laptops, 1 w/ built in webcam (don’t try to be a hero by mixingMac and PC for this one)• Reflector App on computer (download for $15.99)• Webex w/ VOIP option (no, not GoToMeeting. Webex.)• Screen capture software (we like Camtasia for Mac, Morae for PC)• An iPhone (no other kind of smartphone)• An app, website, or prototype• Belief in a higher power (While the number of elements for thissetup is decreased, the chaos factor higher, due to the requirement of creatingan additional wireless network which must perform for an audience, flawlessly,for 8 hours straight 2 days in a row.)34
  • 35. Schematic – Smartphone Setup 2ImageCaptureLaptop(w/ built-in webcam)SmartphoneVideoRecordingLaptop(w/ lots of free space)Running:-Reflector App-WebEx (sharing)-VOIP for audioRunning:-WebEx (attending)-CamtasiaObserversLaptop(attendingWebEx)Phoneor VOIPLab35
  • 36. Setup ComparisonObserversLaptop(attendingWebEx)Phoneor VOIP36Setup 1 Setup 2
  • 37. • Litmus test: If you need to go to iPhone settings to select your newnetwork “by hand”, or if you need to say “create network”on yourcomputer to get the phone to connect: This option isn’t right for you.• Is 1 of your 2 laptops more powerful than the other? Let that one do thescreen recording.• Do you know the video format your business partners can use on theirmachines? For example, if you’re recording with Camtasia for Mac, youcan’t convert to .wmv without lengthy file conversion resulting in terriblequality. Pick the right recording software for the job.• Are all your observers able to log into WebEx? (Some company’s firewallsget crabby about it, requiring special IT interventions.)• Are you able to permanently turn off your screensavers on BOTHmachines? Neither of them will have any keyboard action for the wholesessions, and will eventually go to sleep. 60 minutes of screensaver videois no fun.Smartphone Setup 2 (same as Setup 1)1 – Plan for the end result37
  • 38. • (Put the video recording computer aside for a minute.)a) On your laptop, launch the Reflector appb) On your iPhone, go to the homescreen.c) Double-tap the button and scroll all the way to the left, past the musicscreen, until you see the volume bar (admit it, you didn’t even know thatwas there, right?)d) To the right of the volume bar, an incomprehensible icon should haveappeared. Tap it.e) On the screen select “dialog” and “turn mirroring on”f) Say a little prayer.g) The iPhone screen should magically appear on your computer screen –PC or mac.Smartphone Setup 2Step 2 – Reflect your iPhone screen38
  • 39. 39a) On your laptop, launch the Reflector app
  • 40. 40b) On your iPhone, go to the homescreen.c) Double-tap the main button and scroll all theway to the left, past the music screen, until yousee the volume bar (admit it, you didn’t evenknow that volume control was there, right?)d) To the right of the volume bar, anincomprehensible icon should have appeared.Tap it.
  • 41. 41e) On the screen select the new networkand “turn mirroring on.”f) Say a little prayer.
  • 42. 42g) Voila! The iPhone screen should magically appear on your computer screen – PC or Mac.Little icon on your phonelets you know that you’restill connected to AirPlay’snetwork.
  • 43. a) On the phone image sharing laptop, launch a browser & startyour WebEx meeting and phone call.b) On WebEx, turn on the video cam option next to your name(this will capture the participant’s face). Now you should see2 windows: the participant’s face (small window) and thesmartphone screen (larger window). Share your screen.c) Drag images where you want them. Optional: Minimize thesmaller window with the face image so the participantdoesn’t have to watch themselves. WebEx: Participant menu > AssignPrivileges > Participants Tab > Uncheck Participant List checkbox > Assign buttond) Use static electricity to stick a piece of printer paper onto thescreen of the participant laptop*Smartphone Setup 2Step 3 – Fire up WebEx43
  • 44. Once WebEx** is running on the Image Sharingmachine….a) Launch a browser on the video recording laptop andjoin the meeting. Make sure everything looks howyou want it to look.b) Launch Camtasia (or your favorite screencapture/editing software) on the video recordinglaptop.c) Start recording your sessionSmartphone Setup 2Step 4 – Fire up the machine doing the screen capture44
  • 45. Networks get dropped. It’s a fact of life.• When your network gets dropped, sometimesReflector app has a “reluctance” to put the iPhoneback onto the network you just made.• To recover:– In iPhone Settings: Forget Network.– On Computer: Quit Reflector App.– On Computer: Restart Reflector App.– In iPhone: Follow procedure to mirror displaySmartphone Setup 2Helpful Tip Learned from(horrible)Experience45
  • 46. Setup options (I’m sure you can find more, too!)1. Smartphone Setup 1 can be used identicallyfor iPad testing setup2. Wireless SetupiPad46
  • 47. Pros & Cons of iPad Setup 1 versus 247Setup 1 Setup 2Pros ProsMajor: You can see ANYTHINGon the screenLook mom, no wires!Cool annotation featuresCons ConsCamera contraption can be alittle harder to deal with on aniPad than a smartphone.Can only share content in abrowser – not an app.Hands/fingers can get in theway of the screenCan’t minimize theparticipants’ facePerformance hit on iPad
  • 48. Schematic – Wireless iPad SetupiPad(w/ built-in webcam)VideoRecordingLaptop(w/ lots of free space)Running:-GoToMeeting-GTM VOIP for audio-GTM video for faceRunning:-GoToMeeting (attending)-CamtasiaObserversLaptop(attendingWebEx)Phoneor VOIPLab48
  • 49. iPadMaterials you need• 1 laptop• GoToMeeting (no, not WebEx, GoToMeeting.)• Screen capture software (we like Camtasia for Mac, Morae forPC)• iPad 2 or 3 (not iPad 1 because it doesn’t have a camera to captureparticipant’s face)• An app, website, or prototype49
  • 50. a) (Put the video recording computer aside for a minute.)b) On your iPad, launch GoToMeeting– Make sure your call is working by calling into the VOIPnumber from another phone– Enable the participant videocam feature (if you want it)– Share your screenc) Unbelievably – that’s it.iPad setupStep 2 – Fire up GoToMeeting50
  • 51. 51a) On your iPad, launch GoToMeeting. Enable the participantvideocam feature (if you want it). Share your screen
  • 52. 52Cool Annotation Features!
  • 53. iPad setupStep 3 – Fire up the machine doing the screencapture53Once GoToMeeting is running on the Image Sharingmachine….a) Launch a browser on the video recording laptop andjoin the meeting. Make sure everything looks howyou want it to look.b) Launch Camtasia (or your favorite screencapture/editing software) on the video recordinglaptop.c) Start recording your session
  • 54. • In theory, you can run Display Recorder Appon the iPad during your session to recordaudio, video and even users’ face, whilescreen-sharing the usability session.• Word to the wise: This is a seriousperformance hit on the iPad, but it’s beendone.– Tip: Forgo the video option in favor of betterperformance on the screen.Adventure Club for iPad54
  • 55. • Rather than sitting in the traditional“moderator” position looking over theparticipant’s shoulder, sit across fromthem, and watch the session via screensharingon your own (3rd) laptop.– Your business partners can IM spontaneousquestions to you.– You can take your own notes if you need to, fromright thereUber-Efficiency Mode55
  • 56. Questions?56

×