UR on the Cheap December 2012


Published on

This is the talk that Chris LaRoche and I gave at the Boston UXPA monthly meeting last night

Published in: Technology
1 Comment
  • A very well thought-out presentation with lots of great suggestions.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • So here we are in the typical tricked out usability lab. We’ve got a dedicated room with a mirrored window, a dedicated test system, speakerphone, camera, and microphone all hard-wired to an observation room next door.
  • As an employee, faster studies & results mean that you can conduct more studies. As a consultant, more studies can == more $$. As a f() of stakeholders’ abilities to implement your recommendations, that comes down to how effective you are as a practitioner – what value are you providing? So sometimes cheap == fast.
  • In the 1990’s, the business joke was you could “pick two” Good and Fast (but not cheap), cheap and fast (but not good). We’ll show you tools and methods so you can have all three.
  • If you only need to share your screen with the one system in conference room where your observers will be sitting, you can use Skype. It ‘s free, includes VoIP calling, and works on Mac, PC and mobile platforms.
  • And if you need to share your screen with multiple observers, you can try join.me. Like Skype, it can provide a VoIP phone number, but allows you to share your screen with up to 250 people. Free. And it works on PC, Mac, and Mobile platforms.
  • But for the Skype call to be free, everyone has to have a Skype-enabled device, like a computer or smart phone. What if they just have phones? Not a problem. You can set up a reservationless conference with free conference call dot com.
  • The observers can hear over the phone , but I recommend adding a webcam to your test system, to improve the sound quality of your recordings. You can get one for around $30. Because it’s likely your laptop mic isn’t great.
  • Now what about a note-taker? If you can’t afford to hire one, and you can’t afford one of those super-cool pens that records conversations while you write on special paper, there are options.
  • Say you already have an iPad – you can purchase a stylus for between $2 and $30. Live Note is an app that records while you write for $6. The notes play back as the same time as the audio, so you can hear what was being said when you wrote them.
  • Evernote is free, works on the Mac, PC and mobile platforms, and your notes get stored in the cloud. It will record the audio as you write – it just won’t be synchronized with the text as it plays back. That’s what the $6 for LiveNote gets you.
  • Rather than having a dedicated lab, you can get the same high quality results of a dedicated lab at a much lower cost and greater portability, so you can take the lab to where your participants work and live, or even conduct research from the comfort of your home.
  • http://www.reflectorapp.com/
  • Now, I have to be honest -- I do love the ease and convenience of having a permanent usability lab. But if you’re an independent or if your company doesn’t have the budget for a traditional usability lab, you probably have budget for these options.
  • Now, I have to be honest -- I do love the ease and convenience of having a permanent usability lab. But if you’re an independent or if your company doesn’t have the budget for a traditional usability lab, you probably have budget for these options.
  • UR on the Cheap December 2012

    1. 1. User Research on the Cheap The views Jen McGinn Chris LaRoche,expressed in this User Researcher Usability Consultantpresentation are my own and do Oracle MIT not necessarilyreflect the views of Oracle
    2. 2. Agenda๏ Recruiting & incentives on the cheap๏ Inexpensive methods๏ Agile testing on the cheap๏ Tools and technologies for under $100๏ Mobile testing on the cheap๏ Q &A
    3. 3. Camera Mirror Test systemMicrophone Connections to control room Speaker phone
    4. 4. PiP Monitor PiPMonitor Recording Sound system mixer Microphone Camera controller
    5. 5. What other costs are there?๏ Recruiting - Recruiters charge ~$150 per participant + an administrative fee for each study๏ Incentives - $100 - $150 per participant๏ Time - As an employee - As a consultant - As a function of whether or not the stakeholders can take action on your results
    6. 6. Remember the 1990’s? Good Fast Cheap
    7. 7. Recruiting๏ Your existing customers - Get contacts from sales or support - Licensee databases๏ Friends & family recruit - Send an email to your company alias - Include dates, general recruiting criteria, incentives, and description of the activity๏ Employees - Same process as friends & family
    8. 8. Recruiting: Creating a DB๏ User group meetings or conferences - By industry or product - Set up a table - Get permission, have them announce you - Hand out chocolates or branded swag - Gather contact info or business cards from people who are willing to opt into future research๏ Link on the company home page, in the forums, or at the end of the installer flow - Have them complete a screening survey
    9. 9. Considerations for Recruiting๏ Conflict with other departments or orgs๏ Do not contact lists๏ Legal responsibilities - Massachusetts 201 CMR 17.00 - Protection & storage - Maintaining and updating personal information
    10. 10. Incentives๏ Customers - Often not allowed to offer $$ - Company-branded pens, caps, water bottles, etc. under $20 if not government employees - $5 coffee cards๏ Employees - Will often opt in for no incentive - Company-branded swag - Lunch on us - $25 gift cards
    11. 11. Incentives๏ Depends on the (in)convenience of the research๏ Ways to drive down the inconvenience (and therefore cost of the incentive) - Go to them (eg. Engineers at work) - Go to them (eg. Social site users at home) - Turn it into a phone call - Remote meeting/test/feedback session using screen sharing software
    12. 12. Methods: Competitive Analysis๏ Compare features or tasks across products - Great for background data - No users required - Uncovers mental models, current experience - Great if competitors exist - Great if those products are obtainable - Great if the competitors are going after the same customers or business space
    13. 13. Methods: Data Mining๏ Customer support data (bugs filed) & helpdesk (live) - Great if you can get access to it - Great if it’s being collected & you know where to look for it๏ Usage (click stream) data - Great if you are allowed to gather usage data - Great if you can install code on the website - Great if data is complete enough that you can draw conclusions from it
    14. 14. Methods: Surveys ๏ Data Collection - Profile: when you want to get a snapshot of the composition of your current user population - Satisfaction: when you want to find out what works well and what doesn’t - Value: when you want to know what people find important ๏ Click stream analysis - Adding hidden code to the survey can turn it into an unmoderated testing tool: Demohttp://www.slideshare.net/Andrea_Evans/andrea-evans-upa- 2012-ignite
    15. 15. Methods: Interviews/Site Visit๏ In person - Great if you are within driving distance - Gives you rich, contextual data - Can view them in their environment, see their performance support tools - Recommendation: pair up with a colleague๏ On the phone - Great for quick scheduling and execution - Great for geographically distributed participants
    16. 16. Methods: Remote Testing๏ Moderated remote testing - Just like testing in the lab - Using an online meeting and conference call - Any length๏ Unmoderated remote testing - No facilitator - Participants screen themselves - Participants can be instructed to think aloud - Usually tasks under 30 minutes total - Fast! Same day results
    17. 17. Methods: Agile Testing ๏ RITE-Krug combines 2 “discount” test methods - RITE = Rapid Iterative Test Environment - Krug = Steve Krug’s approach in his books ๏ Formative: testing designs or code about to change ๏ Representatives from PM, Dev & Design are required to attend all sessions + debrief ๏ 3 – 4 participants within 24 hours ๏ Prototypes can change between participants ๏ Debrief happens ASAP after last participant ๏ No formal report, just observations & action items that stakeholders will take as a resultSubmitted for publication, Copyright 2012 Oracle
    18. 18. Usability Testing Lab Costs ๏ Facilities (participant room, control room, giant wall of mirror in between) ๏ Recording hardware (camera and mic) ๏ Web-conferencing software ๏ Recording software ๏ Observation system (computer, optional second monitor) ๏ Conference call
    19. 19. Portable Lab Speaker Microphone Test system phone Camera
    20. 20. Conference room /observation room Web conferencing software Recording softwareConference call Second monitor for observers
    21. 21. Screen Sharing 1:1 http://www.skype.com/intl/en-us/features/allfeatures/screen-sharing/
    22. 22. Screen Sharing 1:M https://join.me/ (About link)
    23. 23. Conference Callhttps://www.freeconferencecall.com/loginaccount.asp
    24. 24. Participant Camera & Michttp://www.logitech.com/en-us/product/hd- pro-webcam-c920?crid=34
    25. 25. Note-taking http://www.livescribe.com/en-us/
    26. 26. Audio Recording http://store.sony.com/c/Recorders-and- Microphones/en/c/S_Voice_Recorders
    27. 27. Note-taking: Live Note Bamboo stylushttp://itunes.apple.com/us/app/live-notes/id383300181?mt=8#
    28. 28. Note-taking: EverNote http://evernote.com/download/
    29. 29. Web join.mecam FreeConferencecall.com
    30. 30. Analyticshttp://www.google.com/analytics/features/index.html
    31. 31. Heat Maps (~$100) http://www.crazyegg.com/
    32. 32. Unmoderated Testing http://www.usertesting.com/
    33. 33. Mobile: Before the Testhttp://4ourth.com/wiki/4ourth%20Mobile%20Touch%20Template
    34. 34. Mobile Screen-casting http://www.reflectorapp.com/
    35. 35. Mobile Testinghttp://www.ipevo.com/prods/Point-2-View-USB-Camera
    36. 36. Hardware Review < $100 MobileTemplate 4th Mobile ($5) Mobile Camera IPevo ($69) Sony Digital Recorder Audio recording ($29 - 79) Audio-video recording Logitech webcam ($30 – 100)Note taking & audio recording LiveScribe ($89 and up) Note-taking input iPad Stylus ($2-30)
    37. 37. Software Review Free < or ~$100 Web conferencing Skype, join.me VoIP Skype, join.me Conference call for FreeConferenceCall.com Skype, join.me phones Screen recording QuickTime Player (Mac) SnagIt ($50)Note taking & audio Evernote Live Note ($6) recording Survey tool (assuming you UserTesting.comUnmoderated testing already pay for one) ($39)Mobile device screen Reflector casting ($15 or $50) Analytics Google Analytics NewEgg ($102)
    38. 38. Questions?
    39. 39. The viewsexpressed in this Thank you!presentation are Jen McGinn my own and do not necessarily jen dot mcginn at oracle dot comreflect the views Chris LaRoche of Oracle c dot laroche at neu dot edu