Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Remote User Testing


Published on

How it works; Advantages & Disadvantages of Remote User Testing; Time Estimates & Cost;

Published in: Design, Technology
  • Be the first to comment

Remote User Testing

  1. 1. Evaluation for 2008<br />Remote User Testing<br />Michael Gaigg<br />March, 18th 2009<br />
  2. 2. Agenda<br />Show how <br />User-centered Design helps teams to save:<br />Time<br />Money<br />Aggravation from clients<br />Usability testing early in the design process identifies:<br />Areas of confusion and frustration <br />Where the user can’t complete a task <br />While it is still cheap and easy to mitigate these problems<br />03/18/2009<br />Remote User Testing<br />2<br />
  3. 3. 03/18/2009<br />Remote User Testing<br />3<br />Agenda<br />User-centered Design<br />User Testing<br />Output (Qualitative & Quantitative)<br />Software & Usage<br />Time Estimates & Cost<br />Advantages & Disadvantages<br />
  4. 4. Introduction to User-centered Design<br />UCD is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process.<br />Its measures are:<br />Effectiveness (accuracy and completeness to achieve goals)<br />Efficiency (resources expended in relation to the accuracy and completeness)<br />Satisfaction (comfort and acceptability of the work system to its users)<br />03/18/2009<br />Remote User Testing<br />4<br />
  5. 5. User Testing is a UCD Method<br />03/18/2009<br />Remote User Testing<br />5<br />
  6. 6. Overview<br />User testing involves: <br />A design<br />A user (participant) <br />An observer (facilitator).<br />Observe user behavior & receive user feedback.<br />Test tasks (not functions).<br />03/18/2009<br />Remote User Testing<br />6<br />User<br />Design<br />Observer<br />
  7. 7. Qualitative Output<br />Understand mental model of the user (navigation).<br />Validate requirements and design assumptions (focus on important functionality early).<br />Discover non-functional design mistakes and areas of improvement (unforeseen user requirements).<br />Evaluate effectiveness of design elements.<br />03/18/2009<br />Remote User Testing<br />7<br />
  8. 8. Quantitative Output<br />Success rates / Error rates<br />Task time<br />Satisfaction ratings<br />03/18/2009<br />Remote User Testing<br />8<br />Task Success<br />Task Time<br />
  9. 9. User Testing Software<br />TechSmithMorae<br />Manager: $ 1,295.-<br />Bundle: $ 1,495.-<br />Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements.<br />03/18/2009<br />Remote User Testing<br />9<br />
  10. 10. Remote User Testing Software<br />TechSmithUserVue<br />$ 149.- / Month or<br />$ 1,495. / Year<br />Web-based app enabling a Facilitator to remotely observe (and record) a Participant using a phone line for communication. <br />Multiple Observers can passively join the Session and share their observations with the Facilitator.<br />03/18/2009<br />Remote User Testing<br />10<br />
  11. 11. The Session<br />Setting up a session.<br />Connecting (download of <br /> software bundle required).<br />Calling participant.<br />Starting the session.<br />Viewing (and recording) the participant.<br />Removing software bundle from computers of involved parties.<br />03/18/2009<br />Remote User Testing<br />11<br />
  12. 12. Time Estimates & Cost<br />Preparation (18 hours)<br />User screening: 8 hours<br />Task creation: 8 hours<br />Environment: 2 hours<br />User testing (10 hours)<br />2 hours per user<br />5 users per round<br />Post-test (32 hours)<br />Test report: 16 hours<br />Implementation: 12 hours<br />Communication: 4 hours<br />03/18/2009<br />Remote User Testing<br />12<br />Our time:<br />1 Round = 60 hours<br />2 Rounds = 110 hrs.<br />3 Rounds = 160 hrs.<br />+ User comps<br />+ Time additional<br /> observers<br />
  13. 13. Advantages of remote UT<br />Reduced (or no) travel time and expenses.<br />Higher exposure through easy screen sharing (managers can sneak in easily).<br />Actual user environment, familiar and comfortable.<br />Possibly fewer drop-outs.<br />03/18/2009<br />Remote User Testing<br />13<br />
  14. 14. Disadvantages of remote UT<br />Facilitator not physically present (degree of separation can be challenging).<br />Can&apos;t see facial expressions or non-verbal cues.<br />Difficult to build rapport and trust.<br />Difficult to control environment.<br />Possibly technical difficulties (firewall, etc.).<br />Setup and use of software or usability lab might be challenging and requires a liaison.<br />03/18/2009<br />Remote User Testing<br />14<br />
  15. 15. Summary<br />Increased usability<br />Higher degree of client satisfaction<br />Continued business = Higher revenues<br />Lesser client aggravation (“This is completely unusable”)<br />Assist PM during requirements and design<br />Focus and prioritize on important functionality<br />Identify unforeseen user requirements early!<br />Reduced costs<br />Training and service<br />Viewer Help-Desk calls<br />03/18/2009<br />Remote User Testing<br />15<br />