Remote User Testing


Published on

How it works; Advantages & Disadvantages of Remote User Testing; Time Estimates & Cost;

Published in: Design, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Projects are initiated by USER needs!Our biggest mistake as a SW provider is to not validate our assumptions, requirements and design against our target audience = users.
  • Highest cost-gain ratio of all methods
  • Goal is to observe user behavior.Goal can be achieved by testing tasks VS. use cases
  • - User model NOT EQUALS Organization view Requirement: Signup required to make purchase Input Form has high drop-out rate (investigate why) Form is filled out completely but never submitted
  • Correlate with our measurements mentioned earlier: effectiveness, efficiency and satisfaction.1.) Your boss told you to upload a raster file [actual file will be provided] to the SPF application (http://mikeg05/usfs)2.) Create an alias for the ‘Fire Potential 2006’ layer and reclassify it using the following values [list of values].3.) A high-level executive wants to see a detailed overview of the analysis “US 2006 National” you are working on. Send an email to that outlines the data source and scenarios used in this analysis.4.) Create an analysis that is based on a theme called ‘Political Sensitivity’ and includes the following data layers: [3 layers given]5.) Create a new analysis that is based on the data of an already existing analysis called “Redlands Tree Mortality”6.)Create a new scenario called “Spring 2006” as part of the already existing analysis “Redlands Tree Mortality” and add weights to the layers of this new scenario.
  • In summary: The implementation of User-centered Design and especially User Testing early in the project management cycle will result in:
  • Remote User Testing

    1. 1. Evaluation for 2008<br />Remote User Testing<br />Michael Gaigg<br />March, 18th 2009<br />
    2. 2. Agenda<br />Show how <br />User-centered Design helps teams to save:<br />Time<br />Money<br />Aggravation from clients<br />Usability testing early in the design process identifies:<br />Areas of confusion and frustration <br />Where the user can’t complete a task <br />While it is still cheap and easy to mitigate these problems<br />03/18/2009<br />Remote User Testing<br />2<br />
    3. 3. 03/18/2009<br />Remote User Testing<br />3<br />Agenda<br />User-centered Design<br />User Testing<br />Output (Qualitative & Quantitative)<br />Software & Usage<br />Time Estimates & Cost<br />Advantages & Disadvantages<br />
    4. 4. Introduction to User-centered Design<br />UCD is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process.<br />Its measures are:<br />Effectiveness (accuracy and completeness to achieve goals)<br />Efficiency (resources expended in relation to the accuracy and completeness)<br />Satisfaction (comfort and acceptability of the work system to its users)<br />03/18/2009<br />Remote User Testing<br />4<br />
    5. 5. User Testing is a UCD Method<br />03/18/2009<br />Remote User Testing<br />5<br />
    6. 6. Overview<br />User testing involves: <br />A design<br />A user (participant) <br />An observer (facilitator).<br />Observe user behavior & receive user feedback.<br />Test tasks (not functions).<br />03/18/2009<br />Remote User Testing<br />6<br />User<br />Design<br />Observer<br />
    7. 7. Qualitative Output<br />Understand mental model of the user (navigation).<br />Validate requirements and design assumptions (focus on important functionality early).<br />Discover non-functional design mistakes and areas of improvement (unforeseen user requirements).<br />Evaluate effectiveness of design elements.<br />03/18/2009<br />Remote User Testing<br />7<br />
    8. 8. Quantitative Output<br />Success rates / Error rates<br />Task time<br />Satisfaction ratings<br />03/18/2009<br />Remote User Testing<br />8<br />Task Success<br />Task Time<br />
    9. 9. User Testing Software<br />TechSmithMorae<br />Manager: $ 1,295.-<br />Bundle: $ 1,495.-<br />Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements.<br />03/18/2009<br />Remote User Testing<br />9<br />
    10. 10. Remote User Testing Software<br />TechSmithUserVue<br />$ 149.- / Month or<br />$ 1,495. / Year<br />Web-based app enabling a Facilitator to remotely observe (and record) a Participant using a phone line for communication. <br />Multiple Observers can passively join the Session and share their observations with the Facilitator.<br />03/18/2009<br />Remote User Testing<br />10<br />
    11. 11. The Session<br />Setting up a session.<br />Connecting (download of <br /> software bundle required).<br />Calling participant.<br />Starting the session.<br />Viewing (and recording) the participant.<br />Removing software bundle from computers of involved parties.<br />03/18/2009<br />Remote User Testing<br />11<br />
    12. 12. Time Estimates & Cost<br />Preparation (18 hours)<br />User screening: 8 hours<br />Task creation: 8 hours<br />Environment: 2 hours<br />User testing (10 hours)<br />2 hours per user<br />5 users per round<br />Post-test (32 hours)<br />Test report: 16 hours<br />Implementation: 12 hours<br />Communication: 4 hours<br />03/18/2009<br />Remote User Testing<br />12<br />Our time:<br />1 Round = 60 hours<br />2 Rounds = 110 hrs.<br />3 Rounds = 160 hrs.<br />+ User comps<br />+ Time additional<br /> observers<br />
    13. 13. Advantages of remote UT<br />Reduced (or no) travel time and expenses.<br />Higher exposure through easy screen sharing (managers can sneak in easily).<br />Actual user environment, familiar and comfortable.<br />Possibly fewer drop-outs.<br />03/18/2009<br />Remote User Testing<br />13<br />
    14. 14. Disadvantages of remote UT<br />Facilitator not physically present (degree of separation can be challenging).<br />Can&apos;t see facial expressions or non-verbal cues.<br />Difficult to build rapport and trust.<br />Difficult to control environment.<br />Possibly technical difficulties (firewall, etc.).<br />Setup and use of software or usability lab might be challenging and requires a liaison.<br />03/18/2009<br />Remote User Testing<br />14<br />
    15. 15. Summary<br />Increased usability<br />Higher degree of client satisfaction<br />Continued business = Higher revenues<br />Lesser client aggravation (“This is completely unusable”)<br />Assist PM during requirements and design<br />Focus and prioritize on important functionality<br />Identify unforeseen user requirements early!<br />Reduced costs<br />Training and service<br />Viewer Help-Desk calls<br />03/18/2009<br />Remote User Testing<br />15<br />