Remote User Testing

1,399 views
1,317 views

Published on

How it works; Advantages & Disadvantages of Remote User Testing; Time Estimates & Cost;

Published in: Design, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,399
On SlideShare
0
From Embeds
0
Number of Embeds
236
Actions
Shares
0
Downloads
24
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Projects are initiated by USER needs!Our biggest mistake as a SW provider is to not validate our assumptions, requirements and design against our target audience = users.
  • Highest cost-gain ratio of all methods
  • Goal is to observe user behavior.Goal can be achieved by testing tasks VS. use cases
  • - User model NOT EQUALS Organization view Requirement: Signup required to make purchase Input Form has high drop-out rate (investigate why) Form is filled out completely but never submitted
  • Correlate with our measurements mentioned earlier: effectiveness, efficiency and satisfaction.1.) Your boss told you to upload a raster file [actual file will be provided] to the SPF application (http://mikeg05/usfs)2.) Create an alias for the ‘Fire Potential 2006’ layer and reclassify it using the following values [list of values].3.) A high-level executive wants to see a detailed overview of the analysis “US 2006 National” you are working on. Send an email to mgaigg@esri.com that outlines the data source and scenarios used in this analysis.4.) Create an analysis that is based on a theme called ‘Political Sensitivity’ and includes the following data layers: [3 layers given]5.) Create a new analysis that is based on the data of an already existing analysis called “Redlands Tree Mortality”6.)Create a new scenario called “Spring 2006” as part of the already existing analysis “Redlands Tree Mortality” and add weights to the layers of this new scenario.
  • In summary: The implementation of User-centered Design and especially User Testing early in the project management cycle will result in:
  • Remote User Testing

    1. 1. Evaluation for 2008<br />Remote User Testing<br />Michael Gaigg<br />March, 18th 2009<br />
    2. 2. Agenda<br />Show how <br />User-centered Design helps teams to save:<br />Time<br />Money<br />Aggravation from clients<br />Usability testing early in the design process identifies:<br />Areas of confusion and frustration <br />Where the user can’t complete a task <br />While it is still cheap and easy to mitigate these problems<br />03/18/2009<br />Remote User Testing<br />2<br />
    3. 3. 03/18/2009<br />Remote User Testing<br />3<br />Agenda<br />User-centered Design<br />User Testing<br />Output (Qualitative & Quantitative)<br />Software & Usage<br />Time Estimates & Cost<br />Advantages & Disadvantages<br />
    4. 4. Introduction to User-centered Design<br />UCD is a design philosophy and a process in which the needs, wants, and limitations of the end user of an interface or document are given extensive attention at each stage of the design process.<br />Its measures are:<br />Effectiveness (accuracy and completeness to achieve goals)<br />Efficiency (resources expended in relation to the accuracy and completeness)<br />Satisfaction (comfort and acceptability of the work system to its users)<br />03/18/2009<br />Remote User Testing<br />4<br />
    5. 5. User Testing is a UCD Method<br />03/18/2009<br />Remote User Testing<br />5<br />
    6. 6. Overview<br />User testing involves: <br />A design<br />A user (participant) <br />An observer (facilitator).<br />Observe user behavior & receive user feedback.<br />Test tasks (not functions).<br />03/18/2009<br />Remote User Testing<br />6<br />User<br />Design<br />Observer<br />
    7. 7. Qualitative Output<br />Understand mental model of the user (navigation).<br />Validate requirements and design assumptions (focus on important functionality early).<br />Discover non-functional design mistakes and areas of improvement (unforeseen user requirements).<br />Evaluate effectiveness of design elements.<br />03/18/2009<br />Remote User Testing<br />7<br />
    8. 8. Quantitative Output<br />Success rates / Error rates<br />Task time<br />Satisfaction ratings<br />03/18/2009<br />Remote User Testing<br />8<br />Task Success<br />Task Time<br />
    9. 9. User Testing Software<br />TechSmithMorae<br />Manager: $ 1,295.-<br />Bundle: $ 1,495.-<br />Morae Manager uses the collected data (observation markers and notes, video, keyboard and mouse inputs) to analyze and calculate task times, error quotes and other common measurements.<br />03/18/2009<br />Remote User Testing<br />9<br />
    10. 10. Remote User Testing Software<br />TechSmithUserVue<br />$ 149.- / Month or<br />$ 1,495. / Year<br />Web-based app enabling a Facilitator to remotely observe (and record) a Participant using a phone line for communication. <br />Multiple Observers can passively join the Session and share their observations with the Facilitator.<br />03/18/2009<br />Remote User Testing<br />10<br />
    11. 11. The Session<br />Setting up a session.<br />Connecting (download of <br /> software bundle required).<br />Calling participant.<br />Starting the session.<br />Viewing (and recording) the participant.<br />Removing software bundle from computers of involved parties.<br />03/18/2009<br />Remote User Testing<br />11<br />
    12. 12. Time Estimates & Cost<br />Preparation (18 hours)<br />User screening: 8 hours<br />Task creation: 8 hours<br />Environment: 2 hours<br />User testing (10 hours)<br />2 hours per user<br />5 users per round<br />Post-test (32 hours)<br />Test report: 16 hours<br />Implementation: 12 hours<br />Communication: 4 hours<br />03/18/2009<br />Remote User Testing<br />12<br />Our time:<br />1 Round = 60 hours<br />2 Rounds = 110 hrs.<br />3 Rounds = 160 hrs.<br />+ User comps<br />+ Time additional<br /> observers<br />
    13. 13. Advantages of remote UT<br />Reduced (or no) travel time and expenses.<br />Higher exposure through easy screen sharing (managers can sneak in easily).<br />Actual user environment, familiar and comfortable.<br />Possibly fewer drop-outs.<br />03/18/2009<br />Remote User Testing<br />13<br />
    14. 14. Disadvantages of remote UT<br />Facilitator not physically present (degree of separation can be challenging).<br />Can&apos;t see facial expressions or non-verbal cues.<br />Difficult to build rapport and trust.<br />Difficult to control environment.<br />Possibly technical difficulties (firewall, etc.).<br />Setup and use of software or usability lab might be challenging and requires a liaison.<br />03/18/2009<br />Remote User Testing<br />14<br />
    15. 15. Summary<br />Increased usability<br />Higher degree of client satisfaction<br />Continued business = Higher revenues<br />Lesser client aggravation (“This is completely unusable”)<br />Assist PM during requirements and design<br />Focus and prioritize on important functionality<br />Identify unforeseen user requirements early!<br />Reduced costs<br />Training and service<br />Viewer Help-Desk calls<br />03/18/2009<br />Remote User Testing<br />15<br />

    ×