Narayanan Thiruvengadam, profile picture
Uploaded byNarayanan Thiruvengadam
PDF, PPTX3,435 views

Usability_Evaluation

Usability evaluation is aimed at finding usability problems in user interfaces to provide qualitative and quantitative data about user behavior. There are two main types of evaluations: formative evaluations conducted during design to gather feedback, and summative evaluations conducted after completion to identify problems. Common methods include heuristic evaluation where experts judge compliance with usability principles, cognitive walkthroughs to simulate user behavior, and usability testing where representative users perform tasks. Usability evaluations provide benefits for both technology improvements and business outcomes.

Embed presentation

Download as PDF, PPTX
measure with users USABILITY EVALUATION USABILITY EVALUATION methods and techniques Narayanan Thiruvengadam tnarayan@gmail.com2013
measure with users USABILITY EVALUATION Usability Definition Usability Evaluation Overview • Formative Vs Summative • History Usability Inspection Methods • Heuristic Evaluation - Overview - Heuristics Principles - What is a Usability Review? - When & How of a Usability review - What to include in a review? - Computing severity ratings - Advantages of a Usability Review - Things to Remember • Cognitive Walkthrough - Overview - How To Do It • Pluralistic Walkthrough - Overview Usability Testing Overview • Techniques to Gather Data • User Testing- How to Do It • Quick Tips User- Centered Design Process Usability Evaluation Benefits • Technology • Business Index
measure with users USABILITY EVALUATION “The extent to which the product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Usability Definition International Organization for Standardization (ISO) Usability 5 Quality Components/Usability Measure
measure with users USABILITY EVALUATION Usability Evaluation is aimed at finding usability problems in the user interface/ System, providing qualitative and quantitative data about people and their behavior or attitudes. Usability evolution are accomplished by identifying representative users, representative tasks and developing a procedure for capturing usability problems with the interface/system. During the design/testing/development cycle of software development. There are two types of user evaluations are carried out • Formative evaluations • Summative evaluations “A usability defect is a product characteristic that makes it difficult or unpleasant for users to accomplish tasks supported by the product.” “Kahn and Prail” Overview
measure with users USABILITY EVALUATION Conducted during design as part of an iterative process, allowing the designers to see how their target user/audience responds to the interface. These methods provide qualitative data. Heuristic Evaluation, User Interface Inspections, Thinking-aloud Testing, Pluralistic Usability walkthroughs, and Cognitive Walkthroughs are some methods that can be used for formative evaluation. Conducted after completion of products, used in product comparisons, to identify problems prior to a redesign. Summative studies are designed to yield quantitative data that can be tested for statistical significance. Formative Summative "When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative.“ - Robert Stakes Formative Vs Summative
measure with users USABILITY EVALUATION Jean Scholtz, National Institute of Standards and Technology Timeline for usability evaluations in the last 30 years. Users were first used as the source of usability feedback but models have been also used for over 20 years. Expert feedback was developed in heuristic reviews and cognitive walkthroughs and has been used since the early 90s. History
measure with users USABILITY EVALUATION Heuristic Evaluation/Usability Review: is the most informal method and involves having usability specialists/experts to judge whether interface design established usability principles. (the “heuristics” ) Cognitive Walkthrough: is a step-by-step method exploring/inspecting user interface with a set of questions and creating success/failure stories on the users’ behavior to simulate users problem, understandability and ease of learning. Pluralistic Walkthroughs: uses group meetings where users, developers, and usability expert step through a scenario, discussing each dialogue element. Formal Usability Inspections: Evaluators walkthrough tasks with the user's goals and purpose in mind, similar to cognitive walkthroughs, although the emphasis is less on cognitive theory and more on encountering usability problems. Usability Inspection Methods There’s no such thing as a “user error”! Cognitive: thought process toward awareness or knowledge
measure with users USABILITY EVALUATION Feature Inspection: lists sequence of features used to accomplish typical tasks, checks for long sequences, cumbersome steps, steps that would not be natural for users to try, and steps that require extensive knowledge/ experience in order to assess a proposed feature set. Consistency Inspection: has designers representing multiple projects inspect an interface to see whether it does things in the same way as their own designs. Standards Inspection: has an expert on some interface standard ‘inspect the interface for compliance. Usability Inspection Methods There’s no such thing as a “user error”!
measure with users USABILITY EVALUATION Heuristic evaluation is a method for finding the usability problems in a user interface design so that they can attend to as part of an iterative design process. Heuristic evaluation involves having a set of evaluators (3 - 5) examine the interface and judge its compliance with recognized usability principles (“the heuristics") Heuristic evaluation Evaluators VS Problem found – report Nielsen’s
measure with users USABILITY EVALUATION Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation 1 2 3 4 5 6 7 8 9 10 The Heuristics Principles Nielsen’s Heuristics Ref. Link: http://www.useit.com/papers/heuristic/heuristic_list.html
measure with users USABILITY EVALUATION A Usability Review (more formally known as ‘Heuristic Evaluation’ is a technique for identifying usability issues. A Usability Review: • Is cheaper to conduct than formal usability testing • Can be completed in a very short period • Can be conducted at any stage of the design. • Usability Reviews are somewhat subjective, because no real users are involved. Objectives of a review: • Identify likely trouble-spots • Recommend directions and concepts for improved design (both tactical and strategic) • Finds ‘obvious’ problems that are NOT really obvious. What is a Usability Review? Usability Review
measure with users USABILITY EVALUATION Usability Review When ? A Usability Review is particularly appropriate if an application has a large number of serious usability problems, or if an application is not sufficiently mature for usability testing with real users. In a hostile environment, a Usability Review is open to the accusation of being only one opinion against the opinion of others. In such circumstances, consider usability testing instead. How? At least two people should independently review the application, to maximize the number of issues identified. In general, avoid being given a walk-through of the application before beginning, as it is best to approach the task with a fresh and un-biased viewpoint. However, you should know: • The purpose of the application • The characteristics of the intended users and their levels of domain knowledge • The context of use. Usability Review
measure with users USABILITY EVALUATION Before you start, create a working document for your notes. This allows you to capture first impressions, which you may not readily recall once you have begun working with the application. Once you begin your review, use a standard notation for capturing issues. For each issue, note: • An exact description of the issue. Write this in the voice that you will use in your report, to facilitate subsequent 'cutting- and-pasting' with a minimum of re-writing. • Occurrence point of the issue • The usability principle (heuristic) violated/ standard in interface • An estimate of severity. • A recommendation What to include in a Review Usability Review
measure with users USABILITY EVALUATION • Frequency with which the problem occurs – is it common or rare? • Impact of the problem if it occurs – will it be easy or difficult for users to overcome? • Persistence of the problem – is it a one-time problem that users can overcome once they know about it or will users repeatedly be bothered by the problem? Computing Severity Ratings Usability Review
measure with users USABILITY EVALUATION Navigation & ease of information retrieval Maximize the ease with which site visitors can find both the information they require and the content you want them to access Page layout & calls-to-action Make sure there are enough calls-to-actions and that they're appropriate and noticeable Persuasiveness & cross-selling Ensure the site and individual pages persuade site visitors to do what you want them to do (make a purchase, contact you etc.) Removing obstacles Make it as easy as possible for people to do what they - and more importantly you - want them to do Content, marketing & brand Ensure that the site's content engages and persuades users and that the site's USPs are communicated clearly and effectively Search Enhance the design and performance of the site's search engine so users can find what they want with ease Advantages of a Usability Review Usability Review
measure with users USABILITY EVALUATION • Precisely why you consider each issue to be relevant from a usability perspective - avoid personal opinions that may be grounded in prejudice. • Acknowledge the positive aspects of the system under review - remember that some readers may be hostile to or upset by your findings. • Give careful, meaningful and reasoned comments. • Provide clear recommendations. E.g.: Decide whether the application can be 'patched', or whether more fundamental changes are required. • Summarize your findings in the beginning of the report. Do not expect all readers to have time to go through the issues in detail. • Combine and print the issues from each reviewer, and group like issues together. This should give you the basic structure for the body of your report. • Review your working notes to ensure that all the issues you noted have been included as issues and not as recommendations. • Peer review the checklists to see if all reviewers are in agreement, and examine any discrepancies. • Do NOT make a mockery of the design of the application. Things to Remember Usability Review
measure with users USABILITY EVALUATION Its a more explicitly detailed procedure where evaluators asks himself a set of questions about the expectations of users’ behavior to assesses interface design and come up with a success/failure stories to simulate user’s problem and solving process at each step through the dialogues. This technique is best used in the design stage of development. The evaluators may include usability experts, designers, software developers or people from marketing, etc. Cognitive Walkthrough Cognitive: thought process toward awareness or knowledge
measure with users USABILITY EVALUATION Prototype Prepare Low / High fidelity prototype Choose a user ( representative user) Explore their skill levels with both technology and the problem domain Evaluation Tasks Define the Goal & Tasks to be achieved in the prototype Action Sequence Determine the step by step actions the user ideally should take to complete the task. Every step should be explicitly listed 1 Cognitive Walkthrough How To Do It
measure with users USABILITY EVALUATION Task Walkthrough The modulator walk through (discuss) the tasks with respect to the interface/mockups and action sequences/ scenarios with the evaluators/ participants and create a credible success /failure story for each individual step. Ask the following questions, for each step, while looking at the interface and create a credible story based around these questions • Will the user try to achieve the right effect? • Will the user notice that the correct action is available? • Will the user associate the correct action with the effect that user is trying to achieve? • If the correct action is performed, will the user see that progress is being made towards their goal? 2 Cognitive Walkthrough How To Do It
measure with users USABILITY EVALUATION Reports (Actionable Data ) Create report with actionable data include steps where the user might not know what to do, steps where they don't know if they were successful, and gaps in the design, including success/failure stories Cognitive Walkthrough How To Do It 3
measure with users USABILITY EVALUATION In the design stage, when paper prototype is available, a group of users, developers, and usability expert meet together to step through a set of tasks, discussing and evaluating the usability of a system. The more people looking for problems, the higher the probability of finding problems. Five Characteristics • Inclusion of representative users, product developers, and Usability professionals • The application’s screens are presented (as hard copy) in the same order as they would appear to the user • All participants are asked to assume the role of the user • Participants write down what actions they, as users, would take for each screen before the group discusses the screens • When discussing each screen, the representative users speak first Pluralistic Walkthrough
measure with users USABILITY EVALUATION In a usability evaluation, you are predicting the problems or successes that users will have with the system/interface. A usability test tells you whether your predictions are valid and also helps to find new usability problems. The aim of the Usability testing is to gather qualitative user data; combines interviewing and observation of a user in a real-time. Usability testing is especially effective in determining: • Naming (section/button labels/Words) • Organization (information grouped into meaningful categories/ better placed) • First-time use and discoverability • Effectiveness • Emotional Response Techniques popularly used to gather data during a usability test include think aloud protocol, Co-discovery Learning, eye tracking, Remote Testing, Question-asking Protocol, Clickstream analysis, Facial Recognition and Psychometric Data Gathering and Analysis Overview
measure with users USABILITY EVALUATION Think Aloud Protocol During the course of a usability test, the test users are asked to verbalize/speak out their thoughts, feelings, and opinions while interacting with the system. It is very useful in capturing a wide range of cognitive/ mental processes of activities. Co-discovery Learning During a usability test, two test users attempt to perform tasks together while being observed. They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product. They are encouraged to explain what they are thinking about while working on the tasks. Techniques to Gather Data
measure with users USABILITY EVALUATION Eye Tracking Eye tracking is the process of measuring either the point of gaze ("where we are looking") or the motion of an eye relative to the head. Watching what people are looking at adds a level of interest to usability studies. Following the eye at work makes you feel as though you are in the user’s head, thinking with him and able to understand how user looks the interface. Eye tracking solutions Eye Tracking is a combined software and hardware package designed to capture the conscious and unconscious eye gaze movement of a respondent. Eye Tracking is a Unique and Reliable Method for Assessing Design Impact or Usability of websites, print media, package design, virtual store shelving, etc. Hardware Solutions & Support noldus.com, smi.de , faceLAB, Tobii TX300 Eye Tracker, EyeTech, etc. Techniques to Gather Data
measure with users USABILITY EVALUATION Eye Tracking Heatmap
measure with users USABILITY EVALUATION Remote Testing Remote usability testing is used when testers are separated in space and/or time from the participants. This means that the testers cannot observe the testing process directly and that the participants are usually not in a formal usability laboratory. Another is different-time different-place testing, where the user's test session is guided and logged through a special piece of software. Tools : Morae, usertesting.com, intuitionhq.com, CrazyEgg, etc. Question-Asking Protocol During a usability test, besides letting the test users to verbalize their thoughts as in the thinking aloud protocol, the testers prompt them by asking direct questions about the product, in order to understand their mental model of the system and the tasks, and where they have trouble in understanding and using the system. Techniques to Gather Data
measure with users USABILITY EVALUATION Clickstream Clickstream analysis/clickstream analytics is the process of collecting, analyzing, and reporting aggregate data about which pages visitors visit in what order There are two levels of clickstream analysis, traffic analysis and e-commerce analysis. Facial Recognition facial recognition technique are used to capture information about the shape of a face and this information is then used for identification and the verification of identity. Psychometric Data Gathering and Analysis Psychometric(Psychology + Metrics) is the study of behavior and mental processes. Psychometric testing or psychometric analysis is to measure or assess the psychology of a person. Techniques to Gather Data Clicksteam Facial Recognition
measure with users USABILITY EVALUATION Plan and Strategize on: • The Goals for the test • The Screens & System for the test • Tasks for the user to be tested • Data to be collected and analyzed • Methodology to collect the data • Location of the test • Recording Setup • Role of the participants Identify Test Users User should be as representative as possible of the intended users of the system. One should take additional care to involve average users. If a larger group of testers are to be engaged, one should select users from several different subpopulations to cover the main different categories of expected users. (Novice to Expert users) 2 “You’re testing the system not the User Knowledge “ 1How To Do It User Testing
measure with users USABILITY EVALUATION Test Location Usability testing is preferably conducted in User’s environment. Else, out of fear of seeming dumb, incompetent, many users may avoid talking about software behaviors that they find problematic. Design Usability Test • Pre test user briefing points, questions & confidential agreements • Participant Tasks • Goals & Tasks • Test User Questions • Post Test Questions & Survey Pilot Test Pilot testing can also be used to refine the experimental procedure and to clarify the definitions, Test tasks, Test plan, Questionnaires. How To Do It User Testing 3 4
measure with users USABILITY EVALUATION Perform User Testing (UT) • Explain the User about the aim of the test • Explain User if any recording, keystroke logging, or other monitoring that is used • Explain and get the confidential agreements if required • Brief the goal and tasks which the tester needs to perform • Choose appropriate techniques/method to gather data ( Think aloud method/Eye Tracking/Question-Asking Protocol) • Define the roles for each participant: • User: Performs the task as planned • Observer: Takes note of every action and reactions of the user • Moderator: Probes with questions and helps the user to overcome any system limitations • Upon completion of the test, analyze the data captured and identify Usability problems How To Do It User Testing 5
measure with users USABILITY EVALUATION • Observe, be quiet, watch, understand • Don't explain, suggest, defend the design • Don't ask the test user's opinion • Help user to overcome the limitations of the prototype • Hand out the test tasks one at a time • Keep a relaxed atmosphere in the test room • Avoid disruptions • Never indicate in any way that the user is making mistakes or is too slow • If necessary, stop the test if it becomes too unpleasant • End by stating that the user has helped you find areas of improvement • Never report results in such a way that individual users can be identified • Cash or an appropriate 'Thank You' gift User Testing – Quick Tips While UT After UT
measure with users USABILITY EVALUATION INITIATEBUILDTEST DELIVER & MONITOR MODEL DISCOVERYEVALUATE FRAME WORK User Centered Design
measure with users USABILITY EVALUATION System / Interface Design • Usability evaluation will help you in identifying system’s Learnability, Efficiency, Memorability, Errors & Users Satisfaction. • Risks are managed and reduced by introducing usability evaluation in early stage of the software development cycle. • Evaluation provides qualitative feedback and helps to improve your system / interface design. • Helps you identify issues and potential problems before the development starts. • Help to discover real needs and tasks of the user’s in the early stage of the design process. • Decrease user acclimation (accustomed to a new environment) time, errors and increased user’s productivity. • Provide a competitive advantage • Help to analyze user’s goals, which results in the business meeting its targets. • Will help in reducing the rework and help in meeting customer requirements (80 percent of software rewrites are due to the fact that important functionality was missed the first time) • Help to build customer and users confidence.
measure with users USABILITY EVALUATION Business • Gain a competitive advantage. • Generate rapid ROI and gain quicker time to market by avoiding big changes late in development. • An easy-to-use system means fewer calls to customer service • Early prediction of potential problems will reduce the project lifecycle cost • Provide more follow-on business due to satisfied customers & users • Look better, demo better and sell better, as they are more intuitive to use, faster and more effective results in profitability and revenue • Loyal customers provide free word-of-mouth exposure results in the business revenue
measure with users USABILITY EVALUATION Usability Inspection Methods- Jakob Nielsen, Robert L Mack Usability Engineering- Jakob Nielsen Measuring The User Experience- Tom Tullis, Bill Albert About Face 3.0- Alan Cooper, Robert Reimann, and Dave Cronin Beyond The Usability Lab - Bill Albert, Tom Tullis, Donna Tedesco usability.gov useit.com usabilityfirst.com stcsig.org wikipedia.org usabilityhome.com webcredible.co.uk Enhancing the Explanatory Power of Usability Heuristics – Jakob Nielsen Usability inspection methods - research and practice- David G. Novick Usability Inspection Methods- Jakob Nielsen The Pluralistic Usability Walk-Through Method- S. Riihiaho Eyetracking Methodology- Kara Pernice and Jakob Nielsen The cognitive walkthrough method: A practitioner’s guide. In Nielsen, J., and Mack, Usability inspection methods. New York, NY: John Wiley & Sons, Inc. Wharton, C., Rieman, J., Lewis, C., and Polson, P. (1994). Courtesy : istockphoto.com Books Online Whitepapers USABILITY EVALUATION References
measure with users USABILITY EVALUATION USABILITY EVALUATION measure with users Narayanan Thiruvengadam tnarayan@gmail.com2013

More Related Content

PDF
Cognitive elements of an effective UI/UX design
byShabnamShahfar
 
PPTX
Ten Usability Heuristics by Jakob Nielsen.pptx
bysharmiladevi941
 
PDF
An Introduction to Usability Testing
byLennart Overkamp
 
PPTX
Design process design rules
byPreeti Mishra
 
PPTX
Usability test
byAnsviaLab
 
PDF
Benchmarking Using SUS
byCake and Arrow
 
PDF
Usability vs. User Experience: What's the difference?
byDomain7
 
PDF
Usability Testing 101 - an introduction
byElizabeth Snowdon
 
Cognitive elements of an effective UI/UX design
byShabnamShahfar
 
Ten Usability Heuristics by Jakob Nielsen.pptx
bysharmiladevi941
 
An Introduction to Usability Testing
byLennart Overkamp
 
Design process design rules
byPreeti Mishra
 
Usability test
byAnsviaLab
 
Benchmarking Using SUS
byCake and Arrow
 
Usability vs. User Experience: What's the difference?
byDomain7
 
Usability Testing 101 - an introduction
byElizabeth Snowdon
 

What's hot

PDF
Ux design process
byJunying Chang
 
PDF
Research Methods in UX
byBrad Orego (he/they)
 
PDF
Simple Steps to UX/UI Web Design
byKoombea
 
PPTX
evaluation techniques in HCI
bysawsan slii
 
PDF
UX & UI Design - Differentiate through design
byDMI
 
PDF
UX Prototyping (UXiD) - Slide by Anton Chandra and Bahni Mahariasha
byAnton Chandra
 
PPTX
i/o extended: Intro to <UX> Design
byGDGKuwaitGoogleDevel
 
PDF
Introduction to User Experience Design
byRavi Bhadauria
 
PDF
UX/UI Design 101
byJasmine Phan
 
PPTX
Conducting usability test
byHaider Ali Warraich
 
PPT
User centered Design
byShalini Shingari
 
PPT
HCI 3e - Ch 5: Interaction design basics
byAlan Dix
 
PPT
Lecture4 requirement engineering
byShahid Riaz
 
PDF
Human Computer Interaction Evaluation
byLGS, GBHS&IC, University Of South-Asia, TARA-Technologies
 
PPTX
Unit2 hci
bypradeepgupta266
 
PPS
Testing techniques
byRaginiRohatgi
 
PDF
What is a User Experience?
byDotinum
 
PPT
HCI 3e - Ch 12: Cognitive models
byAlan Dix
 
PDF
UX RULES: 10 ESSENTIAL PRINCIPLES
byJeremy Robinson
 
PPTX
Heuristic evaluation
bySem Gebresilassie
 
Ux design process
byJunying Chang
 
Research Methods in UX
byBrad Orego (he/they)
 
Simple Steps to UX/UI Web Design
byKoombea
 
evaluation techniques in HCI
bysawsan slii
 
UX & UI Design - Differentiate through design
byDMI
 
UX Prototyping (UXiD) - Slide by Anton Chandra and Bahni Mahariasha
byAnton Chandra
 
i/o extended: Intro to <UX> Design
byGDGKuwaitGoogleDevel
 
Introduction to User Experience Design
byRavi Bhadauria
 
UX/UI Design 101
byJasmine Phan
 
Conducting usability test
byHaider Ali Warraich
 
User centered Design
byShalini Shingari
 
HCI 3e - Ch 5: Interaction design basics
byAlan Dix
 
Lecture4 requirement engineering
byShahid Riaz
 
Human Computer Interaction Evaluation
byLGS, GBHS&IC, University Of South-Asia, TARA-Technologies
 
Unit2 hci
bypradeepgupta266
 
Testing techniques
byRaginiRohatgi
 
What is a User Experience?
byDotinum
 
HCI 3e - Ch 12: Cognitive models
byAlan Dix
 
UX RULES: 10 ESSENTIAL PRINCIPLES
byJeremy Robinson
 
Heuristic evaluation
bySem Gebresilassie
 

Viewers also liked

PPTX
Usability modeling and measurement
byXBOSoft
 
PPT
Dan Brown's Communicating Design Presentation to DOE
byDan Brown
 
PPT
Usability Evaluation in Educational Technology
byAlaa Sadik
 
PPT
Usability evaluations (part 2)
byAndres Baravalle
 
PPTX
Usability evaluation methods (part 2) and performance metrics
byAndres Baravalle
 
PPTX
Planning and usability evaluation methods
byAndres Baravalle
 
PDF
A Review on Usability Features for Designing Electronic Health Records
byIvan Mauricio Cabezas Troyano
 
Usability modeling and measurement
byXBOSoft
 
Dan Brown's Communicating Design Presentation to DOE
byDan Brown
 
Usability Evaluation in Educational Technology
byAlaa Sadik
 
Usability evaluations (part 2)
byAndres Baravalle
 
Usability evaluation methods (part 2) and performance metrics
byAndres Baravalle
 
Planning and usability evaluation methods
byAndres Baravalle
 
A Review on Usability Features for Designing Electronic Health Records
byIvan Mauricio Cabezas Troyano
 

Similar to Usability_Evaluation

PDF
Evaluation methods
byEva Durall
 
PPTX
Design and Evaluation techniques unit 5
byKrishnaVeni451953
 
PPTX
Evaluation in hci
bysajid rao
 
PPT
Usability and evolution Human computer intraction.ppt
bySyedGhassanAzhar
 
PDF
Unit5Unit1-Part1Unit1-Part1Unit1-Part1Unit1-Part1.pdf
byVaghaniSmit
 
PPT
Uid formative evaluation
byPen Lister
 
PDF
USABILITY ENGINEERING LIFE CYCLE IN HCI.pdf
byFentyEkaMuzayyanaAgu
 
PPTX
Lesson 6 - HCI Evaluation Techniques.pptx
bycjanegenovana
 
PPTX
evaluation -human computer interaction.pptx
byNanMdSahar
 
PPTX
USABILITY TESTING TO IMPROVE USER EXPERIENCE
byMalang QA Community
 
PDF
Principles of Health Informatics: Evaluating medical software
byMartin Chapman
 
PPTX
Usability testing
byArchana Krushnan
 
PDF
195
byvivatechijri
 
PDF
THE USABILITY METRICS FOR USER EXPERIENCE
byvivatechijri
 
PPTX
Usability Inspection, Human computer intraction.pptx
bySyedGhassanAzhar
 
PPT
Tech. Writing Usability Presentation
bymhobren
 
PPTX
ICS3211 lecture 10
byVanessa Camilleri
 
PPT
classmar2.ppt
byRangothriSreenivasaS
 
PPSX
Majestic MRSS Usability Engineering
byMajestic MRSS
 
PPTX
L7 Usability testing lecture of usability
byjlajizhangh
 
Evaluation methods
byEva Durall
 
Design and Evaluation techniques unit 5
byKrishnaVeni451953
 
Evaluation in hci
bysajid rao
 
Usability and evolution Human computer intraction.ppt
bySyedGhassanAzhar
 
Unit5Unit1-Part1Unit1-Part1Unit1-Part1Unit1-Part1.pdf
byVaghaniSmit
 
Uid formative evaluation
byPen Lister
 
USABILITY ENGINEERING LIFE CYCLE IN HCI.pdf
byFentyEkaMuzayyanaAgu
 
Lesson 6 - HCI Evaluation Techniques.pptx
bycjanegenovana
 
evaluation -human computer interaction.pptx
byNanMdSahar
 
USABILITY TESTING TO IMPROVE USER EXPERIENCE
byMalang QA Community
 
Principles of Health Informatics: Evaluating medical software
byMartin Chapman
 
Usability testing
byArchana Krushnan
 
195
byvivatechijri
 
THE USABILITY METRICS FOR USER EXPERIENCE
byvivatechijri
 
Usability Inspection, Human computer intraction.pptx
bySyedGhassanAzhar
 
Tech. Writing Usability Presentation
bymhobren
 
ICS3211 lecture 10
byVanessa Camilleri
 
classmar2.ppt
byRangothriSreenivasaS
 
Majestic MRSS Usability Engineering
byMajestic MRSS
 
L7 Usability testing lecture of usability
byjlajizhangh
 

Usability_Evaluation

  • 1.
    measure with users USABILITY EVALUATION USABILITY EVALUATION methodsand techniques Narayanan Thiruvengadam tnarayan@gmail.com2013
  • 2.
    measure with users USABILITY EVALUATION UsabilityDefinition Usability Evaluation Overview • Formative Vs Summative • History Usability Inspection Methods • Heuristic Evaluation - Overview - Heuristics Principles - What is a Usability Review? - When & How of a Usability review - What to include in a review? - Computing severity ratings - Advantages of a Usability Review - Things to Remember • Cognitive Walkthrough - Overview - How To Do It • Pluralistic Walkthrough - Overview Usability Testing Overview • Techniques to Gather Data • User Testing- How to Do It • Quick Tips User- Centered Design Process Usability Evaluation Benefits • Technology • Business Index
  • 3.
    measure with users USABILITY EVALUATION “Theextent to which the product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” Usability Definition International Organization for Standardization (ISO) Usability 5 Quality Components/Usability Measure
  • 4.
    measure with users USABILITY EVALUATION UsabilityEvaluation is aimed at finding usability problems in the user interface/ System, providing qualitative and quantitative data about people and their behavior or attitudes. Usability evolution are accomplished by identifying representative users, representative tasks and developing a procedure for capturing usability problems with the interface/system. During the design/testing/development cycle of software development. There are two types of user evaluations are carried out • Formative evaluations • Summative evaluations “A usability defect is a product characteristic that makes it difficult or unpleasant for users to accomplish tasks supported by the product.” “Kahn and Prail” Overview
  • 5.
    measure with users USABILITY EVALUATION Conductedduring design as part of an iterative process, allowing the designers to see how their target user/audience responds to the interface. These methods provide qualitative data. Heuristic Evaluation, User Interface Inspections, Thinking-aloud Testing, Pluralistic Usability walkthroughs, and Cognitive Walkthroughs are some methods that can be used for formative evaluation. Conducted after completion of products, used in product comparisons, to identify problems prior to a redesign. Summative studies are designed to yield quantitative data that can be tested for statistical significance. Formative Summative "When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative.“ - Robert Stakes Formative Vs Summative
  • 6.
    measure with users USABILITY EVALUATION JeanScholtz, National Institute of Standards and Technology Timeline for usability evaluations in the last 30 years. Users were first used as the source of usability feedback but models have been also used for over 20 years. Expert feedback was developed in heuristic reviews and cognitive walkthroughs and has been used since the early 90s. History
  • 7.
    measure with users USABILITY EVALUATION HeuristicEvaluation/Usability Review: is the most informal method and involves having usability specialists/experts to judge whether interface design established usability principles. (the “heuristics” ) Cognitive Walkthrough: is a step-by-step method exploring/inspecting user interface with a set of questions and creating success/failure stories on the users’ behavior to simulate users problem, understandability and ease of learning. Pluralistic Walkthroughs: uses group meetings where users, developers, and usability expert step through a scenario, discussing each dialogue element. Formal Usability Inspections: Evaluators walkthrough tasks with the user's goals and purpose in mind, similar to cognitive walkthroughs, although the emphasis is less on cognitive theory and more on encountering usability problems. Usability Inspection Methods There’s no such thing as a “user error”! Cognitive: thought process toward awareness or knowledge
  • 8.
    measure with users USABILITY EVALUATION FeatureInspection: lists sequence of features used to accomplish typical tasks, checks for long sequences, cumbersome steps, steps that would not be natural for users to try, and steps that require extensive knowledge/ experience in order to assess a proposed feature set. Consistency Inspection: has designers representing multiple projects inspect an interface to see whether it does things in the same way as their own designs. Standards Inspection: has an expert on some interface standard ‘inspect the interface for compliance. Usability Inspection Methods There’s no such thing as a “user error”!
  • 9.
    measure with users USABILITY EVALUATION Heuristicevaluation is a method for finding the usability problems in a user interface design so that they can attend to as part of an iterative design process. Heuristic evaluation involves having a set of evaluators (3 - 5) examine the interface and judge its compliance with recognized usability principles (“the heuristics") Heuristic evaluation Evaluators VS Problem found – report Nielsen’s
  • 10.
    measure with users USABILITY EVALUATION Visibilityof system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation 1 2 3 4 5 6 7 8 9 10 The Heuristics Principles Nielsen’s Heuristics Ref. Link: http://www.useit.com/papers/heuristic/heuristic_list.html
  • 11.
    measure with users USABILITY EVALUATION AUsability Review (more formally known as ‘Heuristic Evaluation’ is a technique for identifying usability issues. A Usability Review: • Is cheaper to conduct than formal usability testing • Can be completed in a very short period • Can be conducted at any stage of the design. • Usability Reviews are somewhat subjective, because no real users are involved. Objectives of a review: • Identify likely trouble-spots • Recommend directions and concepts for improved design (both tactical and strategic) • Finds ‘obvious’ problems that are NOT really obvious. What is a Usability Review? Usability Review
  • 12.
    measure with users USABILITY EVALUATIONUsability Review When ? A Usability Review is particularly appropriate if an application has a large number of serious usability problems, or if an application is not sufficiently mature for usability testing with real users. In a hostile environment, a Usability Review is open to the accusation of being only one opinion against the opinion of others. In such circumstances, consider usability testing instead. How? At least two people should independently review the application, to maximize the number of issues identified. In general, avoid being given a walk-through of the application before beginning, as it is best to approach the task with a fresh and un-biased viewpoint. However, you should know: • The purpose of the application • The characteristics of the intended users and their levels of domain knowledge • The context of use. Usability Review
  • 13.
    measure with users USABILITY EVALUATION Beforeyou start, create a working document for your notes. This allows you to capture first impressions, which you may not readily recall once you have begun working with the application. Once you begin your review, use a standard notation for capturing issues. For each issue, note: • An exact description of the issue. Write this in the voice that you will use in your report, to facilitate subsequent 'cutting- and-pasting' with a minimum of re-writing. • Occurrence point of the issue • The usability principle (heuristic) violated/ standard in interface • An estimate of severity. • A recommendation What to include in a Review Usability Review
  • 14.
    measure with users USABILITY EVALUATION •Frequency with which the problem occurs – is it common or rare? • Impact of the problem if it occurs – will it be easy or difficult for users to overcome? • Persistence of the problem – is it a one-time problem that users can overcome once they know about it or will users repeatedly be bothered by the problem? Computing Severity Ratings Usability Review
  • 15.
    measure with users USABILITY EVALUATION Navigation& ease of information retrieval Maximize the ease with which site visitors can find both the information they require and the content you want them to access Page layout & calls-to-action Make sure there are enough calls-to-actions and that they're appropriate and noticeable Persuasiveness & cross-selling Ensure the site and individual pages persuade site visitors to do what you want them to do (make a purchase, contact you etc.) Removing obstacles Make it as easy as possible for people to do what they - and more importantly you - want them to do Content, marketing & brand Ensure that the site's content engages and persuades users and that the site's USPs are communicated clearly and effectively Search Enhance the design and performance of the site's search engine so users can find what they want with ease Advantages of a Usability Review Usability Review
  • 16.
    measure with users USABILITY EVALUATION •Precisely why you consider each issue to be relevant from a usability perspective - avoid personal opinions that may be grounded in prejudice. • Acknowledge the positive aspects of the system under review - remember that some readers may be hostile to or upset by your findings. • Give careful, meaningful and reasoned comments. • Provide clear recommendations. E.g.: Decide whether the application can be 'patched', or whether more fundamental changes are required. • Summarize your findings in the beginning of the report. Do not expect all readers to have time to go through the issues in detail. • Combine and print the issues from each reviewer, and group like issues together. This should give you the basic structure for the body of your report. • Review your working notes to ensure that all the issues you noted have been included as issues and not as recommendations. • Peer review the checklists to see if all reviewers are in agreement, and examine any discrepancies. • Do NOT make a mockery of the design of the application. Things to Remember Usability Review
  • 17.
    measure with users USABILITY EVALUATION Itsa more explicitly detailed procedure where evaluators asks himself a set of questions about the expectations of users’ behavior to assesses interface design and come up with a success/failure stories to simulate user’s problem and solving process at each step through the dialogues. This technique is best used in the design stage of development. The evaluators may include usability experts, designers, software developers or people from marketing, etc. Cognitive Walkthrough Cognitive: thought process toward awareness or knowledge
  • 18.
    measure with users USABILITY EVALUATION Prototype PrepareLow / High fidelity prototype Choose a user ( representative user) Explore their skill levels with both technology and the problem domain Evaluation Tasks Define the Goal & Tasks to be achieved in the prototype Action Sequence Determine the step by step actions the user ideally should take to complete the task. Every step should be explicitly listed 1 Cognitive Walkthrough How To Do It
  • 19.
    measure with users USABILITY EVALUATION TaskWalkthrough The modulator walk through (discuss) the tasks with respect to the interface/mockups and action sequences/ scenarios with the evaluators/ participants and create a credible success /failure story for each individual step. Ask the following questions, for each step, while looking at the interface and create a credible story based around these questions • Will the user try to achieve the right effect? • Will the user notice that the correct action is available? • Will the user associate the correct action with the effect that user is trying to achieve? • If the correct action is performed, will the user see that progress is being made towards their goal? 2 Cognitive Walkthrough How To Do It
  • 20.
    measure with users USABILITY EVALUATION Reports(Actionable Data ) Create report with actionable data include steps where the user might not know what to do, steps where they don't know if they were successful, and gaps in the design, including success/failure stories Cognitive Walkthrough How To Do It 3
  • 21.
    measure with users USABILITY EVALUATION Inthe design stage, when paper prototype is available, a group of users, developers, and usability expert meet together to step through a set of tasks, discussing and evaluating the usability of a system. The more people looking for problems, the higher the probability of finding problems. Five Characteristics • Inclusion of representative users, product developers, and Usability professionals • The application’s screens are presented (as hard copy) in the same order as they would appear to the user • All participants are asked to assume the role of the user • Participants write down what actions they, as users, would take for each screen before the group discusses the screens • When discussing each screen, the representative users speak first Pluralistic Walkthrough
  • 22.
    measure with users USABILITY EVALUATION Ina usability evaluation, you are predicting the problems or successes that users will have with the system/interface. A usability test tells you whether your predictions are valid and also helps to find new usability problems. The aim of the Usability testing is to gather qualitative user data; combines interviewing and observation of a user in a real-time. Usability testing is especially effective in determining: • Naming (section/button labels/Words) • Organization (information grouped into meaningful categories/ better placed) • First-time use and discoverability • Effectiveness • Emotional Response Techniques popularly used to gather data during a usability test include think aloud protocol, Co-discovery Learning, eye tracking, Remote Testing, Question-asking Protocol, Clickstream analysis, Facial Recognition and Psychometric Data Gathering and Analysis Overview
  • 23.
    measure with users USABILITY EVALUATION ThinkAloud Protocol During the course of a usability test, the test users are asked to verbalize/speak out their thoughts, feelings, and opinions while interacting with the system. It is very useful in capturing a wide range of cognitive/ mental processes of activities. Co-discovery Learning During a usability test, two test users attempt to perform tasks together while being observed. They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product. They are encouraged to explain what they are thinking about while working on the tasks. Techniques to Gather Data
  • 24.
    measure with users USABILITY EVALUATION EyeTracking Eye tracking is the process of measuring either the point of gaze ("where we are looking") or the motion of an eye relative to the head. Watching what people are looking at adds a level of interest to usability studies. Following the eye at work makes you feel as though you are in the user’s head, thinking with him and able to understand how user looks the interface. Eye tracking solutions Eye Tracking is a combined software and hardware package designed to capture the conscious and unconscious eye gaze movement of a respondent. Eye Tracking is a Unique and Reliable Method for Assessing Design Impact or Usability of websites, print media, package design, virtual store shelving, etc. Hardware Solutions & Support noldus.com, smi.de , faceLAB, Tobii TX300 Eye Tracker, EyeTech, etc. Techniques to Gather Data
  • 25.
  • 26.
    measure with users USABILITY EVALUATION RemoteTesting Remote usability testing is used when testers are separated in space and/or time from the participants. This means that the testers cannot observe the testing process directly and that the participants are usually not in a formal usability laboratory. Another is different-time different-place testing, where the user's test session is guided and logged through a special piece of software. Tools : Morae, usertesting.com, intuitionhq.com, CrazyEgg, etc. Question-Asking Protocol During a usability test, besides letting the test users to verbalize their thoughts as in the thinking aloud protocol, the testers prompt them by asking direct questions about the product, in order to understand their mental model of the system and the tasks, and where they have trouble in understanding and using the system. Techniques to Gather Data
  • 27.
    measure with users USABILITY EVALUATION Clickstream Clickstreamanalysis/clickstream analytics is the process of collecting, analyzing, and reporting aggregate data about which pages visitors visit in what order There are two levels of clickstream analysis, traffic analysis and e-commerce analysis. Facial Recognition facial recognition technique are used to capture information about the shape of a face and this information is then used for identification and the verification of identity. Psychometric Data Gathering and Analysis Psychometric(Psychology + Metrics) is the study of behavior and mental processes. Psychometric testing or psychometric analysis is to measure or assess the psychology of a person. Techniques to Gather Data Clicksteam Facial Recognition
  • 28.
    measure with users USABILITY EVALUATION Planand Strategize on: • The Goals for the test • The Screens & System for the test • Tasks for the user to be tested • Data to be collected and analyzed • Methodology to collect the data • Location of the test • Recording Setup • Role of the participants Identify Test Users User should be as representative as possible of the intended users of the system. One should take additional care to involve average users. If a larger group of testers are to be engaged, one should select users from several different subpopulations to cover the main different categories of expected users. (Novice to Expert users) 2 “You’re testing the system not the User Knowledge “ 1How To Do It User Testing
  • 29.
    measure with users USABILITY EVALUATION TestLocation Usability testing is preferably conducted in User’s environment. Else, out of fear of seeming dumb, incompetent, many users may avoid talking about software behaviors that they find problematic. Design Usability Test • Pre test user briefing points, questions & confidential agreements • Participant Tasks • Goals & Tasks • Test User Questions • Post Test Questions & Survey Pilot Test Pilot testing can also be used to refine the experimental procedure and to clarify the definitions, Test tasks, Test plan, Questionnaires. How To Do It User Testing 3 4
  • 30.
    measure with users USABILITY EVALUATION PerformUser Testing (UT) • Explain the User about the aim of the test • Explain User if any recording, keystroke logging, or other monitoring that is used • Explain and get the confidential agreements if required • Brief the goal and tasks which the tester needs to perform • Choose appropriate techniques/method to gather data ( Think aloud method/Eye Tracking/Question-Asking Protocol) • Define the roles for each participant: • User: Performs the task as planned • Observer: Takes note of every action and reactions of the user • Moderator: Probes with questions and helps the user to overcome any system limitations • Upon completion of the test, analyze the data captured and identify Usability problems How To Do It User Testing 5
  • 31.
    measure with users USABILITY EVALUATION •Observe, be quiet, watch, understand • Don't explain, suggest, defend the design • Don't ask the test user's opinion • Help user to overcome the limitations of the prototype • Hand out the test tasks one at a time • Keep a relaxed atmosphere in the test room • Avoid disruptions • Never indicate in any way that the user is making mistakes or is too slow • If necessary, stop the test if it becomes too unpleasant • End by stating that the user has helped you find areas of improvement • Never report results in such a way that individual users can be identified • Cash or an appropriate 'Thank You' gift User Testing – Quick Tips While UT After UT
  • 32.
    measure with users USABILITY EVALUATION INITIATEBUILDTEST DELIVER& MONITOR MODEL DISCOVERYEVALUATE FRAME WORK User Centered Design
  • 33.
    measure with users USABILITY EVALUATIONSystem / Interface Design • Usability evaluation will help you in identifying system’s Learnability, Efficiency, Memorability, Errors & Users Satisfaction. • Risks are managed and reduced by introducing usability evaluation in early stage of the software development cycle. • Evaluation provides qualitative feedback and helps to improve your system / interface design. • Helps you identify issues and potential problems before the development starts. • Help to discover real needs and tasks of the user’s in the early stage of the design process. • Decrease user acclimation (accustomed to a new environment) time, errors and increased user’s productivity. • Provide a competitive advantage • Help to analyze user’s goals, which results in the business meeting its targets. • Will help in reducing the rework and help in meeting customer requirements (80 percent of software rewrites are due to the fact that important functionality was missed the first time) • Help to build customer and users confidence.
  • 34.
    measure with users USABILITY EVALUATIONBusiness • Gain a competitive advantage. • Generate rapid ROI and gain quicker time to market by avoiding big changes late in development. • An easy-to-use system means fewer calls to customer service • Early prediction of potential problems will reduce the project lifecycle cost • Provide more follow-on business due to satisfied customers & users • Look better, demo better and sell better, as they are more intuitive to use, faster and more effective results in profitability and revenue • Loyal customers provide free word-of-mouth exposure results in the business revenue
  • 35.
    measure with users USABILITY EVALUATION UsabilityInspection Methods- Jakob Nielsen, Robert L Mack Usability Engineering- Jakob Nielsen Measuring The User Experience- Tom Tullis, Bill Albert About Face 3.0- Alan Cooper, Robert Reimann, and Dave Cronin Beyond The Usability Lab - Bill Albert, Tom Tullis, Donna Tedesco usability.gov useit.com usabilityfirst.com stcsig.org wikipedia.org usabilityhome.com webcredible.co.uk Enhancing the Explanatory Power of Usability Heuristics – Jakob Nielsen Usability inspection methods - research and practice- David G. Novick Usability Inspection Methods- Jakob Nielsen The Pluralistic Usability Walk-Through Method- S. Riihiaho Eyetracking Methodology- Kara Pernice and Jakob Nielsen The cognitive walkthrough method: A practitioner’s guide. In Nielsen, J., and Mack, Usability inspection methods. New York, NY: John Wiley & Sons, Inc. Wharton, C., Rieman, J., Lewis, C., and Polson, P. (1994). Courtesy : istockphoto.com Books Online Whitepapers USABILITY EVALUATION References
  • 36.
    measure with users USABILITY EVALUATION USABILITY EVALUATION measurewith users Narayanan Thiruvengadam tnarayan@gmail.com2013