Heuristic Evaluation Tutorial                                1
Heuristic Evaluation• Goal: Identifying usability problems in existing  systems• Checking compliance with a set of usabili...
Heuristic Evaluation• Freely explore the interface or perform  certain tasks• You can go through the interface several tim...
IT security management (ITSM) tools• IT security management tools are components  in the design, development, and maintena...
ITSM HEURISTICS                  5
1 – Visibility of activity statusProvide users with awareness about the status of the activity  distributed over time and ...
2- History of actions and changes on                artifactsAllow capturing the history of actions and  changes on tools ...
3- Flexible representation of               informationAllow changing the representation of  information to suit the targe...
4- Rules and constraintsPromote rules and constraints on ITSM activities, butprovide freedom for users to choose different...
5- Planning and dividing work between                 usersFacilitate dividing work between the users involved in  an acti...
6- Capturing, sharing, and discovery of              knowledgeAllow users to capture and store their knowledgeexplicitly b...
7- Verification of knowledgeFor critical ITSM activities, tools should help SPs validate their  knowledge about the action...
NIELSEN’S HEURISTICS                       13
1 - Visibility of system statusThe system should always keep users informed  about what is going on, through appropriate  ...
2 - Match between system and the real                 worldThe system should speak the users language,with words, phrases ...
3 - User control and freedomUsers often choose system functions by mistakeand will need a clearly marked "emergency exit"t...
4 - Consistency and standardsUsers should not have to wonder whetherdifferent words, situations, or actions mean thesame t...
5 - Error preventionEven better than good error messages is acareful design which prevents a problem fromoccurring in the ...
6 - Recognition rather than recallMinimize the users memory load by makingobjects, actions, and options visible. The users...
7 - Flexibility and efficiency of useAccelerators -- unseen by the novice user -- mayoften speed up the interaction for th...
8 - Aesthetic and minimalist designDialogues should not contain information whichis irrelevant or rarely needed. Every ext...
9 - Help users recognize, diagnose, and            recover from errorsError messages should be expressed in plainlanguage ...
10 - Help and documentationEven though it is better if the system can beused without documentation, it may benecessary to ...
Upcoming SlideShare
Loading in...5
×

Itsm training

246

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
246
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The ITSM version
  • The goal of heuristic evaluation is to identify usability problems in a system. The heuristic evaluation method employs a set of usability principles, called heuristics, to guide the evaluators in identification of usability problems. Each evaluator inspects the interface and checks the compliance of the interface with the heuristics. Heuristic evaluation is performed individually by each evaluator and then the results from different evaluators are aggregated into a set of usability problems.
  • During heuristic evaluation, you can freely explore an interface and identify problems or you can use scenarios to focus on a specific part of the system. In this study, we will use scenarios to limit the scope of the evaluation. You can walk through the steps of each scenario and perform them on the interface once or multiple times. Also, you are free to explore other parts of the interface to become familiar with the interface. But we are only interested in identifying problems related to the provided scenarios. For each problem that you identify, you should specify 3 components: First, the problem itself. Second, the scenario in which you identified the problem. Third, the heuristic (or heuristics) with which you found the problem. You might find problems that can’t be associated to a heuristic. For those problems, just record the problem, and scenario.
  • As we are going to evaluate an IT security management tool, I want to give you a quick background about this class of applications. IT security tools are components in the design, development, and maintenance of a secure information technology infrastructure. Tools like network firewalls, intrusion detection systems, and identity and access management systems are examples of IT security management tools. The environment in which these tools work has certain characteristics. First, it is complex. Everyday, new security issues arise in this environment that need to be addressed in a timely fashion. Many tasks of security administrators are not routine, and require knowledge and expertise. Second, this environment is collaborative. It means that different stakeholders in the organization need to collaborate with each other to perform their tasks. Third, the stakeholders involved in IT security management have different background. There are security administrators, managers, employees, and external contractors who need to collaborate with each other to perform IT security management tasks.
  • Now I will present the heuristics you will be using today. For each heuristic, I will describe it and give an example of how it can be applied to an IT security management tool, in this case, a network firewall. There are 7 heuristics that I will describe. They are called ITSM heuristics. I will go through them in this tutorial and give examples of how they can be applied to IT security tools.
  • IT security management activities are distributed over time and space and they include multiple users. These users work in an organizational environment and must comply with the rules in the organization (e.g., the security policy, privacy policy, and other organizational rules) and standards. Therefore, the tool should provide the required awareness about the status of the activity, the other users involved, the progress toward activity objectives, rules, available tools, etc., about the activity, to all of the stakeholders of the tool. It is important to only provide the status that users really need to know, not more.
  • Historical information could be in the form of use-histories by other people or the subject himself. Use histories can be employed to reflect on work and getting feedback from peers. In IT security, reflecting on work is important as the actions are performed on the system by different stakeholders. Moreover, security regulations require the system to keep a history of actions.
  • As IT security tools have different stakeholders, tools should be able to present information in the format suitable to the target audience. Furthermore, to address complexity, providing different presentation of data allows the user to view information from different perspectives and help them understand complex scenarios. From a different view point, security admins like to combine their tools together to address different problems. Therefore, tools should be able to present their information in a way that can be used by other tools and also accept inputs from different tools.
  • As ITSM tool designers can’t predict all the conditions that a tool user might face, they should provide freedom for users to choose the way they want to perform the activities. Therefore, while there should be multiple ways for users to perform activities, constraints should be enforced by the tool so the user can only choose those paths that are not violating any of the constraints.
  • Because the use of ITSM tools involves multiple stakeholders, the tool should provide facilities for dividing work between different stakeholders. For those tasks that have a routine procedure, incorporation of a workflow in the tool would be a good idea. But for unknown conditions, tool should provide ways for users to generate plans for performing the activity (for example, showing who is available to perform a task or allow a workflow to be created dynamically)
  • To address problems in the complex and evolving environment of ITSM, a subject needs to use the knowledge and experience of other stakeholders involve in the activity. To facilitate accessing distributed knowledge, ITSM tools should enable their users to express their knowledge in a form of a document, web-page, or script that can be used by other users and also facilitate identification and access to the required knowledge sources for accomplishing the activity. In cases that documenting knowledge is not feasible, a method for finding and starting collaboration with the person who possesses the knowledge should be provided.
  • Many actions in ITSM are responses to new, unseen and complex situations. These actions should be performed on systems that are critical to the organization. Moreover, the actions are distributed in time and space and the result of an action can't be evaluated in real time. Therefore, the cost of errors in these actions is huge. To find a solution to a new or complex problem, a security admins usually consults different information sources and combines them into a single plan (a plan, a guide document, a check list, etc.) This plan extracted from different sources might not be correct. Therefore, it should be verified before applying it to the system. Therefore, ITSM tools should allow users to rehearse the action on a non-critical, test system, evaluate the outcome of the action, and then apply it on the critical system. If something goes wrong in the rehearsal, the user can re-examine his or her interpretation of the external sources. After successful rehearsal, users can perform rehearsed actions on the critical artifact. To facilitate this process, tools should help creation of a non-critical system from a critical system, and help the process of applying rehearsed plan on the critical system.
  • Now I will present the heuristics you will be using today. For each heuristic, I will describe it and give an example of how it can be applied to an IT security management tool, in this case, a network firewall. There are 7 heuristics that I will describe. They are called ITSM heuristics. I will go through them in this tutorial and give examples of how they can be applied to IT security tools.
  • The system should keep users informed about what is going on in the system, through appropriate feedback within reasonable time, so that users can accurately determine the right action to perform on the system and also can determine the result of their actions.
  • Users can understand the system if it matches what the users know from the real world. [Nielsen heuristic itself with modifications] Therefore, the system should speak the users' language, with words, phrases, and concepts familiar to the user, rather than using system-oriented terms. It should follow real-world conventions, making information appear in a natural and logical order.
  • Users often make mistakes in choosing their actions. Therefore there should be a way for users to exit an undesirable state. Supporting undo or redo is an example of a shortcut for exiting such an unwanted state.
  • When using a system, users should not have to wonder whether different words, situations, or actions mean the same thing. It is better if the system can follow platform conventions and terminology which are familiar to the user.
  • Even better than good error messages is a careful design that prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
  • Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
  • Different users might use system in different ways. For example a novice user might prefer to go through more steps that help him do the tasks successfully, but an expert user might prefer a very straightforward way for performing an action that he is familiar with. Therefore, it would be good if the tool provides customization in a way that helps users tailor their frequent actions and choose the way they want to perform their tasks.
  • Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
  • Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
  • Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
  • Transcript of "Itsm training"

    1. 1. Heuristic Evaluation Tutorial 1
    2. 2. Heuristic Evaluation• Goal: Identifying usability problems in existing systems• Checking compliance with a set of usability principles• Performed individually• Results will be aggregated 2
    3. 3. Heuristic Evaluation• Freely explore the interface or perform certain tasks• You can go through the interface several times• You should specify: – The problem – The heuristic(s) – The severity 3
    4. 4. IT security management (ITSM) tools• IT security management tools are components in the design, development, and maintenance of a secure information technology infrastructure. – Examples: network firewall, intrusion detection system, identity and access management system• Characteristics of the ITSM environment – Complex, collaborative, people with different backgrounds 4
    5. 5. ITSM HEURISTICS 5
    6. 6. 1 – Visibility of activity statusProvide users with awareness about the status of the activity distributed over time and space, including the other users involved in the activity, their actions, and distribution of work between them; rules that govern the activity; tools, information, and material that are used in the activity; and progress toward the activity objective. Provide communication channels for transferring the status of the activity. While providing awareness is crucial, provide awareness only about what a user needs to know to complete his actions. 6
    7. 7. 2- History of actions and changes on artifactsAllow capturing the history of actions and changes on tools or other artefacts such as policies, logs, and communication between users. Provide a means for searching and analyzing historical information. 7
    8. 8. 3- Flexible representation of informationAllow changing the representation of information to suit the target audience and their current task. Support flexible reports. Allow tools to change the representation of their input/output for flexible combination with other tools. 8
    9. 9. 4- Rules and constraintsPromote rules and constraints on ITSM activities, butprovide freedom for users to choose different pathsthat respect the constraints. Constraints can beenforced in multiple layers. For example, a tool couldconstrain the possible actions based on the task, thechosen strategy for performing the task (e.g., the orderof performing actions), the social and organizationalstructure (e.g., number of subjects involved in the task,policies, standards), and the competency of the user. 9
    10. 10. 5- Planning and dividing work between usersFacilitate dividing work between the users involved in an activity. For routine and pre-determined tasks, allow incorporation of a workflow. For unknown conditions, allow generation of new work plans and incorporation of new users. 10
    11. 11. 6- Capturing, sharing, and discovery of knowledgeAllow users to capture and store their knowledgeexplicitly by generating documents, web-pages, scripts,and notes or implicitly by providing access to a historyof their previous actions. Tools could then facilitatesharing such knowledge with other users. Furthermore,tools should facilitate discovery of the requiredknowledge source including artefacts or a person whopossess the knowledge and provide means ofcommunicating with the person who possesses theknowledge. 11
    12. 12. 7- Verification of knowledgeFor critical ITSM activities, tools should help SPs validate their knowledge about the actions that are required to perform the activity. Allow users to validate their knowledge by performing actions and validating the results on a test system before applying them to the real system. Allow users to document the required actions in the form of a note or a script; this helps the users or their colleagues to review the required actions before applying them on the system. 12
    13. 13. NIELSEN’S HEURISTICS 13
    14. 14. 1 - Visibility of system statusThe system should always keep users informed about what is going on, through appropriate feedback within reasonable time. 14
    15. 15. 2 - Match between system and the real worldThe system should speak the users language,with words, phrases and concepts familiar to theuser, rather than system-oriented terms. Itshould follow real-world conventions, makinginformation appear in a natural and logicalorder. 15
    16. 16. 3 - User control and freedomUsers often choose system functions by mistakeand will need a clearly marked "emergency exit"to leave the unwanted state without having togo through an extended dialogue. Support undoand redo. 16
    17. 17. 4 - Consistency and standardsUsers should not have to wonder whetherdifferent words, situations, or actions mean thesame thing. Follow platform conventions andterminology which are familiar to the user. 17
    18. 18. 5 - Error preventionEven better than good error messages is acareful design which prevents a problem fromoccurring in the first place. Either eliminateerror-prone conditions or check for them andpresent users with a confirmation option beforethey commit to the action. 18
    19. 19. 6 - Recognition rather than recallMinimize the users memory load by makingobjects, actions, and options visible. The usershould not have to remember information fromone part of the dialogue to another. Instructionsfor use of the system should be visible or easilyretrievable whenever appropriate. 19
    20. 20. 7 - Flexibility and efficiency of useAccelerators -- unseen by the novice user -- mayoften speed up the interaction for the expertuser such that the system can cater to bothinexperienced and experienced users. Allowusers to tailor frequent actions. 20
    21. 21. 8 - Aesthetic and minimalist designDialogues should not contain information whichis irrelevant or rarely needed. Every extra unit ofinformation in a dialogue competes with therelevant units of information and diminishestheir relative visibility. 21
    22. 22. 9 - Help users recognize, diagnose, and recover from errorsError messages should be expressed in plainlanguage (no codes), precisely indicate theproblem, and constructively suggest a solution. 22
    23. 23. 10 - Help and documentationEven though it is better if the system can beused without documentation, it may benecessary to provide help and documentation.Any such information should be easy to search,focused on the users task, list concrete steps tobe carried out, and not be too large. 23

    ×