The document discusses various usability evaluation methods including cognitive walkthroughs and heuristic reviews, emphasizing their importance in identifying user interface issues to enhance user experience. It highlights the process of expert usability reviews, which can be conducted quickly and economically with limited resources, and produces actionable recommendations for improvement. It illustrates specific findings from product reviews and cognitive walkthroughs, noting challenges such as unclear navigation and accessibility issues.

2. Catherine Weeks SeaChange International

3. &  Product Evaluation

4. Expert Review Heuristic Review Cognitive Walkthrough “ This is a more specific version of a design walkthrough, focusing on cognitive principles . Based on a user's goals, a group of evaluators steps through tasks, evaluating at each step how difficult it is for the user to identify and operate the interface element most relevant to their current subgoal and how clearly the system provides feedback to that action. Cognitive walkthroughs take into consideration the user's thought processes that contribute to decision making, such as memory load and ability to reason .” – Usability First (http://www.usabilityfirst.com/methods/cogwalk.txl) “ Heuristic evaluations usually are conducted by a small set (one to three) of evaluators . The evaluators independently examine a user interface and judge its compliance with a set of usability principles . The result of this analysis is a list of potential usability issues or problems. The usability principles, also referred to as usability heuristics, are taken from published lists . Ideally, each potential usability problem is assigned to one or more heuristics to help facilitate fixing the problem. As more evaluators are involved, more true problems are found.” – Usability.gov (http://www.usability.gov/methods/heuristiceval.html) “ Expert usability reviews are a fast and economical route to improving the usability of a Web site... The report outlines the criteria used to evaluate the site with reliable recommendations, which are based on extensive Human-Computer Interaction (HCI) research and experience .” – Mich. State Usability & Accessibility Center (http://usability.msu.edu/services_expertRev.asp)

5. Expert Review Experience Research Many evaluators Heuristic Review Principles Non-expert Comparable Cognitive Walkthrough Task centric Working prototype Timely

6. Limited or no product team access Limited UX resources Limited time No access to users Access to working applications This led me to create a process based on a twist of the product evaluation types…

7. Identify User Product Review Gives 1,000 foot view Looks at branding, product placement, end user satisfaction Cognitive Walkthrough Look through application based on user tasks Find interaction problems, layout issues, and even bugs! Create Recommendations Produce Report

8. Case Intro: No team "Pet" project No requirements, no business case Natural extension to current products Identify User: Technical Business solution

9. Product Review Findings “ Customers will not be able to use [Product name] outside of [Parent product name], therefore creating a barrier to use…” “… providers could alter the recommendations without any indication to the user that these are ‘sponsored’ rather than purely calculated recommendations.” Cognitive Walkthrough Findings If the user selects the incorrect operation, in order to fix their mistake, the user must delete the new rule, and add another rule… The colors do not support color blindness (red and green used)

10. Case Intro: Offshore office (Asia) Customer buy-in Responsive team Identify User: Technical Business solution

11. Product Review Findings It is unclear where the user should begin in order to make the system work, and would require a vast amount of training in order for the users to become proficient in the system. Correct SeaChange product branding was not used. Cognitive Walkthrough Findings It is unclear how to access this application when receiving the hardware. The content area pane is laid out well and supports the user’s work. The whole page or continuing page look helps users identify if they need to scroll to see more information.

12. Requires few resources Can be as minimal or extensive as needed Produces a report that is easy for the product team to work with

13. Relies on experience and research, not user input! Reliance on few people to catch many issues Requires understanding of business needs Limited team collaboration opportunities

14. Access to a working interface or prototype Product Review: 1 ~ 3 days Cognitive Walkthrough: 3 ~ 5 days Recommendations: 2 ~ 5 days Report: 2 ~ 4 days Total: 2 ~ 4 weeks

15. Product is perceived as valuable to company strategy Limited resources No access to users

16. Works well in limited resource situations Can provide minimal to extensive reviews Should always opt to have users involved if possible Results in a clear report for the product team

17. Product Review outline Example of Cognitive Walkthrough Report layout example [email_address]

18. OK/Cancel Comics: http://www.ok-cancel.com/comic/18.html , http://www.ok-cancel.com/comic/82.html Cognitive Walkthrough definition: http://www.usabilityfirst.com/methods/cogwalk.txl Heuristic Review definition: http://www.usability.gov/methods/heuristiceval.html Expert Review definition: http://usability.msu.edu/services_expertRev.asp