Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Power of the UX Evaluation

448 views

Published on

Presented 5/11/17 @LOCO_UX by @jkooda of @liminaUX
This talk covers the anatomy of a UX Eval, how to use it as a business development tool, and how to ensure you have a logical and most importantly beneficial return on your client's investment.

Published in: Design
  • Be the first to comment

The Power of the UX Evaluation

  1. 1. The Power of The UX Evaluation a LOCO | UX Meetup TONIGHT
  2. 2. A FEW SHORT WORDS LOCO | UX Community Mission Build a network of UX practitioners local to the Longmont/Boulder area for the purpose of: • Access to opportunities (seeking, hiring, colabos) • Sharing Ideas, Tools, Methods, Techniques • Building community for mutual enhancement/benefit • each other • local businesses • Longmont municipality
  3. 3. Who Tweets During Meetups? @jkooda of @liminaUX at @loco_ux
  4. 4. Expert Heuristic (huh?) Evaluation
  5. 5. heu·ris·tic hyo͞oˈristik/ adjective enabling a person to discover or learn something for themselves. "a “hands-on” or interactive heuristic approach to learning" noun a heuristic process or method.
  6. 6. Is a usability engineering method for finding usability problems in a user interface design, thereby making them addressable and solvable. It involves a small set of expert evaluators who examine the interface and assess its compliance with “heuristics,” or recognized usability principles. Heuristic Evaluation
  7. 7. 10 Usability Heuristic Principles Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Helps users recognize, diagnose, and recover from errors Help and documentation Examples in the Wild: http://designingwebinterfaces.com/6-tips-for-a-great-flex-ux-part-5 Source: https://www.nngroup.com/articles/ten-usability-heuristics/
  8. 8. Donald A. Norman Jacob Nielson “DISCOUNT USABILITY ENGINEERING SERVICE”
  9. 9. CAVEAT A heuristic evaluation should not replace usability testing. Although the heuristics relate to criteria that affect your site’s usability, the issues identified in a heuristic evaluation are different than those found in a usability test.
  10. 10. PROS • Provides quick and relatively inexpensive (cheap) • Systematic and Intuitive, applying a set of predefined rules/heuristics • Requires very little planning • Can be used early in the design process • Does not require a finished interface • Provides corrective measures • Can be used together with other usability testing methods • Can help identify and prioritize issues in need of further usability testing
  11. 11. CONS • Requires knowledge and experience to apply the heuristics effectively • Trained usability experts are sometimes hard to find and can be expensive • Requires multiple experts and aggregate their results • The evaluation may identify more minor issues and fewer major issues • Tends to focus on problems rather than solutions
  12. 12. 1. Set Up The Team 2. Agree on Evaluation Criteria 3. Conduct Cognitive Walk Through 4. Analyze Interface & Record Issues 5. Internal Review 6. Present to Client 7. … Next steps & business development 6 MAJOR PARTS
  13. 13. 1.WHO DO YOU NEED?
  14. 14. PICK 3 UX LEAD USER RESEARCHER INTERACTION DESIGNER VISUAL DESIGNER DEVELOPER CONTENT SPECIALIST
  15. 15. WHY 3?
  16. 16. Rows = 19 evaluators Column = 16 usability problems = Problem found Economics of the Evaluation Team Source: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/ Weak Evaluator Strong Evaluator Hard Easy 0% 25% 50% 75% 100% 1 5 10 15 Number of Evaluators PercentIssuesFound Strength in Numbers 0% 10% 20% 30% 40% 50% 60% 70% 1 5 10 15 Number of Evaluators Cost/BenefitRatio ROI Eval Quants
  17. 17. Freak-o-nomics 1.Emergence over authority – Talks about the shift in authority from the top-down to the bottom-up. Think Wikipedia. 2.Pull over push – Another example of bottom-up demand. User-driven instead of mass-marketing. Think on-demand instead of broadcast. 3.Compasses over maps – Methodology not mission. Teach a man to fish. A play for Seymour Papert’s constructionism. 4.Risk over safety – When the cost of innovation is low, like it is now, it’s easier take more risks and try more things. Negroponte famous MediaLab motto was, “Demo or Die”, Ito says “Deploy.” 5.Disobedience over compliance – “You don’t win a Nobel Prize by doing what you’re told” (Joi Ito). Ask forgiveness, not permission. 6.Practice over theory – “In theory there is no difference between theory and practice. In practice there is” (Yogi Berra). Both should inform one other. 7.Diversity over ability – Crowdsourcing (a word Howe coined) can solve some problems that dedicated professions can’t solve internally. 8.Resilience over strength – They quote the old adage that an oak tree is strong, but can be blown over in a storm; while the weaker but flexible willow bends with it and can survive. Akin to what Angela Duckworth popularized as “grit” in her great book with the same title. 9.Systems over objects – The value of interdisciplinary networking.
  18. 18. 2. SO, WHO DOES WHAT?
  19. 19. • Workflow • Information Architecture • Ease of Use • Taxonomy / Terminology • Interface Elements • Window/Pane Management • Error Handling • Access to Help • Page Layout • Color Use • Visual Language • Data Visualization • Iconography BREAKDOWN OF RESPONSIBILITY USER RESEARCHER INTERACTION DESIGNER VISUAL DESIGNER
  20. 20. 3. COGNITIVE WALKTHROUGH
  21. 21. Getting Domain Expertise BUSINESS REPRESENTATIVES — Wrangle business objectives and understand opportunities for the business. Understand how to make or save money for the business. Understand how the design meets the needs of the product domain and its users. PROJECT MANAGER — Ensure the client can implement this design in time and when/what the project milestones are. TECHNICAL EXPERT — Understand technical framework (opportunities and limitations). Validate the client can implement the product recommendations (feasibility / time intensity). YOUR TEAM - provides the domain expertise in “USER EXPERIENCE & USABILITY”, they provide industry domain expertise.
  22. 22. SETTING TARGETS KEY PRODUCT FEATURES Plan to evaluate all key features that differentiate this product from it’s competitors. Ask who else is doing this, is any one doing this better? KEY TASKS / WORKFLOWS Plan to evaluate all key workflows in the system. Unless problematic, avoid evaluating common tasks (ex: login, logout) KEY PERFORMANCE INDICATORS Understand what metrics are important to the business: (ex. time to completion, task completion ratios, user registration, engagement volume, etc.) BE A SPONGE: • Take this opportunity to understand the product from the business and user perspective. • Remain objective, do not adopt or project your cognitive bias. • Take screenshots and where possible, record the session (voice and screen)
  23. 23. 4. GETCHO ANALYSIS ON!
  24. 24. Ingredients SHARED REPOSITORY • Directory: (Recording, Screenshots, Documents, Tracking Sheets) • Collaboration Workspace: Google Slides, InvisionApp, 3rd Party Tools EVALUATION STRUCTURE • Outline by Issue Category or Functional Areas • Issue category overview • Individual issue slides • Category findings SCREEN MARKUPS • Point out problem areas • Make note of successes • Assign heuristic (1-13) • Assign Issue Category • Assign Severity / Prevalence (H, M, L) • Cite example solutions and recommendations KEY TAKEAWAYS (Executive Summary) • Findings • Key Issues • Severity/Priority
  25. 25. THANK YOU XEROX • 22 Pages • 13 Principles • 288 Check Points Source: http://users.polytech.unice.fr/~pinna/MODULEIHM/ANNEE2010/CEIHM/XEROX%20HE_CKLST.pdf
  26. 26. THANK YOU CHRIS GALLELLO • Chrome Plugin • 10 Principles • - Heuristic • - Severity • - Notes • - Recommendations • In-Line Annotations • Word.doc Export Source: : http://www.uxcheck.co/ Reference: https://medium.com/@cgallello/low-cost-usability-testing-61b5f8a2a1be
  27. 27. 5. PRE-GAME HUDDLE
  28. 28. Draft Run Through GAP ANALYSIS – Any areas need more attention? Use your KPI’s, Key Workflows, Key Product Features INTERIM CLIENT CHECKPOINT – Depending on confidence and success, it’s always a good idea to check in with the client around the 60% complete mark ORGANIZE FINDINGS — Look across what everyone has analyzed, group logically (by issue category or functional areas – avoid redundancy (cite instances and severity/prevalence) DRIVE VALUE – Without giving up free work, provide broad recommendations an cite examples of successful implementations. ASSIGN PRESENTERS / NOTE TAKERS — Depending on skills mix you may want to mix it up, unless you’re confident with one presenter ESTIMATE PRESENTATION TIME — These reports can be dense and packed with rich discussion, you will need to account for extra time per issue.
  29. 29. 6. SHOW TIME
  30. 30. Empathy BE SENSITIVE TO POLITICAL / EMOTIONAL IMPLICATIONS <VERY IMPORTANT> • Always start on a positive note it’s important to acknowledge where they hit the mark and showed innovative, unique and successful design • Often times the interfaces you’re evaluating represent a significant amount of commitment, time, and/or cost investment on your client’s part • Realize that every observation can be perceived as a personal critique by any number of client team members BREAK THE NEWS IN THE BEST LIGHT POSSIBLE • Your perspective represents an objective 3rd party observations that should not be charged with subjectivity and aesthetic/technical preference • This is particularly important for the visual analysis • Your observations should be supported by empirical evidence and sound solutions and recommendations • Make room for conversations, check the pulse of how each observation and recommendation is landing
  31. 31. Engage YOU MAY HAVE CLARIFYING QUESTIONS • Make sure you’re taking time to validate your observations and recommendations THIS IS NOT A PASSIVE SESSION • Create opportunity for the client to articulate priority and effort levels against your observations and recommendations (H, M, L) this will be useful in backlog management • You are proxy for the user, be sure to advocate for the usability impact of resolving or deprioritizing a fix, they are relying on you to be the expert • Continue to articulate your recommendations and solutions in terms of KPI’s and what the would be most valuable to the business without negatively impacting ease of use and user value.
  32. 32. Drive Momentum IF YOU’VE DONE YOUR JOB… • The client would have agreed with 60%-80% of your findings • They will already be thinking about low handing fruit and how to parse the enhancement into near term sprints and long term release cycles – this is your opportunity to step up. THE EVALUATION REPORT IS THE BEGINNING, NOT THE END • You now own the issue log and recommendations with client feedback • Ranked by: Client Priority, User Benefit, Technical Complexity… • This is the making of what we call a Feature Matrix: a useful tool for UX prioritization, sprint planning and backlog management
  33. 33. SAMPLE UX EVAL <ATTENDEES ONLY>
  34. 34. FEATUREMATIX (sneakpeek)
  35. 35. • Catalogue UX enhancements or requirements into swim- lanes • Assign each cell with User Benefit, Business Benefit and Technical Complexity • Use this information to build the priority ranking for sprint planning and backlog management.
  36. 36. YOU SAID POWER.
  37. 37. POWER FOR YOU YOUR BUSINESS DEVELOPMENT TOOL FOR UX WORK Brings you top of mind for follow on detailed design contract work Clearly articulates the value of UX in terms that matter to the client Wets the appetite of the client without giving up detailed design work Strong business planning tool – showing UX as a strategic practice
  38. 38. POWER FOR YOUR CLIENT TACTICAL HITLIST In most cases your observations and recommendations can dovetail into their existing development plan – the client can yield immediate value from the report For a relatively low cost, the client has gained critical external feedback for near and long term product enhancements – outputs from the report are typically folded into long term development plans
  39. 39. THANK YOU! Questions?
  40. 40. LOCO | UX Community News? Announcements? Next Topic Ideas?
  41. 41. THANKS TO OUR SPONSORS! https://twitter.com/loco_ux + CoSolve

×