The Why and How of Usability and User Experience (UX) Testing


Published on

Although usability and user experience may seem synonymous, they are separate and much different concepts. While usability is well defined in standards, UX has no agreed upon definition because it relates to a more nebulous attribute-user satisfaction. Both are, however, key ingredients for successful system deployment. Because they don’t know how to measure and evaluate UX, many teams ignore this important attribute until the end of development. Philip Lew discusses how to model both usability and UX by breaking each attribute down into measurable characteristics-learnability, user effectiveness, user efficiency, content quality, user errors, and more. Phil shows you how to derive measurements and metrics that your development and team can employ to benchmark, analyze, and improve both usability and UX. Beyond the measurements, Phil discusses case studies in which measurements have driven significant usability and user experience improvements.

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Why and How of Usability and User Experience (UX) Testing

  1. 1.           BT12 Concurrent Session  11/8/2012 3:45 PM                "The Why and How of Usability and User Experience (UX) Testing"       Presented by: Philip Lew XBOSoft               Brought to you by:        340 Corporate Way, Suite 300, Orange Park, FL 32073  888‐268‐8770 ∙ 904‐278‐0524 ∙ ∙
  2. 2. Philip Lew XBOSoft, Inc. After working for twenty years in various management and technical positions in product development and management, CEOPhilip Lew now leads the direction and strategy of XBOSoft ( He has helped hundreds of organizations assess the quality of their software, examine software quality processes, and set forth measurement plans so they can consistently improve software quality using systematic methods. Phil has authored articles in IEEE and ACM journal publications and trade journals; presented at several conferences on web application usability, user experience, and quality evaluation; and is conducting postdoctorate research focused on user experience measurement frameworks and software learnability.
  3. 3. The Why and How of Usability and User Experience (UX) Testing Philip Lew Agenda • What is usability and UX and why important? i t t? • Usability and UX – What are they • How to set up models and measurements • Case study and examples in testing and evaluating applications 1
  4. 4. What is Usability-UX and Why is it Important The Year of Mobile? • Last year was the year of mobile in terms of th growth i b th mobile usage and th f the th in both bil d the availability of mobile sites and mobileapps. • Time to redesign your mobile site, because your existing version is probably far below users growing expectations for users' user experience quality. • What do you think users expect? 4 2
  5. 5. History Repeats Itself • In the beginning, the Web was experimental —it was acceptable to have a somewhat shaky, experimental website. • Many sites were crippled by misguided design advice • Many didn't know any better (because they didn't do usability studies). Now, people simply expect websites to work. • For mobile, last year, it might have been cool simply to mobile year have an app. • Now, that app better be good. Expectations have gone up. 5 Web and Mobile User Expectations • Business models have changed – Instead of paying upfront and ‘owning’ the software – Pay as you go, pay by p subscription • Cloud and mobile converge • Behavior and expectations have changed 3
  6. 6. Basic Usability Concepts Usability  Design Usability  Effect User  Experience 7 Design-Test and Evaluate • What will the  mobileapp do? mobileapp do? • Is it a conversion  of existing app? Test and  • What functions  Evaluate will a user really  access? Design Test and  Evaluate Release 8 4
  7. 7. Usability - Design Perspective • • • • • • Understandability Learn-ability Operability Attractiveness Navigation Responsiveness-performance What else can you think  of? 9 Usability-Effect Degree t which specified D to hi h ifi d users can achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. Source: ISO 25010 10 5
  8. 8. Usability-Effect “Context” and “Specified” specified users • User role • Objective specified goals • Task • Environment specified context of use • Domain What else can you think  •… of? 11 Usability-Effect User Experience • Satisfaction – The degree to which users are g satisfied in a specified context of use. Satisfaction is further subdivided into subcharacteristics: • Likability (cognitive satisfaction) • Pleasure (emotional satisfaction) • Comfort (physical satisfaction) • Trust • Including many other factors experienced over time and other channels Source: ISO 25010 12 6
  9. 9. Task Ease (effectiveness and efficiency) • Buttons are a key tool in the user experience designer’s box of tricks. • Prioritize tasks - Some tasks more important than others. • Understand the objectives of the application • Understand which tasks are really important. • Majority of your app’s value is provided by a small number of tasks. • Remove any friction that inhibits the user’s progress. • Choose your words carefully to make it clear what the buttons or functions do What could  add ‘friction’ ? 13 Design and Test For the Task • Highly specific tasks. – You are in an electronics store looking at the Minolta GS234 camera. It costs $120 25 in the store Check camera $120.25 store. to see if you can get a better price online. • Directed, but less specific. – Find a moisturizer with SPF 45 or above that is suitable for your skin." (Use Walmart app.) • Open-ended, but restricted to a predetermined site-app. – "See if you can find any interesting pictures related to today's news." (While using the USA Today app.) • Web-wide tasks that let users go anywhere. – "Find out which is the tallest building in the world." 14 7
  10. 10. Usability Basic Design Issues and Concepts Design Testing and Evaluation 15 Design Concepts  Test and  Evaluation Criteria • • • • Context Positioning Color usage Size usage Size usage 8
  11. 11. User Context is King • Mis-Targeted user background can b k d cause loss of effectiveness • Know your users. Who are your users? What are their special  needs/characteristics? 17 Positioning • Effectiveness of application buttons pp is affected by position. • Emphasis is gained by – removing clutter – placing the button where the user’s eyes will mostly likely be. 18 9
  12. 12. Using Color • Color creates emphasis. • Dependent on the importance of the application’s function. • Make easy to for user to see the most important stuff. • Color can affect usefulness. usefulness 19 LinkedIn - Color Create an  account!!! • The ‘View Full Profile’ button – LinkedIn wants you to click that. – You’ll be prompted to create an account! • Button has a unique color that isn’t shared with any other part of the design. • Page has tons of information, this button still stands out. 20 10
  13. 13. Don’t Overuse a Color 21 Size Matters • Time taken to point at an object is directly influenced by the size of that object. • Big is beautiful depending on function-what you want the user to do. • Size can dictate the button’s importance over everything else on the page. • Take the Firefox page for example, Mozilla don’t mess around with subtlety here. y • Good design communicates priority. • With one massive button on the page, that priority is obvious. 22 11
  14. 14. Using Size AND Color 23 Multivariate Testing • Small changes can make a big difference. difference • Laura Ashley arrived at this design following multi-variate testing of 5 different options • Differences in the testing included: g – Link colors and locations • The "Go to checkout" button was dark gray instead of green • Achieved 11% increase in checkouts 24 12
  15. 15. Let’s Go Mobile Usability and UX For Mobile 25 Usability - Desktop to Mobile • Just loading websites on the phone is not enough • What matters for users – Time to load the website – User interface – Accessibility of various functions available • Usability design needs to change, otherwise usability effect and UX will suffer 26 13
  16. 16. Different Platforms Need Different Design • Horizontal swiping now used to "flip" through deck-ofcards or carousel features. – Include a visible cue when people can swipe • Avoid swipe ambiguity: don't use the same swipe gesture to mean different things on different areas of the same screen. – Use the same meanings for mobile phones and tablets – Mobile-device users typically expect to horizontally swipe while desktop websites avoid horizontal scrolling • Mobile sites should perform better than full sites when used on a mobile device. Nov 2012 Better Software East 2012 27 Let’s Step Through a Few Examples Usability and UX For Mobile 28 14
  17. 17. Laura Ashley Mobile Site • Same as web version • G Green sticks out ti k t 29 Laura Ashley • What is wrong here? 30 15
  18. 18. Laura Ashley – 9/2012 31 LA MS • Consistent green • Ob i Obviously want you t l t to click and add to the bag 32 16
  19. 19. LA Shopping bag • Clear green signals… BUY 33 LA MobileApp • Main Screen • D Decent size b tt t i buttons • No priority or importance weighting 34 17
  20. 20. LA Mobile App • Big buttons • G d for big fi Good f bi fingers • Semi-meaningful icons on the bottom – Note contextual location 35 Buy or Share? • No placement of importance or priority 36 18
  21. 21. Select Size • Good navigation • G d size b tt Good i buttons • No priority 37 In my Basket • What should I do now? • Buy or share? 38 19
  22. 22. Shopping Basket • Empty or Checkout? 39 Bloomberg • Simple Information App • Simple colors • Meaningful Icons • Big enough to see 40 20
  23. 23. Fidelity • Easy to use scroll – Thumb Friendly • Simple colors • Very limited task based functionality 41 Fidelity • Size and Importance • P tt easy to see Pretty t what they want you to do • Consistent icons on bottom – Si l Simple – Not everything 42 21
  24. 24. Yelp • Easy on the eyes • S i Semi-meaningful i f l icons 43 Yelp • Consistent • Fi Finger friendly f i dl • Consistent location of icons on bottom Similar to Fidelity What does this mean? 44 22
  25. 25. Making Assumptions • Web sites automatically sense the platform and switch t a ‘ bil site’ it h to ‘mobile it ’ (for instance, United Airlines, Google, and others) – Provide mobile users with more efficient web e pe e ce experience. • Has the potential of making too many assumptions regarding the users’ expectations. 45 Web-Mobile Integration • Overall integration of web-apps needs to be thought out carefully with mobile scenarios in mind. • Friend wanted to show me a product at • He sent me a link from his iPhone. I was at my desktop, and opened the link and got the mobile version. Nov 2012 Better Software East 2012 46 23
  26. 26. Give a Choice to Avoid Making the Wrong Assumption • Managing user expectations of how the application should behave needs to be thought out carefully. • Context of the user and their expectations is key • LinkedIn gives you a choice when accessing their site; an optimized mobile experience via an mobileapp, or just their ordinary website. Nov 2012 Better Software East 2012 47 Determine the Needs at Hand • What need is the mobile application attempting to mobilize? • How can the workflow on a mobile device be designed more efficiently to accomplish the task? • How can the specific features and characteristics of a mobile device improve and complement the experience in contrast to the normal web-based p application or other mobile platforms? Nov 2012 Better Software  East 2012 48 24
  27. 27. Determine the Needs at Hand • Fidelity implements top scenarios their users want and optimizing efficiency for p g y those user scenarios ONLY. • Filling an order was a user scenario optimized just for the mobile platform. • Note that it only has 4 data items to fill in with 2 being in, scroll buttons, while the biggest button indicates precisely what the purpose of the task is. Nov 2012 Better Software  East 2012 49 Mobile Usability Design Best Practices Summary • Quick - Can you read the text without zooming or scrolling? • Simple Navigation-Task Oriented • Thumb Friendly Design for the small screen. Don’t make users struggle to click tiny areas that are • Visibility much smaller than their fingers. • Easy to Convert/Complete the task • Contextual • Seamless with main webapp • When you have a smaller screen, you must limit the number of features to those that matter the most for the mobile use case. 50 25
  28. 28. What Is Designed, We Test Understanding Design Leads to Better U d t di B tt Understanding, E l ti Evaluation, and Testing What does want to achieve in the Usability UX Standpoint? 52 26
  29. 29. Introduction to • iMedTablet - patient care coordination for all types of long term care providers long-term providers. • Allows the user to securely access and document patient care information in realtime utilizing Cloud technology, GPS technology and intra-facility gy y communications. • Designed to seamlessly integrate with many pharmacy and long-term care software systems. 53 - Login Screen 54 27
  30. 30. -Med Admin Record 55 • Users are _________. • Heavily context oriented based on user  expectations and habits. – Best design is not equal to the best  usability. 28
  31. 31. Setting Up Usability Measurements Measure and Improve 57 Big Picture on Usability and UX Usability  Design Usability  Effect User  Experience 58 29
  32. 32. Design-Test and Evaluate Design Test and  Evaluate Test and  Evaluate Release 59 Let’s Define Usability Product (Design) Viewpoint Usability Characteristic 1 Navigation Subcharacteristic 1 Control Stability Position Attribute 1 • • • Characteristic 2 Simpleness Subcharacteristic 2 Button Visability # Buttons Attribute 2 Characteristic n Subcharacteristic  n Color Usage Attributes expressed hierarchically Any number of sub-levels is OK Achieving the sub-attributes=achieving the high level attribute -measurable 30
  33. 33. Defining Usability from an Effect-Real usage Point of View Quality y y Usability Effectiveness Efficiency Accuracy Errors Satisfaction Completeness Attribute 2 Attribute 3 Characteristic n Subcharacteristic  n Attribute 1 Potential Attributes Measurements for ‘Effect’ Usability • Effectiveness – Completion rates – Error rate – Help usage • Efficiency – Task time/Speed • Learnability – Learning rate – Task time deviation – Change in task times over time period 31
  34. 34. Measurable Attributes • • • • • • • • Attribute name Description and purpose Need a model before you can How to measure start measuring… What is measured Measurement/Calculation Range (min, max) Objective Current Usability Measurement Methods Focus  groups Walk Throughs Satisfaction S if i Surveys Labs Heuristic  Evaluation Logging Let’s get started 32
  35. 35. Usability Measurement Attribute Scale How Measure or Objective Current Calculation Help Access Percent  of Server Log  Users  Users files Accessing Help Task  Simplicity % 30% 40% Keystrokes  Heuristic  % to measurement find/use a  feature/fu nction/inf ormation 90% < 3 50% Task Times Time  Server log  finish‐time  files start % 90% < 3 50% Accuracy Number errors Integer or  % <5 10 Server Log  files 65 Sample Heuristic Evaluation Usability 1. Understandability 1.1  Label/icon recognizable 1.2  Function‐task recognizable 2. Navigation 2.1  Back‐Forth 2.2  Full‐mobile 3. Swiping 3.1  Consistency 3.2  User Awareness y 4  Learnability 4.1  Error prevention 4.2  Error detection Total Measure Elementary 2 4 50% 100% 3 2 75% 50% 3 2 75% 50% 1 1 Partial 75% 33% 33% 63% 63% 33% 58% Through defining the model, then benchmarking with heuristic evaluation, we were able to determine weak areas and make recommendations for significant improvements Nov 2012 66 33
  36. 36. Usability Logging Measurement and Data Collection • Identify users by using session ID to identify a unique user. • Iteratively insert code into the application • Collect data • Analyze the data for each attribute in different dimensions and aggregations • Determine the need for further calculations and what attributes to measure further • Revise the data we are collecting, adding or decreasing granularity 67 Satisfaction Surveys The process: 1. Calculate the usability score (satisfaction) of  1 Calculate the usability score (satisfaction) of version X 2. Do the survey 3. Change to version X.1 – make changes to the  software directly correlated to the usability  factors to either increase or decrease the  factors to either increase or decrease the usability score 4. Do the survey again  5. See if differences made change impact the  survey results 68 34
  37. 37. Putting It All Together Usability Design User  Satisfaction Usability in  Use User  Experience 69 Getting Started with Measurement • Produce an action plan – What usability attributes are important to your organization? • Develop a model – What data can you collect/Which technique can you use • Maybe some elements of the model drop out-can’t be measured that easily – Start collecting and developing benchmarks 70 35
  38. 38. Conclusion-Usability and UX • Paramount for today’s applications with users’ short attention spans. • Designs should be as simple as possible for the user user. • UX is heavily influenced by expectations. • Make navigation easy on a small screen, thumb friendly, and as intuitive as possible. • If the user cannot figure it out in 30 seconds, they are gone. • Mobile-app versus your web-app - differentiate and discern access from each platform. • Determine the features you really need and optimize screens for certain workflows rather than trying to do it all. 71 Conclusion-Measurement • Defining UX is different for each organization y g • Need a model for your organization – What is most important to you depends on your users and their behavior and expectations • The model is the foundation of what to measure, evaluate and improve 72 36
  39. 39. Thanks Questions and Answers 408-350-0508 Philip Lew 73 37