PxS'12 - week 12 - ux evaluation


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

PxS'12 - week 12 - ux evaluation

  1. 1. EPFL, spring 2012 - week 12!UX evaluation
  2. 2. today’s schedule➝  review of UX evaluation plans➝  test run of scripts➝  individual walk throughs
  3. 3. overview➝  about UX evaluations➝  UX evaluation criteria➝  UX evaluation methods➝  tips and hints➝  for more information ✱Attribution: parts of this presentation are based on " Forum Nokia’s UX Evaluation series
  4. 4. why UX evaluation?Success factor ➝  UX is a critical success factor for products and services ➝  We need confirmation during design and development that we are on the right track ➝  It is challenging for the software developer to keep anThe designer is objective view of the developed software not the user ➝  It is human nature to become blind of your own outcomes ➝  Thus it is important to get a second opinion of the UX, from outside of the development teamSimulate real ➝  Preferably from real users trying it in a realisticuse situations environment
  5. 5. usability and product quality SW Product Quality (ISO-9126) Functionality Reliability Efficiency Usability Portability Maintainability Suitability Maturity Time behavior Operability Installability Testability Accuracy Fault tolerance Resource Learnability Adaptability Stability utilization … … ... … … … Implemented Code Number of sw Function point Source lines of functional coverage bugs analysis code requirements➝  there are many metrics to measure software performance and quality
  6. 6. UX and product quality In this definition, UX is not a part of product quality. A SW SW Product Quality (ISO-9126) engineering point-of-view? Functionality Reliability Efficiency Usability Portability Maintainability Suitability Maturity Time behavior Operability Installability Testability Accuracy Fault tolerance Resource Learnability Adaptability Stability utilization … … ... … … …➝  product quality Expectations, motives, ➝  measuring UX is not straight- has a strong actions, interpretations, forward impact on UX … ➝  there are many intervening issues User Experience
  7. 7. UX evaluation criteria 1/5➝  Nielsen (1993) on usability ➝  learnability ➝  Novices’ ability to reach a reasonable level of performance rapidly UX ➝  efficiency ➝  Expert user’s level of performance ➝  errors utility usabi social enjoy lity value ment ➝  Number of errors users make ➝  User’s ability to recover from errors pleasure stimulation useful relia ease- effici acces connec identifi sibility ting cation ➝  Existence of catastrophic errors? ness bility of-use ency people ➝  memorability ➝  Casual user’s ability to remember drivers tailored issues how to use a system as such for your detailed for ➝  satisfaction product your product ➝  User’s subjective assessment of the system basic usability heuristic UX drivers derived from the UX elements
  8. 8. UX evaluation criteria 2/5 Melvin, 35! Jason, 25! Louise, 27! use case storyboard accessing phone from web browser •  Engineer from Germany •  Customer Service •  Web designer, graduate •  Lives with his wife, 2 kids Assistant. student on university. and a dog. •  Keeps a blog for his friends •  SMS addict - 50 •  Both parents need to and family messages every day. sometimes travel due their Uses his device for capturing •  Likes arts, especially indie jobs, and kids have lots of images to blog. movies and old films. hobbies. Most of the images •  Is currently organizing •  Would like better manage surprise parties for her related to snowboarding. his everyday schedules friend. •  Plays guitar in a wildly with family. not-yet-so-popular band.➝  Evaluate the design against the specified UX artefacts
  9. 9. UX evaluation criteria 3/5➝  evaluate against benchmark ➝  products and experiences
  10. 10. UX evaluation criteria 4/5 Bar Slide Full hw qwerty Full sw qwerty Touch screen Touch only screen Resistive screen Capacitative screenClamshell Swivel➝  Run your tests with all the target devices and form factors
  11. 11. UX evaluation criteria 5/5 people places context Things time Indirect, unspoken criteria Direct, spoken criteria - originating from the context ➝  the ultimate criterion: ➝  potential users try the product in their mobile context
  12. 12. planning the UX evaluations 1/2 A step-by-step process would specify needs be easy to and context of define the UX create UI evaluate the UI manage. use concept designs designs In practice, design and evaluation are specify needs define the UX often in a parallel create UI designs and iterative and context of use concept relationship. evaluate concepts and designs➝  for any UX artifacts, you can think can and should it be evaluated, somehow
  13. 13. planning the UX evaluations 2/2My cool travel-mate concept ➝  What artefacts do we want to evaluate ➝  Sketches, visualisations, prototypes,... ➝  What issues are we evaluating? ➝  Concept, interaction, grahical design, ... ➝  Who will do the evaluation? ➝  Experts, end users,... Evaluate concepts and designs
  14. 14. UX evalution: expert evaluation ➝  Usability experts evaluate a system based on common heuristics (i.e. design principles) and personal expertise ➝  Common heuristics are based on knowledge gained through expertise and experience ➝  Knowledge about perceptional and cognitive processes - such as the function of memory – are utilized ➝  At least two experts take part in the evaluation to ensure reliable results ➝  At the beginning of an expert evaluation, the value of the system to its user and buyer/provider is determined ➝  Based on these values, usability criteria are defined ➝  At the end, the findings documented and prioritized. Solutions proposals are made ➝  Also user tasks can be defined and walkthroughs conducted as part of the evaluation
  15. 15. UX evaluation: focus groups ➝  a group of people (4-7) that has something in common will be invited to semi-structured group discussion session to share their views on certain topic. Lasts usually 2 hours ➝  face-to-face sessions ➝  Sometimes on-line ➝  variations of the method ➝  Exploratory ➝  Feature prioritization ➝  Competitive analysis ➝  Trend explanation
  16. 16. UX evaluation: usability testing➝  the test user is asked to conduct certain tasks with the user interface under design ➝  paper prototypes can be used ➝  users are asked to think aloud Special equipment can be useful to catch the Moderator actions on mobile Observer devices Tested sw Test user Test cases➝  sometimes good to test the UI with pairs of users to trigger more discussion or to test social aspects➝  usability problems are identified and prioritized based on the user data gathered during the usability tests
  17. 17. UX evaluation: observation ➝  the researcher does not interrupt the user but just observes the user s actions ➝  the user is followed usually shorter time than when shadowing in her/his own environment, and usually only certain activities/time period are observed ➝  good for studying users that cannot be interrupted ➝  reveals task flows and possible detectableIf observation is not possible (e.g. in hospital) video problems in them and behavioral patternsrecording can be alternative. but not motivations, attitudes, values, Video can be gone through with the participant(s) concerns afterwards
  18. 18. dont listen to users➝  watch users attempting to perform tasks in the UI ➝  watch what people actually do ➝  be careful with what people say they do ➝  dont believe what people predict they may do in the future.
  19. 19. when you listenbeware:➝  users often tell you what they think you want to hear➝  often based only on recall➝  rationalized behavior
  20. 20. UX evaluation: make a mix ➝  an expert evaluation and a usability test can be used as complementary methods ➝  an expert evaluation require less time and preparation than a usability test ➝  expert evaluations are recommended before usability tests in order to support designing the test tasks ➝  a satisfaction questionnaire can be combined with a usability test
  21. 21. UX evaluation: tips and hints ➝  the UX evaluation criteria should be in-line with the defined UX drivers, targets and requirements ➝  make sure to share your UX evaluation criteria with the designers before they start their work ➝  do evaluations with real users ➝  expert evaluations can provide quick useful findings ➝  try to have a end user mindset ➝ ideally your evaluation methods should scale to weekly (or designevaluate even daily) evaluations ➝  design & evaluation iteration loops do not have to be iterate big monolithic steps ➝  UX evaluation(s) should be an integral part of software development process, just like software testing