Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Statistical user experience measurement 1

1,763 views

Published on

Published in: Technology
  • Be the first to comment

Statistical user experience measurement 1

  1. 1. User Experience Measurement<br />Prototype Comparison<br />Sharon Harper - Taxonomist & User Experience Consultant<br />
  2. 2. Sharon Harper - Taxonomist & User Experience Consultant<br />User Experience MeasurementTaxonomy Structure, Organizational, Navigational Difference In Prototype Designs<br />
  3. 3. Sharon Harper - Taxonomist & User Experience Consultant<br />Statistical Measurement Strategy Compare Means - task completion time<br />3 or more prototypes<br />Across different participants<br />(independent samples)<br />≥ 30 participants<br />z-test<br />2 prototypes<br />Within same group of participants<br />(paired samples)<br />< 30 participants<br />t-test<br />
  4. 4. Null Hypothesis . there is no difference in the usability (performance & satisfaction) of the new user interface and the previous interface. Neither the taxonomy integration, organization or navigation efforts have improved the user experience.<br />(New user interface) – (Earlier user interface) = 0<br />Measure. What difference in the values of the means of two interfaces can be obtained that would disprove this hypothesis? The difference of the paired samples means by the same participants is relative to the spread or variability of their scores (time on successful task completion between prototype interfaces). This is the t value<br />Sharon Harper - Taxonomist & User Experience Consultant<br />T – test: Paired Two Samples for Means Comparison<br />
  5. 5. Compare this result, the t-value, to determine whether the result is statistically significant. Assign risk factor of 5% (α= 0.05)to ensure 95% significance.<br />Number of paired measurements (participants) minus 1 is the degree of freedom, df. <br />With the acceptable risk factor of 5% or 95% significance, df, and t-value the hypothesis is tested.<br />The resulting p value is less than 0.05 then the null hypothesis is NOT TRUE The time on task for the two interfaces is significantly different, that is the new interface is better<br />Sharon Harper - Taxonomist & User Experience Consultant<br />T – test: Paired Two Samples for Means Comparison (cont.)<br />
  6. 6. Sharon Harper - Taxonomist & User Experience Consultant<br />Excel™ Data Analysis Add In<br />Other software packages available : SPSS, SAS , MiniLab, http://usablestats.com …<br />Explanatory resources: <br />The Cartoon Guide to Statistics by Gonick & Smith<br />Introductory Statistics by Ross<br />Web Center for Social Research Methods<br />http://www.socialresearchmethods.net/kb/stat_t.php<br />
  7. 7. Sharon Harper - Taxonomist & User Experience Consultant<br />Install Excel Data Analysis Add In<br />
  8. 8. Sharon Harper - Taxonomist & User Experience Consultant<br />Time on Task t-Test paired sample for means<br />
  9. 9. Sharon Harper - Taxonomist & User Experience Consultant<br />Result<br />
  10. 10. Misinformation about statistical measurement is strongly ingrained, ‘math anxiety’<br />‘school’ marketing presenters, anticipate<br />KIS(s) , use examples from others domains: medicine, linguistics, UX development<br /> NOT pages of formulas, jargon<br />Do not show p-value = .001293051, use .00129 or .001<br />Usability analysts should have rudimentary statistical knowledge and able to read power tables identifying the minimum sample size and acceptable risk from that sample size.<br /> Other methods should accompany quantitative methods<br />Sharon Harper - Taxonomist & User Experience Consultant<br />Report/Presentation Elements<br />
  11. 11. UX Resources<br />Tullis & Albert. (2008) Measuring The User Experience: Collecting, Analyzing, and Presenting Usability Metrics.<br />Sauro. A Practical Guide to Measuring Usability: 72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software<br />Sauro & Lewis. (2012) Practical Statistics for User Research.<br />http://www.measuringusability.com<br />Sharon Harper - Taxonomist & User Experience Consultant<br />Questions?<br />
  12. 12. Sharon Harper - Taxonomist & User Experience Consultant<br />Questions?<br />

×