Statistical user experience measurement 1

User Experience Measurement
Prototype Comparison
Sharon Harper - Taxonomist & User Experience Consultant

User Experience MeasurementTaxonomy Structure, Organizational, Navigational Difference In Prototype Designs

Statistical Measurement Strategy Compare Means - task completion time
3 or more prototypes
Across different participants
(independent samples)
≥ 30 participants
z-test
2 prototypes
Within same group of participants
(paired samples)
< 30 participants
t-test

Null Hypothesis . there is no difference in the usability (performance & satisfaction) of the new user interface and the previous interface. Neither the taxonomy integration, organization or navigation efforts have improved the user experience.
(New user interface) – (Earlier user interface) = 0
Measure. What difference in the values of the means of two interfaces can be obtained that would disprove this hypothesis? The difference of the paired samples means by the same participants is relative to the spread or variability of their scores (time on successful task completion between prototype interfaces). This is the t value
T – test: Paired Two Samples for Means Comparison

Compare this result, the t-value, to determine whether the result is statistically significant. Assign risk factor of 5% (α= 0.05)to ensure 95% significance.
Number of paired measurements (participants) minus 1 is the degree of freedom, df.
With the acceptable risk factor of 5% or 95% significance, df, and t-value the hypothesis is tested.
The resulting p value is less than 0.05 then the null hypothesis is NOT TRUE The time on task for the two interfaces is significantly different, that is the new interface is better
T – test: Paired Two Samples for Means Comparison (cont.)

Excel™ Data Analysis Add In
Other software packages available : SPSS, SAS , MiniLab, http://usablestats.com …
Explanatory resources:
The Cartoon Guide to Statistics by Gonick & Smith
Introductory Statistics by Ross
Web Center for Social Research Methods
http://www.socialresearchmethods.net/kb/stat_t.php

Install Excel Data Analysis Add In

Time on Task t-Test paired sample for means

Result

Misinformation about statistical measurement is strongly ingrained, ‘math anxiety’
‘school’ marketing presenters, anticipate
KIS(s) , use examples from others domains: medicine, linguistics, UX development
NOT pages of formulas, jargon
Do not show p-value = .001293051, use .00129 or .001
Usability analysts should have rudimentary statistical knowledge and able to read power tables identifying the minimum sample size and acceptable risk from that sample size.
Other methods should accompany quantitative methods
Report/Presentation Elements

UX Resources
Tullis & Albert. (2008) Measuring The User Experience: Collecting, Analyzing, and Presenting Usability Metrics.
Sauro. A Practical Guide to Measuring Usability: 72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software
Sauro & Lewis. (2012) Practical Statistics for User Research.
http://www.measuringusability.com
Questions?

Questions?

Statistical user experience measurement 1

by Sharon Harper on Jun 26, 2011

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 53

Statistics

Statistical user experience measurement 1 Presentation Transcript