Unobtrusive Usability Testing: Creating Measurable Goals to Evaluate a Website
The Unobtrusive “Usability Test”: Creating Measurable Goals to Evaluate a Website Tabatha Farney Web Services Librarian University of Colorado Colorado Springs Paper at: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferen ces/confsandpreconfs/2013/papers/Farney_Unobtrusive.pdf Presented at the 2013 ACRL Conference.
Usability is a constant processHow can we keep up with the pace of changing library users?
What is Web Analytics?The process of collecting, reporting, and analyzing websiteusage data to reveal how users are actually interacting witha website in order to make improvements.
Common Web Analytics Metrics Basic Usage Engagement Goals Visits Duration of Visit Conversions Visitors Page Depth Page Views Bounce Rate …and the list goes on…
All that data is gatherseamlessly into a web analytics tool withoutbothering the end user.
Usability Testing Web Analytics + = “Validation” Testing
Validation testing takes placeafter the website is finishedand evaluates that the sitemeets its desired goals basedon set benchmarks.Requires a large group of users andsimple usability benchmarks.
Applying Web Analytics to Usability Testing Usability Process* Applying WA1. Understand test’s Unchangedpurpose and goals2. Determine the research Unchangedquestions3. ID a target audience Website users (can further segment this group) Tasks must be measurable4. Select the tasks in website use data.5. Define the usability Select the web analyticsmetrics and reports metrics and reports.6. Report the findings Unchanged * Modified from Rubin and Chisnell (2008).
Identifying Usability MetricsTask 1: Find a research database withinfour web pages of the visit. Task Completion Success Rate Metric: Conversion Metric: Successful Conversions Total Conversions Report: Goal w/ custom Report: Goal w/ custom segment segment Time to Complete Task Common Pathways Metric: Average Time on Site (average); Visit Duration (mean) Metric: Page Views Report: Audience Overview; Report: Reverse Goal Path Audience Engagement
Example Report Sample report in Google Analytics
Reporting September – November 2012 Task 1Total Conversions 10154 Any success Total Successes 6778 under 65% Total Failures 3376 triggers the Success Rate 66.84% error analysis process.Average Time on Site 0:09:40Visit Duration 0-10 seconds 2187 11-30 seconds 2132 31-60 seconds 798 61-180 seconds 1153 181-600 seconds 1343 601-1800 seconds 1587 1801+ seconds 954
Drawbacks & Benefits Drawbacks Benefits Error analysis is a Easy to regularly long process assess website usage Inability to track Simple to implement individual users Many WA tools have automatic export Only data from website users Can be combined with other usability efforts Data focuses on actions, not Many libraries already motivations use a WA tool
Recommendations for You Reuse Keep test Invest in error previous goals and analysis and usability tests Test the test. tasks reporting to guide manageable. steps. goal/task creation.
Summary• Apply the structure of usability testing to your website use data• Make your tasks measurable using website use data• Be prepared to report the data regularly• Error analysis is an essential process• Make changes based on the data and test again!
ResourcesCitations for Slides Images Used “Students Moving through HUB,” pennstatenews, Flickr,Barnum, Carol M. Usability Testing Essentials:Ready, Set—Test! Boston: Morgan Kaufmann http://www.flickr.com/photos/pennstatelive/4950729405Publishers, 2011. (slide 2). “Web Analytics Framework,” BeantinFarney, Tabatha and Nina McHale. Web Analytics webbkommunikation, Flickr,Strategies for Information Professionals: A LITA http://www.flickr.com/photos/beantin/7649183772/Guide. Chicago: ALA TechSource, 2013. (slide 3).Nielsen, Jakob and Hoa Loranger. Prioritizing Web “Peanut Butter,” sparktography, Flickr,Usability. Berkeley: New Riders, 2006. http://www.flickr.com/photos/sparktography/410446108/ (slide 6).Rubin, Jeffrey and Dana Chisnell. Handbook ofUsability Testing: How to Plan, Design, and “Kshocolat Dark Chocolate Bar,” Boz Bros, Flickr,Conduct Effective Tests, 2nd ed. Indianapolis:Wiley, 2008. http://www.flickr.com/photos/sparktography/410446108/ (slide 6).Tullis, Tom and Bill Albert. Measuring the User “Giant Reese’s Peanut Butter Cup,” camknows, Flickr,Experience: Collecting, Analyzing, and http://www.flickr.com/photos/camknows/8265569228/Presenting Usability Metrics. Boston: MorganKaufmann Publishers, 2008. (slide 6). “Labs are Full!” clemsonunivlibrary, Flickr,Turner, Steven J. “Demonstrating Success: Web http://www.flickr.com/photos/clemsonunivlibrary/6096116Analytics and Continuous Improvement.” Journal 865/ (slide 7).of Web Librarianship 6, no.1 (2012): 45-55. “Thumbs Up,” P1r, Flickr, http://www.flickr.com/photos/p1r/4730426448/ (slide 14).