Data Driven Design: Using Web Analytics to Improve Information Architectures

Slideshow Transcript

1. Slide 1: Data Driven Design Using Web Analytics to Improve Information Architectures Andrea Wiggins IA Summit 2007 SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
2. Slide 2: Mo tivatio n: What Info rmatio n Archite cts Want to Kno w Interviewees said: s – Context for making design decisions – Validation of heuristic assumptions – Understand why visitors come to the site & what they seek SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
3. Slide 3: Age nda Overview for Context s Insert show of hands here! (topic, tools, data) s What is web analytics (WA)? How is it done? s – major WA concepts – what the data look like IA questions to answer s Rubinoff’s user experience audit s Some WA measures for heuristic validation s SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
4. Slide 4: What is w e b analytics? Data mining from web traffic logs s – Web server log files – Page tag logs from client-side data collection (end up in server logs) – Cookies to identify “unique visitors” What for? s – Proving web site value (ROI) – Marketing campaign evaluation – Executive decision making - markets & products – Web site design parameters – More… SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
5. Slide 5: Ho w do yo u do it? Vendor analysis solutions s • Hosted ASP – Currently most popular model – Provides traffic stats “on-demand” • Software – Runs on dedicated servers – Scalability: requires significant data storage space and data maintenance • Costs – Starts at FREE for Google Analytics and goes way, way up – Large organizations spend $50K/yr and up Open source: not a robust option s SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
6. Slide 6: Ve ry Quick Majo r Co nce pts Sessionizing (cookie > IP & UA) s Hits: all server requests s Pageviews: all server requests for page s filetypes, variously defined Visits & Visitors: stronger measures from s sessionizing, sensitive to time periods SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
7. Slide 7: Sample Lo gs #Software: Microsoft Internet Information Services 6.0 #Version: 1.0 #Date: 2005-08-01 00:00:35 #Fields: date time cs-method cs-uri-stem cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc- status sc-bytes 2005-08-01 00:10:05 GET /index.htm - 216.xx.76.7 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+98) http://search.yahoo.com/search?p=purple+rose+theater&sm=Yahoo%21+Search&fr=FP-tab-web-t- 280&toggle=1&cop=&ei=UTF-8 200 13099 2005-08-01 00:10:29 GET /current.html - 216.xx.76.7 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+98) http://www.purplerosetheatre.org/ 200 17985 2005-08-01 00:11:24 GET /tickets.html - 216.xx.76.7 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+98) http://www.purplerosetheatre.org/current.html 200 15689 2005-08-01 00:18:06 GET /index.htm - 152.xxx.100.11 HTTP/1.0 Mozilla/4.0+(compatible;+MSIE+6.0;+AOL+9.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) http://www.guide2detroit.com/arts/stage-calendar.shtml 304 300 2005-08-01 00:20:18 GET /index.htm - 68.xx.117.55 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) http://www.google.com/search?hl=en&q=purple+rose+theatre 200 13099 2005-08-01 00:20:21 GET /classes.html - 68.xx.117.55 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) http://www.purplerosetheatre.org/ 200 15296 SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
8. Slide 8: Spide rs 2005-08-01 00:49:32 GET /robots.txt - 68.xxx.251.159 HTTP/1.0 s Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/y search/slurp) - 200 319 2005-08-01 00:49:32 GET /plays/completing_dahlia.html - 68.xxx.249.67 s HTTP/1.0 Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/y search/slurp) - 200 3507 SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
9. Slide 9: A Fe w Go o d Me trics Information Architects want to know: s – Confirmation of heuristics • Do users leave at first glance of this awful page? • Where do they click? • What position on the screen or layout produces the most clicks for the same content? • Do the users “pogo-stick” back and forth between pages? What are they comparing? – Ambient findability measures • At what hierarchy depth do visitors enter the site? How do they get in on deep pages? • Do they ever see the home page? • Can they find their way to where we want them to go? SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
10. Slide 10: Se arching fo r IA Answ e rs On-site search behaviors s – How many searches do users make? – Do users refine their search results? – What type of queries do users make? – How often are search results the last page? – From what pages are searches initiated? – Do the search terms have context in the page from which the search is initiated? – Why are users querying about chimpanzees?!? SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
11. Slide 11: What IAs Want Good navigation and content make the online s world go ‘round – Where in a process do users leave? Where do they go? Do they re-enter the process? – How do users move through the site? Is there a better route? – What pages don’t get visited? What pages get unexpectedly high visits? – What prompts conversion? – Where do search engine spiders go in the site? Is the best content being indexed? SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
12. Slide 12: Eve rybo dy Lo ve s Rubino ff UX audit quantifies subjective measures s – Offers structure for comparing properties of the site – Completely customizable, use strategically In a perfect world: s – Analyst & IA work together to set key performance indicators (KPI) and measurable heuristics – Each independently evaluates the site on the same points and compare the IA’s heuristics to user data for validation – They set before-and-after measures to prove value for the entire project SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
13. Slide 13: Rubino ff’s Fo ur Cate go rie s Using a sample of statements from Rubinoff’s s model: 1. Branding a) Engaging, memorable brand experience b) Value of multimedia & graphics 2. Functionality a) Server response time & technical errors b) Security & privacy practices 3. Usability a) Error prevention & recovery b) Supporting user goals & tasks 4. Content a) Navigation & site structure b) Search & referrals SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
14. Slide 14: 1a: Branding Me mo rable & Engaging Expe rie nce s Ratio of new to returning visitors is key; set s target KPI specific to site business goals Track trends over time and in relation to cross- s channel marketing Median visit length in minutes s Average visit length in pages viewed s Depth, breadth of visits s Segment new and returning visitors to examine s visit trends for different audiences SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
15. Slide 15: 1b: Branding Value o f Multime dia & Graphics Flash & AJAX require deciding upon what to s measure, programming appropriate data collection, and configuring analysis tools Plan to include measures when designing s multimedia applications to prove value Compare clickthrough rates for clickable s graphics to rates for standard navigation links Great tools like Crazy Egg’s heatmap - easy! s (also relevant to navigation, of course) SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
16. Slide 16: Crazy Egg He atmap Example SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
17. Slide 17: Crazy Egg Ove rlay Example SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
18. Slide 18: Crazy Egg List Example SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
19. Slide 19: 2a: Functio nality Re spo nse Time & Te chnical Erro rs Response time is a default log field, easy to s measure Check at peak load time to make sure site is s responding quickly enough Monitor the rate of 500 (server) errors: this s should be an extremely low number SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
20. Slide 20: 2b: Functio nality Se curity & Privacy Practice s A matter of design for measurement, not s measurement of design: considerations for designing a site that will be measured – Privacy best practices: • Give a short, accurate, easy to understand privacy statement and stand by your word • True first-party cookie – Security best practices: (from an IA/analytic POV) • SSL encryption on any transactional forms: lead generation, ecommerce, surveys • Secure file transfer for & restricted access to raw web analytic data; password restrictions at minimum SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
21. Slide 21: 3a: Usability Erro r Pre ve ntio n & Re co ve ry Percentage of visits experiencing 404 and 500 s errors: errors should be < 0.5% of all hits Percentage of visits including an error, that s end with an error - frustrated into leaving Where do 404 errors occur? s – Use to build a redirect page list to ensure (temporary) continuity of service to bookmarked URLs – Path/navigation analysis: how did users arrive at 404? What did they do after? User errors: identify problems & re-enact or s test SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
22. Slide 22: 3b: Usability Suppo rting Use r Go als & Tasks Scenario/conversion analysis s – Define tasks and procedures supporting user goals – Examine completion rates, step by step, intervals & overall • A to B, B to C, C to D; A to C, B to D; A to D – Look at leakage points • Where did they go when they left the process? Did they come back later? – Shopping cart analysis • Keep in mind that users shop online for offline purchases • Do behaviors suggest a need for a tool like a shipping calculator or product comparison? SCHO O L O F INFO RMATIO N – Online form completion UNIVERSITY O F MICHIGAN
23. Slide 23: 4a: Co nte nt Navigatio n & Site Structure Pogo-sticking: jumping back & forth between s content or hierarchy levels (what about tabs?) – Need a comparison tool, can’t identify product: not enough detail at the right level of site hierarchy or step of the purchase decision process Compare page-level traffic statistics for larger s trends, broad navigation analysis: the usual #s Path analysis on navigation tools (by type) to s pinpoint navigation and labeling problems – Extensive use of supplemental navigation may indicate need for updates to global navigation SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
24. Slide 24: 4b: Co nte nt Mining Se arch & Re fe rrals Popularity = value? What about findability? If s it’s not findable, it probably won’t be popular. – Compare the content’s value (against similar content) with proportions of returning visitors, average page viewing length, external referrals - especially search referrals Search log analysis: what do your users s value? – Does user query language match site contents? Are users searching for pantie s when you’re selling pants? SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
25. Slide 25: Validate the Match Be tw e e n the Site & the Re al Wo rld More ways to use search log analysis: s – Does user vocabulary match site vocabulary? – Do different audiences have different vocabularies, and does the site support them equally? – Brand measurement returns • product and industry terminology usage • “accuracy” of brand queries: spelling, inclusion of competitor’s brands, advertising slogans – Did users find what they expected? How many visits end on search results? Null results are revealing. SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
26. Slide 26: Language Validatio n SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
27. Slide 27: Co nclusio ns Not much out there in the academic literature s on using web analytics (hopefully to change!) WA data is flawed and tough to handle, but s ultimately pays off in developing holistic understanding of user behavior Best-suited to case studies s WA is ripe for adoption into formal usability s frameworks, particularly for persona design and determining design parameters Best used iteratively: beginning, middle, end, s annual follow-up… SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN
28. Slide 28: Thanks! Q ue stio ns? SCHO O L O F INFO RMATIO N UNIVERSITY O F MICHIGAN