Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UXD v. Analytics - IAS13 Baltimore


Published on

**This version of our presentation is for The Information Architecture Summit, Apr 7 2013 in Baltimore** Chris and Farris expose the differences between how user experience designers and analytics practitioners think. While UXD weave best practices and user research into their designs, digital analysts spend their time confirming or refuting hypotheses in a data-driven way. One approach is decidedly qualitative, the other decidedly quantitative. In this presentation you will learn through their conversations how it is possible to leverage both enlightened design and deep data to continuously optimize user experiences. If you work on either side of this debate, this is how to better state your case… and get along with the other side.

Published in: Technology
  • Be the first to comment

UXD v. Analytics - IAS13 Baltimore

  1. 1. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure UXD v. Analytics What have YOU got for me? No, what have you got for ME? Information Architecture Summit Baltimore, MD | April 7th 2013 Chris Farnum, Senior User Experience Designer, ProQuest Farris Khan, Analytics Lead, ProQuest & BobbleheadGuru blogger 1
  2. 2. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDramatis Personae Digital Analytics Lead..…Farris UXD/IA………………………..Chris *Identities to be revealed upon proposal acceptance. ;-) 2
  3. 3. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureIntroduction – UXD v. Digital Analytics Where do these specialties overlap? How are the roles different? Why should they collaborate? Got a question or comment? Tweet: @BobbleheadGuru or @crfarnum Hashtags: #ias13 #measure 3
  4. 4. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe ProQuest Search Platform 4
  5. 5. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure 5
  6. 6. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe UXD Role at ProQuest Core UXD Design Processes Prioritized Develop/Test requirements Planning Iteration release Product release 6
  7. 7. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe UXD Approach, In General… 7
  8. 8. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureA newcomer to the department… 8
  9. 9. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureOverlapping Territories? Competition? 9
  10. 10. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureWhat is Digital Analytics? 10
  11. 11. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEcosystem of Analytics Object of Analysis: Document, Library, System Metrics: What? UsageObject of Analysis: Object of Analysis:Visitor Attitudinal/ Behavioral/ VisitorMetrics: Persona Observational Metrics:Who? Why? Where? When? Common Unit of Analysis 11
  12. 12. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureMonthly Analytics Heatmap Notes: 1. Data from March 2012 through February 2013. 2. Source: WebTrends and Internal data collection. 12
  13. 13. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureVisits: Year-Over-Year Comparison 13
  14. 14. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureVisits/Visitors 14
  15. 15. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureWeekly Analytics Heatmap 15
  16. 16. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureYear Over Year Visits Per Week Thanksgiving (US) Easter Christmas 16
  17. 17. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDaily Analytics Heatmap 17
  18. 18. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureVisits per Day(Since February 13, 2011)
  19. 19. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measurePercentage of Visits with Error Pages Notes: 1. Data from January 1, 2012 through March 17, 2013. 2. Source: WebTrends 3. Four “Spike Days”: Feb. 1, 2012, March 17, 2012 and March 1, 2013. 19
  20. 20. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measurePercentage of Visits with Error Pages Notes: 1. Hour-By-Hour Analysis of March 1, 2013. 2. Source: WebTrends Analysis: 1. This event occurred over a two hour period (10am through 12noon) 20
  21. 21. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureCharts directly from tools can look… well… 21
  22. 22. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureLike Uncooked Spaghetti 22
  23. 23. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureJust one strand… 23
  24. 24. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConfidence Interval Analysis 24
  25. 25. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure80% Interval Analysis 25
  26. 26. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureIncreasing Confidence - 80%, 90%, 95% 26
  27. 27. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe DA Approach, In General… 27
  28. 28. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd of scene 28
  29. 29. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #1 – three weeks later… 29
  30. 30. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureIt’s monthly report day, and Farris is looking hard at the data… 30
  31. 31. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureFarris’ Question “Why do some of our segments have significantly higher satisfaction scores than others?” 31
  32. 32. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure80% Interval Analysis with Narrow Scale 32
  33. 33. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure1st Order Report 33
  34. 34. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure2nd Order Report 34
  35. 35. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure3rd Order Report: NO High School or Librarians 35
  36. 36. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD Clue: Persona Driven Design 36
  37. 37. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureInterpreting Satisfaction Data I. 2nd year students are intermediate researchers, therefore have highest satisfaction. II. 3rd year students are advanced searchers and more likely to need advanced search tools. Satisfaction drops in 3rd year, so they may need more attention. 37
  38. 38. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd of scene 38
  39. 39. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #1 – Summary DA offers intelligence via segment analysis UXD offers context via personas 39
  40. 40. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #2 – two weeks later… 40
  41. 41. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureOne morning Chris opened his email to find… 41
  42. 42. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureChris’ Question “There’s a customer service request to improve the ease of narrowing results. Is there any user data that can give me the background I need to design some new approaches?” 42
  43. 43. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD: Do we need more options, less,…? 43
  44. 44. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDA Clue: Satisfaction Summary 44
  45. 45. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureModel Question SummaryHigher Scores  Legend: Accuracy of Information (1.2, 8.1) (Impact, Score) Quality of Information (1.2, 8.1) Breath of Content (1.2, 8.0) How Quickly Pages Load (0.5, 7.9) Consistency of Speed Page to Page (0.5, 7.9) Ability to Load Pages without Errors (0.5, 7.9) Readability of Graphics (0.5, 7.9) How Well Site Is Organized (1.7, 7.5) Balance of Graphics and Text (0.5, 7.3) Area of Focus How Well Site Layout Helps Find (1.7, 7.2) Relevance of Search Results (2.8, 7.2) Organization of Search Results (2.8, 7.2) How Well Search Results Help Selection (2.8, 7.2) Options for Navigation (1.7, 7.1) How Well Search Options Help Narrow Results (2.8, 7.0) Visual Appeal (0.5, 6.9) Number of Clicks (1.7, 6.9) Higher Impact  45
  46. 46. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureOpen Ended Comments With Respondent Level Data Narrow down their searches to more specifics Provide Tips for narrowing Search Provide more user friendly narrowing devices Be able to narrow search results to only find articles with full text 46
  47. 47. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConfirmed… I. Comment was not an outlier. II. Improving narrowing options should significantly impact satisfaction. 47
  48. 48. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd of scene 48
  49. 49. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #2 – Summary UXD needs background info before design. Analytics offers quantitative  qualitative data 49
  50. 50. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #3 – four weeks later… 50
  51. 51. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureFarris’ Question “I was preparing my monthly report and noticed something new. Page views per visit dropped, but other statistics remained constant. What does this mean?” 51
  52. 52. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDA: Weekly Heatmap 52
  53. 53. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD: Updates to site navigation, database selection 53
  54. 54. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureAnswers and More Questions I. Hypothesis - dynamic page element and/or other design changes caused page counts to decrease. II. No increase in satisfaction… yet. Further adjustments needed? 54
  55. 55. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd scene… 55
  56. 56. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #3 – Summary DA spots changes efficiently. UXD can offer hypotheses. 56
  57. 57. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD v. Analytics - Differences Digital Analytics User Experience Design Quantitative Qualitative “What did most users do?” “Why did you do that?” Reporting trends Design decisions Broad & shallow view Deep & narrow view Big data analysis Live user research 57
  58. 58. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD & Analytics - Common Goals Digital Analytics User Experience Design Deep interest in user behavior Want to increase user satisfaction Desire to influence strategic decisions By leveraging each other…. Does 1 + 1 = 3? 58
  59. 59. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureWorking Better Together • Give UXD access to analytics tools • Project collaboration • Refine survey questions and KPIs • Design informed by data insights • Deeper analysis based on a richer understanding of user personas and motivation • Smarter user research – finding respondents, asking better questions. 59
  60. 60. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureQuestions and Comments Farris Khan @Bobbleheadguru Chris Farnum @crfarnumSlideshare 60