UXD v. Analytics - IAS13 Baltimore


Published on

**This version of our presentation is for The Information Architecture Summit, Apr 7 2013 in Baltimore** Chris and Farris expose the differences between how user experience designers and analytics practitioners think. While UXD weave best practices and user research into their designs, digital analysts spend their time confirming or refuting hypotheses in a data-driven way. One approach is decidedly qualitative, the other decidedly quantitative. In this presentation you will learn through their conversations how it is possible to leverage both enlightened design and deep data to continuously optimize user experiences. If you work on either side of this debate, this is how to better state your case… and get along with the other side.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Image: via Chris Farnum and Farris Khan
  • Customers: college, university, corporate, and government libraries worldwideSubscription databases of articles in journals, newspapers, magazines, trade pubs, books…Academic migration complete
  • Summary – the bootstrap days are done, and now we are extending and refining.Image: via Chris FarnumSound: http://ihalo.org/Aerodev/Renders/Sound/effects/grunt_birthday_party/grunt_birthday_party_explosion.mp3
  • We do projects to translate business requirements into user experiencesAgile methodology12 team membersWork closely with product management, technical teams, and QA
  • Qualitative User ResearchFocused on “Why did you do that?”Design decisionsDeep and narrow perspective on usersDeliverables and toolsWireframes and prototypesUser stories (aka specifications)User testing reportsPersonasHeuristicsDesign patternsImage: http://upload.wikimedia.org/wikipedia/commons/b/bf/Mistery_stub.pngImage: http://cnx.org/content/m20220/latest/graphics1.png
  • UXD was recently placed within the same team as our digital analytics lead, Farris.The big question…Who is this guy and what’s this report he’s publishing about the platform we designed?Image: https://upload.wikimedia.org/wikipedia/commons/5/53/Gray_-_replace_this_image_male.svg
  • UXD and DA both claim expertise about usersQuantitative v. Qualitative?Influence over strategic direction?Image: http://farm1.staticflickr.com/39/94326793_4b8d06c4a0_b.jpg
  • Common Unit of Analysis = Day, Week, Month
  • Image: http://pixabay.com/static/uploads/photo/2012/07/27/14/13/ropes-53133_640.jpg?i
  • Quantitative data collection and analysisFocused on “What did most users do?”Reporting trendsBroad and shallow perspective on usersDeliverables and toolsInternal data warehouseBehavioral analytics (WebTrends)Attitudinal analytics (ForeSee)System response testing (Gomez)Weekly and monthly reportsImage: http://upload.wikimedia.org/wikipedia/commons/3/32/Caliper_full_view.jpegImage: http://www.flickr.com/photos/mayaevening/138372058/
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://revolutionarygardens.com/wp-content/uploads/2013/01/Flying-Calendar.jpg
  • Image: http://www.flickr.com/photos/mayaevening/138372058/
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://revolutionarygardens.com/wp-content/uploads/2013/01/Flying-Calendar.jpg
  • Image: http://pixabay.com/en/fire-night-glow-orange-colorful-49954/
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://revolutionarygardens.com/wp-content/uploads/2013/01/Flying-Calendar.jpg
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://www.freefoto.com/preview/33-06-54/Brick-Texture
  • Image: http://commons.wikimedia.org/wiki/File:Bechyne-Duha-kresba.png
  • Image: http://upload.wikimedia.org/wikipedia/commons/3/39/Reeses-PB-Big-Cup.jpg
  • UXD v. Analytics - IAS13 Baltimore

    1. 1. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure UXD v. Analytics What have YOU got for me? No, what have you got for ME? Information Architecture Summit Baltimore, MD | April 7th 2013 Chris Farnum, Senior User Experience Designer, ProQuest Farris Khan, Analytics Lead, ProQuest & BobbleheadGuru blogger 1
    2. 2. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDramatis Personae Digital Analytics Lead..…Farris UXD/IA………………………..Chris *Identities to be revealed upon proposal acceptance. ;-) 2
    3. 3. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureIntroduction – UXD v. Digital Analytics Where do these specialties overlap? How are the roles different? Why should they collaborate? Got a question or comment? Tweet: @BobbleheadGuru or @crfarnum Hashtags: #ias13 #measure 3
    4. 4. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe ProQuest Search Platform 4
    5. 5. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure 5
    6. 6. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe UXD Role at ProQuest Core UXD Design Processes Prioritized Develop/Test requirements Planning Iteration release Product release 6
    7. 7. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe UXD Approach, In General… 7
    8. 8. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureA newcomer to the department… 8
    9. 9. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureOverlapping Territories? Competition? 9
    10. 10. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureWhat is Digital Analytics? 10
    11. 11. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEcosystem of Analytics Object of Analysis: Document, Library, System Metrics: What? UsageObject of Analysis: Object of Analysis:Visitor Attitudinal/ Behavioral/ VisitorMetrics: Persona Observational Metrics:Who? Why? Where? When? Common Unit of Analysis 11
    12. 12. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureMonthly Analytics Heatmap Notes: 1. Data from March 2012 through February 2013. 2. Source: WebTrends and Internal data collection. 12
    13. 13. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureVisits: Year-Over-Year Comparison 13
    14. 14. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureVisits/Visitors 14
    15. 15. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureWeekly Analytics Heatmap 15
    16. 16. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureYear Over Year Visits Per Week Thanksgiving (US) Easter Christmas 16
    17. 17. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDaily Analytics Heatmap 17
    18. 18. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureVisits per Day(Since February 13, 2011)
    19. 19. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measurePercentage of Visits with Error Pages Notes: 1. Data from January 1, 2012 through March 17, 2013. 2. Source: WebTrends 3. Four “Spike Days”: Feb. 1, 2012, March 17, 2012 and March 1, 2013. 19
    20. 20. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measurePercentage of Visits with Error Pages Notes: 1. Hour-By-Hour Analysis of March 1, 2013. 2. Source: WebTrends Analysis: 1. This event occurred over a two hour period (10am through 12noon) 20
    21. 21. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureCharts directly from tools can look… well… 21
    22. 22. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureLike Uncooked Spaghetti 22
    23. 23. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureJust one strand… 23
    24. 24. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConfidence Interval Analysis 24
    25. 25. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure80% Interval Analysis 25
    26. 26. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureIncreasing Confidence - 80%, 90%, 95% 26
    27. 27. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureThe DA Approach, In General… 27
    28. 28. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd of scene 28
    29. 29. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #1 – three weeks later… 29
    30. 30. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureIt’s monthly report day, and Farris is looking hard at the data… 30
    31. 31. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureFarris’ Question “Why do some of our segments have significantly higher satisfaction scores than others?” 31
    32. 32. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure80% Interval Analysis with Narrow Scale 32
    33. 33. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure1st Order Report 33
    34. 34. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure2nd Order Report 34
    35. 35. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measure3rd Order Report: NO High School or Librarians 35
    36. 36. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD Clue: Persona Driven Design 36
    37. 37. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureInterpreting Satisfaction Data I. 2nd year students are intermediate researchers, therefore have highest satisfaction. II. 3rd year students are advanced searchers and more likely to need advanced search tools. Satisfaction drops in 3rd year, so they may need more attention. 37
    38. 38. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd of scene 38
    39. 39. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #1 – Summary DA offers intelligence via segment analysis UXD offers context via personas 39
    40. 40. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #2 – two weeks later… 40
    41. 41. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureOne morning Chris opened his email to find… 41
    42. 42. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureChris’ Question “There’s a customer service request to improve the ease of narrowing results. Is there any user data that can give me the background I need to design some new approaches?” 42
    43. 43. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD: Do we need more options, less,…? 43
    44. 44. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDA Clue: Satisfaction Summary 44
    45. 45. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureModel Question SummaryHigher Scores  Legend: Accuracy of Information (1.2, 8.1) (Impact, Score) Quality of Information (1.2, 8.1) Breath of Content (1.2, 8.0) How Quickly Pages Load (0.5, 7.9) Consistency of Speed Page to Page (0.5, 7.9) Ability to Load Pages without Errors (0.5, 7.9) Readability of Graphics (0.5, 7.9) How Well Site Is Organized (1.7, 7.5) Balance of Graphics and Text (0.5, 7.3) Area of Focus How Well Site Layout Helps Find (1.7, 7.2) Relevance of Search Results (2.8, 7.2) Organization of Search Results (2.8, 7.2) How Well Search Results Help Selection (2.8, 7.2) Options for Navigation (1.7, 7.1) How Well Search Options Help Narrow Results (2.8, 7.0) Visual Appeal (0.5, 6.9) Number of Clicks (1.7, 6.9) Higher Impact  45
    46. 46. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureOpen Ended Comments With Respondent Level Data Narrow down their searches to more specifics Provide Tips for narrowing Search Provide more user friendly narrowing devices Be able to narrow search results to only find articles with full text 46
    47. 47. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConfirmed… I. Comment was not an outlier. II. Improving narrowing options should significantly impact satisfaction. 47
    48. 48. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd of scene 48
    49. 49. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #2 – Summary UXD needs background info before design. Analytics offers quantitative  qualitative data 49
    50. 50. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #3 – four weeks later… 50
    51. 51. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureFarris’ Question “I was preparing my monthly report and noticed something new. Page views per visit dropped, but other statistics remained constant. What does this mean?” 51
    52. 52. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureDA: Weekly Heatmap 52
    53. 53. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD: Updates to site navigation, database selection 53
    54. 54. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureAnswers and More Questions I. Hypothesis - dynamic page element and/or other design changes caused page counts to decrease. II. No increase in satisfaction… yet. Further adjustments needed? 54
    55. 55. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureEnd scene… 55
    56. 56. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureConversation #3 – Summary DA spots changes efficiently. UXD can offer hypotheses. 56
    57. 57. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD v. Analytics - Differences Digital Analytics User Experience Design Quantitative Qualitative “What did most users do?” “Why did you do that?” Reporting trends Design decisions Broad & shallow view Deep & narrow view Big data analysis Live user research 57
    58. 58. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureUXD & Analytics - Common Goals Digital Analytics User Experience Design Deep interest in user behavior Want to increase user satisfaction Desire to influence strategic decisions By leveraging each other…. Does 1 + 1 = 3? 58
    59. 59. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureWorking Better Together • Give UXD access to analytics tools • Project collaboration • Refine survey questions and KPIs • Design informed by data insights • Deeper analysis based on a richer understanding of user personas and motivation • Smarter user research – finding respondents, asking better questions. 59
    60. 60. UXD v. Analytics @BobbleHeadGuru @crfarnum #ias13 #measureQuestions and Comments Farris Khan Farris.Khan@ProQuest.com @Bobbleheadguru Chris Farnum Chris.Farnum@ProQuest.com @crfarnumSlideshare http://www.slideshare.net/ChrisFarnum/ 60