UXD v. Analytics - eMetrics 2013 San Francisco


Published on

***This version of the presentation is for the eMetrics Summit in SF, April 17 2013. 11:15am in Room: Salon 7*** Chris and Farris expose the differences between how user experience designers and analytics practitioners think. While UXD weave best practices and user research into their designs, digital analysts spend their time confirming or refuting hypotheses in a data driven way. One approach is decidedly qualitative, the other decidedly quantitative. Learn how it is possible to leverage both enlightened design and deep data to continuously optimize user experiences. If you work on either side of this debate, this is how to better state your case… and get along with the other side.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Image: via Chris Farnum and Farris Khan
  • Customers: college, university, corporate, and government libraries worldwideSubscription databases of articles in journals, newspapers, magazines, trade pubs, books…Academic migration complete
  • Summary – the bootstrap days are done, and now we are extending and refining.Image: via Chris FarnumSound: http://ihalo.org/Aerodev/Renders/Sound/effects/grunt_birthday_party/grunt_birthday_party_explosion.mp3
  • Image: http://semanticstudios.com/publications/semantics/000029.php
  • Image / Source: http://www.jjg.net/elements/pdf/elements.pdf
  • We do projects to translate business requirements into user experiencesAgile methodology12 team membersWork closely with product management, technical teams, and QA
  • We do projects to translate business requirements into user experiencesAgile methodology12 team membersWork closely with product management, technical teams, and QA
  • We do projects to translate business requirements into user experiencesAgile methodology12 team membersWork closely with product management, technical teams, and QA
  • We do projects to translate business requirements into user experiencesAgile methodology12 team membersWork closely with product management, technical teams, and QA
  • Qualitative User ResearchFocused on “Why did you do that?”Design decisionsDeep and narrow perspective on usersDeliverables and toolsWireframes and prototypesUser stories (aka specifications)User testing reportsPersonasHeuristicsDesign patternsImage: http://upload.wikimedia.org/wikipedia/commons/b/bf/Mistery_stub.pngImage: http://cnx.org/content/m20220/latest/graphics1.png
  • UXD was recently placed within the same team as our digital analytics lead, Farris.The big question…Who is this guy and what’s this report he’s publishing about the platform we designed?Image: https://upload.wikimedia.org/wikipedia/commons/5/53/Gray_-_replace_this_image_male.svg
  • UXD and DA both claim expertise about usersQuantitative v. Qualitative?Influence over strategic direction?Image: http://farm1.staticflickr.com/39/94326793_4b8d06c4a0_b.jpg
  • Common Unit of Analysis = Day, Week, Month
  • Image: http://pixabay.com/static/uploads/photo/2012/07/27/14/13/ropes-53133_640.jpg?i
  • Quantitative data collection and analysisFocused on “What did most users do?”Reporting trendsBroad and shallow perspective on usersDeliverables and toolsInternal data warehouseBehavioral analytics (WebTrends)Attitudinal analytics (ForeSee)System response testing (Gomez)Weekly and monthly reportsImage: http://upload.wikimedia.org/wikipedia/commons/3/32/Caliper_full_view.jpegImage: http://www.flickr.com/photos/mayaevening/138372058/
  • Image: http://revolutionarygardens.com/wp-content/uploads/2013/01/Flying-Calendar.jpg
  • Image: http://www.flickr.com/photos/mayaevening/138372058/
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://revolutionarygardens.com/wp-content/uploads/2013/01/Flying-Calendar.jpg
  • Image: http://pixabay.com/en/fire-night-glow-orange-colorful-49954/
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://revolutionarygardens.com/wp-content/uploads/2013/01/Flying-Calendar.jpg
  • http://commons.wikimedia.org/wiki/File:Clap.jpg
  • Image: http://www.freefoto.com/preview/33-06-54/Brick-Texture
  • Image: http://commons.wikimedia.org/wiki/File:Bechyne-Duha-kresba.png
  • Image: http://upload.wikimedia.org/wikipedia/commons/3/39/Reeses-PB-Big-Cup.jpg
  • UXD v. Analytics - eMetrics 2013 San Francisco

    1. 1. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD UXD v. Analytics What have YOU got for me? No, what have you got for ME? eMetrics Summit San Francisco, CA| April 17th 2013 11:15am, Room: Salon 7 Chris Farnum, Senior User Experience Designer, ProQuest Farris Khan, Analytics Lead, ProQuest & BobbleheadGuru blogger 1
    2. 2. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDDramatis Personae Digital Analytics Lead..…Farris UXD/IA………………………..Chris 2
    3. 3. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDIntroduction – UXD v. Digital Analytics Where do these specialties overlap? How are the roles different? Why should they collaborate? Got a question or comment? Tweet: @BobbleheadGuru or @crfarnum Hashtags: #eMetrics #UXD 3
    4. 4. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDThe ProQuest Search Platform 4
    5. 5. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD 5
    6. 6. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDFacets of the User Experience From User Experience Design by Peter Morville 6
    7. 7. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDWhat is “User Experience Design” From The Elements of User Experience by Jesse James Garret 7
    8. 8. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDEcosystem of Information Architecture Business goals Brand strategy Technology Politics Resources ContextDocument types IAApps / FunctionalityContent ObjectsVolume Content Users AudienceStructure Personas Tasks Needs Info seeking behavior Based on Rosenfeld and Morville – Information Architecture for the World Wide Web 8
    9. 9. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDElements of IA/UXD • Organization Systems How is content organized? By User Task, Subject Category, Audience Type, Metaphor… • Labeling Systems Links, buttons, labels, calls-to-action… • Navigation Systems Sitewide navigation, local navigation, utility navigation… • Search Systems Site search interfaces, results pages, indexing… • Thesauri, Controlled Vocabularies, and Metadata Systems of terms used to organize and tag content Based on Rosenfeld and Morville – Information Architecture for the World Wide Web 9
    10. 10. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDWireframes and Specifications FIG 2: My Saved Articles FIG 2 Notes: 1. Page title 2. Count of all items in the list. • Increments as items added 1 • Decrements as items deleted 2 4 3. Link back to last set of search results 3 5 4. Sort options: • By date added – reverse chron • By date published – reverse chron • Alphabetical by title 6 5. Checkbox to select all items in the list • Checking selects all items • Unchecking deselects all items 6. Articles. Each item includes: • Checkbox • Number in list • Citation – in same style as in search results • Date added – DD Mon YYYY 10
    11. 11. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDThe UXD Role at ProQuest Core UXD Design Processes Prioritized Develop/Test requirements Planning Iteration release Product release 11
    12. 12. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDThe UXD Approach, In General… 12
    13. 13. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDA newcomer to the department… 13
    14. 14. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDOverlapping Territories? Competition? 14
    15. 15. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDWhat is Digital Analytics? 15
    16. 16. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDEcosystem of Analytics Object of Analysis: Document, Library, Sy stem Metrics: What? UsageObject of Analysis: Object of Analysis:Visitor Attitudinal/ Behavioral/ VisitorMetrics: Persona Observational Metrics:Who? Why? Where? When? Common Unit of Analysis 16
    17. 17. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDMonthly Analytics Heatmap Notes: 1. Data from March 2012 through February 2013. 2. Source: WebTrends and Internal data collection. 17
    18. 18. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDVisits: Year-Over-Year Comparison 18
    19. 19. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDVisits/Visitors 19
    20. 20. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDWeekly Analytics Heatmap 20
    21. 21. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDYear Over Year Visits Per Week Thanksgiving (US) Easter Christmas 21
    22. 22. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDDaily Analytics Heatmap 22
    23. 23. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDVisits per Day(Since February 13, 2011)
    24. 24. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDPercentage of Visits with Error Pages Notes: 1. Data from January 1, 2012 through March 17, 2013. 2. Source: WebTrends 3. Four “Spike Days”: Feb. 1, 2012, March 17, 2012 and March 1, 2013. 24
    25. 25. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDPercentage of Visits with Error Pages Notes: 1. Hour-By-Hour Analysis of March 1, 2013. 2. Source: WebTrends Analysis: 1. This event occurred over a two hour period (10am through 12noon) 25
    26. 26. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDCharts directly from tools can look… well… 26
    27. 27. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDLike Uncooked Spaghetti 27
    28. 28. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDJust one strand… 28
    29. 29. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConfidence Interval Analysis 29
    30. 30. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD80% Interval Analysis 30
    31. 31. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDIncreasing Confidence - 80%, 90%, 95% 31
    32. 32. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDThe DA Approach, In General… 32
    33. 33. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConversation #1 – three weeks later… 33
    34. 34. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDIt’s monthly report day, and Farris is looking hard at the data… 34
    35. 35. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDFarris’ Question “Why do some of our segments have significantly higher satisfaction scores than others?” 35
    36. 36. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD80% Interval Analysis with Narrow Scale 36
    37. 37. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD1st Order Report 37
    38. 38. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD2nd Order Report 38
    39. 39. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXD3rd Order Report: NO High School or Librarians 39
    40. 40. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDUXD Clue: Persona Driven Design 40
    41. 41. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDInterpreting Satisfaction Data I. 2nd year students are intermediate researchers, therefore have highest satisfaction. II. 3rd year students are advanced searchers and more likely to need advanced search tools. Satisfaction drops in 3rd year, so they may need more attention. 41
    42. 42. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDEnd of scene 42
    43. 43. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConversation #1 – Summary DA offers intelligence via segment analysis UXD offers context via personas 43
    44. 44. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConversation #2 – two weeks later… 44
    45. 45. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDOne morning Chris opened his email to find… 45
    46. 46. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDChris’ Question “There’s a customer service request to improve the ease of narrowing results. Is there any user data that can give me the background I need to design some new approaches?” 46
    47. 47. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDUXD: Do we need more options, less,…? 47
    48. 48. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDDA Clue: Satisfaction Summary 48
    49. 49. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDModel Question SummaryHigher Scores  Legend: Accuracy of Information (1.2, 8.1) (Impact, Score) Quality of Information (1.2, 8.1) Breath of Content (1.2, 8.0) How Quickly Pages Load (0.5, 7.9) Consistency of Speed Page to Page (0.5, 7.9) Ability to Load Pages without Errors (0.5, 7.9) Readability of Graphics (0.5, 7.9) How Well Site Is Organized (1.7, 7.5) Balance of Graphics and Text (0.5, 7.3) Area of Focus How Well Site Layout Helps Find (1.7, 7.2) Relevance of Search Results (2.8, 7.2) Organization of Search Results (2.8, 7.2) How Well Search Results Help Selection (2.8, 7.2) Options for Navigation (1.7, 7.1) How Well Search Options Help Narrow Results (2.8, 7.0) Visual Appeal (0.5, 6.9) Number of Clicks (1.7, 6.9) Higher Impact  49
    50. 50. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDOpen Ended Comments With Respondent Level Data Narrow down their searches to more specifics Provide Tips for narrowing Search Provide more user friendly narrowing devices Be able to narrow search results to only find articles with full text 50
    51. 51. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConfirmed… I. Comment was not an outlier. II. Improving narrowing options should significantly impact satisfaction. 51
    52. 52. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDEnd of scene 52
    53. 53. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConversation #2 – Summary UXD needs background info before design. Analytics offers quantitative  qualitative data 53
    54. 54. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConversation #3 – four weeks later… 54
    55. 55. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDFarris’ Question “I was preparing my monthly report and noticed something new. Page views per visit dropped, but other statistics remained constant. What does this mean?” 55
    56. 56. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDDA: Weekly Heatmap 56
    57. 57. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDUXD: Updates to site navigation, database selection 57
    58. 58. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDAnswers and More Questions I. Hypothesis - dynamic page element and/or other design changes caused page counts to decrease. II. No increase in satisfaction… yet. Further adjustments needed? 58
    59. 59. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDEnd scene… 59
    60. 60. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDConversation #3 – Summary DA spots changes efficiently. UXD can offer hypotheses. 60
    61. 61. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDUXD v. Analytics - Differences Digital Analytics User Experience Design Quantitative Qualitative “What did most users do?” “Why did you do that?” Reporting trends Design decisions Broad & shallow view Deep & narrow view Big data analysis Live user research 61
    62. 62. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDUXD & Analytics - Common Goals Digital Analytics User Experience Design Deep interest in user behavior Want to increase user satisfaction Desire to influence strategic decisions By leveraging each other…. Does 1 + 1 = 3? 62
    63. 63. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDWorking Better Together • Give UXD access to analytics tools • Project collaboration • Refine survey questions and KPIs • Design informed by data insights • Deeper analysis based on a richer understanding of user personas and motivation • Smarter user research – finding respondents, asking better questions. 63
    64. 64. UXD v. Analytics @BobbleHeadGuru @crfarnum #eMetrics #UXDQuestions and Comments Farris Khan Farris.Khan@ProQuest.com @Bobbleheadguru Chris Farnum Chris.Farnum@ProQuest.com @crfarnumSlideshare http://www.slideshare.net/ChrisFarnum/ 64