Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UXD v. Analytics - IUE12 Ann Arbor


Published on

**This version of our presentation is for Internet User Experience, July 2012 in Ann Arbor** Chris and Farris sit next to each other at ProQuest. However, their jobs are very different. Each one has something that the other wants... but they don't always know how to ask for it.

Chris Farnum and Farris Khan will demonstrate the differences between their goals for the first part of this presentation. They will go back and forth on what are the right questions to ask and which issues are more important. In the second half of the presentation, they will seek to understand how to find common ground and understand each other better. They will illustrate how ProQuest is able to leverage BOTH enlightened design and deep data to continuously optimize their products. If you work on either side of this debate, we hope you will give you some insights to better state your case... and get along with the other side.

Published in: Technology, Business
  • Be the first to comment

UXD v. Analytics - IUE12 Ann Arbor

  1. 1. UXD v. Analytics What have YOU got for me? No, what have you got for ME? Internet User Experience 2012 July 18, 2012 Chris Farnum and Farris KhanTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 1
  2. 2. Background – The ProQuest Platform • Search application for journals, news, magazines, and many other types • Academic and corporate audiences – one platform, two skins • Majority of academic customers now have migrated, corporate migrations underway • Summary – the bootstrap days are done, we are now extending and refiningTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 2
  3. 3. The UXD Role At ProQuest • Agile methodology Core UXD • 12 team members Design Processes • Work on large scale projects – Global Prioritized Develop/Test – Multi-year requirements • We do projects to translate business Planning requirements into user experiences Iteration release Product • Work closely with release platform management, technical teams, and QATwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 3
  4. 4. The UXD Approach, In General… • Mostly Qualitative - Often focused on one-on- one sessions, with a small group of carefully chosen users • Focused on “Why did you do that?” • Often used for tactical design decisions • Often offers a deep and narrow perspective on users • Practitioners develop a gut feel, instinct for what will work for their users’ preferencesTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 4
  5. 5. A New Development For UXD UXD was recently placed within the same platform management team as our digital analytics lead, Farris. The big question… Who is this guy and what’s this report he’s publishing about the platform we designed?Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 5
  6. 6. Overlapping Territories? Competition? • UXD and Digital Analytics both claim lots of expertise about users • Quantitative v. Qualitative? • Influence over strategic direction?Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 6
  7. 7. What is Analytics? Actionability Smart System Strategy and Operations are largely driven by Analytics Decision Support Decisions are made based on the data Predictive Model is Intelligence established The trend-line is long Measure enough to determine what is in the acceptable A credible set of v. is in the “needs “report card” metrics improvement” range established MaturityTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 7
  8. 8. Daily HeatmapTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 8
  9. 9. Weekly HeatmapTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 9
  10. 10. Monthly HeatmapTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 10
  11. 11. Trend-line = Uncooked SpaghettiTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 11
  12. 12. Trend-line: GoodTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 12
  13. 13. BetterTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru
  14. 14. Even BetterTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 14
  15. 15. Comparing UXD and DA Digital Analytics User Experience Design Mostly Quantitative Mostly Qualitative Page or widget level statistics, Often focused on one-on-one can be enhanced via surveys sessions, with a small group of and segmentation carefully chosen users Focused on “What did most Focused on “Why did you do users do?” that?” Often used for performance Often used for tactical design reporting, strategic decision decisions making Often offers a broad and Often offers a deep and shallow perspective on users narrow perspective on users Practitioners develop a keen Practitioners develop a gut eye for trends, changes and feel, instinct for what will patterns in user data work for their users’ preferencesTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 15
  16. 16. Farris and Chris work in back-to-back cubes and this leads to interesting conversations…Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 16
  17. 17. Conversation #1 – Interpreting Report Data One day Farris asked, “Why do some of our segments have significantly higher satisfaction scores than others?”Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 17
  18. 18. Conversation #1 - Why Is Satisfaction Higher For Some Segment v. Others? NOTE: Data has been alteredTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 18
  19. 19. Conversation #1 – Interpreting Report Data UXD: Personas and SegmentationTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 19
  20. 20. Conversation #1 – Summary • DA interprets heartbeat reports: – Important to understand institutional baselines – Analysis of KPIs by audience segment • UXD can offer: – Insight into personas and design priorities – Help on whether deltas are expected or unexpectedTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 20
  21. 21. Conversation #2 – Narrowing Options One day Chris said, “There’s a customer service request to improve the ease of narrowing results. Is there any user data that can give me the background I need to design some new approaches?”Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 21
  22. 22. Conversation #2 – Narrowing Options UXD: Many existing options. Do we need more, less,…?Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 22
  23. 23. Conversation #2 - Open Ended Comments, Area of Focus “Narrow Results” The following pages contain analysis from “representative comments” from November data. The data is a subset of 272 comments from respondents with comments who were in the bottom middle quartile in satisfaction, meaning they were slightly below the median in satisfaction. This group is considered an area of focus because their scores are “on the fence” and it is realistically possible to “win over” these visitors (moving them from the bottom middle quartile to the upper middle quartile). The 24 comments that are highlight the concept of “narrowing results” which was identified to be the largest topic associated with the areas of focus (search) in this group. NOTES: 1. This topic was selected based on the frequency of the term. We can conduct similar analysis for other topics. 2. The first page contains the actual open ended “verbatim” comments. The second page contains respondent level detail about the visitor. This is an abridged version of the larger set of data that we can obtain.Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 23
  24. 24. Conversation #2 – Summary • UXD needs background info before beginning to design solutions: – Does the issue impact many users or just a few? – Observed behavior patterns, ratings and comments can provide insight • DA can offer both qualitative and quantitative data: – Satisfaction scores based on surveys – Analysis of a segment of the data – Review comments – A virtual user research studyTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 24
  25. 25. Conversation #3 – Interpreting Emerging Trends One day Farris said, “I was preparing my monthly report and noticed something new. Page views per visit dropped, but other statistics remained constant. What does this mean?”Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 25
  26. 26. Conversation #3 – Monthly HeatmapTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 26
  27. 27. Conversation #3 – Weekly HeatmapTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 27
  28. 28. Conversation #3 – Interpreting Emerging Trends UXD: Updates to site navigation, database selectionTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 28
  29. 29. Conversation #3 – Summary • DA spots trends in regular reports: – Determines whether changes in KPIs are significant – Trends may suggest priorities for planning site development • UXD can offer: – Possible explanations for changes – Communication about upcoming site updates – Assistance in identifying important KPIs, creating and refining survey questions…Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 29
  30. 30. Conversation #4 – Is This Design Successful? One day Chris said, “We show price notifications to corporate customers for each transaction. How often do users turn off these notifications?”Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 30
  31. 31. Conversation #4 – Is This Design Successful? Price notifications in the corporate interface – Pay per document when not subscribed – Design question - should we show notifications for every paid transaction? – Is option to “do not show” sufficient for users who don’t want the interruption? – Are further refinements needed or is the current design working fine?Twitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 31
  32. 32. Conversation #4 - Comparative AnalysisTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 32
  33. 33. Conversation #4 – Also… • Path tracking – needs development effort to embed tags • Session ReplayTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 33
  34. 34. Conversation #4 – Summary • UXD would like to evaluate designs and features: – We want to know if a feature is serving it’s purpose or needs refining – If features aren’t used or are unpopular, we may want to remove or replace them • DA can offer deep analysis – However, extra effort is needed – Prioritization relative to other projects will be an issueTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 34
  35. 35. Next Steps - Evolving Our Process & Working Together • Strategic decision making - transition from gut feel  data driven • Deeper analysis based on a richer understanding of user personas and motivation • Measure clicks and paths, not just pagesTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 35
  36. 36. Small Group Tweet Up • Brainstorm how you would like to see analytics and UxD fit together in your organization • Everyone contribute • Pick top 3 • Post on Twitter: #IUE12 #MeasureTwitter: #IUE12 #Measure @crfarnum @BobbleHeadGuru 36
  37. 37. Questions and Comments Farris Khan @Bobbleheadguru Chris Farnum @crfarnum Slideshare #IUE12 #Measure @crfarnum @BobbleHeadGuru 37