It's Not the Size of Your Data, It's How You Use It

Uploaded on

After leading usability sessions and ethnographic interviews on hundreds of sites, and hearing plenty of client conjecture about “how tech use relates to age” or “this is a best practice for that …

After leading usability sessions and ethnographic interviews on hundreds of sites, and hearing plenty of client conjecture about “how tech use relates to age” or “this is a best practice for that feature,” I thought, “Well, I actually have a lot of data about that.” Then, I decided to start aggregating and categorizing the data from all our studies to get a better sense of trends.

Learn how to transform the insights from your hand-crafted, small-batch studies for maximum impact. Why bother? Because bigger data can let you identify trends, benchmark against competitors, and track your site over time.

More in: Data & Analytics
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. It’s Not the Size of Your Data It’s How You Use It
  • 2. Pamela Pavliscak (don’t even try it) @paminthelab Change Sciences
  • 3. I love experiments
  • 4. This speaks to me This speaks to me
  • 5. Outline of today’s talk data despair reality WTF experiments enlightenment
  • 6. 294 billion emails sent a day 1 billion Google searches a day 230+ million tweets a day 9 million Paypal payments a day Source: IBM, Google, Paypal
  • 7. Source: Pew Internet 100 terabytes of data uploaded to Facebook every day. Source: Facebook
  • 8. 1 baby born 5 mobile phones activated Source: ITU & US Census
  • 9. Not that babies are to blame for this
  • 10. Correlation, causality, who’s to say?
  • 11. big data is really awesome it’s really big of course not totally sure what qualifies as big data could be a lot of things well of course because it’s big like there are 50 tweets a minutes about big data every single day in fact I may just tweet about big data right now actually Obama likes big data you know the federal government invested $200 million in big data projects so that’s something right there it does seem like big data will solve all our problems and we have a lot of problems like the volume of business data doubling each year yeah that must be why my boss seems really stressed out she should really try yoga classes but no time to think about that because I have to read hundreds of articles about big data that keep popping up it’s a real phenomenon after all it is not stopping it just keeps going and
  • 12. Being Nothingness DATA
  • 13. More = Better (or does it?)
  • 14. But let’s get some perspective
  • 15. “Big data is like teenage sex: everybody talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it. -Dan Ariely
  • 16. 90% of teenagers don’t even know or care about big data
  • 17. 0% damn What I Care About The Data We Have
  • 18. #Homescreen2014 EXPERIMENT ONE
  • 19. A series of interviews about home screens in 2014
  • 20. Scraping #homescreen2014
  • 21. BIG (kind of) SMALL 87% iOS 89% use standard texting app 86% have the standard phone app 85% have Twitter related apps 69% have Facebook/Instagram 62% have Google apps Folders to see the picture Folders because folders Most important stuff first screen Default stuff put on last screen Some people search every time! Most important apps at the bottom N=65 N=1000
  • 22. BIG (kind of) SMALL 87% iOS 89% use standard texting app 86% have the standard phone app 85% have Twitter related apps 69% have Facebook/Instagram 62% have Google apps 12% use folders to see the picture 10% like folders because folders 68% put most important first 82% put most defaults last 3% search every time! 45% put important apps at the bottom N=65 N=1000
  • 23. Big doesn’t mean there is no BIAS
  • 24. HEISENBERG UNCERTAINTY PRINCIPLE (small data) Get over it, bias is a factor SIGNAL BIAS (big data)
  • 25. There is always a filter
  • 26. HUMAN CONTACT NICE Nice people just ruin it for all of us
  • 27. TV Network Site ONLINE Study 0 20 40 60 80 100 TV Network Site LAB Study 0 20 40 60 80 100 Proof, a 10% nice factor Source: Change Sciences
  • 28. Big data is mostly about COUNTING stuff
  • 29. They saw stuff (page views) They hung out (time) They did stuff (clicks)
  • 30. What was their goal? How did that work out for them? How did they feel about it? How did that fit into the big picture? Still so many questions
  • 31. Analytics basically tell us this story…
  • 32. <something, something> They came They left
  • 33. <something, something> UX is really good at the
  • 34. Analytics are the traces left behind
  • 35. But wait, our users are alive!
  • 36. Small data is all about STORIES
  • 37. Let’s look at UX DATA
  • 38. Our data is small, but extraordinarily detailed
  • 39. We connect the dots between data & design
  • 40. Yes, five is often just fine
  • 41. But not for PATTERNS & TRENDS
  • 42. counts, correlations, credibility (or patterns, relationships, and value)
  • 43. Bring All the Small Data Together Aggregate Standardize Categorize Connect
  • 44. AGGREGATE to get a big picture
  • 45. It has to work TOGETHER
  • 46. csv, plain text, xml (the simpler the better)
  • 47. Excel is an easy way to get started
  • 48. “Big data is too big to fit in your available memory, or too big to store on your own hard drive, or too big to fit into an Excel spreadsheet. -Hilary Mason
  • 49. Less formatting, more cool stuff to try
  • 50. STANDARDIZE to make it easier on yourself
  • 51. Align SCALES
  • 52. Keep some aspects of studies the same
  • 53. CATEGORIZE to see trends and patterns
  • 54. 1. Features 2. UI Elements 3. Interactions/Gestures 4. Concepts 5. Naming
  • 55. Count and track across projects
  • 56. You may need some helpers N=65
  • 57. Tallies Really it’s about in this step
  • 58. Counting!
  • 59. CONNECT it up with a framework
  • 60. Broad Metrics Across Datasets How easy is it? USABILITY How much do people do? ENGAGEMENT What is their takeaway? CONVERSION
  • 61. Enough to Drill-Down Time on Task Success Rate Perceived Success USABILITY METRICS Ease of Use Rating Stumbling Blocks Normalized Time Scrolling, Clicks, Back Button Fails
  • 62. SUBJECTIVE what people say OBJECTIVE what people do Balance it out to get the full picture
  • 63. Let’s look at CAROUSELS
  • 64. Aggregate 30 Studies EXPERIMENT TWO N=525
  • 65. Click-through varies by site type 12% 9% 5% 4% 3% 2% 2% Music Television E-commerce Academic Financial Real Estate Careers Source: Change Sciences
  • 66. “If the image gets my attention, I might try it. I’m always looking for new music. - M, Millennial, Pragmatist Source: Change Sciences
  • 67. “I’m looking for something under $2500/mo with 2 beds, 2 baths. I focused only on the search. - F, GenX, Wired Source: Change Sciences
  • 68. 56% click on whatever is immediately below the carousel (or hero), if it fills the top of the page. 18% wait for more than one featured item or use arrows to cycle through. Source: Change Sciences
  • 69. Click through differs by cycle rate Fast Slow No •  12% 78% 10% Source: Change Sciences
  • 70. Felt happy Clicked Didn’t Time Exploring Likely to Return Felt happyTime Exploring Likely to Return Source: Change Sciences
  • 71. And Mobile?
  • 72. “What seemed tedious on desktop, seems like no big deal on mobile. - M, GenX, Influencer Source: Change Sciences
  • 73. 67% will swipe, 45% tap a video, 5% tap otherwise Source: Change Sciences
  • 74. Consider new PARADIGMS for small data
  • 75. Citizen science crowdsourcing observations
  • 76. Social science and control groups
  • 77. The data PLEDGE
  • 78. All the Data deck, reading list, posts, experiments @paminthelab