Data-Driven Design (MX '10)


Published on

A five minute rant on the importance of letting data be your guide when making tactical design decisions. An introduction for managers of design teams who are driven from a heuristic, or "genius" perspective.

Published in: Design, Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Hello, I’m Ryan Freitas from Second Verse.
  • Enough with the Oxbridge pleasantries. This is a 5 minute minute madness which means...
  • I’ve got to get to my point rather quickly. When I worked with Peter at AP his usual guidance when writing a presentation to was to be sure there was a recognizable point to what you were trying to say. He called it the “so what”, and it was one of the most helpful lessons he ever taught me. So what’s the “so what”?
  • Arguments over tactical design decisions take up too much time for all experience design teams. Minute decisions around messaging, structure and even color derail our efforts from ensuring what should be our priority, creating the best possible experience for our audience.
  • This may be the most successful homepage on the web. Two actionable areas, sign in or register. Minimal messaging... tactical decisions focused on GETTING PEOPLE IN THE DOOR. The real heavy lifting, the ‘genius design’ is the experience once you’re through the gates...
  • Which is really what its all about. Our time and energy should not be spent fighting out tactical battles over the inventory and positioning of elements on the home page.
  • We should be inventing and refining, focusing on decisions that create deeper, more impressive levels of interaction with our audience.

  • I’m hoping that you’re at least getting occasional reports on visits, views and your bounce-rate. There might even be somebody who’s compelling you to increase one while reducing another.
  • But there’s a whole wealth of information you can gain access to if you know where to look.
  • This is Eric Reis, author, speaker, and advocate of the lean startup movement. And he’s got a very good point.
  • It’s all in there. Yes of course you can do research and ask individual users and groups about their experience. But the numbers tell their own story, and while it may not be as deep or nuanced, it is ACTIONABLE, and potentially more valuable than you understand.
  • You can increase conversion by experimenting with your calls to action. In a genius design methodology, we come up with many possible messages and roll out ONE. In data driven design, for tactical choices WE ROLL OUT ALL OF THEM.
  • If you’re not already convinced of the value of data, you might be asking this question.
  • The best answer is from Avinash Kaushik of Google. He’s their Analytics Evangelist, and one of the smartest people talking about the roll of data in design right now. (you should follow him on twitter)
  • What if putting things out in to the world to get REAL reactions from users was trivial? What if it didn’t require writing a protocol, recruitment, or any other element of research?

  • This is Hiten Shah, founder of KISSMetrics and another lean startup advocate. He’s also brilliant. (you should follow him on twitter)
  • The technical requirements to do multivariate or split testing on your site are minimal - there’s server and client side solutions, and both can ensure that only a sliver of your audience sees any given tactical change. Just enough to get feedback on the failure or success of that change.
  • Split testing opens the way to experiment on our products in the real world and track the impact.
  • Testing yields bigger, more actionable data in less time for prep and deployment than any comparable method, including remote usability.
  • Why am I telling you this? Because if you don’t believe it already, you need to soon. You are the Highest Paid Person’s Opinion in many of the tactical design decisions that are getting made. And we all know that HiPPOs usually get their way. I’d like to encourage you to be advocates for data as the determiner of how to make tactical design decisions.
  • ... because I think that if you don’t, the next year or so is going to get very, very difficult for you and your team. This isn’t a prediction. This is how business is getting done.
  • So focus. Find the areas of the site where you can move the dial with minor changes. See what impact you can have, and be sure to measure it appropriately.
  • Many of you already have these, but they are as outdated as the pageview model for advertising that’s being thrown out as we speak. Focus on conversion and engagement, and align them into your OKRs.
  • I can’t stress this enough. If you decide to go with only one field for passwords, and don’t make users enter them twice, you might drive down the time to complete a registration form. You’ll never know if it was the right decision until you pay attention to the fact that your “I forgot my password” clicks just doubled since the change. Experiences bleed over.
  • This is simple. Don’t try to test everything at once. Don’t push your designers out of the product loop by testing without their input. And most important, don’t overtest the same issue into the realm of ridicule.
  • For most of you, 4 shades will probably do.
  • Complete information is rare in our line of work. Even with the best research on our audience, we cannot assume we know everything they need. Allow their ACTUAL BEHAVIOR to be your guide, and refine the experience for them based on their interaction with your product.
  • If you’re looking for a place to get started, pick an underperforming landing page and see what can be accomplished with just a few changes to how the page looks, and what it says. Come up with 3 alternatives, and give them to 5% of your audience each.
  • Remember that sometimes the solution will lie outside of the realm of the constraints you set up for yourself. Try not to go too nuts. It’s a balancing act.
  • Past design decisions are a sunk cost. Move past them as quickly as you can.
  • You should be more afraid of what you don’t know than what might happen if you make the wrong decision. Which leads me to my final piece of advice...
  • Unless you’ve got a perfect conversion rate, you can always improve. The best design teams constantly iterate, because someone out there is building something without all the compromises that created the current solution.

  • Data-Driven Design (MX '10)

    1. 1. Show & Prove Balancing Data-Driven & ‘Genius’ Design Ryan Freitas March 8th, 2010
    2. 2. Five minutes of fury.
    3. 3. “So what?”
    4. 4. Creative freedom is not defined by getting to pick where the “sign up” button goes.
    5. 5. No genius required.
    6. 6. ‘Genius design’ should focus upon articulating the holistic experience.
    7. 7. It should build the novel, the elegant, and the compelling.
    8. 8. For all the rest, default to data.
    9. 9. What do you do with your data
    10. 10. How closely are you paying attention to the metrics of conversion, discovery and contribution?
    11. 11. “Metrics are people, too.”
    12. 12. Each of these data points tells a story about the success or failure of the experience you’ve built for your audience.
    13. 13. Each of these data points can be positively impacted by data- driven design decisions.
    14. 14. What role can data play in helping teams create better experiences for users?
    15. 15. “Removing risk!”
    16. 16. Data can empower user experience teams to design more creatively “risky” things than they believed to be possible.
    17. 17. They can implement the most audacious design decisions, while at the same time lowering the risk of those decisions.
    18. 18. “Try insane sh*t.”
    19. 19. How? By only showing 5% of your web audience.
    20. 20. Split testing is your new BFF.
    21. 21. Split testing and user surveys on live sites can move metrics quickly and effectively, with less expense than traditional user testing.
    22. 22. HiPPOs
    23. 23. Behavioral data will become the primary driver of tactical design decisions within your organization.
    24. 24. Identify, isolate, validate.
    25. 25. “Objectives & Key Results”
    26. 26. Track the lifespan of the design decisions you make.
    27. 27. Caveat tester: Don’t go overboard.
    28. 28. This isn’t about 41 shades of
    29. 29. Data liberates us from the illusion that we completely understand the needs and desires of our audience.
    30. 30. “But what should we test?” Why not start with page structure and messaging?
    31. 31. “You can’t test within the confines of a style guide.”
    32. 32. When we lack the resolve to experiment, we doom ourselves to a series of ongoing compromises.
    33. 33. Blow it up if you have to.
    34. 34. Iterate without sentimentality*
    35. 35. Iterate without sentimentality* * especially when you’re successful.
    36. 36. Thank you for your attention.
    37. 37. Thank you for your attention. You should follow me on Twitter: @ryanchris