Uploaded byLitmus
Data for Designers
We'd all love a data scientist on our team, but the reality is that designers, coders and marketers have to make data-driven decisions about their email campaigns. We'll walk you through how email creators can gather data, analyze it, and sell your ideas through data visualization. We'll show some of our client success stories and lessons we've learned along the way that focus on narrowing the gap between mobile and desktop engagement. Presented by Kristina Huffman and Schuyler Wareham at The Email Design Conference: San Francisco on October 1, 2013.
![Adapting to Consumers' New Definition of Spam [Webinar]](https://cdn.slidesharecdn.com/ss_thumbnails/adaptingtoconsumersnewdefinitionofspam-161116220236-thumbnail.jpg?width=640&height=640&fit=bounds)
Data for Designers
- 1. Data for Designers KristinaHuffman, Creative Services, ExactTarget Schuyler Wareham, Data & Analytics, ExactTarget
- 2. “We promise a50% increase in click-through rates”
- 3. What: Inputs availableto designers Why: Making decisions and using data to drive change. How: Data hacks for designers
- 5. Source Context Validity Consider: Designers can askthese questions!
- 6. “It’s called artwork Notartfuckingaround”
- 7. Benchmarks are bullshit (butbosses love them)
- 8. Inputs available todesigners 1. Device usage data Informs: • Design/Code workflow • Audience profile 2. Performance data Informs • Content: content strategy • Design: layout, ui
- 9. Devices affect workflow Outlook-heavyaudience Adjust design for preview pane, code for limited support Image-blocking email clients Code with styled alt text and bgcolors to create interest Large Blackberry environment Code using a fluid layout for legibility Large number of retina devices Adjust photoshop file for retina sizing Large Lotus notes audience Start looking for another job
- 10. Research data: IDCWorldwide Various Survey Data Research data: Quantcast ,Piper Jaffray Research data: emailclientmarketshare.com, Litmus ExactTarget research: one customer, 1m+ dataset ExactTarget research: one customer, 1m+ dataset
- 11. Use data tobuild an audience profile Stop looking at devices, start looking at customers.
- 12. People click withoutopening ExactTarget primary research – one retailer, one send, 1m+ dataset 16% of one brand’s clickthroughs were from people who did not open
- 13. People do notopen later on desktop Mobile triage is a myth. 2-3% of emails are opened on two different platforms. ExactTarget primary research – one retailer, one send, 1m+ dataset
- 14. Peak mobile opentimes are early morning and late evening ConScore
- 15. 55% 43% 2% 32% 62% 6% 0% 10% 20% 30% 40% 50% 60% 70% Mobile Desktop Multiple Open Click Peopleclick less on mobile Many factors in play: usability, trust, time ExactTarget primary research – one retailer, one send, 1m+ dataset Opens Clicks Opens Clicks
- 16. 13% 25% 0% 5% 10% 15% 20% 25% 30% Mobile clickthrough rateDesktop clickthrough rate People click less on mobile Many factors in play: usability, trust, time ExactTarget primary research – one retailer, one send, 1m+ dataset Mobile Desktop
- 17. What platforms aremy subscribers using to open, click AND Convert? Mobile? Desktop?
- 18. ExactTarget primary research– one retailer, one send, 1m+ dataset
- 19. What devices aremy subscribers using to open my emails? Convert? Apple Android Other
- 20. ExactTarget primary research– one retailer, one send, 1m+ dataset
- 21. ExactTarget primary research– one retailer, one send, 1m+ dataset
- 22. For one retailer, mobileemail clickers were 61% less likely to purchase than desktop email clickers. The Bad News:
- 23.
- 24. Coke or Pepsi? Mobileonly? Both?Desktop only? One list of subscribers - multiple sends over a 30 day period What’s your platform affinity?
- 25. ExactTarget primary research– one retailer, daily sender, 15m+ dataset 60% of people who open emails are opening only on mobile Platform stickiness - opens
- 26. ExactTarget primary research– one retailer, daily sender, 15m+ dataset Platform stickiness - clicks 60% of people who click emails are clicking only on mobile
- 27. ExactTarget primary research– one retailer, weekly sender, 1m+ dataset Conclusions • Most users stick to a platform • Platform-ambidextrous users prefer desktop for clicking Insight • Mobile must stand alone as a complete brand experience • Ensure clicks/conversion dropoff is not due to reduced content options or poor ui
- 28. ExactTarget primary research– one retailer, weekly sender, 1m+ dataset Which data points should I look at? Every send? Every quarter?
- 29. Delivered Open Click Convert Opt-in practices Deliverability Relationship history/ Brand trust From Name Subject Line content Preheader content Preview pane with images off Relationship history / Brand trust Visual design Content: Value prop, copy, photo, etc. User interface/clickability Landing page content/ui Conversion opportunity: form/checkout/etc.
- 30. Every send: 1. Clickoverlays Did they click where I designed them to? 2. Conversions How close was I to meeting our goals?
- 31. Click overlays Did theyclick where I designed them to? Rate and review3rd party promo Abandoned cart newsletter
- 32. Click overlays bydevice? Apple Android
- 34.
- 35.
- 36. iPhone: Shrinks message More clickson CTA More headline clicks Navigation ignored Android: Crops message Navigation-heavy Few go below viewport
- 37. Quarterly (or beforea redesign): 1. Your email client market share % breakdown: yahoo, outlook, iPad, etc. both from Litmus Analytics and website analytics - ensure your workflow and layout/code align with email clients 2. New devices/technologies in market Priority Inbox, Z10, Gmail tabs… selectively freak out 3. Clicks by device Are mobile clicks underperforming? Should we adjust? 4. Click overlays Did they click where I designed them to? Any surprises? 5. Conversions How close was I to meeting our goals? Open rate? Clickthrough rate?
- 38. What: Inputs availableto designers Why: Making decisions and using data to drive change. How: Data hacks for designers
- 39. Making decisions: Expertise-driven Overall aesthetics,brand Color palettes Typefaces Content Layout Responsive approach Data-driven Button Language Button color Subject Lines Email Length Responsive approach Opt-in forms Metrics Driven Design – Joshua Porter
- 40. Making decisions: Design innovation: Whatis the best possible model? Optimization: What works best in the current model? Metrics Driven Design – Joshua Porter
- 41. A culture oftesting “Sure, go ahead and test what's testable. But the real victories come when you have the guts to launch the untestable.” - Seth Godin
- 42. Lessons on Testing 1.You have a large dormant audience They’re masking the true impact of improvements 2. Purest way to test is with new subscribers They haven’t been tainted 3. Change occurs over time Retrain your subscribers
- 43. Ammunition for change you’rewelcome.
- 44. “More whitespace!” - Everydesigner
- 45. Whitespace increases reading speed,comprehension, and is preferred by readers. Margins, Optimal Leading No Margins, Sub-Optimal Leading Reading Online Text: A Comparison of Four White Space Layouts Barbara Chaparro, J. Ryan Baker, A. Dawn Shaikh, Spring Hull, & Laurie Brady 7.5% increase in reading speed 16% increase in comprehension Most preferred layout from readers when asked
- 46. “There’s too muchcontent in this email” - designers ExactTarget primary research – one retailer, one send, 1m+ dataset
- 47. You get oneopen. On average, 72% of opens are unique ExactTarget primary research – one retailer, one send, 1m+ dataset You get one click. On average, 81% of clicks are unique What will you use it on?
- 48. “We should accommodateretina displays so this looks better on my mac.” - You when you get a new computer
- 49. Average pixel densityfor mobile devices is: 200%And smartphone owners have more education and higher incomes: we need to come across in the most brand-positive way to our highest potential customers. Screensiz.es | Pew research: smartphone ownership by demographic
- 50. “Let’s think mobilefirst.” - you
- 51. 76% of openers opened onmobile at some point. One retailer over the course of 30 sends found: ExactTarget primary research | Emailclientmarketshare.com
- 52. “We should makemore videos.” - Everyone with iMovie
- 53. People watch videoslonger on mobile devices. Video can help decrease the barrier to click on mobile. ExactTarget primary research Tablet 2:59 Phone 2:50 Desktop 2:09
- 54. What: Inputs availableto designers Why: Making decisions and using data to drive change. How: Data hacks for designers
- 55. 3 Things youcan do with:
- 56. Visualize a clickoverlay with brush sizes: 40% of clicks = 40px brush, etc.
- 57. 1. Export Litmustracking data
- 58. 1. Export Litmustracking data
- 59. 2. Merge Litmusdata with clickthrough data EmailAddress Opener Open OS/Device Open Time DesignBadass@TEDC.com yes desktop 10/1/13 11:13 AnalyticsNinja@dojo.com yes mobile 10/2/13 12:01 sue@sue.me yes desktop 10/1/13 14:10 sky@yahoo.com yes desktop 10/3/13 9:57 Dood@FarOut.org yes mobile 10/1/13 12:07 Gov't@shutdown.dumb yes mobile 10/2/13 12:01 kHuff@gmail.com yes mobile 10/2/13 20:33 StuBeef@et.com yes desktop 10/2/13 16:01 EmailAddress Clicker Gov't@shutdown.dumb yes sky@yahoo.com yes no@match.com yes DesignBadass@TEDC.com yes kHuff@gmail.com yes
- 60. 2. Merge Litmusdata with clickthrough data EmailAddress Opener Open OS/Device Open Time DesignBadass@TEDC.com yes desktop 10/1/13 11:13 AnalyticsNinja@dojo.com yes mobile 10/2/13 12:01 sue@sue.me yes desktop 10/1/13 14:10 sky@yahoo.com yes desktop 10/3/13 9:57 Dood@FarOut.org yes mobile 10/1/13 12:07 Gov't@shutdown.dumb yes mobile 10/2/13 12:01 kHuff@gmail.com yes mobile 10/2/13 20:33 StuBeef@et.com yes desktop 10/2/13 16:01 EmailAddress Clicker Gov't@shutdown.dumb yes sky@yahoo.com yes no@match.com yes DesignBadass@TEDC.com yes kHuff@gmail.com yes
- 61. 2. Merge Litmusdata with clickthrough data EmailAddress Opener Open OS/Device Open Time Clicker ? DesignBadass@TEDC.com yes desktop 10/1/13 11:13 yes AnalyticsNinja@dojo.com yes mobile 10/2/13 12:01 sue@sue.me yes desktop 10/1/13 14:10 sky@yahoo.com yes desktop 10/3/13 9:57 yes Dood@FarOut.org yes mobile 10/1/13 12:07 Gov't@shutdown.dumb yes mobile 10/2/13 12:01 yes kHuff@gmail.com yes mobile 10/2/13 20:33 yes StuBeef@et.com yes desktop 10/2/13 16:01
- 62.
- 63. What: Inputs availableto designers Why: Making decisions and using data to drive change. How: Data hacks for designers
- 64. What about the50% lift in click-through?
- 65. Delivered Open Click Convert Opt-in practices Deliverability Relationship history/ Brand trust From Name Subject Line content Preheader content Preview pane with images off Relationship history / Brand trust Visual design Content: Value prop, copy, photo, etc. User interface/clickability Landing page content/ui Conversion opportunity: form/checkout/etc.
- 66. Use your lenses. Askquestions.
Editor's Notes
- #2 Ask questions throughout
- #3 Who has been in this position?This is when sky and I started to appreciate each other’s fieldsWe were tasked with redesigning for liftWe realized it wasn’t just ui buttons vs. underlined links. The industy makes you think that 50% clickthrough from button color! Symbols in subject lines! Are you using a preheader?!We were too miopic.It’s broader than that. … (blending of Art and Science…Kristina (Design) and I (Data) we came to appreciate that there’s a confluence of disciplines that when combined are greater than the sum of its parts. We rely on each other, and the outcome of a combined approach is highly valuable. “…as a result of that partnership, we want to share with you (audience) what we’ve learned”. …over time, we combined our specialized expertise and collaborated to mutual benefit. Through the course of our relationship and collaboration, we’ve gained more than the sum of our parts…so we’ll share some of that tasty goodness with you today.
- #4 SkyWhy are we here? To show you What data you can/should be looking at (to assess your audience and performance of your designs/campaigns, etc); Why it’s important – ie studies/research we’ve done that lend credibility to why you should expend energy seeking the “what” data; How to get your hands on it, produce the insights. These new lenses can impact how you shape workflow and evaluate effectiveness of your program; provide you ammunition for change (support for new initiatives, design tactics, resources, send times, etc). Finally we focus on “things you can do” – our views on how you can accommodate the evolving landscape and maximize performance.
- #5 Sky…but not in a boring “agenda” kind of way (which I’m not great at). Why we’re here: to show you What data you can/should be looking at (to assess your audience and performance of your designs/campaigns, etc); Why it’s important – ie studies/research we’ve done that lend credibility to why you should expend energy seeking the “what” data; How to get your hands on it, produce the insights. These new lenses can impact how you shape workflow and evaluate effectiveness of your program; provide you ammunition for change (support for new initiatives, design tactics, resources, send times, etc). Finally we focus on “things you can do” – our views on how you can accommodate the evolving landscape and maximize performance.
- #6 KHSource: always consider a dataset’s source and how it was gathered. Survey based data to 50 marketers is different than tracking-gathered data from an analytics softwareContext: clicks don’t always clearly communicate intent and satisfaction. Look at every data point holistically before making conclusionsValidity: when comparing, make sure we are apples to apples. Don’t measure your rates against industry benchmarksDesigners visualize business goals. It’s not personal or self-expression. You have a direct affect on metrics, so let’s dig in.
- #7 KH
- #8 SkyBefore we get into the numbers and start quoting all sorts of research…take it with a grain of salt. Here’s why! … Many benchmarks are self-reported. And reported by whom (and to whom ?– to impress the boss)? These are virtually never apples to apples comparisons to your business - same vertical, yeah, but different size org (Best Buy vs. Mom + Pop wine shop). Lies, Damned Lies and statistics. The point is: MEASURE YOURSELF!!!
- #9 KH
- #10 KH
- #11 SKYSKY – alright, some numbers for youAt the top, we see some of the most highly cited statistics.If you focus on “global shipments” only, you’d be lead to believe you only need to pay attention to AndroidIf you look at mobile opens, you’d be lead to believe you only need to render well on iPhone. A closer look at clicks and conversion show the HIDDEN ANDROID audience.Look holisticallyDon’t ignore androidLook at your data first, not industryThings aren’t apples to applesData perspective: look at TRUE metrics that even the playing field. Not opens. Clicks and conversions by device. Don’t be fooled, even though android shipped looks big, and apple web browse looks big, you still have to look at it for you. Outcome: prioritize Reinforces the mobile first device agnostic design. Graceful degradation. Android: opens vsiphone opens. Not a true represenation. Look holisticallyDon’t ignore androidLook at your data first, not industryThings aren’t apples to applesData perspective: look at TRUE metrics that even the playing field. Not opens. Clicks and conversions by device. Don’t be fooled, even though android shipped looks big, and apple web browse looks big, you still have to look at it for you. Outcome: prioritize Reinforces the mobile first device agnostic design. Graceful degradation. Android: opens vsiphone opens. Not a true represenation.
- #12 KH
- #13 KHThings aren’t always what they seem.You can’t just look at opens because you’re missing a huge % of your audience by looking at one data point – remember that hidden android audience.
- #14 SKY
- #15 KH
- #16 SKYSKYThe size of the mobile audience isn’t enough; it’s not the full picture, but so too the gap between you mobile open rate and mobile click rate. Add the uzh open and click rates by mobile and desktop – show the GAP
- #17 SKY
- #18 KH
- #19 SKYSKYView your performance through the “conversion lens”. This is not a level playing field. Reiterate barrier to click, greater barrier to convert. Perhaps we’ll never get to parity, but surely we can do better than (26% mobile conversion vs. 60% open). WE’VE SEEN BRANDS NARROW THE GAP.
- #20 KH
- #21 SKY
- #22 SKYReiterate hidden Android audience – though it’s not as big as you might have thought. AND, dropoff/barrier exists regardless of device.
- #23 KHThe soundbite for all of this is: X, and that’s scary for a marketer.This is for one retailer
- #24 KHRemember: there’s a broader story than just your email program – customer-centric approach to tracking. This is a challenge to you: find out more about your subscriber journey outside of email.
- #25 SKYThis looks at Platform usage from a longitudinal perspective, beyond just the open rate for a single email.
- #26 SKYThis is a measure of the INTENSITY of platform usage (beyond measuring the rate). That large mobile audience, they really love consuming email on their mobile devices.
- #27 SKYClicks show more of a divergence. Only 10% of subscribers in this case click on both platforms over time. Most either prefer all mobile or all desktop.
- #28 KHLook holisticallyDon’t ignore androidLook at your data first, not industryThings aren’t apples to applesData perspective: look at TRUE metrics that even the playing field. Not opens. Clicks and conversions by device. Don’t be fooled, even though android shipped looks big, and apple web browse looks big, you still have to look at it for you. Outcome: prioritize Reinforces the mobile first device agnostic design. Graceful degradation. Android: opens vsiphone opens. Not a true represenation.
- #29 KH
- #30 KHLike a tube of toothpaste.
- #31 KHLook holisticallyDon’t ignore androidLook at your data first, not industryThings aren’t apples to applesData perspective: look at TRUE metrics that even the playing field. Not opens. Clicks and conversions by device. Don’t be fooled, even though android shipped looks big, and apple web browse looks big, you still have to look at it for you. Outcome: prioritize Reinforces the mobile first device agnostic design. Graceful degradation. Android: opens vsiphone opens. Not a true represenation. You’ll notice I don’t have opens or clickthrough rates on here. I’ve seen AB tests where the version that won on clicks did NOT win on convesions. AND the other way around: the one that lost on clicks actually one on conversions. Why do you do what you do? Focus on money
- #32 KHNewsletter is keeping people interested above the fold, but maybe work on hierarchy a bit. If evertything is even it tells me you didn’t really have anything unique, exclusive or special to say, it’s just your month’s worth of promotions and things shoved into an emailAbandoned cart – perfect example of giving secondary options on an abandoned cart – I’d reconsider the algorithems behind the right two suggestion boxes.3rd party promo: most clicks were on vawp link – this was because the email looked very different than most, had other branding on it. Clickers are distributed, even though it was one promotion – there is no clear CTA: work on concise clear CTA language and designRate and review; proves navigation isn’t detracting from main click – one big image, one big button. Really clear CTA. You know what’s going to happen when you click.
- #33 KH
- #34 KH
- #35 SKYSKYKey takeaways here are…main CTA is taken by iPhone openers at twice the rate of Android openers! Conversely, Android are way more likely to gravitate toward Nav. It’s intuitive. Lesson for images off optimization. Android users are NOT TAKING THE MAIN CALL TO ACTION.
- #36 SKYKey takeaways here are…main CTA is taken by iPhone openers at twice the rate of Android openers! Conversely, Android are way more likely to gravitate toward Nav. It’s intuitive. Lesson for images off optimization. Android users are NOT TAKING THE MAIN CALL TO ACTION.
- #37 TogetherTakeaways:Image based content was a barrier to click for android – esp for the main CTAThey’re only seeing your nav, not your main CTA
- #38 Look holisticallyDon’t ignore androidLook at your data first, not industryThings aren’t apples to applesThese 5 things are ACTIONABLE. Is open rate actionable? Is clickthrough rate actionable? NO. A click overlay is. Data perspective: look at TRUE metrics that even the playing field. Not opens. Clicks and conversions by device. Don’t be fooled, even though android shipped looks big, and apple web browse looks big, you still have to look at it for you. Outcome: prioritize Reinforces the mobile first device agnostic design. Graceful degradation. Android: opens vsiphone opens. Not a true represenation.
- #39 KH
- #41 KH
- #42 KH“Netflix tests everything. They're very proud that they A/B test interactions, offerings, pricing, everything. It's almost enough to get you to believe that rigorous testing is the key to success.Except they didn't test the model of renting DVDs by mail for a monthly fee.And they didn't test the model of having an innovative corporate culture.And they didn't test the idea of betting the company on a switch to online delivery.The three biggest assets of the company weren't tested, because they couldn't be.Sure, go ahead and test what's testable. But the real victories come when you have the guts to launch the untestable.”
- #43 Most orgs have a significant dormant subscriber segment. I ususally see it between 1/3rd and 2/3rd -You don’t have to put complex test environments in place, but can control the audience you view in the back end analysis. FPL example – changes toward Responsive were made in the Winter, by Summer, each consecutive month was seeing a reduction of that GAP between mobile open and mobile click.
- #44 A gift from us to you. We’ve translated common designer requests into data-backed arguments. Remember, benchmarks are bullshit but bosses like them. (useful when you have nothing else and when you can ensure it’s an apples to apples comparison to your business) Because when you put a very large, nicely-kerned number on a powerpoint, it becomes the word of God. Stop complaining, start presenting data. You would make these based on expertise, but you may not have the data to back it up when it gets to the c-level. Trust gap on mobile. “Cut out all of this shitty content - nobody cares about our xyy” - you get one open and one click, generally. on average we see 70-80% of clicks single clickers.“Don’t ignore android” - post click breakdown - opens are low, yes, but
- #45 KH
- #46 KHThis study was funded by Microsoft, ironically enough.
- #47 SKY
- #48 SKYLook at unique opens vs total opens. Which basket will you put your eggs in? And how about for that images-off audience?
- #50 KHAccommodate mobile now. We can also derive demographic data from this – smartphone owners, according to Pew research, have more education and more income. we need to come across in the most brand-positive way to our high potential customers.
- #51 SKY
- #52 SKY…here’s at lens (or statistic) to hit home the magnitude of the situation. Not just one send, where the avg is 48%, some sends have 60% mobile on a single email. This is the total size of the mobile audience. The POTENTIAL mobile open rate.
- #53 KH
- #54 KHProduction time needs to be a factor, and making sure the content is best met in the video channel, yes, but video is a very natural experience on a mobile device: full screen, audio built in, ui is familiar, etc.
- #55 KH…but not in a boring “agenda” kind of way (which I’m not great at). Why we’re here: to show you What data you can/should be looking at (to assess your audience and performance of your designs/campaigns, etc); Why it’s important – ie studies/research we’ve done that lend credibility to why you should expend energy seeking the “what” data; How to get your hands on it, produce the insights. These new lenses can impact how you shape workflow and evaluate effectiveness of your program; provide you ammunition for change (support for new initiatives, design tactics, resources, send times, etc). Finally we focus on “things you can do” – our views on how you can accommodate the evolving landscape and maximize performance.
- #56 KH
- #57 I’m a visual person – I have to see clicks. This is an easy way to visualize them. You’ll start to see where the dropoffs are to see how short you should keep your email
- #63 Kh
- #64 …but not in a boring “agenda” kind of way (which I’m not great at). Why we’re here: to show you What data you can/should be looking at (to assess your audience and performance of your designs/campaigns, etc); Why it’s important – ie studies/research we’ve done that lend credibility to why you should expend energy seeking the “what” data; How to get your hands on it, produce the insights. These new lenses can impact how you shape workflow and evaluate effectiveness of your program; provide you ammunition for change (support for new initiatives, design tactics, resources, send times, etc). Finally we focus on “things you can do” – our views on how you can accommodate the evolving landscape and maximize performance.
- #66 KHDesign is mostly a post-open tactic.
- #67 There should be an analytical perspective to design. Source: always consider a dataset’s source and how it was gathered. Survey based data to 50 marketers is different than tracking-gathered data from an analytics softwareContext: clicks don’t always clearly communicate intent and satisfaction. Look at every data point holistically before making conclusionsValidity: when comparing, make sure we are apples to apples. Don’t measure your rates against industry benchmarks