Metrics that Matter: The 360-Degree Customer


Published on

presentation from Cassie Lancellotti-Young at NY Internet Week 2013

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • ASK ROOM who is from startups vs. established brandsbusinessanalytics are a science, but devising which metrics are mission-critical for your business is really something of an art
  • 38% of US adults are always addressable2/3+ of people under 45 are always addressable
  • industry definition -4Vs = volume, velocity, variety and veracity
  • Improving the signal-to-noise ratio
  • Lets you look at things like – X% lift in Facebook followers last Tuesday, but did that also result in increase to subscribers? Near-term purchases?
  • And of course, there are myriad tools that will plug into database and let you slice/dice on your own – Tableu, ProClarity, etc.Think of these as pivot tables on steroids
  • User lookup – basically a B2C CRM tool to see everything about the customer in one place – devices, whatever custom vars the client was to track, etc.
  • 1/0 approach is great for synthesizing data via pivot tables – we’ll talk more about 0/1 scoring later
  • Someone can feasibly use OpenTable multiple times per week, the same cannot be said for airplane ticketsDay1 purchase behavior – if that were relevant in B2B, many sales/AMs wouldn’t have jobs
  • everyone should (hopefully) be looking at the first 2, but analytics superstars are hyper-focused on the thirdratios/relationships are really the crux of non-siloedbusiness analytics
  • Vanity metric would be aggregate customer conversion rateDifferent email treatments can be a cohort of users – how does the behavior of someone who receives welcome series A differ from someone who gets Series B?Relates to product level as well – how does someone who gets a product tour differ from someone who does not?Can also use on the B2B side and look at churn rates…
  • Not always about immediate results – think about what happens to someone who gets a welcome series with a discount – might activate sooner, but downstream LTV might be lower – in this example, we don’t care if people only buy with free shipping if the amount they always spend makes up for it profit lost
  • Elevator analysis – “it went up” or “it went down” – might be interesting to hear, but completely uselessReiterates importance of ratios/relationships – explains the WHY behind those metrics…
  • In case you still have no clue what I am talking about…Going to highlight a # of examples – these are by no means comprehensive, but will help you think about what to look at
  • In early days of live chat at TLC we saw a 13% increase in conversion from basic to premium product for group that live chatted with a representative
  • Post-dining survey to users after each experience asking them to rate experience at restaurantWe looked at feedback scores from first-time users and then mapped that to repeat rate and learned that some restaurants were more likely to inspire better customers, so only featured these restos in our early marketing materials
  • Do more calls/higher time spent mean more $$?
  • Revtrax – ties online coupons to online spendingIdentify cohorts of users who came in during the time a TV spot aired
  • Facebook – 7 friends within 10 days of signing upZynga – D1 retention – if someone comes back a day after signing up, good indicator… SUPER IMPORTANT because “1 and doners” are big problem
  • 0/1 – for every X increase in reservations, risk of churn was reduced by Y%Savored example – likelihood of churn based on # reservations in first 30 days. MULTICOLLINEARITYIf there is one thing to know about stats, its SIGNIFICANCE TESTINGI’m not going to stand here and talk about actionable ordered logistics though, want to jump into quicker more actionable metricsHOW WOULD YOU GET STARTED, people ask. The answer is Dave McClure!
  • There are so many possible metrics we can look at, how we do think about approaching business analytics in an organized way?Going to borrow a page from Dave McClure.
  • Different channels will interact differently with landing page examples – branded search terms vs. fleeting message in display adsIt’s helpful to isolate these metrics (e.g. compare CTR for creative), but it’s also important to look at what is happening downstream – look at cash/keyword ROI, etc.
  • Are they not signing up because the form is so long?WHAT THIS ALL BOILS DOWN TO…
  • Understand the difference of different landing pages – reg/chall example from TheLadders with no paid option on first stepTHIS IS NOT NECESSARILY A PURCHASE CONVERSION PROBLEM!
  • Think about difference between B2B and B2C!
  • some companies (especially B2B) will even take this a step further:what is the TOTALcost of retaining a customer (account management, etc.)?COMPARE THIS # TO YOUR LTV to understand the unit economics of a customer
  • This is important because app owners may have higher LTV, etc.People most likely aren’t going to submit a lead gen form for a bakery, so click-to-call is a nice way to measure efficacy of spend
  • Downstream metrics – CPA might be higher, but LTV may also be much higher
  • Facebook example – can’t say the channel isn’t valid, might just be that the East Village ad eating 80% of your budget does not perform but the AARP-targeted ad at 5% allocation is crushing it
  • focus on mechanics of the lower funnel before pumping the gas on acquisition – squeeze every nickel until it screams
  • i.e. forcing users to follow things is not a valuable behavior
  • Same thing goes for content sites – how much is available/depth of content when someone first comes to site?
  • Beyond supply and demand, think about the staging of your productBirchboxor any “waitlist” product – would beinteresting to regress wait time vs. lifetime purchase behavior
  • Discount offers for cart abandonment or product re-targetingLots of people doing retargeting – Facebook, etc. Can also do push notifications, etc.
  • We called people at Savored to understand why they weren’t booking, then scored those users to understand downstream behaviorTLC – welcome call to premium subscribers, lift on renewals
  • Important to understand the trade-offs between price promotion and long-term value – let’s think about a sample subscription site
  • Accomplished via email/mobile/ads (push), product marketing (pull) and developing positive customer satisfaction
  • Historically marketing was about 1:many and 1:1, where the latter mostly dealt meant transactional emails or cold callsThis is why there has been an explosion of companies focused on persuasion architecture or personal experiences – monetate, etc. that were focused on 1:FEW.NOW there are tools to let us customize our content with low development resourcesPeople focus on a lot of types of targeting – geographic (so can put in things about the weather), etc. but behavioral tends to render the biggest bang for the buck
  • Seamless used Device Targeter to double CTR on app download prompts and to increase app downloads by 50%....And because we love DOWNSTREAM metrics, they also saw that clickers placed 90% more orders after clickers
  • I had opened the app, but not redeemed any deals
  • This is a model I made up, so you don’t have to listen to it, but…RFM analyses say nothing about what is driving customers to make purchases… so many precursors – how often are they being emailed, do they have app, etc.
  • A lot of people will ask me what are some easy retention wins? I want to stay focused on metrics but will quickly power through a few examples that leverage email but keep in mind you can also leverage what we talked about earlier – remarketing across the web, etc.
  • But make sure you are taking all of their behaviors into account – i.e. what if they haven’t been on the site, but have been in the app?MEASURING efficacy – 3 groups – this segment, segment with this email and no offer, segment that gets nothing Isn’t this kind of like RFM? DON’T JUST DO IT IN BATCH SENDING – set this up as a trigger
  • Engagement – what else they look at/how deep is sessionLift – do people go on to buy more, etc.Beware of the filter bubble – NYT example
  • Warby Parker example?Particularly worth trying with people who viewed a ton of pages and bought nothing
  • “hurry,” “last chance,” etc. always do well
  • At Sailthru, the typical client sees a 6% in monthly email revenue when they use cart abandonmentConsulting for HA – drove over $7MM in incremental annual revenue via booking flow abandonment
  • Can create drip campaigns so you’re always coming back to see what friends are doing – Quora does this well
  • Seems like a reasonable place to talk about email measurement, where much of the science is pretty antiquated
  • So Twitter could have “follows” as a goal in GA and tracks conversion of this email
  • either business goal related (get revenue)Or could be engagement goal related (get dormant people to do something, etc.) – talk about Savored game
  • If the product isn’t converting, email won’t do anythingAlso look at source – saw 0% conversion from iPhone – there was a bug!Is the funnel as simple as it could be?
  • VIRAL is not a strategy because you can’t guarantee it, but it can helpIs LTV higher for customers who were referred? ENCOURAGE MORE REFERRALS – if there are different ways to spread the word, which are most effective?
  • Social APIs
  • Revenue is a function of much of the behaviors we’ve already discussed
  • Jason Goldberg’s blog – 10% purchase conversion in first week from iPad users – and they are forecasted to be worth 2x more over the lifetime
  • If I were a betting woman – app users use more often after last purchase, so how to drive more usage through thatCAVEAT:Understand what is truly causality (for instance, do people spend more because they have an iPad, or are they already power users so will just leverage any vehicle possible to buy more?)
  • Metrics that Matter: The 360-Degree Customer

    1. 1. Cassie Lancellotti-Young VP Client Analytics, Sailthru May 2013 @dukecass Metrics that Matter: The 360-Degree Customer Transcending Data Silos for Holistic Marketing
    2. 2. About Me: The “Reader‟s Digest” Version media/tech banker at Citigroup acquisition and subscription analytics at TheLadders independent analytics consultant while MBA‟ing marketing and analytics at Savored (exited to Groupon) intrapreneurship at Gerson Lehrman Group (GLG) 2013 2005 client optimization/analytics at Sailthru
    3. 3. to get metrics, we need data …and that data has become big
    4. 4. big data proliferation: the good, the bad and the ugly
    5. 5. the good: data is everywhere CUSTOMER mobile CRM website emailPOS support social
    6. 6. more of the good: meet the “always addressable” customer
    7. 7. the bad: with big data come big silos
    8. 8. the ugly: the ignorant marketer “Cassie, check out our updated mobile app”  Cassie‟s account is already linked to an iPhone app  No options to upgrade AMEX app in App Store, so Cassie likely already has this version
    9. 9. 39% of marketers can‟t turn data into action Stat Source: Columbia Business School: “Marketing ROI in the Era of Big Data,” 2012. Image Source: Dilbert, 29 July 2012.
    10. 10. 360-degree marketing = “big data alchemy” Image Source: Cambridge in Colour, 2013.
    11. 11. how to become an alchemist?
    12. 12. the API economy is key
    13. 13. API aggregation example: SumAll
    14. 14. API aggregation example: GoodData
    15. 15. and of course, there are tools to let you try this at home
    16. 16. this is precisely what sailthru‟s “smart data” is all about
    17. 17. …but hacks can work, too (e.g. bringing user level into GA)
    18. 18. as can clunky “scrappy” excel files!
    19. 19. thought this talk was about metrics? well, here we go.
    20. 20. give yourself a sanity check! (b2b vs. b2c, price point, etc.)
    21. 21. measurement happens via 3 lenses  user level – what are users doing?  product/transaction level – when is site conversion highest? which products yield the strongest repeat rates?  relationships/ratios – how do certain experiences impact user behaviors (e.g. first purchase type vs. NPS)?
    22. 22. cohort analysis vs. vanity metrics
    23. 23. always have a pulse on what happens on downstream Source: Monetate
    24. 24. just say no to “elevator” analysis understand the why?  product/marketing: deliberate changes to messaging, site, etc.  business ecosystem: i.e. inventory issues, technical problems  “macro” factors: i.e. industry trends, economic climate, press
    25. 25. relationship analytics, you say?
    26. 26. live chat usage vs. conversion TheLadders saw a fairly immediate 13% increase in premium conversion for a test group that live-chatted with a representative.
    27. 27. feedback vs. subsequent use cases Some Savored restaurants were absolutely horrid for driving repeat usage.
    28. 28. account management vs. spend Does more time spent on account management yield upsells? Higher contract value? Does it reduce churn?
    29. 29. tying together online/offline behaviors Customers who try the brand offline will oftentimes have higher AOVs (average order value) online downstream.
    30. 30. “magic” numbers define tipping point for engagement Facebook considers a user to be “engaged” if s/he gets 7 friends within 10 days of signup.
    31. 31. understanding relationships is as easy as 0,1 (0=no, 1=yes)  Data science/predictive models are most ideal, but you can get started (directionally) on your own with simple binomial regressions.  e.g. Savored regressed restaurant churn propensity against reservations in first 30 days.
    32. 32. McClure‟s Startup Metrics for Pirates Acquisition Activation Retention Referral Revenue link to pre
    33. 33. Acquisition Where do we get new users and how much does it cost us to get them?
    34. 34. key to acquisition is understanding scale/efficiency trade-off
    35. 35. efficiency is a function of spend, creative, multiple conversions
    36. 36. (understand what‟s causing these gaps)
    37. 37. resulting in two CPAs: CPAR + CPAC …and efficiency is really about the latter  CPAR: cost per registrant (or CPL/lead) $100 spend / 20 signups = $5 CPAR  CPAC: cost per customer (or engaged user) 20 signups >> 1 buyer = $100 CPAC
    38. 38. can‟t predict the future? enter the “intake curve”
    39. 39. use the intake curve to predict CPAC >
    40. 40. what kind of conversion are you trying to drive?
    41. 41. use downstream metrics to inform/optimize your acquisition strategy Image Source:
    42. 42. that said, the tighter the tracking, the better
    43. 43. Activation Do users do as we ask them to do? If so, how quickly do they do it?
    44. 44. define what actually qualifies as “USAGE”
    45. 45. so your customers don‟t pay you? not a problem…  Develop proxies for revenue – a post, a Tweet – but make sure those proxies are truly valuable behaviors.  For advertising-driven businesses, still think about key value – i.e. PV/user yields $X in ad revenue?
    46. 46. know if you‟re dealing with a “bow-tie” marketplace…
    47. 47. what„s a marketplace? it„s about balancing supply and demand
    48. 48. health of the marketplace ecosystem has a material impact on the metrics
    49. 49. is restricted access precluding eventual activation?
    50. 50. important to have realistic expectations for customer behavior Image Source: Max Woolf
    51. 51. …don„t jump the gun! Image Source: Max Woolf
    52. 52. when you can‟t activate via email, leverage other channels!
    53. 53. even picking up the phone works (for both b2b and b2c!) Image Source: SalesNexus
    54. 54. please, just be careful with discounts! Cost of First Month Month 1 Renewal Rate $40 (full price) 70% $36 70% $32 65% $28 55% ∙ ∙ ∙ ∙ ∙ ∙ $0 (free trial) 40%
    55. 55. (random aside) use your customers to get ideas
    56. 56. Retention How do we keep users engaged with our product/content over time?
    57. 57. the democratization of 1:1; mass messaging no longer cuts it
    58. 58. case study: and movable ink‟s device targeter Source: Movable Ink
    59. 59. case study: scoutmob app win-back email
    60. 60. old school segmentation has evolved (because customers aren‟t segments) MANY PEOPLE Recency Frequency Monetary Value “80% of your revenue comes from 20% of your customers” CASSIE‟S MODEL Behavioral Usage Situational “what else do we know about that 20% segment?”
    61. 61. quickly, some easy retention tactics (email-centric)
    62. 62. “win-backs” for good customers who are MIA
    63. 63. recommendations (measure by CTR, engagement and lift)
    64. 64. inventory updates (how many disengaged can you recapture?)
    65. 65. urgency will always move product
    66. 66. page or user flow abandonment
    67. 67. leveraging outside APIs for social proof
    68. 68. promotion (but be strategic about it)
    69. 69. big data vs. big brother “your friend Amanda dined at Zengo with Savored” Disengaged Segment 10% open rate 15% CTR Dormant Segment 30% open rate 10% CTR
    70. 70. email measurement
    71. 71. open rates are dead. metrics such as RPM, PVM define success
    72. 72. understand how email impacts on site/in app behavior
    73. 73. again, the right metrics are often more of an art than a science Twitter‟s success metric for this email is likely something like “incremental follows”
    74. 74. make it easy: which ONE thing do you want the user to do? ?!?!?!?
    75. 75. this is pretty straightforward (probably optimized for day1 buyers?)
    76. 76. don„t forget about creative optimization Savored – increase from 4-15 restaurants per email increased RPM by over 300%
    77. 77. even creative is about relationships! component CTR vs. revenue
    78. 78. product retention
    79. 79. watch conversion closely
    80. 80. data-driven product optimization (GA click map) Source:
    81. 81. Referral (and “Social,” “Viral,” etc.) sharing = traffic = $$$
    82. 82. understand which types of content produce highest sharing lift Source: AddThis
    83. 83. make it easy for people to invite others (and incentivize them to do so)
    84. 84. Revenue How do we make money (and lots of it)?
    85. 85. everything we‟ve already said, plus one more thing…
    86. 86. understand which factors underpin revenue propensity Source: betashop, 16 March 2011.
    87. 87. …and then drive behavior in those directions
    88. 88. to summarize? the 360-degree customer is nothing without situational understanding (relationships!)
    89. 89. questions?