Metrics and Analytics, Guest Lecture, UCLA
Upcoming SlideShare
Loading in...5
×
 

Metrics and Analytics, Guest Lecture, UCLA

on

  • 829 views

Provided hour lecture on metrics and analytics for Jaime Levy's UX Design course at UCLA Extension.

Provided hour lecture on metrics and analytics for Jaime Levy's UX Design course at UCLA Extension.

Statistics

Views

Total Views
829
Views on SlideShare
822
Embed Views
7

Actions

Likes
0
Downloads
7
Comments
0

2 Embeds 7

http://www.linkedin.com 6
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • You rarely here of companies going out of business because technology was to difficult to build. Usually it’s about figuring out a business model and pivoting, which comes from understanding metrics. <br />
  • No, it’s your job , knowledge is power. <br />
  • When you guys go to the doctor, what metrics do you pay attention to? Anyone watch sports….what metrics do you guys pay attention to?Is the product working, viable or is it a useless piece of crap? <br />
  • You know these guys? The big metric here was the purity of their meth. <br />
  • …for the purpose of this session, we’re gonna touch on anlalytics but the focus will be on metrics. <br />
  • It’s not that GA is bad, it’s actually incredibly powerful, you just have to realize it’s a great place to start, but shouldn’t do everything. <br />
  • If you read lean analytics, chapter 1 is about “we’re all liars” delusional entrepreneurs. It’s good for entrepreneurs to be able to make a strong pitch, but stop selling to yourself, pay attention to the data and learn when you’re full of it. <br />
  • You could probably say Ali G is lying to himself, and he might be. His data is all assertions. Is this a bad product? <br /> Who knows, maybe he should get something on the market and see. <br />
  • So where does metrics fit in the lean startup “build measure learn” – it’s the Data that you learn from, enabling you to pivot or stay the course. <br />
  • Traffic – one day it’s 10000, next day it’s 10001, so whattime on site – is that good or bad, is site broken?Overall conversion rate – not really so useful especially when prices are a major factor <br />
  • Especially for startups, less so far larger businessesone month focus is on acquiring and optimizing traffic, then reg, then optimizing activation <br />
  • Starting at top of the funnel with acquisition, you can continue to push focus as you progress. <br />
  • LTV must be greater than CPA, ARPU can be less than CPA (but hurts cashflow) <br />
  • Add a little image of a price test (similar to the AB test) <br />
  • Now add a spreadsheet showing the $6 as a $4 option but showing the $6 winning in terms of ARPU and Net, but executive winning on Gross <br />
  • Show groups of things that are testable (from AB testing book) <br />
  • November 2012 Amazon quietly switched their pricing to offer a monthly and annual, then switching it off several days later. <br />
  • November 2012 Amazon quietly switched their pricing to offer a monthly and annual, then switching it off several days later. <br />
  • Quantitative can’t tell you your web site looks like shit; ask them if they ever speak to their customers. <br />
  • Now add a spreadsheet showing the $6 as a $4 option but showing the $6 winning in terms of ARPU and Net, but executive winning on Gross <br />
  • Show groups of things that are testable (from AB testing book) <br />

Metrics and Analytics, Guest Lecture, UCLA Metrics and Analytics, Guest Lecture, UCLA Presentation Transcript

  • Analytics & Metrics UCLA Extension (Guest Lecture) Darren Levy - Senior Director of Revenue, Retention at Mylife.com - Founder of Gatherspace.com darrenjlevy@gmail.com
  • True or False: Technology is the hardest part of building an online business?
  • True or False: Knowing the metrics is someone else’s job.
  • Breakdown of today’s discussion - Part 1 – Definitions, why they are so critical - Part 2 – What metrics to focus on? - Part 3 – Optimizing metrics - Part 4 – Case Studies
  • I. Why do we need metrics?
  • What are metrics, why do we need them? - Small units of measurement, typically ratio’s. - How are we doing? - Is business healthy? - Is the customer happy, is the product working?
  • 99.1% Pure
  • …and what about analytics? - Combining metrics, finding patterns of behavior. - What should we be working on? - Should we pivot?
  • Google Analytics – be cautious - Storing all of your proprietary data with Google Limitations on customizations When shit hits the fan, getting customer support You NEED to slice and dice and query your data
  • Why are metrics so important for YOU?! - Be the Expert, be the Hero, the “go to” person - Don’t rely so much on others - Because we’re all Liars
  • Ice Cream Glove
  • Tightly ingrained within Lean Startup Process Validation Hypothesis Experiment Metrics
  • II. What metrics to focus on?
  • Vanity Metrics - Not actionable - Not understandable - Examples: • traffic • time on site • cumulative registrations • # of visits • Overall conversion rate
  • Single Key Metric OMTM - Analysis Paralysis (don’t overdo it!) - For startups, business rally around a metric - Changes over time as business progresses
  • Customer Lifecycle – 5 Steps to Success AARRR!
  • Dave McClure’s Pirate Metrics AARRR!
  • AARRR Lifecycle Category Lifecycle Experience Metrics * CPC * Keyword Rank * Ad CTR Est Value Acquisition Visit Site (or landing page, or widget) Activation Jump through initial hoop, signup (“a ha” moment) * CTR Retention Repeat Visitor, Open up emails * Avg # of Visits * Engagement Score CTR ~2% Value=$5 Customer generates revenue * RPV * AOV * ARPU CTR ~1% Value=$25 Revenue CTR ~100% Value=.01 CTR ~5% Value=$3
  • Lean Analytics Growth Stages (“Gates”)
  • Single Key Metrics For Startups How much do we spend to get a customer? Cost per Acquisition ($) CPA How much do our customers pay us? Average Revenue per User ($) ARPU At what rate are customers cancelling? Churn (%) Churn How much is a customer worth over time? Lifetime Value per Customer ($) LTV
  • Subscription v. Transactional Models Subscription Business VPR (value per reg) Churn / LTV / ARPU Activation & Engagement Transactional Business Shopping Cart Abandonment Average Order Value (AOV) Upsell & Product Page CTR
  • III. How do we do this?
  • How do we get Metrics? - Google Analytics DIY – recording events, activities in a SQL db AB testing Other tools – Heatmaps
  • Types of AB Testing - Simple AB testing - Multivariate testing - Price testing
  • Simple AB Testing - Changing 1 element at a time? - Effecting CTR at any point in a funnel
  • Multivariate Testing - More simultaneous variables - Takes longer to test - Better results than “simple”
  • Price Testing - Testing different prices or terms Not measuring CTR, measuring RPV Pay attention to “Accumulation of Revenue” Pricing Strategies (freemium, subsc., bundling)
  • Ugly Friend Concept
  • Famous Price Test (Ugly Friend) - Decoy (ugly friend) pushes price selection - In this example, decoy pushed sales by 30%
  • Apple iPhone Pricing ~67% ~20% ~13%
  • 10 Things you MUST Test 1. Calls to Action (CTA) 2. Propositions 3. Copy & Content 4. Visual Media 5. Funnel testing 6. Forms 7. Emails 8. Pricing 9. Shipping 10. Personalization
  • AB Test Example
  • Cohort Analysis Measuring engagement and value over time
  • Optimization via Heatmaps
  • Quantitative v. Qualitative Metrics - Example of qualitative (fire alarm in middle of night, 99.97% up time, yet pissed off customers) - Why you need both. (user testing, surveys, call your customers)
  • IV. Case Studies
  • Price Test @ Dollarshaveclub.com Control Test 4 4.4% higher 24 hr Net 7 7% higher CLTV
  • Case Study: Netflix & House of Cards - Goal to be “HBO” of online TV - Cost of $100m on production - Analytics told them * Most customers watched David Fincher’s social network all the way * British version of “House of Cards” was highly watched * Significant overlap of British House of Cards watch Kevin Spacy films and Fincher films. - Data so far * Q1 2013, brought in 3m new custmers * in Q1 alone, paid for cost of production * Increasing CLTV
  • Q&A