Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measure Right the First Time - Infusionsoft Partnercon

5,903 views

Published on

  • Be the first to comment

Measure Right the First Time - Infusionsoft Partnercon

  1. 1. MEASURE RIGHT THE FIRST TIMEInfusionsoft Partnercon 2012 vuurr.com
  2. 2. WHAT WE’LL COVER• What you need• How to track• How to design experiments• What to experiment on• Putting it all together
  3. 3. TOOLS
  4. 4. TOOLS YOU MUST HAVE • Google Analytics (or SiteCatalyst, or similar) • Optimizely (or similar, but you need killer statistics) • Call tracking => analytics platform of choice • Link builder appropriate for your analytics platform of choice • AdWords/AdCenter/Facebook properly configured (if advertising)
  5. 5. DETERMINE YOUR MODEL CPA ROI MARGIN Minimize Maximize Minimize ad the cost of revenue for spend as a each every dollar percent of customer spent revenue acquisition
  6. 6. DETERMINE YOUR MODEL CPA ROI MARGIN Ad Spend ($) Revenue - Spend Revenue - Spend Conversion (qty) Spend Revenue answer is in dollars ($) per all in dollars ($), answer is % all in dollars ($), answer is % conversion
  7. 7. DETERMINE YOUR MODEL -FINDING ROI & MARGIN MARGIN ROI
  8. 8. DETERMINE YOUR MODEL - FINDING CPA (MANUALLY)Advertising => AdWords Completed=> Campaigns => Site Usage Conversions Total Cost Advertising => AdWords => Campaigns => Clicks
  9. 9. DETERMINE YOUR MODEL -FINDING CPA (AUTOMATICALLY) • Use the Google Analytics API to query for total number of conversion by source & medium, and then properly divide them into the total marketing expense for each source & medium. • Use a third party tool
  10. 10. TRACKING
  11. 11. INSTALLING ANALYTICS• Follow your vendor’s instructions• Don’t just dump the tracking code in and “go”• Setup proper funnels, events, goals, and filters• Know the difference between accounts, properties, and profiles• Attach GWT and AdWords (if applicable)• Track ALL OF THE THINGS
  12. 12. GOALS & ECOMMERCE• Assign relevant goal values. A video view != purchase• Track eCommerce as accurately as possible• Explicit funnels are only useful if the supporting steps are relevant and provide information - use sparingly
  13. 13. TRACKING CAMPAIGNS • Every click is valuable Name/ • Every traffic source Content matters • Proper campaign Campai tracking is crucial to adequately gauging gn traffic valueSource Medium
  14. 14. PROPER URL TAGGING
  15. 15. TRACKING RICH MEDIA:VIDEOS• Instantiate video with YouTube/Vendor JavaScript API• Track state changes, namely Plays and Completes• On these changes, push GA events
  16. 16. TRACKING RICH MEDIA:SOCIAL BUTTONS_gaq.push([_trackSocial, network, socialAction, opt_target,opt_pagePath]); Instantiate by subscribing to edge.create and message.send Google Analytics handles for you Instantiate by subscribing to the ‘tweet’ intent event
  17. 17. TRACKING RICH MEDIA:FORM ABANDONMENT • Track full and empty exits as events • Determine fields causing abandonment
  18. 18. TRACKING OFFLINE:PHONE CALLS• Build GA gif request and HTTP GET on phone call• Push calls as pageviews ( /call/ +14805551234/abcdefg )• Use Twilio & Twimlbin or another 3rd party service• Consider filtering out those pageviews from main profile• Setup goals in Google Analytics to fire on calls• Give each ad, landing page, social source, different number
  19. 19. TRACKING PAID CLICKS • Properly configure AdWords to push to Analytics • Verify all cost data is applied to Analytics account • Put MSN AdCenter {AdId} & {Keyword} in destination URL, then use AdCenter API to pull the cost and associate with relevant clicks from Analytics API
  20. 20. EXPERIMENT DESIGN
  21. 21. INTERNET MARKETING IS ABOUTEXPERIMENTATIONy = f( x1, x2, x3, . . . . xn ) y CTR CTR X1= Page or Ad Copy Page or Ad Copy CR CR X2= Page Layout Page Layout X3= Button Color & Size Button Color & Size Goals Goals Xn= Other Factors Other Factors
  22. 22. DESIGNING EXPERIMENTS -ONE FACTOR AT A TIMELet’s run an experiment where we test two different buttoncolors. Run Button Color ( x1 ) CTR ( y ) 1 Green 5.2% 2 Orange 7.7%Factors are tested in series, one at a time. This is extremelytime intensive, requiring many tests for very gradualimprovement.
  23. 23. DESIGNING EXPERIMENTS -FULL FACTORIAL DESIGNLet’s run an experiment where we test two different buttoncolors, button positions, and button labels simultaneously. 3 FACTORS WITH 2 VARIATIONS EACH Number of Unique Tests =2 =8 3This tests multiple changes in parallel to find optimalcombination of factors (x variables) to optimize results (yvariable).
  24. 24. DESIGNING EXPERIMENTS:FULL FACTORIAL DESIGN Run Color (x1) Position (x2) Text (x3) CTR (y) 1 Green Left Try Now 5.2% 2 Orange Left Try Now 7.7% 3 Green Right Try Now 9.3% 4 Orange Right Try Now 12.7% 5 Green Left Buy Today 21.2% 6 Orange Left Buy Today 19.3% 7 Green Right Buy Today 14.9% 8 Orange Right Buy Today 17.7%
  25. 25. DESIGNING EXPERIMENTS:OFAT VS. FULL FACTORIAL ONE FACTOR FULL FACTORIAL AT A TIME More Complex to Setup Iterative Simple to Setup VS. Requires more visitor data to test all Easy to Measure Results combinations SLOW & GRADUAL QUICK & DRASTIC IMPROVEMENT IMPROVEMENT
  26. 26. DESIGNING EXPERIMENTS:STATISTICAL SIGNIFICANCEPreliminary results - is orange better?Button Color ( Visitors Clicks CTR ( y ) x1 ) Green 38 2 5.2% Orange 39 3 7.7% Too small of a sample size leads to incorrect conclusions.
  27. 27. DESIGNING EXPERIMENTS:STATISTICAL SIGNIFICANCEIt turns out our original conclusion of CTR was incorrect.Button Color ( Visitors Clicks CTR ( y ) x1 ) Green 8,238 486 5.9% Orange 7,893 734 9.3% Too large of a sample size wastes time, effort, & money.
  28. 28. SO WHAT QUANTITY OF VISITORS IS ENOUGH?
  29. 29. DETERMINING MINIMUM SAMPLESIZEx = Sample mean E=x-μμ = Population meanZ = Number of standard σdeviations above meanα = Confidence distance from100% E=Z α/2σ = Standard deviationn = Minimum sample size √n
  30. 30. DETERMINING MINIMUM SAMPLESIZE - α/2 Z α/2
  31. 31. DETERMINING MINIMUM SAMPLESIZE (CONTINUED) σ Standard Deviation E=Z α/2 √n Minimum Sample Size n= [Z α/2 σ] E
  32. 32. A/B TESTING SOFTWARE
  33. 33. TESTING
  34. 34. EVERY SINGLE CLICK HAS A COST ASSOCIATED WITH IT
  35. 35. KNOW YOUR LEVERS • Product Pricing • Organic Rankings • Ad bids, placement, content • Email subject lines & content • Social content • Site Content
  36. 36. FUNNEL ANALYSIS &OPTIMIZATION IMPRESSIONS Traffic Sources drive impressions & clicks CT • SEM, SEO, Social, Email, & More R CLICKS Optimizing those sources get more clicks (Site Visits) • • Improve Quality Score Split Test Ad Copy • Adjust Keyword Bids • Increase Organic Rankings • Increase Social Reach CR Optimized landing pages & site content increase conversions • Split Test Landing Page Copy • Improve Call to Action • Experiment with Deep CONVERSION Linking S • • Look at Bounce Rates Look at Cart Abandonment (Sales/Leads)
  37. 37. FUNNEL ANALYSIS &OPTIMIZATION ORGANIC OFFLINE S EARC H MARKETING AFFILIATESEMAIL PPC S OC IAL PHONE C ALLS ALL OF THESE SOURCES HAVE LEVERS THAT MANIPULATE THEIR QUALITY & LIKELIHOOD TO BECOME BUYERS
  38. 38. EMAIL OPTIMIZATION LEVERS EMAILS SENT LEVER 1: List Size Increase List Size • Completely • Optimize CR on dependent on subscription page conversion rate of • Increase sales to lure subscription form repeat buyers OPENED EMAILS LEVER 2: Open Rate Test Subject Line LEVER 3: CTR Test Email Copy LINKS CLICKED
  39. 39. EMAIL OPTIMIZATION LEVERS EMAILS SENT LEVER 1: List Size Increase List Size LEVER 2: Open Rate Test Subject Line Quantity • Use an email OPEN = application with A/B RATE Opened testing OPENED EMAILS Quantity Sent • Call to action is important LEVER 3: CTR Test Email Copy LINKS CLICKED
  40. 40. EMAIL OPTIMIZATION LEVERSEMAILS EMAILS SENT SENT LEVER 1: List Size Increase List Size LEVER 2: Open Rate Test Subject Line LEVER 3: CTR Test Email Copy OPENED • Treat as a landing EMAILS Clicks from CTR = Email page • Test layout, buttons, Quantity content, etc. Opened LINKS CLICKED
  41. 41. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget • Directly drive • Optimize for maximum conversions factoring in conversions at current conversion rate & CPC CPC LEVER 2: Bids Optimize for KPI AD CLICKS LEVER 3: Quality Score Maximize QS LEVER 4: CTR Test Ad Copy
  42. 42. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget LEVER 2: Bids Optimize for KPI Actual + Ranking Score of Position Below Cost = + $0.01 Quality Score of Your Ad AD CLICKS LEVER 3: Quality Score Maximize QS LEVER 4: CTR Test Ad Copy
  43. 43. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget LEVER 2: Bids Optimize for KPI LEVER 3: Quality Score Maximize QS • Function of ad content, • Relevant ad landing page content, • Relevant landing page AD CLICKS historical CTR, & more • Display URL LEVER 4: CTR Test Ad Copy
  44. 44. PPC OPTIMIZATION LEVERSAD IMPRESSIONS LEVER 1: Budget Increase Budget LEVER 2: Bids Optimize for KPI LEVER 3: Quality Score Maximize QS LEVER 4: CTR Test Ad Copy AD CLICKS • Continuously A/B test CTR = Clicks from Ad ad Ad Impressions • Keep highest CTR ad • Delete lower CTR ads • Repeat until returns diminish
  45. 45. SOCIAL CONTENT OPTIMIZATION LEVERS SOCIAL IMPRESSIONS LEVER 1: Network Add Networks • Be there in the first • Pinterest place • Reddit • OthersSOCIAL CONTENT LEVER 2: Content Quantity/Quality CLICKS LEVER 3: Interactions Followers/Frequency
  46. 46. SOCIAL CONTENTOPTIMIZATION LEVERS SOCIAL IMPRESSIONS LEVER 1: Network Add Networks LEVER 2: Content Quantity/Quality • Links to deep content • Frequency • Polite, relevant posts • Better content drives more shares/retweetsSOCIAL CONTENT CLICKS LEVER 3: Interactions Followers/Frequency
  47. 47. SOCIAL CONTENTOPTIMIZATION LEVERS SOCIAL IMPRESSIONS LEVER 1: Network Add Networks LEVER 2: Content Quantity/Quality LEVER 3: Interactions Followers/FrequencySOCIAL CONTENT • More interactions • Increase quantity and CLICKS drives more clicks value of followers • Don’t “buy” followers
  48. 48. WEBSITE LAYOUT & CONTENT VISITS LEVER 1: Layout Forms, Phone #s LEVER 2: Content Quantity, Quality LEVER 3: Click Areas Size, Color, Position LEVER X: etc.....CONVERSIONS
  49. 49. EXECUTION
  50. 50. THE WEB IS A SYSTEM• Your marketing strategy is a system of equations• Some levers are independent of others• Some levers manipulate others• Some levers have a larger impact on KPI than others Ad Bid SEO Email Social Spend Subject Effort
  51. 51. THE WEB IS A SYSTEM• Think about moving several levers at once• Always consider what a single and multiple-lever move will do to the system, and accordingly, your end result• Design a good experiment, continue to measure, and repeat y = f( x1, x2, x3, . . . . xn )
  52. 52. EXAMPLE SCENARIO• Current Budget: $7,500 / month ( $250 / day )• Conversions: 48 / month ( 1.6 / day )• Average CPA = $158.17• Average CPC = $2.29• Landing Page CR = 1.45%New Target: 10 Conversions / Day AND Lower CPA
  53. 53. SCENARIO ONE: ADJUSTING BUDGET• Average CPA at $158.17 means 10 conversions per day is $1581.70• $47,000 per month• Easy, but too expensive
  54. 54. SCENARIO TWO: ADJUSTING CPC• Budget stays at $250 / day, conversion rate steady at 1.45%• Need 689 clicks to get 10 conversions• Average CPC must be $0.36• Improbable to bring $2.29 down to $0.36 in this space while maintaining volume
  55. 55. SCENARIO THREE: ADJUSTCONVERSION RATE• Budget stays at $250 / day, CPC at $2.29• Can purchase 110 clicks for $250• To get 10 conversions, landing page needs to convert at 9.1%• Improbable to quickly jump from 1.45% to 9.1%
  56. 56. FINAL SCENARIO: MOVE ALL THREELEVERS• Double the budget to $15,000• Reduce average CPC by 50% ( $2.29 to $1.15 ) via QS & Keywords• Improve landing page CR from 1.45% to 2.3% via A/B testing• ( $15k / $1.15 per click ) at 2.3% drives 300 conversions monthly at a CPA of $50• Used 3 levers simultaneously to improve conversions 6x and reduce average CPA by 68%
  57. 57. ITERATING
  58. 58. CREATE A CULTURE OF CONTINUOUSIMPROVEMENT• Define success• Measure current performance and compare to targets• Determine biggest levers and how they contribute to KPIs• Design an experiment• Test repeatedly• Finalize improvements based on adequate data• Repeat
  59. 59. Questions?Scott Yacko - scott@vuurr.com - @scottmyacko Jonathan Kressaty - jonathan@vuurr.com - @kressaty

×