Campaign Attribution is Broken

So how do we maximise ROI on marketing spend?
Who am I?
    G’day, I’m Peter...
    I am Australian – with a strong aussie accent
    Founder of L3 Analytics
    Also Founder of MeasureCamp




    Page 2                @peter_oneill   28th Nov, 2012
I intend to cover 4 areas*
1.       Review visitor behaviour & touch points
         How campaign attribution treats them
2.       My debate with campaign attribution fans
3.       Always focus on the business questions
4.       My recommended approaches**

* Using an extended version of Matthew Tod’s football (soccer)
  analogy to make my points
** I don’t have a case study to prove these work ***
*** Yet...

 Page 3                        @peter_oneill           28th Nov, 2012
VISITOR BEHAVIOUR




Page 4     @peter_oneill   28th Nov, 2012
The Field of Play




RIO (location of the game)
Who gets the credit for a goal (conversion)?




Page 6             @peter_oneill   28th Nov, 2012
1. Last Click Attribution




                                  Final touch
                                  scores goal &
                                  gets all credit




RIO (location of the game)
Who gets the credit for a goal (conversion)?
    The “Goal Scorer”
        Last click attribution




    Page 8                        @peter_oneill   28th Nov, 2012
2. First Click / Weighted Attribution Models

                            Midfield play
                            set up the goal
                            for the striker




       Multiple players
       contributed to the
       goal & may deserve
       some credit

RIO
Who gets the credit?
    The “Goal Scorer”
        Last click attribution
    Midfield passes
        First click / weighted attribution / descending
         attribution / whatever allows you to pick the winner...




    Page 10                       @peter_oneill   28th Nov, 2012
3. Ad Tracking Models




                  Ad tracking
                  networks don’t
                  capture all online
                  touch points

RIO
Who gets the credit?
    The “Goal Scorer”
        Last click attribution
    Midfield passes
        First click / weighted attribution / descending
         attribution / whatever allows you to pick the winner...
    Ignoring players
        An issue with traditional ad tracking tools




    Page 12                       @peter_oneill   28th Nov, 2012
4. Multiple Devices




       Play started on
       the other side of
       the pitch & these
       players deserve
       credit too
      Work (or smart            Home (computer)
RIO
      phone, tablet, etc)
Who gets the credit?
    The “Goal Scorer”
        Last click attribution
    Midfield passes
        First click / weighted attribution / descending
         attribution / whatever allows you to pick the winner...
    Ignoring players
        An issue with traditional ad tracking tools
    Initiators of the passage of play
        You simply can’t ignore the issue of multiple devices




    Page 14                       @peter_oneill   28th Nov, 2012
5. Offline Touch Points




      Ball forced out by
      defender & other
      players provided
      alternative
      attacking options –
      also deserve credit

RIO
Who gets the credit?
    The “Goal Scorer”
        Last click attribution
    Midfield passes
        First click / weighted attribution / descending
         attribution / whatever allows you to pick the winner...
    Ignoring players
        An issue with traditional ad tracking tools
    Initiators of the passage of play
        You simply can’t ignore the issue of multiple devices
    Defenders caused the error & ran in support
        Offline touch points can’t be captured in any tool,
         however powerful (or big the data is)
    Page 16                       @peter_oneill   28th Nov, 2012
Let’s look at a scenario
    Kevin Hillstrom (MineThatData) wrote this scenario &
    asked – How should this purchase be attributed?




    http://blog.minethatdata.com/2012/10/your-opinion-wanted-attribution.html



    Nine varied answers, most very precise
        Lets have a flick through

     Page 17                                          @peter_oneill             28th Nov, 2012
THE DEBATE




Page 18       @peter_oneill   28th Nov, 2012
Defining the Debate
    Campaign attribution tools only capture a part of
     the visitor journey
        All, half, most, a minority of the journey – it depends...
        If all visitors login, you will get more (multiple devices)
    But it is just not possible to get the full visitor
     journey, however powerful (expensive) the tool is
     or how big the data is

    Let’s clarify what I believe the debate should be

“Is the data being captured sufficient to provide the
 intelligence for informed business decisions?”
    Page 19                     @peter_oneill      28th Nov, 2012
Current Majority Opinion
                                                                 Invest in the right tools &
                                                                 throw enough resources at
                                                                 the problem – and it will
                                                                 can be solved!!
 Image from SmartInsights blog




This is Digital Analytics – we
work with the data we have                                                     Image from Adobe SiteCatalyst blog



                                                                          In sporting terms
                                                                            “Point to the
                                                                            Scoreboard”
                                 Quotes from Tagman case study


  Page 20                                              @peter_oneill          28th Nov, 2012
My Viewpoint - No
    This is not a sampling size issue
        e.g. We don’t have 100% accuracy but we can use the
         trends
    It is incomplete data
    Scale of the problem is unknown
    Data available can be misleading
    Risk of wrong decisions is too large
    Paul Postance: “it could be done better” is not an
     insult. It’s a mindset to make things better.
    Change the conversation from which model is
     most accurate to how to optimise spend
    Page 21                   @peter_oneill    28th Nov, 2012
BUSINESS QUESTIONS




Page 22    @peter_oneill   28th Nov, 2012
What do we really need to know?
    Do we really need campaign attribution models?
        No – they are simply the most obvious solution to the
         business problem
    Let’s focus on the business problems, not the
     technology
    There are three business intelligence requirements...
      1.   What do I report to the business?
      2.   How much do I pay agencies for the last period?
      3.   How do I optimise future marketing spend?
    Why hunt for a single solution – use the best
     approach to answer each requirement
    Page 23                    @peter_oneill    28th Nov, 2012
A 4th Question – from the CEO




         Did we win?



RIO ROI (only location they care about)
RECOMMENDED
 APPROACHES




Page 25   @peter_oneill   28th Nov, 2012
Business Performance reporting
    Pick a single approach & stick to it
        I don’t care which (last click is simplest)
    What is important is that
        The business understands the approach
        A change in performance can be easily identified
             and reacted to!!
    Football analogy – who scored the goals




    Page 26                      @peter_oneill     28th Nov, 2012
Evaluate Agency Performance
    Define business objective for each campaign
        Is it awareness, research, conversion, etc
    Define what success looks like
        KPIs & targets
    Agree payment on this basis
    Football analogy – performance based payments
        Goals, tackles, minutes played, crosses, accurate
         crosses
    Business
        Impressions, TVRs, visits, non bounce visits, visits that
         create a basket, leads, sales, increase in NPS

    Page 27                    @peter_oneill      28th Nov, 2012
Don’t say this can’t be done...




    Not saying it would be easy
    But wouldn’t a campaign
     equivalent of this be incredibly
     useful/actionable?


    Page 28                @peter_oneill   28th Nov, 2012
Optimise Marketing Spend
    Remind me – how do we optimise again?
        Objectives | Evaluation | Hypothesis | Test
    Football analogy – simultaneous games




    Page 29                   @peter_oneill    28th Nov, 2012
Optimise Marketing Spend
    Business equivalent – test campaigns
    How to test campaigns
        Different campaigns in different geographical regions
        Hold out tests
        Switch off/on keywords
        Pick similar trending products & promote half
        Measure impact from offline on online & vice versa
    Learn what impact of campaign really is
    Optimise spend using these learnings



    Page 30                    @peter_oneill    28th Nov, 2012
HAVE I CONVINCED
 ANYONE???




Page 31       @peter_oneill   28th Nov, 2012
Summary
    Campaign attribution is broken
        Simply can’t capture all touch points
    Instead of wasting money on impossible, invest
     resources in the difficult
    Focus on the real business intelligence
     requirements
    Be consistent in internal reporting
    Evaluate performance against predefined
     objectives & targets specific to campaign
    Test to discover what works best
    Adjust spend to maximise profitability
    Page 32                    @peter_oneill     28th Nov, 2012
THANK YOU

   I can be found at
   • peteroneill@l3analytics.com
   • @peter_oneill
   • +44 7843 617 347
   • www.linkedin.com/in/peteroneill




Page 33                            @peter_oneill   28th Nov, 2012

Campaign attribution is broken

  • 1.
    Campaign Attribution isBroken So how do we maximise ROI on marketing spend?
  • 2.
    Who am I?  G’day, I’m Peter...  I am Australian – with a strong aussie accent  Founder of L3 Analytics  Also Founder of MeasureCamp Page 2 @peter_oneill 28th Nov, 2012
  • 3.
    I intend tocover 4 areas* 1. Review visitor behaviour & touch points  How campaign attribution treats them 2. My debate with campaign attribution fans 3. Always focus on the business questions 4. My recommended approaches** * Using an extended version of Matthew Tod’s football (soccer) analogy to make my points ** I don’t have a case study to prove these work *** *** Yet... Page 3 @peter_oneill 28th Nov, 2012
  • 4.
    VISITOR BEHAVIOUR Page 4 @peter_oneill 28th Nov, 2012
  • 5.
    The Field ofPlay RIO (location of the game)
  • 6.
    Who gets thecredit for a goal (conversion)? Page 6 @peter_oneill 28th Nov, 2012
  • 7.
    1. Last ClickAttribution Final touch scores goal & gets all credit RIO (location of the game)
  • 8.
    Who gets thecredit for a goal (conversion)?  The “Goal Scorer”  Last click attribution Page 8 @peter_oneill 28th Nov, 2012
  • 9.
    2. First Click/ Weighted Attribution Models Midfield play set up the goal for the striker Multiple players contributed to the goal & may deserve some credit RIO
  • 10.
    Who gets thecredit?  The “Goal Scorer”  Last click attribution  Midfield passes  First click / weighted attribution / descending attribution / whatever allows you to pick the winner... Page 10 @peter_oneill 28th Nov, 2012
  • 11.
    3. Ad TrackingModels Ad tracking networks don’t capture all online touch points RIO
  • 12.
    Who gets thecredit?  The “Goal Scorer”  Last click attribution  Midfield passes  First click / weighted attribution / descending attribution / whatever allows you to pick the winner...  Ignoring players  An issue with traditional ad tracking tools Page 12 @peter_oneill 28th Nov, 2012
  • 13.
    4. Multiple Devices Play started on the other side of the pitch & these players deserve credit too Work (or smart Home (computer) RIO phone, tablet, etc)
  • 14.
    Who gets thecredit?  The “Goal Scorer”  Last click attribution  Midfield passes  First click / weighted attribution / descending attribution / whatever allows you to pick the winner...  Ignoring players  An issue with traditional ad tracking tools  Initiators of the passage of play  You simply can’t ignore the issue of multiple devices Page 14 @peter_oneill 28th Nov, 2012
  • 15.
    5. Offline TouchPoints Ball forced out by defender & other players provided alternative attacking options – also deserve credit RIO
  • 16.
    Who gets thecredit?  The “Goal Scorer”  Last click attribution  Midfield passes  First click / weighted attribution / descending attribution / whatever allows you to pick the winner...  Ignoring players  An issue with traditional ad tracking tools  Initiators of the passage of play  You simply can’t ignore the issue of multiple devices  Defenders caused the error & ran in support  Offline touch points can’t be captured in any tool, however powerful (or big the data is) Page 16 @peter_oneill 28th Nov, 2012
  • 17.
    Let’s look ata scenario Kevin Hillstrom (MineThatData) wrote this scenario & asked – How should this purchase be attributed? http://blog.minethatdata.com/2012/10/your-opinion-wanted-attribution.html  Nine varied answers, most very precise  Lets have a flick through Page 17 @peter_oneill 28th Nov, 2012
  • 18.
    THE DEBATE Page 18 @peter_oneill 28th Nov, 2012
  • 19.
    Defining the Debate  Campaign attribution tools only capture a part of the visitor journey  All, half, most, a minority of the journey – it depends...  If all visitors login, you will get more (multiple devices)  But it is just not possible to get the full visitor journey, however powerful (expensive) the tool is or how big the data is  Let’s clarify what I believe the debate should be “Is the data being captured sufficient to provide the intelligence for informed business decisions?” Page 19 @peter_oneill 28th Nov, 2012
  • 20.
    Current Majority Opinion Invest in the right tools & throw enough resources at the problem – and it will can be solved!! Image from SmartInsights blog This is Digital Analytics – we work with the data we have Image from Adobe SiteCatalyst blog In sporting terms “Point to the Scoreboard” Quotes from Tagman case study Page 20 @peter_oneill 28th Nov, 2012
  • 21.
    My Viewpoint -No  This is not a sampling size issue  e.g. We don’t have 100% accuracy but we can use the trends  It is incomplete data  Scale of the problem is unknown  Data available can be misleading  Risk of wrong decisions is too large  Paul Postance: “it could be done better” is not an insult. It’s a mindset to make things better.  Change the conversation from which model is most accurate to how to optimise spend Page 21 @peter_oneill 28th Nov, 2012
  • 22.
    BUSINESS QUESTIONS Page 22 @peter_oneill 28th Nov, 2012
  • 23.
    What do wereally need to know?  Do we really need campaign attribution models?  No – they are simply the most obvious solution to the business problem  Let’s focus on the business problems, not the technology  There are three business intelligence requirements... 1. What do I report to the business? 2. How much do I pay agencies for the last period? 3. How do I optimise future marketing spend?  Why hunt for a single solution – use the best approach to answer each requirement Page 23 @peter_oneill 28th Nov, 2012
  • 24.
    A 4th Question– from the CEO Did we win? RIO ROI (only location they care about)
  • 25.
    RECOMMENDED APPROACHES Page 25 @peter_oneill 28th Nov, 2012
  • 26.
    Business Performance reporting  Pick a single approach & stick to it  I don’t care which (last click is simplest)  What is important is that  The business understands the approach  A change in performance can be easily identified  and reacted to!!  Football analogy – who scored the goals Page 26 @peter_oneill 28th Nov, 2012
  • 27.
    Evaluate Agency Performance  Define business objective for each campaign  Is it awareness, research, conversion, etc  Define what success looks like  KPIs & targets  Agree payment on this basis  Football analogy – performance based payments  Goals, tackles, minutes played, crosses, accurate crosses  Business  Impressions, TVRs, visits, non bounce visits, visits that create a basket, leads, sales, increase in NPS Page 27 @peter_oneill 28th Nov, 2012
  • 28.
    Don’t say thiscan’t be done...  Not saying it would be easy  But wouldn’t a campaign equivalent of this be incredibly useful/actionable? Page 28 @peter_oneill 28th Nov, 2012
  • 29.
    Optimise Marketing Spend  Remind me – how do we optimise again?  Objectives | Evaluation | Hypothesis | Test  Football analogy – simultaneous games Page 29 @peter_oneill 28th Nov, 2012
  • 30.
    Optimise Marketing Spend  Business equivalent – test campaigns  How to test campaigns  Different campaigns in different geographical regions  Hold out tests  Switch off/on keywords  Pick similar trending products & promote half  Measure impact from offline on online & vice versa  Learn what impact of campaign really is  Optimise spend using these learnings Page 30 @peter_oneill 28th Nov, 2012
  • 31.
    HAVE I CONVINCED ANYONE??? Page 31 @peter_oneill 28th Nov, 2012
  • 32.
    Summary  Campaign attribution is broken  Simply can’t capture all touch points  Instead of wasting money on impossible, invest resources in the difficult  Focus on the real business intelligence requirements  Be consistent in internal reporting  Evaluate performance against predefined objectives & targets specific to campaign  Test to discover what works best  Adjust spend to maximise profitability Page 32 @peter_oneill 28th Nov, 2012
  • 33.
    THANK YOU I can be found at • peteroneill@l3analytics.com • @peter_oneill • +44 7843 617 347 • www.linkedin.com/in/peteroneill Page 33 @peter_oneill 28th Nov, 2012