GA is a powerful tool but has limitations when it comes to accurately assessing FB’s true value. How can we measure the incremental impact of Facebook ads effectively? Consider the three following aspects: Geo Treatment Test, Facebook Lift Analysis, Markov Cross Channel

1. Gianluca Binelli
Managing Director,
Booster Box
Come misurare l'impatto incrementale dei tuoi ad
Facebook nelle campagne di digital marketing

2. Who am I?
● 14y experience in online advertising, 6.5y at Google
between Dublin, NYC & London
● Advisor for Google’s own equity fund Capital G
● Voted 2nd Most Influential PPC Person in 2019
Gianluca Binelli- Booster Box

3. What do we do at Booster Box?
● Uber awesome PPC
● Complex client accounts
● Build Silicon Tuscany
Gianluca Binelli- Booster Box

4. Our philosophy: Paid Media is a science
● With millions of metrics & endless
experimentation possibilities: Paid
Media is the pinnacle of mathematical
marketing.
● As digital marketing scientists, we
have infused the scientific method
across all of our activities.
Gianluca Binelli- Booster Box

5. Gianluca Binelli- Booster Box
10$ 20$

6. Gianluca Binelli- Booster Box
10$
25$
20$

7. Gianluca Binelli- Booster Box
Which one should we trust?

8. The reality is that we should consider both
data points and bear in mind the
differences between the 2 platforms
Gianluca Binelli- Booster Box

9. GA is a powerful tool but has limitations
when it comes to accurately assessing
FB’s true value. Why?
● Google Analytics is cookie based!
● Limited cross-device capabilities
● No visibility on ad impressions
Gianluca Binelli- Booster Box

10. Some of the differences between
Facebook and Google Analytics
1. Facebook tends to be too optimistic sometimes:
● Facebook pixel doesn’t consider other channels, while GA does (by
default, in a last non-direct click attribution model)
● Post view conversions are considered, while GA doesn’t
2. Facebook has more cross-device measurement capability
Gianluca Binelli- Booster Box

11. In a nutshell: you cannot compare apples
to oranges! Or at least, don’t expect to see
the same thing :)
Gianluca Binelli- Booster Box

12. So how can we measure the incremental
impact of Facebook ads effectively?
1. Geo Treatment test
2. Facebook Lift Analysis
3. Markov cross channel analysis
Gianluca Binelli- Booster Box

13. So how can we measure the incremental
impact of Facebook ads effectively?
1. Geo Treatment test
2. Facebook Lift Analysis
3. Markov cross channel analysis
Gianluca Binelli- Booster Box

14. 1. Geo Treatment Test
Use a Randomized Controlled Trial (RCT) as if we were testing a new drug
Gianluca Binelli- Booster Box
● The patients: geographical locations (e.g. cities,
DMAs, regions, countries ecc.)
● The new drug: the introduction of Facebook
advertising (this means that in some locations we
will show the FB ads and in the remaining ones we
will not.

15. USA DMAs
Gianluca Binelli- Booster Box

16. We can use a simple OLS regression to
estimate the impact of introducing
Facebook in our media mix:
Gianluca Binelli- Booster Box
Here more info: Estimating Ad Effectiveness using Geo Experiments in a Time-Based Regression Framework. Google, Inc.

17. Limitations of Geo Treatment Test
1. There is a city mismatch from FB data to GA data:
Pick your cities/towns wisely and make sure they are defined in the
same way on both platforms!
2. Not necessarily applicable for operations with only a few
geographic units available for testing.
Gianluca Binelli- Booster Box

18. Limitations of Geo Treatment Test
3. Ensure there aren’t any specific functions that would affect the
significance of the test like city specific promotions, physical
events or offline promos.
4. Seasonality interference. Try to run the test over a period of
relatively flat seasonality so that your results are not skewed.
Gianluca Binelli- Booster Box

19. Limitations of Geo Treatment Test
5. The Facebook Algorithm performs best with bigger audiences so be
careful not to over segment the cities and hinder the performance of
campaigns.
6. In the pre-launch phase, you need to double check that your
randomization was balanced in terms of characteristics that could
affect your experiment.
Gianluca Binelli- Booster Box

20. So how can we measure the incremental
impact of Facebook ads effectively?
1. Geo Treatment test
2. Facebook Lift Analysis
3. Markov cross channel analysis
Gianluca Binelli- Booster Box

21. 2. Facebook Lift Analysis
Gianluca Binelli- Booster Box
Conversion lift compares the
actions of real people in
randomised test and control
groups to measure the additional
business driven by Facebook

22. Limitations of Facebook Lift Analysis
1. You can’t be certain that there are no external factors affecting the
results as you don't have full control over the randomised groups
while the experiment is running.
2. Testing can sometimes result in smaller-sized test audiences with
comparatively higher CPMs.
Gianluca Binelli- Booster Box

23. So how can we measure the incremental
impact of Facebook ads effectively?
1. Geo Treatment test
2. Facebook Lift Analysis
3. Markov cross channel analysis
Gianluca Binelli- Booster Box

24. 3. Markov Cross Channel
Gianluca Binelli- Booster Box
● Use Markov chains to see what happens to
your conversion rate if you remove certain
parts of your conversion paths.
● Perform the analysis at channel level in order
to assess the impact of Facebook across all
of your media mix.
Here is additional information on how the Markov model works

25. Gianluca Binelli- Booster Box
Here you can find a sample
of a script to implement the
Markov attribution model

26. Markov Chains: the ingredients
1. a CSV file containing
conversion paths (you can
use the one from GA in the
Multi-Channel-Funnel level)
2. the R software to launch the
script
Gianluca Binelli- Booster Box

27. Conclusions
1. There is no right or wrong way to measure the impact of Facebook Ads
on your digital marketing! This is definitely not an exact science:)
2. Considering these 3 different aspects will help you better gauge the
incremental value that Facebook brings to the whole operation!
Gianluca Binelli- Booster Box

28. Gianluca Binelli
gianluca@boosterboxdigital.com
Come misurare l'impatto incrementale dei tuoi ad
Facebook nelle campagne di digital marketing

29. 4-5-6 JUNE 2020 / ONLINE
For further information, you can write to
speaker@webmarketingfestival.it
www.ilfestival.it