Web analytics is a big part of search marketing and therefore, it’s essential to have a good understanding of how to interpret and convey numbers. This session will walk through the most valuable statistical concepts to anyone working with PPC.

1. Statistical Concepts Every PPC
Specialist Should Know
Frederik Hyldig
Head of PPC | s360 | @frederikhyldig

2. Who am I?
Frederik Hyldig
Frederik Hyldig
Head of PPC at s360
- one of the leading search agencies in Denmark
Works with some of the largest advertisers in
Denmark and Scandinavia
Writes for PPC Hero, Wordstream, Moz,
SEMrush, 3Q Digital, and the Certified Knowledge blog

3. SEO & PPC in
all the Nordic
countries
www.s360.dk

4. Some of the companies we work with
Frederik Hyldig

5. In this presentation:
- Statistics: a sexy skill
- The Pareto principle
- The law of large numbers
- Relative and absolute numbers
- Simpson’s paradox
- How (not) to lie with statistics
Frederik Hyldig

6. A sexy skill
Frederik Hyldig

7. Frederik Hyldig
At least according to this guy...

8. Frederik Hyldig
I keep saying the sexy job in the next ten years will be
statisticians. People think I’m joking, but who would’ve
guessed that computer engineers would’ve been the sexy
job of the 1990s? The ability to take data—to be able to
understand it, to process it, to extract value from it, to
visualize it, to communicate it—that’s going to be a
hugely important skill in the next decades [...]
- Hal Varian, Chief Economist at Google

9. The ability to take data
- to be able to understand it
- to process it
- to extract value from it
- to visualize it
- to communicate it
Frederik Hyldig
Reflected in the quality of
your work
Reflected in the relationships
with clients (or bosses)

10. Frederik Hyldig
Data isn’t going anywhere. Our top skill category last year,
statistical analysis and data mining, is still sitting
comfortably at #2.
It is the only skill category that is consistently ranked in
the top 4 across all of the countries we analyzed.
We still live in an increasingly data-driven world, and
businesses are still aggressively hiring experts in data
storage, retrieval and analysis.
-The 25 Skills That Can Get You Hired in 2016
https://blog.linkedin.com/2016/01/12/the-25-skills-that-can-get-you-hired-in-2016

11. Frederik Hyldig
Meaning businesses will be like...

12. Statistical Concepts Every PPC
Specialist Should Know
Frederik Hyldig

13. The Pareto principle
Frederik Hyldig
Concept 1

14. The Pareto principle Frederik Hyldig
For many events, roughly 80% of the results
come from 20% of the effort.

15. The Pareto principle Frederik Hyldig
Vilfredo Pareto
Italian economist
Also known as the 80/20 rule

16. The Pareto principle Frederik Hyldig
stickfigureeconomics.blogspot.com/2012/06/understanding-pareto-principle-aka-8020.html

17. The Pareto principle Frederik Hyldig
stickfigureeconomics.blogspot.com/2012/06/understanding-pareto-principle-aka-8020.html

18. The Pareto principle Frederik Hyldig

19. How is this relevant in PPC?
Frederik Hyldig

20. The Pareto principle Frederik Hyldig
Focus on the few most important campaigns, ad groups, or keywords.
● Improving ads
● Restructuring accounts
● Negative keyword
● Etc.

21. The Pareto principle Frederik Hyldig
Example:
477 campaigns
367 conversions
According to the Pareto principle:
293 conversions
(80% of the 367 conversions)
should come from
95 campaigns
(20% of 477 total campaigns)

22. Frederik Hyldig
92% of the conversions (338) came
from 20% of the campaigns
The Pareto principle

23. The Pareto principle
Frederik Hyldig
Concept:
Take away:
A very large part of the results will often come from a small part of the activities.
Prioritize your work on the few keywords/campaigns with the largest impact.
For many events, roughly 80% of the results come from 20% of the effort.

24. The law of large numbers
Frederik Hyldig
Concept 2

25. The law of large numbers Frederik Hyldig
If you repeat a random experiment often
enough, the average of the outcomes will
converge towards the expected value.

26. The law of large numbers Frederik Hyldig
Coin tossing example: 10 tosses

27. The law of large numbers Frederik Hyldig
Coin tossing example: 1,000 tosses

28. The law of large numbers Frederik Hyldig

29. The law of large numbers Frederik Hyldig
>

30. Frederik Hyldig

31. Frederik Hyldig
If the statisticians who reported to the Gates Foundation
had asked about the characteristics of the worst schools,
they would have found that bad schools also tend to be
smaller than average. The truth is that small schools
are not better on average; they are simply more
variable.
- Daniel Kahneman

32. How is this relevant in PPC?
Frederik Hyldig

33. The law of large numbers Frederik Hyldig
Sorted by highest
conversion rate
These are all very
small cities

34. The law of large numbers Frederik Hyldig
Sorted by lowest
conversion rate
These are also
small cities

35. The law of large numbers Frederik Hyldig
Seems like we have our most valuable hours at night!

36. Frederik Hyldig

37. Frederik Hyldig

38. The law of large numbers Frederik Hyldig
No.... It’s just more variable due to low volume

39. Frederik Hyldig
ROAS
#1 #2 #3 #4 #5 #6 #7 #8
Look at the return in country #1!
Let’s allocate more of our spend to this account!
Countries

40. Frederik Hyldig
Ad spend
#1 #2 #3 #4 #5 #6 #7 #8
Not so fast!
The volume is much lower than the other countries and
therefore we should expect more variance.
Countries

41. The law of large numbers
Frederik Hyldig
Concept:
Take away:
When randomness is involved (i.e: always) you should expect more
variance from low volume.
Put more trust in numbers when volume is higher.
If you repeat a random experiment often enough, the average of the outcomes will
converge towards the expected value.

42. Relative and absolute numbers
Frederik Hyldig
Concept 3

43. Relative and absolute numbers
A new drug reduces the risk of getting a
dangerous disease by 25%
Frederik Hyldig

44. Relative and absolute numbers Frederik Hyldig
No drug: 20 in 1,000 get disease (2.0% risk)
With drug: 15 in 1,000 get disease (1.5% risk)
25 percent fewer people get the disease
Actual risk of getting disease is only 0.5 percentage points lower
What you will read in the
newspapers
What you really want to
know when deciding

45. How is this relevant in PPC?
Frederik Hyldig

46. Relative and absolute numbers Frederik Hyldig

47. Relative and absolute numbers Frederik Hyldig

48. Relative and absolute numbers
Frederik Hyldig
Concept:
Take away:
Percentage must not be confused with percentage points.
Percentages can be misleading when the absolute numbers are omitted.

49. Simpson's paradox
Frederik Hyldig
Concept 4

50. Simpson’s paradox Frederik Hyldig
A trend appears in different groups of data
but disappears or reverses when these
groups are combined.

51. Simpson's paradox Frederik Hyldig
1973 admission figures for the University of California, Berkeley

52. Simpson's paradox Frederik Hyldig
1973 admission figures for the University of California, Berkeley - by departments

53. How is this relevant in PPC?
Frederik Hyldig

54. Simpson’s paradox Frederik Hyldig
Variant 1: Low prices on XX Variant 2: Save 20% on XX
Ad group 1
Ad group 2
Ad group 3
Ad group 4

55. Simpson’s paradox Frederik Hyldig
Ad 2 is winning in every ad group
But when aggregating the
numbers, ad 2 is actually worse!

56. Simpson’s paradox Frederik Hyldig
Ad 1 volume is concentrated in the two
ad groups with generally higher CTRs.

57. Simpson’s paradox Frederik Hyldig

58. Simpson’s paradox Frederik Hyldig
Even though ads are shown
evenly, a group with high volume
can still cause Simpson’s paradox

59. Simpson’s paradox
Frederik Hyldig
Concept:
Take away:
Beware when aggregating data.
Make sure to run properly randomized trials (rotating ads evenly).

60. How to lie with statistics
Frederik Hyldig
Concept 5
not

61. How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs

62. How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs

63. How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs

64. How (not) to lie with statistics Frederik Hyldig
Graph is from “How to Lie With Statistics” by Darrell Huff
Manipulating graphs
Same numbers. Different impressions.

65. How (not) to lie with statistics Frederik Hyldig
Manipulating graphs
Pay attention to the vertical axis in AdWords
- especially when looking at shorter intervals.

66. How (not) to lie with statistics Frederik Hyldig
Manipulating graphs
A graph should start a zero.

67. How (not) to lie with statistics
An increase by 20% followed by a decrease by 20% does not bring you back
to the starting value.
● 100 + 20% is 120
● 20% of 120 is 24
● 120 - 20% is 96
Frederik Hyldig
Shifting baseline

68. How (not) to lie with statistics Frederik Hyldig
Shifting baseline

69. How not to lie with statistics
Frederik Hyldig
Concept:
Take away:
Pay attention to the vertical axis when looking at (or creating) graphs.
Be aware of shifting bases when increasing and decreasing.

70. Summary
- The Pareto principle
- The law of large numbers
- Relative and absolute numbers
- Simpson’s paradox
- How (not) to lie with statistics
Frederik Hyldig

71. Thanks for listening!
Frederik Hyldig
Head of PPC
E-mail: frederik@s360.dk
Twitter: @frederikhyldig
s360
SEO & PPC across
all the nordic countries
Web: www.s360.dk