With the recent launch of the Split Testing and Lift APIs, applying the scientific method to your marketing is now a reality.

1. Measurement
is Crucial

2. 74
“Half the money I spend on
advertising is wasted; the trouble
is I don’t know which half”
John Wanamaker

3. Users are exposed to ads through multiple channels
75

4. Correlation doesn't imply causation
Source: http://www.tylervigen.com/spurious-correlations
400
500
600
700
800
9
9.5
10
10.5
11
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
Civil engineering doctorates
awarded
Per capita consumption
of mozzarella cheese
76

5. Testing Facebook Ads
Using marketing API
Split testingConversion lift measurement
Test
Control
Ad impressions
No Ad impressions
Test
Control
Facebook population0 100%
0 50%Split #1 50% 100%Split #2
Test group #1
(Campaign IDs or ad set IDs)
Test group #2
(Other campaign IDs or ad set IDs)
77

6. Test
Conversion Lift and the scientific method
Control
78

7. Conversion lift
AD IMPRESSION
NO AD IMPRESSION
TEST
CONTROL
TEST
CONTROL
79

8. Defining an experiment
CAMPAIGN A NO CAMPAIGNVERSUS
Single cell study
Multi cell study
CAMPAIGN A NO CAMPAIGNVERSUS
CAMPAIGN B NO CAMPAIGNVERSUS
CAMPAIGN A
+
CAMPAIGN B
NO CAMPAIGNVERSUS

9. Study duration and cooldown periods
Pre-study
quiet period
Post-campaign
quiet period
Campaign
duration
APRIL 15 ‘17 MAY 1 ‘17 MAY 8 ‘17TODAY
MEASUREMENT PERIOD
81

10. How is percentage lift calculated?
TEST CONTROL
3k conversions (exposed)
2k conversions (unexposed)
4k conversions (scaled)
82

11. Formula
reached conv. - incremental conv.
(3000) (1000)
incremental conv.
(1000)
50%=
percentage lift
83

12. 84
How do I get started?

13. 85
Set up conversion lift study
SET UP A LIFT STUDY WITH ONE TEST GROUP
curl
-F 'name="new study"'
-F 'description="description of my study"'
-F 'start_time=1435622400'
-F 'end_time=1436918400'
-F 'cooldown_start_time=1433116800'
-F 'observation_end_time=1438300800'
-F 'viewers=[<USER_ID1>, <USER_ID2>]'
-F 'type=LIFT'
-F 'cells=[{name:"test group",description:"description of my test group",treatment_percentage:89.75,control_percentage:
10.25,adaccounts:[<ACCOUNT_ID1>,<ACCOUNT_ID2>]}]'
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]'
-F 'confidence_level=0.9'
-F 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies

14. 86
Multiple groups
curl
-F 'name="new study"'
-F 'description="description of my study"'
-F 'start_time=1435622400'
-F 'end_time=1436918400'
-F 'cooldown_start_time=1433116800'
-F 'observation_end_time=1438300800'
-F 'viewers=[<USER_ID1>, <USER_ID2>]'
-F 'type=LIFT'
-F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns:
[<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage:
10,campaigns:[<CAMPAIGN_ID2>]}]'
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]'
-F 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
SET UP A LIFT STUDY WITH MULTIPLE TEST GROUPS
-F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns:
[<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage:
10,campaigns:[<CAMPAIGN_ID2>]}]'

15. 87
Lift results API
curl -G
-d 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>?fields=results

16. 88
Lift results sample
{
"id": "<STUDY_OBJECTIVE_ID>",
"results": [{
"population.test": 1349292,
"population.control": 336567,
"polutation.reached": 738219,
"impressions": 3349184,
"spend": 23443.12,
"frequency": 4.5368434028385,
"buyers.test": 212300,
"buyers.control": 50823,
"buyers.scaled": 203748.63642603,
"buyers.incremental": 8551.3635739689,
"buyers.delta": 1536.450929913,
"buyers.pValue": 0,
"buyers.isStatSig": 1,
"buyers.reached": 111199,
"buyers.reachedPercent": 0.52378238341969,
"buyers.baseline": 102647.63642603,
"buyers.lift": 0.08330794426164,
"conversions.test": 1473655,
"conversions.control": 362552,
"conversions.scaled": 1453465.4710177,
"conversions.incremental": 20189.528982342,
"conversions.delta": 29532.640850362,
"conversions.pValue": 0.26076775078954,
"conversions.isStatSig": 0,
"conversions.reached": 735235,
"conversions.reachedPercent": 0.49891935358004,
"conversions.baseline": 715045.47101766,
"conversions.lift": 0.028235307824002,
"incrementalROAS": -0.1998564046528,
"sales.test": 2112564.35,
"sales.control": 528126.12,
"sales.scaled": 2117249.607677,
"sales.incremental": -4685.2576770443,
"sales.delta": 113045.85347285,
"sales.pValue": 0.94564373701777,
"sales.isStateSig": 0,
"sales.reached": 1103736.31,
"sales.baseline": 1108421.567677,
"sales.lift": -0.0042269636514411,
}],
}

17. 89
Ad studies breakdowns
curl -G
-d 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>?fields=results&breakdowns=['cell_id']
{
"id": "<STUDY_OBJECTIVE_ID>",
"results": [
{
"cell_id": "<CELL_ID1>",
...
Default fields where the values are specific to the <CELL_ID1> breakdown
...
},
{
"cell_id": "<CELL_ID2>",
...
Default fields where the values are specific to the <CELL_ID2> breakdown
...
}],
}

18. • Can run multiple studies at
once
• Use findings to apply
adjustments to attribution
figures
• Retesting period depends
on purchase consideration
cycle length
90

19. 91
How do I get better
results from my
Facebook campaigns?

20. One of these has 3x the conversions?
Same CTR
92

21. Group A
A/B test
Group B
93

22. Ad studies and cells
Split testing provides an easy way to run experiments on how different factors affect the performance of
Facebook ads
Ad formats Landing pages Creatives Bid types Targeting
9494

23. Defining an experiment
HEADLINE A HEADLINE BVERSUS
Single cell A/B test
FACEBOOK INSTAGRAMVERSUS VERSUS
FACEBOOK
+
INSTAGRAM
Multi cell A/B test
95

24. One of these has 3x the conversions?
Same CTR
96

25. 97
Ad studies and cells
Creation example
curl
-F 'name="Grumpy Faces Study"'
-F 'description="Will test which planet is best"'
-F 'start_time=1435622400'
-F 'start_time=1436918400'
-F 'type=SPLIT_TEST'
-F 'cells=[
{name: “Blue Planet", treatment_percentage: 50, adsets: [<AD_SET_ID>]},
{name: “Red Planet", treatment_percentage: 50, adsets: [<AD_SET_ID>]}
]'
-F 'access_token=<ACCESS_TOKEN>'
https://graph.facebook.com/v2.7/<BUSINESS_ID>/ad_studies

26. 98
Key takeaways
• Test using the scientific method
• Conversion Lift to prove FB value
• Split testing for optimisation
within FB