1.
Driving Online Donations:
A Real-World Test
March 2012

2.
During pledge week,
every boy’s and girl’s fancy turns to one thing
2

3.
How do we best use our websites
to increase awareness and conversion?
3

4.
4

5.
5

6.
6

7.
The big questions
 What online ad format is most effective at driving
increased conversion rate and pledge amount?
 Do the intrusive ad formats negatively affect user
engagement (e.g., bounce rate)?
7

8.
The test
 Partnered with Michigan Radio
 A/B test splitting traffic among 3 test formats and a
control
 Same messaging and creative used for all versions
 Deployed for pledge week (October 2011)
 23k total home page visits during the test
8

9.
Test setup
 We split the traffic (using Core Publisher, our Drupal-
based CMS)
• Randomized visitors and assigned a visitor-level cookie for the
duration of the test
 We used DFP (DoubleClick for Publishers) to serve the ads
 Sent the test version in the DFP call
 In Google Analytics, we used a visit/session-level custom
variable to capture the test version
• This custom variable was used to calculate:
• Conversion rate
• Pledge amount
• Bounce rate
9

10.
Control
10

11.
Lightbox
11

12.
Pencil push-down
12

13.
Bottoms up
13

14.
Pick the winner?
14

15.
The results
Control Lightbox Push-down Bottoms up
Conversion: 6.39% 5.65% 5.50% 5.49%
Avg pledge: $126.52 $138.30 $124.90 $132.83
Revenue: $46,181 $44,394 $39,968 $40,779
Bounce: 37% 37% 37% 36%
None of the options resulted in a statistically significant change in performance
15

16.
What does this mean?
 Creating intrusive ads for pledge week is essentially a
waste of time, resources, and money
 The standard 300x250 is at least as effective as doing
something “special”
 Time is probably better spent on more effective ways of
driving donations (e.g., email, mailings, offers)
16

17.
Every test leads to more questions
 What if we tested different messaging or creative?
 Would the results be the same if we didn’t test during
pledge week?
 What about the rest of the pledge funnel? For
example, does the presence of a landing page before the
pledge form help or hurt?
17