1.
Driving Online Donations:A Real-World TestMarch 2012

2.
During pledge week,every boy’s and girl’s fancy turns to one thing 2

3.
How do we best use our websitesto increase awareness and conversion? 3

4.
4

5.
5

6.
6

7.
The big questions What online ad format is most effective at driving increased conversion rate and pledge amount? Do the intrusive ad formats negatively affect user engagement (e.g., bounce rate)? 7

8.
The test Partnered with Michigan Radio A/B test splitting traffic among 3 test formats and a control Same messaging and creative used for all versions Deployed for pledge week (October 2011) 23k total home page visits during the test 8

9.
Test setup We split the traffic (using Core Publisher, our Drupal- based CMS) • Randomized visitors and assigned a visitor-level cookie for the duration of the test We used DFP (DoubleClick for Publishers) to serve the ads  Sent the test version in the DFP call In Google Analytics, we used a visit/session-level custom variable to capture the test version • This custom variable was used to calculate: • Conversion rate • Pledge amount • Bounce rate 9

10.
Control 10

11.
Lightbox 11

12.
Pencil push-down 12

13.
Bottoms up 13

14.
Pick the winner? 14

15.
The results Control Lightbox Push-down Bottoms upConversion: 6.39% 5.65% 5.50% 5.49%Avg pledge: $126.52 $138.30 $124.90 $132.83Revenue: $46,181 $44,394 $39,968 $40,779Bounce: 37% 37% 37% 36%None of the options resulted in a statistically significant change in performance 15

16.
What does this mean? Creating intrusive ads for pledge week is essentially a waste of time, resources, and money The standard 300x250 is at least as effective as doing something “special” Time is probably better spent on more effective ways of driving donations (e.g., email, mailings, offers) 16

17.
Every test leads to more questions What if we tested different messaging or creative? Would the results be the same if we didn’t test during pledge week? What about the rest of the pledge funnel? For example, does the presence of a landing page before the pledge form help or hurt? 17