-
1.
Driving Online Donations:
A Real-World Test
March 2012
-
2.
During pledge week,
every boy’s and girl’s fancy turns to one thing
2
-
3.
How do we best use our websites
to increase awareness and conversion?
3
-
4.
4
-
5.
5
-
6.
6
-
7.
The big questions
What online ad format is most effective at driving
increased conversion rate and pledge amount?
Do the intrusive ad formats negatively affect user
engagement (e.g., bounce rate)?
7
-
8.
The test
Partnered with Michigan Radio
A/B test splitting traffic among 3 test formats and a
control
Same messaging and creative used for all versions
Deployed for pledge week (October 2011)
23k total home page visits during the test
8
-
9.
Test setup
We split the traffic (using Core Publisher, our Drupal-
based CMS)
• Randomized visitors and assigned a visitor-level cookie for the
duration of the test
We used DFP (DoubleClick for Publishers) to serve the ads
Sent the test version in the DFP call
In Google Analytics, we used a visit/session-level custom
variable to capture the test version
• This custom variable was used to calculate:
• Conversion rate
• Pledge amount
• Bounce rate
9
-
10.
Control
10
-
11.
Lightbox
11
-
12.
Pencil push-down
12
-
13.
Bottoms up
13
-
14.
Pick the winner?
14
-
15.
The results
Control Lightbox Push-down Bottoms up
Conversion: 6.39% 5.65% 5.50% 5.49%
Avg pledge: $126.52 $138.30 $124.90 $132.83
Revenue: $46,181 $44,394 $39,968 $40,779
Bounce: 37% 37% 37% 36%
None of the options resulted in a statistically significant change in performance
15
-
16.
What does this mean?
Creating intrusive ads for pledge week is essentially a
waste of time, resources, and money
The standard 300x250 is at least as effective as doing
something “special”
Time is probably better spent on more effective ways of
driving donations (e.g., email, mailings, offers)
16
-
17.
Every test leads to more questions
What if we tested different messaging or creative?
Would the results be the same if we didn’t test during
pledge week?
What about the rest of the pledge funnel? For
example, does the presence of a landing page before the
pledge form help or hurt?
17
http://www.flickr.com/photos/68751915@N05/6848823919/
http://www.flickr.com/photos/mcfarlandmo/4086606502/
Custom Variables can be used to track dimensions that don’t exist in GA out of the box.Examples:Page category, Test Version, Membership statusThree Levels of scope with different persistence:Pageview – lasts only for the page viewVisit/session – Lasts for the duration of the session, expires at end of visitVisitor – Last for 24 months in the visitor cookieCode with 4 parameters: slot (1-5), name, value, scope (2=visit)_gaq.push('_setCustomVar',1,’PledgeTest',’AdVersion1', 2);Only Visitor Level custom variables are built in to standard reportsTo view page or visit level you will need to build a custom report
http://www.flickr.com/photos/kheelcenter/5277921657/
Conversion is on the visit