Setting Up Useful Tests
The How, When, Where and Why
• Move the needle based on program goals
• Build on what’s working, change what isn’t
• Stay one step ahead of your peers
• Testing is FUNdamental!
How To Test?
• Define your goal BEFORE you test
• Test only one thing at a time
• Split data proportionally and randomly
• Ensure there is enough quantity (and gifts)
• Wait long enough for a conclusion
When and Where to Test?
• As often as possible!
• Try and test at least one variable in every campaign
• Tests aren’t limited to just Package A vs. Package B
• Creative/Design elements
• Donor Segmentation
• Prospect Lists (80/20 rule)
Tests That are Making a Difference
Plain vs. Color OE
Plain white envelope raised 16% more $$!
No Photo vs. Photo
Card with no photo more than doubled response and tripled revenue
One vs. Two Stamps
Two stamps increased revenue
by over 30% with minimal cost
Credit Card Security Codes
15% fewer gifts – and
thousands of dollars in
lost revenue – when
asking for security code
Address Labels Still Work!
Including labels renewed 35%
more donors with only a $2 dip
in average gift
Know Your Donors
Referencing donors past giving raised 20% more money!
Getting on a First Name Basis
Added personalization increased response rate and average gift, with
23% more revenue raised
Reminding Them to Renew
Saying “renew” to donors who hadn’t made a gift yet resulted in 341
more gifts (32%) and nearly $12,000 more revenue (37%)
Timing is of the Essence
Following up the
November Appeal with a
year-end ask in December
raised twice as much than
waiting until January for
Pushing for Monthly Giving
Dual Ask Test raised as much
money as the 1x Ask Control
– but generated triple the
number of monthly donors!
Average gift increased by
$5 – resulting in 20% more
Paper or Plastic?
The simple perf out card was a clear winner in all segments – achieving
a higher response rate and significantly lower costs!
Get Donors Engaged
Digital testing let’s you implement winning strategies right
away so you can maximize your results!
• Test with a segment of file and then roll out with the rest!
• If you don’t have enough donors to test a segment, then
resend « winning » campaign to people who didn’t open it, or
those who received « losing » email.
• Or, use your test to help inform future campaigns!
Digital donors are fickle!
• Test, test and retest!
• Tests often have short shelf life online
• Be careful about overusing winning tests because their
impact might wear off
• Pay attention to your inbox and spot trends
• Make sure you are using the right metric to determine a
winner (i.e. if something boosts open rates but lowers
response, it may not be the way to go)
Ask Language Matters!
RENEW vs. GIVE
Segmenting email data can really pay off. Using “renew” versus “give”
with the appropriate segments will almost always raise more.
Test Subject Lines
PERSONAL vs. HYPER PERSONAL
Personalizing the subject line with the donor’s name
increased response and revenue. But, when we added
in their MRC, it was “too personal.”
“Renew Rachel” “Renew for $50 Rachel”
Won for prospects – increasing average gift by $3.04. However, it lost for
donors – lowering page completion rate by 8%.
• Other things to test online:
• Location of donation button – both in email and on website.
• Color of donation button
• Personalization in subject line and throughout email.
• Number of ask options on donation page
• Conversational subject lines (like “Following Up” or “Thought you’d
• Landing page elements (header, image, ask string)
• Make people feel special
• Keep it simple!
• Tried and true techniques
• Not just mail …
Vicki Viera, Account Director – firstname.lastname@example.org
Other example: August—tested What are you waiting for? Vs. What are you waiting for, Name? Name won significantly at the 99.8% level Example showing here—9.23% open vs. 8.08% open