In direct marketing & fundraising testing can help us learn, improve, refine and excel. This session explored some of the 'gold nugget' findings from over 12 years of testing in individual giving fundraising across hundreds of campaigns and programs. If you run an individual giving program, use direct mail, telephone fundraising or digital to raise funds there will be some ideas and learnings for you.

1. Unique test database
reveals gold nuggets
Fiona McPhee

3. Why do we test?
Huge database of test
results
No guarantees – nothing
stays the same
Short-term vs. Long-term
World of myths to bust
Testing for over 12 years

4. Keep In Mind
Have a test strategy & hypothesis for each
Does it have a chance of being valid?
Can you do anything with the potential
results?
Have a control group & test group(s)
Have a test budget
A / B
??

5. What is affecting
response
Digital: What has and
hasn’t worked
Channels: Direct Mail,
Digital, Phone
What is affecting
income
Testing Propositions &
Language
What is on the horizon?
% $

6. The biggest things that influence response rates
Asking
Audience
Contact Rates / Open rates
Ability to contact in future

7. Should we ask?
In a direct mail survey test the
inclusion of an ask at the end of
the survey significantly
increased response rate, return
on investment and net income.
Test Segment Mailed # Resp Resp Rate Avg Gift
Gross
Income
Net Income ROI
Ask 4847 285 5.9% $49.55 $14,123 -$2,308 0.86
No Ask 4855 198 4.1% $57.23 $11,331 -$5,127 0.69

8. Should we make specific asks?
In a direct mail appeal test we asked in
the copy and on the response
mechanism against just on the response
mechanism. The ask in copy and on the
response mechanism significantly
increased response rate
Test Segment Mailed # Resp Income Response Rate AvgGift
Ask in Copy & RM 5248 354 $18,818.00 6.75% $ 53.16
Ask on RM Only 5248 299 $17,765.20 5.70% $ 59.42

9. This is a sample text.
Insert your desired text here.
Sample text
The low multiplier
significantly increased
response rate for a direct
mail regular giving
conversion ask
Test Segment Mailed # RG Resp RG Resp Rate
Avg Annual RG
Value
Annual Value
High Ask 2,241 59 2.6% $246.10 $14,520
Low Ask 2,242 95 4.2% $171.66 $16,308
How high should we ask?

10. Audience Testing
Mail: Response highest from
most recent
Phone: Variables such as
channel and pay type also
have significant impact
Warm
Strong responders
There is a cost to
swapping
Charity Givers
Mail: Quality of data
critical
Phone: Your own
leads can increase
response
Tepid
Cheap = cheap
Know what you are
testing
Cold
ResponseRates

11. Direct Mail Phone Online
Face to Face Media Other
0%
25%
50%
75%
100%
0%
25%
50%
75%
100%
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
MailabilityRate
Contact Rates

12. Contact Rates
Charity A: ~40k
RGs
Year 1 Year 2 Year 3
# Campaigns /
Batches
1 2 5
Conversations 6,960 12,828 19,685
Upgrades 2,288 4,840 5,827
Yr 5 Net Income $1,086,000 $2,719,000 $2,907,000
Charity B: ~16k RGs Year 1 Year 2
# Campaigns / Batches 1 2
Conversations 5,174 10,901
Upgrades 2,070 4,197
Yr 5 Net Income $1,141,000 $1,977,000

13. Contact detail capture: Digital Lead Generation
Mandatory
=
Less sign ups
Higher quality
90% to call
Not Mandatory
=
More volume
25% phone opt
out

14. 0
500
1,000
1,500
2,000
2,500
3,000
0%
10%
20%
30%
40%
50%
60%
70%
Addr1 Addr1
Email
Addr1
Email
Landline
Addr1
Email
Landline
Mobile
Addr1
Email
Mobile
Addr1
Landline
Addr1
Mobile
Email Email
Landline
Email
Mobile
2nd Gift Rate (Year 1) Recruits
Base:2009 - 2014 Recruits
Multi-contact opportunity
donors have the highest
second gift rates. No
address sees much lower
conversion.
Address & Mobile
potentially an age thing
Information
provided by
donors can be
increased over
time

15. Information
provided by
donors can be
increased over
time

16. The biggest things that influence income
Asking
Longer is better
Tangible

17. Asking
Test
Segment
Qty Mailed Resp Resp Rate Ave Gift
Net
Income
ROI
Ask 9,831 1,858 19.9% $57.02 $93,610 8.59
No Ask in
Copy nor
RM
4,240 648 15.3% $47.81 $25,641 5.80

18. Is longer better?
That longer appeal
letters will generate a
lower response
8 campaigns – 6 charities
2 significant results for
response
Only resulted in significantly higher income on
one occasion
Test Mailed Resp# RR% AV G Net Return
4pp 9,415 2,554 27.1% $59.03 $124,769 5.80
2pp 9,415 2,287 24.3% $57.44 $106,129 5.21

19. For an individual test though
Likely to be marginal
difference
Any cost of 4pp vs. 2pp
Would have to be less
than that.
4pp Generates more
large gifts & consistently
more income
Gross Income per ‘000 mailed
An average of an extra $1 per
donor mailed
Is longer better?

20. Is Tied better?
That asking for Tied funds
(vs. General funds) will not
increase RR% or Av Gift
Significant result on RR%
Average gift higher for Tied funds but not
enough to outweigh lower RR% across the
campaign
Test Mailed Response RR% AV G
General 17,324 2,826 16.31% $50.06
Tied 17,325 2,661 15.36% $45.62
“If I can not raise $100,000, I will
have to cut X service.”
Vs.
“If I can not raise $100,000, we will
have to consider cutting the X
program and others like it.”

21. More Gold?
World largest single mass of Gold, 630 pounds

22. Testing Propositions
Three propositions vying for supremacy
Two taken to the phones with warm
donors
One significantly out pulled the other half way
through calling
01
02
03
Welcome to Stroke Foundation Life Savers.
Thank you for taking the time to speak with one of
the team recently and thank you so much for making
the wonderful decision to join Life Savers.- helping
us to run a powerful and sustained national
campaign to stop so many people dying needlessly.
Together we can save a life every hour.
Thank you for taking the time to speak with one of
the team recently and thank you so much for making
the wonderful decision to join us in spreading the
message to save lives - helping us to fill in the gaps
in people’s knowledge about stroke.
By choosing to make a monthly donation you are
continuing to change lives and will be helping to
make sure everyone in New Zealand knows how to
reduce the chance of stroke, how to recognise stroke
and what to do if a stroke happens.

23. Testing Language
Does the use of colloquial language increase response rate?
The use of Kiwi (vs. New Zealand) language in variable copy increased
response rates. Average gifts were down for the colloquial test group – did
we put off posher donors?
Less formal language positively impacting response. Finding the right tone in
relation to each appeal / story important and we are looking at
testing/adaptation for the Mid-Value Donor program.
01
02
03
Test Segment
Mailed
Cash Avg
Gift
RG Avg Gift
Comb
ROI
Comb
RR%
Kiwi 26,534 $46 $13 3.04 4.21%
New Zealand 26,454 $57 $14 3.04 3.83%

24. Digital: Direct to Donation
10 years ago it
worked
Early adopters
getting results
Integrating
warm was
getting traction
Asking to get
married before
the first date?

25. Digital: Direct to Donation
Too high a
barrier to
donating as first
interaction
10 years ago it
worked
Now: Wall to Wall
advertising

26. Lifeline
STEP 1 – Drive to Petition

27. Facebook – best performing creative, Step 1 – Petition
1 2 3

28. Lead Acquisition
Signatures: 65,027
Email opt-ins: 31,609
Phone opt-ins: 8,838

29. Some of the things we are testing in the digi space now
• Digital to phone
• Digital to digitalConversion Step
• Petition
• Survey
• Pledge
Lead Sources
• Targeting within mediaMedia
• Link Ad
• Canvas
• Carousels
• Video Slideshows
Format
• Stories
• Images
• Subject lines
Creative

30. Channel Integration
Does the provision of online options increase survey
response?
Including both the launch email and the post email had the most
significant impact on Survey response. The emails had no significant
impact on financial donations to the Survey.
Email integration had a positive impact on the key objective of the appeal.
Continuing to increase email penetration rates and testing email execution
needed.
01
02
03

31. Ask Tests
Level change
against each
other
Fixed string
against
calculated
Number of
asks
Type of asks
Combining
NL/Survey
with an ask
Highlight asks
Reversing
sequence
Other than
ask1 in copy,
no ask in copy
My choice on
RM

32. Having no calculated
asks on RM and copy
suppresses the
response and results in
lower average gift
Numbers of asks in copy
– insignificant outcomes
Adding ask at end of
survey increases
response rate
5 Charities, 6 tests, 1
significant
Number of Asks

33. Ask Testing
Sample text
Highlight asks
Insignificant
Reverse Sequence Ask 2 As Main
Lowers Response but
appears to increase
average gift
Insignificant Significant

34. Direct Mail Pack
Additions Testing
Does inclusion of a premium (Note pad) vs no premium uplift
response rate?
Inclusion of a Notepad Premium uplifted response rate to the appeal –
however due to the higher cost of this pack, ROI is lower
Premiums continue to uplift response, however cost and audience needs to
be carefully balanced to make it a valid option going forward
01
02
03

35. Pack Additions Testing
Significant higher response rate
and net income
Include a Calendar
Significantly lower
average gift
More responded
Significantly higher response rate
and average gift
Include an
Involvement Device
Significantly lower net
income
Sample text
Significantly higher response rate
Post It Notes
Increased net income
Significantly lower response rate &
net income
Adding Newsletter
Higher average gift
with newsletter
More responded More responded Less responded

36. What is on the horizon?
1
2 Splitting incorrectly & not
sticking with it
What is it? How do you
provide it? How can you
test its impact/value?
Measure the income &
retention vs. total cost
over an extended period
of time
3
Stewardship /
Donor Care
Wrong
Right

37. What is on the horizon?
1
2
Are we under emailing?
Under mailing? Under
communicating
What is appropriate?
What is effective?
One channel / more
channels? Or is it all too
much?
3
Frequency of Contact
Too Little?
Too Much?

38. fiona.mcphee@paretofundraising.com