This document provides tips and guidance on testing in direct marketing. It emphasizes that testing allows marketers to know what works rather than guessing. It outlines various elements that can be tested, from email subject lines and envelopes to prices and incentives. The benefits of testing include making businesses safer by avoiding failures and allowing prediction of results. A step-by-step process for running tests is described, involving selecting groups, promotions, comparing and monitoring results. Repeating tests is advised to gain reliable insights. Various online and offline testing examples are also included.

1. Know, Don’t Guess
Tests - How to Run & Analyze Them?
Direct Marketing tips by Mediapost Hit Mail
#5

2. Why tests are SO important?
The answer is simple. It’s all about knowledge. In direct marketing you
know what works and what doesn’t, so you don’t have to guess, nor to
rely on someone else’s opinion.
There are two major activities that
bring successful results in DM:
testing and building a continuing
relationship with your customers. By
testing you will be sure you are not
losing money (which, in fact, is what
you have to worry first of all).

3. What could be tested in DM?
In fact, almost everything:
 Sender/ Subject line for an e-mail campaign;
 New outer envelope of a mailing;
 New brochure for your product;
 Different piece of copy in the promotion;
 Various order options;
 Different benefits for the customers;
 Different prices of your product/ service;
 The best time to call, send an e-mail or questionnaire;
 The product you will be offering;
 Feedback incentives etc.
Almost any question you have in direct marketing can be
answered cheaply and quickly by a test.

4. Testing benefits
 Failure proof: Tests can protect you from
mistakes and thus make your business
safer. You can hardly fail.
 When it’s over: you can predict your
business results with quite high level of
confidence.
 Tests save a lot of money: if it doesn’t work,
you don’t use it, but you are safe in terms
of profitability, because you didn’t spend a
lot; if it works, you start using it, and you
even try to make it better, thus getting
optimal results.
 Control: You have a full control on your
business, because you know exactly what
are you doing and what is too risky for you.

5. Step by step: –
how to run tests
1. Choose a variable to test. It could be price, insert, sender/subject e-mail
combination, benefit, brochure, approach etc.
2. Select two similar groups from your database (A and B); A is your control
group, B is the test group; in terms of quantity A=B;
3. Define which promotion you will use for A (an existing and working one)
and B (the new one for the test);
4. Send the control promotion to group A and the test to group B at the same
time within a single campaign;
5. When results come, compare A to B to define the initial winner;
6. Keep monitoring the results after that to define the final winner;
7. When you know the winning promotion, re-test it at least once.
8. After that you will know what works better for you.

6. Repetition is crucial:
Don’t rely on results
from a single test.
Repeat tests at least
once, preferably twice.
Direct marketing is a
business for
cautious people.

7. Some on- and offline test examples
 A/B split: half of the mailing has one piece of copy and the other one a
different one. This is very close to statistical perfection. A minus is that you
can only test two to four things at a time.
 Split-run inserts: thus you can test different media and find which will work
best for you. Or you can test different creative. Flexibility is a plus here. A
drawback is that initial printing costs are usually very high.
 Split-run mailings: here you can test new databases before renting them,
new approaches before going for them, and different segments before
choosing which offer/ product to send to them.
 Sender/ Subject test (on an e-mail campaign): it could increase your open
rate dramatically if properly used. A single word added to the subject line
could literally double your results. And it’s amazingly cheap.

8. When reading the results:
 Be careful - initial results may mislead you. Monitor your test
results with utmost attention on the value of the sales you make
and the customers you attract.
 Every media has its own response pattern. The phone, obviously,
gets instant response. E-mails are much faster than direct mail.
Don’t rely on the very initial results; be patient and wait as long
as it is needed.
 The least reliable result is a low response to a small mail quantity.
Tests involving small numbers and modest rates are a complete
waste. Make your test group large enough in order to get reliable
results.

9. S IX
Our five testing tips:
1. Always test new databases with proven existing promotional materials.
2. Every time you test a new approach, make sure you have your best
available offer against it. This is your “control group”.
3. If you want to test higher intensity impact have a control group that you
mail as usual and test group that gets mail more frequently.
4. Don’t test two variables simultaneously. For example two different
benefits and two different prices. It is extremely important. If you do so,
you will never be sure which one made the real difference in results.
5. The cheapest way to test is to use an already existing regular
communication (a monthly statement for example) with your customers.
In this case you can use even some existing printed inserts with no
additional postage costs.
6. Bonus tip: Only test a factor that really matters for you.

10. A point to remember
If you are testing carefully, you
don’t have to worry at all about
creative flair.
You will know. You won’t guess.
Guesswork brings hidden
surprises, as it is shown here:
And your customers will tell you
what works by voting with their
own money, at no additional
costs for you.

11. More to come:
1. Why Do You Need Your Own Database?
2. How to Build a Database?
3. How to Manage Your Database?
4. How to Segment Your Customers?
5. Tests - How to Run and Analyze Them?
6. The Deliverability Problem
7. Personalization
8. Online & SMS Promotions
9. Useful Direct Marketing Tips
10. How to Launch Your CRM Program?
11. The Advantages of Direct Marketing (Summary)

12. two more slides...

13. Our contacts:
9 Sveti Sedmochislenitsi Str.,
Sofia 1421, Bulgaria
Phone: (+359) 2 962 86 29
Fax: (+359) 2 962 86 29
Mobile: (+359) 885 360 715
E-mail: office@mediaposthitmail.bg
Web: www.MediapostHitMail.bg
just one more...

14. Hello,
It’s time to become more practical with the fifth presentation in
the “Direct Marketing Tips” set. I hope it is useful and you liked it.
Please, share your opinion on it with us via e-mail at
office@mediaposthitmail.bg,
or call us (+359 885 360 715). In case you’ve missed the previous
ones, this way you will get all the presentations in our set as soon
as they are available.
Thanks!
Hristo Radichev
Country Manager, Mediapost Hit Mail
P.S. Please, send this file to a friend or colleague who you think
would like these materials, or send us their contacts.
They'll get a polite invitation from us (which they can decline) and
we can help them build their own direct marketing strategy.