This document discusses various techniques for evaluating integrated marketing programs and communication pieces at different stages of development, including concept testing, copytesting, recall tests, recognition tests, attitude/opinion tests, emotional reaction tests, physiological arousal tests, persuasion analysis, behavioral measures, sales/response rates, test markets, and purchase simulation tests. It also covers evaluating public relations efforts and the overall IMC program, as well as challenges in measuring ROI and international implications.

1. 15
Prof. (Dr.) Kao Kveng Hong, PhD. , D. Litt.
Chapter Fifteen
Evaluating an Integrated
Marketing Program

2. Pretesting for Effectiveness
• Rocket analogy
• Decision Analyst
• http://www.decisionanalyst.com
• CopyScreen
• CopyCheck
• What are the pros and cons of
testing ads and marketing
communication pieces at various
stages of development?
15

3. Chapter Overview
• Matching methods with objectives
• Message evaluations
• Evaluation criteria
• Behavioral evaluations
• Evaluating public relations
• Evaluating the IMC program
Evaluating an Integrated
Marketing Program
15

4. Evaluation Categories
• Message evaluation techniques
• Physical design
• Cognitive elements
• Affective elements
• Respondent behavior evaluations
• Conative elements
• Measurable with numbers
• Customer actions

5. Evaluation and IMC Objectives
• Match objectives
• Pre- and posttest analysis
• Levels of analyses
• Short-term
• Long-term
• Product-specific
• Corporate level
• Affective, cognitive, or conative

6. F I G U R E 1 5 . 1
Message Evaluation Techniques
and When to Use Them
• Concept testing
• Copytesting
• Recall tests
• Recognition tests
• Attitude and opinion tests
• Emotional reaction tests
• Physiological tests
• Persuasion analysis
Prior to ad development
Final stages, or finished ad
Primarily after ad has been launched
After ad has been launched
Anytime during or after ad development
Anytime during or after ad development
Anytime during or after ad development
After ad has been launched
Message Evaluation Method When the Test Is Normally Used

7. Concept Testing
• Prior to ad development
• Average cost 30-second TV ad $350,000
• Focus groups
• Concept testing instruments
• Comprehension tests
• Reaction tests

8. AFLAC
Concept Testing
Before launching theAFLAC duck advertising campaign, the agency
conducted concept tests to determine which idea was the best.
Click here to play clip
fromAFLAC

9. Copytesting
• Used when marketing piece is finished
or in final stages
• Methods used
• Portfolio test
• Theater test
• Focus groups
• Mall intercept

10. Copytesting
Copytesting can be
used to determine if
viewers comprehend
this ad and what their
reaction to it is.

11. Copytesting
• Criticisms of copytesting
• Some agencies not using
• Stifles creativity
• Focus groups not good judge
• Support of copytesting
• Issue of accountability
• Majority support because clients want
support for ad decision
Copyright © 2010 by Pearson Education, Inc. publishing as Prentice Hall

12. Recall
Tests
• Day-after recall (DAR)
• Unaided recall
• Aided recall
• Incorrect answers
• Use primarily after ads launched

13. Items Tested for Recall
• Product name or brand
• Firm name
• Company location
• Theme music
• Spokesperson
• Tagline
• Incentive being offered
• Product attributes
• Primary selling point of communication piece
F I G U R E 1 5 . 3

14. Recall Tests: Do Viewers Remember?

15. 25.0%
20.0%
15.0%
10.0%
5.0%
0.0%
Brand name Theme music Spokesperson Tagline Incentive Product
Attribute
Test Ad CompetitorA Competitor B
Sample DAR Test
30-Second TV Advertisement for Pet Food

16. Sample DAR Test
30-Second TV Advertisement for Pet Food
9.4%
16.3%
18.3%
24.6%
8.5%
14.6%
21.4%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
Males Females Pet Dog Ages 18- Ages 36- Ages 51+
Overall Recall 12.9%
Owners Owners 35 50

17. Recall Decay
Magazine Ad vs. Television Ad
Source: Magazines Canada’s Research Archive
100% 100%
86%
65%
75%
43%
40%
20%
0%
60%
80%
100%
DAR Two days later
Magazine Television
Eight days later

18. Recall Tests
Factors That Influence Scores
• Attitude towards advertising
• Prominence of brand name
• Brand used by respondent
• Institutional ads
• Respondent’s age

19. Recognition Tests
• Respondents shown marketing piece
• Often used with recall tests
• Good for measuring
• Reaction
• Comprehension
• Likeability

20. Recognition Tests
• Expression of person’s interest
• Ad liked  + 75%
• Ad interesting  + 50%
• Brand used  + 50%
• Affected by ad size, color, length
• Scores do not decline over time

21. Recognition Tests
Can be used to measure
• Reaction
• Comprehension
• Likeability

22. Attitude and Opinion Tests
• Used in conjunction with other tests
• Recall tests
• Recognition tests
• Closed-ended questions
• Open-ended questions
• Roper Start  ADD + IMPACT

23. Emotional Reaction Tests
• Affective advertisements
• Used for material designed to solicit emotions
• Difficult to measure emotions with questions
• Warmth monitor
• Emotional reaction tests are self-reported
instruments

24. Sample Graph from a Warmth Meter
30-Second TV Advertisement
S am p le A d R atin g
W arm th Meter
S tart 1 0 s e c on ds 2 0 s e c on ds 3 0 s e c on ds
Ad section that elicited negative emotions
Target Audien ce
Total Audience

25. Physiological Arousal Tests
• Measure fluctuations in a person’s body
• Psychogalvanometer – sweat
• Pupillometric test – pupils of eyes
• Psychophysiology – brain waves and
currents
• Cannot be faked easily

26. Persuasion Analysis
• Appraise persuasiveness of marketing item
• Requires pre- and posttests
• ASI Market Research studies

27. Copytesting Principles of PACT
• Testing procedure should be relevant to objectives.
• Researchers should agree on how the results will be used in
advance.
• Multiple measures should be used.
• The test should be based on some model or theory of human
response to communication.
• Testing procedure should allow for more than one exposure.
• In selecting alternate ads to include in the test, they should be at
the same stage in the process as the test ad.
• The test should provide controls to avoid biases.
• Sample used for the test should be representative of the target
sample.
• Testing procedure should demonstrate reliability and validity.
Source: Based on PACT document published in the Journal of Marketing, (1982) ,Vol. 11, No. 4, pp. 4-29.
F I G U R E 1 5 . 6

28. Behavioral Measures
• Sales
• Response rates
• Redemption rates
• Test markets
• Purchase simulation tests
F I G U R E 1 5 . 7

29. Sales and Response Rates
• Measuring sales with UPC codes
• Scanner data
• Retailers
• Manufacturers
• Sales changes can be caused by other factors

30. Difficulties in Evaluating
Advertising
•
•
•
•
•
Influence of other factors on behavior
Delayed impact of advertising
Consumers change their mind in the store
Whether brand is in evoked set
Goal of ad may be to build brand equity,
not increase sales

31. • Changes in sales
• Telephone inquiries
• Response cards
• Internet inquiries
• Direct marketing responses
• Redemption rate of sales promotion offers
• Coupons, premiums, contests, sweepstakes
F I G U R E 1 5 . 9
Responses to Marketing Messages That Can Be Tracked

32. F I G U R E 1 5 . 10
Methods of Measuring Interactive Marketing
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
Source: Adapted from Larry Jaffee, “Follow the Money,” Promo, Vol. 20, No. 11 (November 2007 Sourcebook), pp. 5-10.
51.0%
36.5% 44.5%
41.1% 40.7%
34.2%
26.6% 24.7%
12.2% 16.3%
4.6%

33. Online Metrics
• Adknowledge
• MarketMatch Planner
• Campaign Manager
• Administrative Campaign Manager
• Audience demographics
• MediaMetrix – basic demographics
• NetRatings – GRP and other rating instruments
• SRI Consulting – Psychographic information
• NetGuide – Web site ratings and descriptives
• BPA Interactive – Web traffic audit data

34. Test Markets
• Used to assess:
• Advertisements
• Consumer and trade promotions
• Pricing tactics
• New products
• Cost-effective method of evaluation prior to launch
• Resembles actual situation
• Design test market to model full marketing plan
• Length of test market is a concern
• Competitive actions must be considered

35. Purchase Simulation Tests
• Bias in purchase intention questions
• Simulated purchase tests
• Research Systems Corporation
• Does not rely on opinions and attitudes

36. Evaluating Public Relations
• Number of clippings
• Number of impressions
• Advertising equivalence technique
• Comparison to public relations objectives
F I G U R E 1 5 . 11

37. Evaluating the IMC Program
• Greater demand for accountability
• ROI of advertising and marketing
• Difficult to measure ROI – 70%
• Difficult to define ROI – 70%

38. Source: Paul J. Cough, “Study: Marketers Struggle to Measure Effectiveness,” Shoot, Vol. 45, No. 29
(August 20, 2004), pp. 7-8.
TA B L E 1 5 . 2
Definitions of ROI for Marketing
Definition of ROI Percent Using
Incremental sales from marketing 66%
Changes in brand awareness 57%
Total sales revenue from marketing 55%
Changes in purchase intentions 55%
Changes in market share 49%
Ratio of advertising costs to sales 34%
Reach/frequency achieved 30%
Gross rating points delivered 25%
Post-buy analysis comparing the media plan to its delivery 21%

39. Measures of Overall Health of a Company
• Market share
• Level of innovation
• Productivity
• Physical and financial resources
• Profitability
• Manager performance and development
• Employee performance and attitudes
• Social responsibility
Source: Pete Drucker, Management: Tools, Responsibilities, Practices, New York: Harper and Row, 1974.
F I G U R E 1 5 . 12

40. International Implications
• Assessment of IMC Programs
• Domestic results
• Results in other countries
• Overall organization
• Individual ads and promotional programs
• Local culture
• Across national boundaries
• Multinational – regional offices