This document discusses various methods for measuring the effectiveness of promotional programs, including reasons to measure effectiveness and reasons measurements may not be taken. It describes testing processes such as concept generation and testing, pretesting of finished ads, and market testing. Various testing methods are outlined, including focus groups, consumer juries, portfolio tests, and physiological measures. The document also covers post-testing of print and broadcast ads, as well as factors that determine the success of tracking studies and problems with current research methods.

1. Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of McGraw-Hill
Education.
Chapter 18
Measuring the
Effectiveness
of the
Promotional
Program

2. Arguments for and Against Measuring
Effectiveness
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
Reasons to measure
effectiveness
• Avoid costly mistakes
• Evaluate alternative strategies
• Increased advertising
efficiency
• Determine if objectives are
achieved
Reasons effectiveness measures
are not taken
• Costs involved
• Research problems
• Disagreement on what to test
• Objections of creative
department
• Lack of time
2

3. What, Where, and How to Test
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
3

4. Testing Process
 Concept generation and testing
 Rough art, copy, and commercial testing
 Pretesting of finished ads
 Market testing of ads
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
4

5. Figure 18.5 - Weaknesses Associated
with Focus Group Research
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
5

6. Types of Rough Art, Copy, and
Commercial Tests
 Comprehension and reaction tests: Assess the
reaction an ad generates to ensure that it is not
offensive
 Consumer juries: Use consumers representative of
the target market to evaluate the probable success
of an ad
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
6

7. Limitations of the Consumer Juries
Method
 Consumer may become a self-appointed expert
 Number of ads that can be evaluated is limited
 Halo effect: Overall rating is influenced by the
judgment on one or few characteristics of the ad
 Preferences for types of advertising may
overshadow objectivity
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
7

8. Figure 18.8 - Gallup & Robinson’s
Impact System
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
8

9. Portfolio Tests
 Expose a group of respondents to a portfolio
consisting of control and test ads
 Limitations
 Factors other than advertising creativity and/or
presentation may affect recall
 Ability to recognize the ad when shown may be a
better measure than recall
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
9

10. Readability Tests
 Communications efficiency of the copy in a print
ad is tested without reader interviews
 Flesch formula: Assesses readability of a copy by
determining the average number of syllables per
100 words
 Limitations
 Copy may become too mechanical
 Direct input from receiver is not available
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
10

11. New Print Pretesting Measures
 PreTesting Groups’ People Reader methodology
 Provides mocked-up magazines to consumers and
measures their responsiveness to each ad
 Uses hidden cameras to record behavior
 Link
 Uses a comprehensive set of diagnostic questions to
evoke viewer reactions to the ads
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
11

12. Pretesting Finished Broadcast Ads
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
12

13. Physiological Measures
 Indicate receiver’s involuntary response to the ad
 Pupil dilation
 Pupillometrics: Measures dilation and constriction of the pupils in
response to stimuli
 Galvanic skin response (GSR)
 Electrodermal response (EDR): Measures the skin’s resistance or
conductance to a small amount of current
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
 Brain Waves
 Electroencephalographic (EEG): Determine electrical frequencies in
brain
 Alpha activity: Degree of brain activation
 Hemispheric lateralization: Distinguishes between alpha activity in the
left and right sides of the brain
 Brain scan imaging - Examine physiological reactions to ads and brands
13

14. Posttests of Print Ads
Advantages of inquiry tests
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
• Implementation is
inexpensive
• Provide some feedback
about the general
effectiveness of an ad or
medium used
Disadvantages of inquiry tests
• Inquiries may not be a true
measure of the attention-getting
or information-providing
aspects of an ad
• Ineffective for comparing
different versions or specific
creative aspects of an ad
14

15. Posttests of Broadcast Commercials
Advantages of day-after recall tests
• Natural setting provides a more realistic
response profile
• Provide norms that allow advertisers to
compare how well their ads are performing
• Persuasive and diagnostics measures are
also available
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
Disadvantages of day-after recall tests
• Have limited samples, high costs, and
security issues
• Favor unemotional appeals
• Program content may influence recall
• Respondents are aware of the test and will
be more attentive
• Recall is not a measure of acceptance or
predictive of sales
15

16. Factors That Make or Break Tracking
Studies
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
Properly defined objectives
Alignment with sales objectives
Properly designed measures
Consistency
Random samples
Continuous interviewing
Evaluate measures related to behavior
Critical evaluative questions asked early to eliminate bias
16

17. Factors That Make or Break Tracking
Studies
Measurement of competitors’ performance
Skepticism about questions asking where the ad was seen
Building of news value into the study
Moving averages spot long-term trends and avoid seasonality
Data reported in terms of relationships
Integration of key marketplace events with tracking results
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
17

18. Problems with Current Research
Methods
 Accomplishing some factors important to good
copy testing require more effort
 Most current methods do little more than provide
recall scores
 Lab measures - Artificial and vulnerable to testing
effects
 Field measures - Result in a loss of control
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
18

19. Essentials of Effective Testing
 Establish communications objectives
 Use a consumer response model
 Use both pretests and posttests
 Use multiple measures
 Understand and implement proper research
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
19

20. Measuring the Effectiveness of
Nontraditional Media
• Much less effective than claimed
Ski resort-based media
In-store radio and television
Other media
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
Shopping cart signage
• Measures are combined with sales tracking data to evaluate its
effectiveness
• Software has been introduced to measure the effectiveness
• Resources are available to measure impact of IMC elements
20

21. Measuring the Effectiveness of
Sponsorships
• Monitor the quantity and nature of the media
coverage obtained for the sponsored event
• Estimate direct and indirect audiences
Tracking measures
Copyright © 2014 McGraw-Hill Education. All rights reserved. No reproduction or distribution without the prior written consent of
McGraw-Hill Education.
Exposure methods
• Evaluate the awareness, familiarity, and preferences
produced by sponsorship based on surveys
21