2. Learning Objectives
To understand reasons for measuring
advertising effectiveness.
To know the various dependent
measures that is used in assessing
advertising effectiveness.
To evaluate alternative methods for
conducting measures of advertising
effectiveness.
To understand the requirements for
3. Pros and Cons of Measuring Effectiveness
Advantages
Disadvantages
Avoid costly mistakes
Avoid costly mistakes
Cost of measurement
Cost of measurement
Evaluate alternative
Evaluate alternative
strategies
strategies
Research problems
Research problems
Increase efficiency in
Increase efficiency in
general
general
Disagreement on
Disagreement on
what to test
what to test
Determine if objectives are
Determine if objectives are
achieved
achieved
Objections of creatives
Objections of creatives
Time
Time
5. Measuring Effectiveness: Marketers’ Views
Issue
Agree
MPM is a key priority for today’s
technology companies.
90%
I am dissatisfied with our ability to
demonstrate marketing programs’
business impact and value.
80%
It is important to define, measure, and
take concrete steps in the area of
advertising accountability.
6
37%
6. Measuring Advertising Effectiveness
What to test
What to test
• Source factors
• Source factors
• Message variables
• Message variables
• Media strategies
• Media strategies
• Budget decisions
• Budget decisions
How to test
How to test
• Testing guidelines
• Testing guidelines
• Appropriate tests
• Appropriate tests
Where to test
Where to test
• Laboratory tests
• Laboratory tests
• Field tests
• Field tests
When to test
When to test
• Pretesting
• Pretesting
• Posttesting
• Posttesting
11. Positioning Advertising Copy Test (PACT)
1. Provide measurements relevant to objectives of advertising
2. Require agreement on how results will be used
3. Provide multiple measures
4. Be based on a model of human response to communications
5. Consider multiple versus single exposure to the stimulus
6. Require alternative executions to have same degree of finish
7. Provide controls to avoid biasing effects of exposure context
8. Take into account basic considerations of sample definition
9. Demonstrate reliability and validity
12. Test Points
1.Concept Testing
1.Concept Testing
2.Rough Testing
2.Rough Testing
Occurs at
Occurs at
Various Stages
Various Stages
3.Finished art or
3.Finished art or
commercial pretesting
commercial pretesting
4.Market testing
4.Market testing
(posttesting)
(posttesting)
13. Concept Testing
Objective
Objective
Explores consumers’ responses to ad
concepts expressed in words, pictures, or
symbols
Alternatives are exposed to consumers who
match the target audience
Method
Method
Reactions & evaluations sought through focus
groups, direct questioning, surveys, etc.
Sample sizes depend on the number of
concepts and the consensus of responses
Output
Output
Qualitative and/or quantitative data
evaluating and comparing alternative
concepts
14. Rough Art, Copy, and Commercial Testing
Comprehension and Reaction Tests
Consumer Juries
Advantages
Disadvantages
Control
Consumer may become a
self-appointed expert
Cost effectiveness
Number of ads that can be
evaluated is limited
Endorsements by
independent third parties
A halo effect is possible
Achievement of credibility
Preference for ad types
may overshadow objectivity
16. Pretesting Finished Print Ads
A laboratory method
Portfolio
Portfolio
Tests
Tests
Includes test and control ads
Portfolio test have problems
Readability
Readability
Tests
Tests
Dummy
Dummy
Advertising
Advertising
Vehicles
Vehicles
Based on syllables per 100 words
Other factors also considered
Distributed to random sample homes
Product interest may still bias results
17. Pretesting Finished Broadcast Ads
Theater Tests
On-Air Tests
• Measures changes in
product preferences
• Insertion in TV programs
in specific markets
• May also measure . . .
• Interest in and reaction
to the commercial
• Reaction from an
adjective checklist
• Recall of various
aspects included
• Interest in the brand
presented
• Continuous reactions
• Limitations are imposed
by “day-after recall”
• Physiological Measures
24. Essentials of Effective Testing
Use a consumer
Use a consumer
response model
response model
Establish
Establish
communications
communications
objectives
objectives
Testing
Testing
Understand and
Understand and
implement
implement
proper research
proper research
Use pretests and
Use pretests and
post-tests
post-tests
Use multiple
Use multiple
measures
measures
25. Measuring Effectiveness of Other Programs
Sales
Sales
promotions
promotions
Shopping cart signage
Shopping cart signage
NonNontraditional
traditional
media
media
Ski resort-based media
Ski resort-based media
In-store radio and television
In-store radio and television
Other media
Other media
SponsorSponsorships
ships
Exposure methods
Exposure methods
Tracking measures
Tracking measures
Relation to text
This slide relates to the material on pp. 615-619 of the text.
Summary Overview
This slide shows some of the reasons why companies should measure effectiveness of their communications programs, as well as why they don’t. Reasons why such measures should be taken include:
To avoid costly mistakes
To evaluate strategies implemented
To increase the efficiency of advertising in general
To determine if objectives are achieved
Many managers choose not to measure, citing the following reasons:
The costs associated with measuring effectiveness
Problems with research methods
Disagreement as to what to test
Objections of creative
Time
Use of this slide
This slide can be used to discuss the reasons why managers should conduct research to measure the effectiveness of advertising and other components of the IMC program, as well as some of the reasons why they may resist doing so.
Relation to textThis slide relates to page 615 and Exhibit 19-1 of the text.
Summary OverviewAdvertising can be an expensive proposition. Testing ad effectiveness helped one retailer discover that promoting Tide detergent generated more cross-selling opportunities than did promotions of non-premium detergent brands. This meant getting more bang for the same advertising buck.
Use of this slideUse this slide to give a real-world benefit that came from ad effectiveness testing.
Relation to textThis slide relates to page 617 and Figure 19-1 of the text.
Summary OverviewThis slide presents marketers’ views toward measuring effectiveness, the result of two recent studies. Although advertisers know it’s important to measure effectiveness, with as many as 90 percent considering it a priority, many do not do so. Or, if they do, they are not confident with the results.
Use of this slide
This slide can be used as an introduction to the reasons companies give for not measuring the effectiveness of advertising and promotion strategies.
Relation to text
This slide relates to material on pp. 619-623 of the text.
Summary Overview
This slide presents some of the issues that have to be considered in the measurement of advertising effectiveness. These include:
What to test
Source factors
Message variables
Media strategies
Budgeting decisions
When to test
Pretesting
Posttesting
Where to test
Laboratory tests
Field tests
How to test
Testing guidelines
Appropriate tests
Use of this slide
This slide presents various factors that must be taken into consideration when measuring the effectiveness of advertising and other IMC elements. It can be used to introduce the material on the following slides, which address many of these issues.
Relation to text
This slide relates to material on pp. 620-622 and Figure 19-2 of the text.
Summary Overview
This slide discusses some of the various pre-testing methods available, further classifying them as to where they are to be conducted - laboratory or field. Pre-tests are those measures taken prior to the implementation of the campaign. Laboratory tests are those in which participants are brought to a specific location for testing, while field tests are those conducted in more natural viewing situations.
Use of this slide
This slide can be used to introduce and provide an overview of pre-testing measures as well as laboratory and field testing.
Relation to text
This slide relates to material on pp. 620-622 and Figure 19-2 of the text.
Summary Overview
This slide provides examples of some of the various types of posttests available. Posttests occur after the ad or commercial has been in the field. The field measures presented on this slide are used to determine the effectiveness of the ads once the campaign has been implemented.
Use of this slide
This slide can be used to discuss the types of post-testing methods that are available to marketers to measure the effectiveness of their advertising programs.
Relation to text
This slide relates to material on pp. 621-622 and Exhibit 19-3 of the text.
Summary Overview
This slide shows a NASCAR race car that was sponsored by Lowe’s, the nation’s second largest home improvement center. Lowe’s recently conducted extensive field experiments to determine the effectiveness of its various forms of advertising, including television spots, newspaper ads, circular catalogs, and even its sponsorship of NASCAR auto racing.
Use of this slide
This slide can be used as part of a discussion of how companies such as Lowe’s use field testing to determine the impact of their IMC programs. You might ask students how a company could go about determining the value of a NASCAR sponsorship.
Relation to textThis slide relates to page 622 of the text.
Summary OverviewEffectiveness testing can take place either in the laboratory and in the field. The major advantage of the lab setting is the control it affords the researcher. The major disadvantage is the lack of realism.
Field tests occur under natural viewing situations, complete with the realism of noise, distractions, and comforts of home. The major disadvantage of field testing is the lack of control. It also takes more time in money. So, realism is gained at the expense of other important factors.
It is up to the researcher to determine which trade-offs to make.
Use of this slideUse this slide when discussing where testing should take place.
Relation to text
This slide relates to the material on pp. 622-623 and Figure 19-3 of the text.
Summary Overview
Because measuring effectiveness is not an easy task, twenty-one of the largest U.S. advertising agencies have endorsed a set of principles aimed at “improving the research used in preparing and testing ads, providing a better creative product for clients, and controlling the cost of TV commercials”. This set of nine principles, called Positioning Advertising Copy Testing (PACT), is designed to establish guidelines for good copy testing research. The nine PACT principles are shown here.
Use of this slide
This slide can be used to present the nine PACT principles that were developed to guide advertising copy testing. You might want to discuss each of these, as they are essential to the development of good copy testing methods.
Relation to text
This slide relates to material on p. 623 of the text
Summary Overview
This slide provides an overview of the testing process that may occur at various stages throughout the development of an advertising campaign. The stages where testing may occur and types of testing that might be done include:
Concept generation research
Rough, prefinished art, copy, and/or commercial testing
Finished art or commercial pretesting
Market testing of ads or commercials (posttesting).
Use of this slide
This slide provides an overview of the testing process and can be used to provide an overview of various forms of testing that are used. The subsequent slides provide specific information about each of these.
Relation to text
This slide relates to material on pp. 623-624 and Figure 19-4 of the text.
Summary Overview
This slide summarizes what is involved in concept testing, which is conducted very early on in the research process. It shows the objective, methods and outputs associated with this form of testing. One of the more common methods of concept testing is through the use of focus groups, though field testing is often employed, and the Internet has resulted in an increase of concept testing online.
Use of this slide
This slide can be used to further explain concept testing which is one of the research methods that is used very early on in the campaign development process.
Relation to text
This slide relates to material on pp. 6248-627 of the text.
Summary Overview
This slide shows the methods of rough art, copy and commercial testing available to the marketer and the advantages and disadvantages associated with these. Because of the high costs associated with commercial development and production, many marketers attempt to measure the potential success of the commercial prior to completing the finished product.
Use of slide
This slide can be used to explain the use of the rough art, copy and commercial testing. It should be noted that for messages that do not involve high emotional content, these rough forms correlate highly with the finished product
Relation to text
This slide relates to material on p. 626 of the text and Figure 19-7.
Summary Overview
There are a number of options available for rough testing purposes. The choice of which to use will be based on the type of commercial to be tested.
This slide shows the three broad categories into which rough testing can be classified including:
Animatic rough - includes a succession of drawings/cartoons, rendered artwork, still frames and simulated movement
Photomatic rough -successions of photographs often showing real people/scenery, etc. with still frames and simulated movement
Live-action rough - employs live motion, stand-in involvement often with nonunion crews, limited props and opticals, and location settings.
Use of this slide
This slide can be used to show the various forms of rough copy testing.
Relation to text
This slide relates to material on pp. 627-628 of the text that discusses pretesting.
Summary Overview
This slide shows a number of methods for pretesting finished print advertisements. The purpose of pretesting print ads is to attempt to determine how the finished product will perform, make corrections if needed, and fine-tune the message. As the slide shows, some of the options available for this type of testing include:
Portfolio tests—a laboratory methodology designed to expose respondents to a portfolio consisting of both control and test ads.
Readability tests—measures the communications efficiency of the copy in a print ad
Dummy advertising vehicles—“dummy” magazines are created including articles, and test and non-test ads. The magazines are randomly distributed to homes in a specified area. Readers are interviewed to assess their reactions to both the editorial and the ads.
The determination as to which method will be used will be based on the needs and objectives of the advertiser.
Use of this slide
This slide can be used to demonstrate some of the various research methods available to marketers for the pretesting of print advertisements.
Relation to text
This slide relates to material on pp. 628-632 of the text.
Summary Overview
A variety of methodologies are available for pretesting broadcast commercials. A few of the most commonly used are presented in this slide including:
Theater test - participants are invited to view pilots of new TV programs. Embedded in the programs (or following immediately after) are the test commercials. Theater tests can be used to take a variety of measures.
On-air tests - testing of finished or rough commercials through actual TV programs in specified test markets. On-air tests offer some of the same advantages and disadvantages of theater tests but in a more realistic setting.
Physiological measures - a laboratory method for measuring involuntary responses to commercials such as heartbeat, pupil dilation, eye movement, brain waves and galvanic skin response
Theater tests and on-air testing are commonly employed, physiological measures are used less often.
Use of this slide
This slide can be used to discuss the various methods for testing reactions to finished broadcast commercials.
Relation to text
This slide relates to material on pp. 630-632 of the text.
Summary Overview
This slide shows the various physiological measures, a less common method of pretesting finished commercials. These measures include:
Pupil dilation
Galvanic skin response
Eye tracking
Brain waves
Use of this slide
These measures indicate the receiver’s involuntary response to the ad, theoretically eliminating biases associated with voluntary measures. This slide can be used to discuss how physiological responses are measured.
Relation to text
This slide relates to material on pp. 632-635 and Exhibits 19-4 and 19-5 of the text.
Summary Overview
This slide shows the various methods that are used to determine the effectiveness of print advertising once the ads have been completed and put into the market. A number of methods are available including:
Inquiry tests
Recognition tests
Recall
Tracking studies
These methods are often used by advertisers to determine how well print ads are doing in terms of getting attention, being read, and being remembered.
Use of this slide
This slide can be used to discuss the various options available to marketers for the testing of how well print ads are performing in the market.
Relation to text
This slide relates to material on pp. 634-635 and Figure 19-14 of the text.
Summary Overview
This slide shown an example of a Starch scored ad. The Starch method is one of the more commonly employed posttest measures of print ads. It uses three measures including:
Noted score– the percentage of readers who remember seeing the ad
Seen-associated score—the percentage who remember seeing or reading any part of the ad identifying the product or brand
Read most score—the percentage of readers who report reading at least half of the copy portion of the ad.
Starch measures have been used in the magazine industry for many years, and have become one of most commonly employed measures of advertising effectiveness.
Use of this slide
This slide can be used to provide an example of a Starch scored advertisement.
Relation to text
This slide relates to material on pp. 636-639, Exhibit 19-5 and 19-6, and Figure 19-16 of the text.
Summary Overview
This slide shows the variety of measures available for the market testing of finished broadcast commercials. These include:
Day after recall test - the most commonly performed measure.
Persuasive measures – attempting to measure persuasive impact of the commercial on brand selection.
Diagnostic - measures to determine viewers’ evaluations of the commercial, including comprehension and clarity.
Comprehensive measures - a combination of each of the three previous measures.
Test marketing - testing of finished commercials in specified test markets.
Single source tracking - tracking of the impact of the commercial from TV set to the retail store through the use of scanner data
Tracking studies - measuring the effects on various factors (recall, awareness, and attitudes) over time.
Use of this slide
This slide can be used to demonstrate the various options available for the marketer interested in measuring the effectiveness of a finished commercial.
Relation to text
This slide relates to material on pp. 636-637 and Exhibit 19-5 of the text.
Summary Overview
This slide shows an example of a provider of research service, Ipsos-ASI. The company provides a variety of research services for print and broadcast measurement, including diagnostics and comprehensive measures.
Use of this slide
This slide can be used to provide an example of one of the many research companies whose services are available for commercial testing.
Relation to text
This slide relates to material on pp. 623 and 640, and Figure 19-3 of the text.
Summary Overview
This slide shows the principles important to good copy testing. It is clear that some of these principles are easily accomplished, whereas others require substantially more effort.
Principles 1 and 2: should require minimum effort providing measurements relative to the objectives sought and determining a priori how the results will be used.
Principle 3: slightly more difficult, but largely in control of the researcher, providing multiple measurements
Principle 4: one of the most difficult factors to control. Most current methods do little more than provide recall scores.
Principle 5: slightly difficult, can be accomplished with proper research design.
Principle 6: should require minimum effort, providing equivalent test ads.
Principle 7: one of the most difficult factors to control. Lab measures are artificial and vulnerable. Field measures often lose control.
Principle 8: slightly difficult, requires little more than sound research methodology.
Principle 9: one of the most difficult factors to control due to concerns for reliability and validity.
Use of this slide
This slide can be used to discuss problems with current research methods when compared to the criteria established by PACT.
Relation to text
This slide relates to material on pp. 641 of the text.
Summary Overview
This slide shows the essentials of effective advertising testing. To be truly effective, the testing method should:
Establish communications objectives
Use a consumer response model
Use both pretests and posttests
Use multiple measures
Understand and implement proper research
Solid research should meet these criteria. Research methodologies that meet these criteria will be the most effective for providing insight into the performance of advertising.
Use of this slide
This slide can be used to provide an overview of the criteria for effective measurement.
Relation to text
This slide relates to material on pp. 641-644 of the text.
Summary Overview
This slide shows a number of methods for measuring the effectiveness of other program elements not previously discussed. These measures include:
Sales promotions—a number of organizations measure sales promotions, including MarketSource, Schnucks, Smitty’s Super Valu, Vons and Shopper Trak.
Nontraditional media—examples include shopping cart signage, ski resort-based media, in-store radio and television, and other media.
Sponsorships—essentially, measures of sponsorship can be categorized as exposure-based methods or tracking measures.
Use of this slide
This slide can be used to discuss the increased use of integrated marketing communications programs, which has led to more interest in determining the synergistic effects of all program elements.
Relation to text
This slide relates to material on pp. 644-647 and Figure 19-23 of the text.
Summary Overview
This slide illustrates the importance of measuring effectiveness and efficiency. Based on the belief that integrated marketing communications improves both the efficiency and the effectiveness of a campaign, the research company Integration contends that most traditional measurement techniques focus only on the former of these.
Use of this slide
This slide can be used to discuss the increased demand for marketing managers to prioritize the media vehicles used to promote their brands, and to measure both the efficiency and the effectiveness of media used to establish contacts with consumers.
