Relation to text This slide relates to the material on pp. 615-619 of the text. Summary Overview This slide shows some of the reasons why companies should measure effectiveness of their communications programs, as well as why they don’t. Reasons why such measures should be taken include: To avoid costly mistakes To evaluate strategies implemented To increase the efficiency of advertising in general To determine if objectives are achieved Many managers choose not to measure, citing the following reasons: The costs associated with measuring effectiveness Problems with research methods Disagreement as to what to test Objections of creative Time Use of this slide This slide can be used to discuss the reasons why managers should conduct research to measure the effectiveness of advertising and other components of the IMC program, as well as some of the reasons why they may resist doing so.
Relation to text This slide relates to page 615 and Exhibit 19-1 of the text. Summary Overview Advertising can be an expensive proposition. Testing ad effectiveness helped one retailer discover that promoting Tide detergent generated more cross-selling opportunities than did promotions of non-premium detergent brands. This meant getting more bang for the same advertising buck. Use of this slide Use this slide to give a real-world benefit that came from ad effectiveness testing.
Relation to text This slide relates to page 617 and Figure 19-1 of the text. Summary Overview This slide presents marketers’ views toward measuring effectiveness, the result of two recent studies. Although advertisers know it’s important to measure effectiveness, with as many as 90 percent considering it a priority, many do not do so. Or, if they do, they are not confident with the results. Use of this slide This slide can be used as an introduction to the reasons companies give for not measuring the effectiveness of advertising and promotion strategies.
Relation to text This slide relates to material on pp. 619-623 of the text. Summary Overview This slide presents some of the issues that have to be considered in the measurement of advertising effectiveness. These include: What to test Source factors Message variables Media strategies Budgeting decisions When to test Pretesting Posttesting Where to test Laboratory tests Field tests How to test Testing guidelines Appropriate tests Use of this slide This slide presents various factors that must be taken into consideration when measuring the effectiveness of advertising and other IMC elements. It can be used to introduce the material on the following slides, which address many of these issues.
Relation to text This slide relates to material on pp. 620-622 and Figure 19-2 of the text. Summary Overview This slide discusses some of the various pre-testing methods available, further classifying them as to where they are to be conducted - laboratory or field. Pre-tests are those measures taken prior to the implementation of the campaign. Laboratory tests are those in which participants are brought to a specific location for testing, while field tests are those conducted in more natural viewing situations. Use of this slide This slide can be used to introduce and provide an overview of pre-testing measures as well as laboratory and field testing.
Relation to text This slide relates to material on pp. 620-622 and Figure 19-2 of the text. Summary Overview This slide provides examples of some of the various types of posttests available. Posttests occur after the ad or commercial has been in the field. The field measures presented on this slide are used to determine the effectiveness of the ads once the campaign has been implemented. Use of this slide This slide can be used to discuss the types of post-testing methods that are available to marketers to measure the effectiveness of their advertising programs.
Relation to text This slide relates to material on pp. 621-622 and Exhibit 19-3 of the text. Summary Overview This slide shows a NASCAR race car that was sponsored by Lowe’s, the nation’s second largest home improvement center. Lowe’s recently conducted extensive field experiments to determine the effectiveness of its various forms of advertising, including television spots, newspaper ads, circular catalogs, and even its sponsorship of NASCAR auto racing. Use of this slide This slide can be used as part of a discussion of how companies such as Lowe’s use field testing to determine the impact of their IMC programs. You might ask students how a company could go about determining the value of a NASCAR sponsorship.
Relation to text This slide relates to page 622 of the text. Summary Overview Effectiveness testing can take place either in the laboratory and in the field. The major advantage of the lab setting is the control it affords the researcher. The major disadvantage is the lack of realism. Field tests occur under natural viewing situations, complete with the realism of noise, distractions, and comforts of home. The major disadvantage of field testing is the lack of control. It also takes more time in money. So, realism is gained at the expense of other important factors. It is up to the researcher to determine which trade-offs to make. Use of this slide Use this slide when discussing where testing should take place.
Relation to text This slide relates to the material on pp. 622-623 and Figure 19-3 of the text. Summary Overview Because measuring effectiveness is not an easy task, twenty-one of the largest U.S. advertising agencies have endorsed a set of principles aimed at “improving the research used in preparing and testing ads, providing a better creative product for clients, and controlling the cost of TV commercials”. This set of nine principles, called Positioning Advertising Copy Testing (PACT), is designed to establish guidelines for good copy testing research. The nine PACT principles are shown here. Use of this slide This slide can be used to present the nine PACT principles that were developed to guide advertising copy testing. You might want to discuss each of these, as they are essential to the development of good copy testing methods.
Relation to text This slide relates to material on p. 623 of the text Summary Overview This slide provides an overview of the testing process that may occur at various stages throughout the development of an advertising campaign. The stages where testing may occur and types of testing that might be done include: Concept generation research Rough, prefinished art, copy, and/or commercial testing Finished art or commercial pretesting Market testing of ads or commercials (posttesting). Use of this slide This slide provides an overview of the testing process and can be used to provide an overview of various forms of testing that are used. The subsequent slides provide specific information about each of these.
Relation to text This slide relates to material on pp. 623-624 and Figure 19-4 of the text. Summary Overview This slide summarizes what is involved in concept testing, which is conducted very early on in the research process. It shows the objective, methods and outputs associated with this form of testing. One of the more common methods of concept testing is through the use of focus groups, though field testing is often employed, and the Internet has resulted in an increase of concept testing online. Use of this slide This slide can be used to further explain concept testing which is one of the research methods that is used very early on in the campaign development process.
Relation to text This slide relates to material on pp. 6248-627 of the text. Summary Overview This slide shows the methods of rough art, copy and commercial testing available to the marketer and the advantages and disadvantages associated with these. Because of the high costs associated with commercial development and production, many marketers attempt to measure the potential success of the commercial prior to completing the finished product. Use of slide This slide can be used to explain the use of the rough art, copy and commercial testing. It should be noted that for messages that do not involve high emotional content, these rough forms correlate highly with the finished product
Relation to text This slide relates to material on p. 626 of the text and Figure 19-7. Summary Overview There are a number of options available for rough testing purposes. The choice of which to use will be based on the type of commercial to be tested. This slide shows the three broad categories into which rough testing can be classified including: Animatic rough - includes a succession of drawings/cartoons, rendered artwork, still frames and simulated movement Photomatic rough - successions of photographs often showing real people/scenery, etc. with still frames and simulated movement Live - action rough - employs live motion, stand-in involvement often with nonunion crews, limited props and opticals, and location settings. Use of this slide This slide can be used to show the various forms of rough copy testing.
Relation to text This material relates to material on pp. 624-626 and Figure 19-7 of the text. Summary Overview This slide shows an example of an animatic which is one of the forms of rough testing. The animatic is from the “Action Sport Heroes” campaign that was used for Airwalk shoes in the late ‘90s. Animatics are used to test commercials prior to complete development. This saves on production costs as money does not have to be spent to actually produce the commercial in final form. Animatics can also be used to present advertising concepts to clients to assess their reactions before money is spent producing a commercial. Use of this slide This slide can be used to provide students with an example of an animatic that was actually used and then developed into a finished commercial spot. The actual commercial is shown on the next slide.
Relation to text This slide relates to material on pp. 624-626 and Figure 19-7 of the text. Summary Overview This slide shows the finished Airwalk commercial that resulted from the animatic shown in the previous slide. As noted, animatics are used to provide a rough copy test providing the marketer with insight into whether to proceed with the finished commercial. Use of this slide This slide can be used to demonstrate the effective use of animatics. The finished commercial shown here was tested in animatic form and performed well which resulted in the decision to actually produce the spot.
Relation to text This slide relates to material on pp. 627-628 of the text that discusses pretesting. Summary Overview This slide shows a number of methods for pretesting finished print advertisements. The purpose of pretesting print ads is to attempt to determine how the finished product will perform, make corrections if needed, and fine-tune the message. As the slide shows, some of the options available for this type of testing include: Portfolio tests— a laboratory methodology designed to expose respondents to a portfolio consisting of both control and test ads. Readability tests— measures the communications efficiency of the copy in a print ad Dummy advertising vehicles —“dummy” magazines are created including articles, and test and non-test ads. The magazines are randomly distributed to homes in a specified area. Readers are interviewed to assess their reactions to both the editorial and the ads. The determination as to which method will be used will be based on the needs and objectives of the advertiser. Use of this slide This slide can be used to demonstrate some of the various research methods available to marketers for the pretesting of print advertisements.
Relation to text This slide relates to material on pp. 628-632 of the text. Summary Overview A variety of methodologies are available for pretesting broadcast commercials. A few of the most commonly used are presented in this slide including: Theater test - participants are invited to view pilots of new TV programs. Embedded in the programs (or following immediately after) are the test commercials. Theater tests can be used to take a variety of measures. On-air tests - testing of finished or rough commercials through actual TV programs in specified test markets. On-air tests offer some of the same advantages and disadvantages of theater tests but in a more realistic setting. Physiological measures - a laboratory method for measuring involuntary responses to commercials such as heartbeat, pupil dilation, eye movement, brain waves and galvanic skin response Theater tests and on-air testing are commonly employed, physiological measures are used less often. Use of this slide This slide can be used to discuss the various methods for testing reactions to finished broadcast commercials.
Relation to text This slide relates to material on pp. 630-632 of the text. Summary Overview This slide shows the various physiological measures, a less common method of pretesting finished commercials. These measures include: Pupil dilation Galvanic skin response Eye tracking Brain waves Use of this slide These measures indicate the receiver’s involuntary response to the ad, theoretically eliminating biases associated with voluntary measures. This slide can be used to discuss how physiological responses are measured.
Relation to text This slide relates to material on pp. 632-635 and Exhibits 19-4 and 19-5 of the text. Summary Overview This slide shows the various methods that are used to determine the effectiveness of print advertising once the ads have been completed and put into the market. A number of methods are available including: Inquiry tests Recognition tests Recall Tracking studies These methods are often used by advertisers to determine how well print ads are doing in terms of getting attention, being read, and being remembered. Use of this slide This slide can be used to discuss the various options available to marketers for the testing of how well print ads are performing in the market.
Relation to text This slide relates to material on pp. 634-635 and Figure 19-14 of the text. Summary Overview This slide shown an example of a Starch scored ad. The Starch method is one of the more commonly employed posttest measures of print ads. It uses three measures including: Noted score– the percentage of readers who remember seeing the ad Seen-associated score— the percentage who remember seeing or reading any part of the ad identifying the product or brand Read most score— the percentage of readers who report reading at least half of the copy portion of the ad. Starch measures have been used in the magazine industry for many years, and have become one of most commonly employed measures of advertising effectiveness. Use of this slide This slide can be used to provide an example of a Starch scored advertisement.
Relation to text This slide relates to material on pp. 636-639, Exhibit 19-5 and 19-6, and Figure 19-16 of the text. Summary Overview This slide shows the variety of measures available for the market testing of finished broadcast commercials. These include: Day after recall test - the most commonly performed measure. Persuasive measures – attempting to measure persuasive impact of the commercial on brand selection. Diagnostic - measures to determine viewers’ evaluations of the commercial, including comprehension and clarity. Comprehensive measures - a combination of each of the three previous measures. Test marketing - testing of finished commercials in specified test markets. Single source tracking - tracking of the impact of the commercial from TV set to the retail store through the use of scanner data Tracking studies - measuring the effects on various factors (recall, awareness, and attitudes) over time. Use of this slide This slide can be used to demonstrate the various options available for the marketer interested in measuring the effectiveness of a finished commercial.
Relation to text This slide relates to material on pp. 636-637 and Exhibit 19-5 of the text. Summary Overview This slide shows an example of a provider of research service, Ipsos-ASI. The company provides a variety of research services for print and broadcast measurement, including diagnostics and comprehensive measures. Use of this slide This slide can be used to provide an example of one of the many research companies whose services are available for commercial testing.
Relation to text This slide relates to material on pp. 623 and 640, and Figure 19-3 of the text. Summary Overview This slide shows the principles important to good copy testing. It is clear that some of these principles are easily accomplished, whereas others require substantially more effort. Principles 1 and 2: should require minimum effort providing measurements relative to the objectives sought and determining a priori how the results will be used. Principle 3: slightly more difficult, but largely in control of the researcher, providing multiple measurements Principle 4: one of the most difficult factors to control. Most current methods do little more than provide recall scores. Principle 5: slightly difficult, can be accomplished with proper research design. Principle 6: should require minimum effort, providing equivalent test ads. Principle 7: one of the most difficult factors to control. Lab measures are artificial and vulnerable. Field measures often lose control. Principle 8: slightly difficult, requires little more than sound research methodology. Principle 9: one of the most difficult factors to control due to concerns for reliability and validity. Use of this slide This slide can be used to discuss problems with current research methods when compared to the criteria established by PACT.
Relation to text This slide relates to material on pp. 641 of the text. Summary Overview This slide shows the essentials of effective advertising testing. To be truly effective, the testing method should: Establish communications objectives Use a consumer response model Use both pretests and posttests Use multiple measures Understand and implement proper research Solid research should meet these criteria. Research methodologies that meet these criteria will be the most effective for providing insight into the performance of advertising. Use of this slide This slide can be used to provide an overview of the criteria for effective measurement.
Relation to text This slide relates to material on pp. 641-644 of the text. Summary Overview This slide shows a number of methods for measuring the effectiveness of other program elements not previously discussed. These measures include: Sales promotions— a number of organizations measure sales promotions, including MarketSource, Schnucks, Smitty’s Super Valu, Vons and Shopper Trak. Nontraditional media— examples include shopping cart signage, ski resort-based media, in-store radio and television, and other media. Sponsorships —essentially, measures of sponsorship can be categorized as exposure-based methods or tracking measures. Use of this slide This slide can be used to discuss the increased use of integrated marketing communications programs, which has led to more interest in determining the synergistic effects of all program elements.
Relation to text This slide relates to material on pp. 644-647 and Figure 19-23 of the text. Summary Overview This slide illustrates the importance of measuring effectiveness and efficiency. Based on the belief that integrated marketing communications improves both the efficiency and the effectiveness of a campaign, the research company Integration contends that most traditional measurement techniques focus only on the former of these. Use of this slide This slide can be used to discuss the increased demand for marketing managers to prioritize the media vehicles used to promote their brands, and to measure both the efficiency and the effectiveness of media used to establish contacts with consumers.
Pros and Cons of Measuring Effectiveness Advantages Disadvantages Avoid costly mistakes Avoid costly mistakes Cost of measurement Cost of measurement Evaluate alternative Evaluate alternative Research problems Research problems strategies strategies Increase efficiency in Increase efficiency in Disagreement on Disagreement on general general what to test what to testDetermine if objectives areDetermine if objectives are Objections of creatives Objections of creatives achieved achieved Time Time
Measuring Effectiveness: Marketers’ Views Issue Agree MPM is a key priority for today’s 90% technology companies. I am dissatisfied with our ability to 80% demonstrate marketing programs’ business impact and value. It is important to define, measure, and 61.5% take concrete steps in the area of advertising accountability. I can forecast the impact on sales of a 37% 10% cut in marketing spending.
Measuring Advertising EffectivenessWhat to testWhat to test Where to test Where to test • Source factors • Laboratory tests • Laboratory tests • Source factors • Message variables • Field tests • Field tests • Message variables • Media strategies • Media strategies • Budget decisions • Budget decisionsHow to testHow to test When to test When to test • Testing guidelines • Testing guidelines • Pretesting • Pretesting • Appropriate tests • Appropriate tests • Posttesting • Posttesting
Pretesting Methods Laboratory Field Consumer Juries Dummy Ad Vehicles Portfolio Tests On-air TestsPhysiological Measures Theater Tests Rough Tests Concept Tests Readability Tests Comprehension and Reaction Tests
Posttesting Methods Recall Tests Recall Tests Tracking Tracking Association Association Studies Studies Measures Measures Methods Methods Single- Single-RecognitionRecognition Source Source Tests Tests Systems Systems Inquiry Tests Inquiry Tests
Positioning Advertising Copy Test (PACT)1. Provide measurements relevant to objectives of advertising2. Require agreement on how results will be used3. Provide multiple measures4. Be based on a model of human response to communications5. Consider multiple versus single exposure to the stimulus6. Require alternative executions to have same degree of finish7. Provide controls to avoid biasing effects of exposure context8. Take into account basic considerations of sample definition9. Demonstrate reliability and validity
Test Points 1.Concept Testing 1.Concept Testing 2.Rough Testing 2.Rough Testing Occurs at Occurs atVarious StagesVarious Stages 3.Finished art or 3.Finished art or commercial pretesting commercial pretesting 4.Market testing 4.Market testing (posttesting) (posttesting)
Concept Testing Explores consumers’ responses to adObjectiveObjective concepts expressed in words, pictures, or symbols Alternatives are exposed to consumers who match the target audience Method Method Reactions & evaluations sought through focus groups, direct questioning, surveys, etc. Sample sizes depend on the number of concepts and the consensus of responses Qualitative and/or quantitative data Output Output evaluating and comparing alternative concepts
dvantagesRough Art, Copy, and Commercial Testing Comprehension and Reaction Tests Consumer Juries Consumer may become a Control self-appointed expert Disadvantages Number of ads that can be Cost effectiveness evaluated is limited Endorsements by A halo effect is possibleindependent third parties Preference for ad typesAchievement of credibility may overshadow objectivity
An Animatic Example from Airwalk *Click outside of the video screen to advance to the next slide
Spot Resulting from Animatic Rough *Click outside of the video screen to advance to the next slide
Pretesting Finished Print Ads A laboratory methodPortfolioPortfolio Includes test and control ads Tests Tests Portfolio test have problems Based on syllables per 100 wordsReadabilityReadability Tests Tests Other factors also considered Dummy Dummy Distributed to random sample homesAdvertisingAdvertising Vehicles Vehicles Product interest may still bias results
Test Your Knowledge_____ is a method of testing ads by placing themrandomly in certain copies of regularly distributedmagazines. A) Vehicle source testing B) Burkes reflection test C) A Flesch test D) Dummy advertising vehicle testing E) A contextual test
Pretesting Finished Broadcast Ads Theater Tests On-Air Tests• Measures changes in • Insertion in TV programs product preferences in specific markets• May also measure . . . • Limitations are imposed • Interest in and reaction by “day-after recall” to the commercial • Physiological Measures • Reaction from an adjective checklist • Recall of various aspects included • Interest in the brand presented • Continuous reactions
Starch-Scored Sports Illustrated Ad Noted score– the percentage of readers who remember seeing the ad Seen-associated score— the percentage who remember seeing or reading any part of the ad identifying the product or brand Read most score—the percentage of readers who report reading at least half of the copy portion of the ad.
Posttests of Broadcast Commercials Day after recall Persuasive tests measures Tracking Diagnostics studies TestingSingle-source Comprehensive tracking measures Test marketing
Essentials of Effective Testing Use a consumer Use a consumer response model response model Establish Establish Use pretests and Use pretests andcommunicationscommunications posttests posttests objectives objectives Testing Testing Understand and Understand and Use multiple Use multiple implement implement measures measures proper research proper research
Test Your KnowledgeGood tests of advertising effectiveness must addressthe nine principles established by PACT. One of theeasiest ways to do this is to follow a decision sequencemodel. The first step in the model is to: A) Understand the appropriate research B) Create a model that uses multiple measures C) Establish communication objectives D) Decide whether to use posttests or pretests E) Develop a consumer response model
Measuring Effectiveness of Other Programs Sales Sales promotions promotions Shopping cart signage Shopping cart signage Ski resort-based media Ski resort-based media Non- Non- traditional traditional media In-store radio and television In-store radio and television media Other media Other media Exposure methods Exposure methods Sponsor- Sponsor- ships ships Tracking measures Tracking measures