The document outlines a four-step communications process (research, analysis, communication, evaluation) to effectively plan and evaluate communications efforts, particularly in public relations. It discusses the importance of research in both planning and evaluating campaigns, detailing various research methods and the distinction between goals and objectives. The text emphasizes the integration of evaluation mechanisms into the planning phase to ensure success in achieving specific, measurable outcomes.

1. Evaluating Your Communications Efforts Robin Mayhall, APR Corporate Communications Blue Cross and Blue Shield of Louisiana Originally Presented 2.11.09 | Updated 2.04.10

2. About Me Baton Rouge native Journalism degree with PR concentration from the University of Texas at Austin PRSA member since 1991 Austin Chapter, 1991-2001 BR Chapter, 2001-present Joined PRAL after I moved back to Baton Rouge in 2001 Accredited in Public Relations Named a SPRF Senior Practitioner in 2004 Named 2008 Practitioner of the Year by PRAL-BR Have been with Blue Cross since August 2003 as one of three writers in Corporate Communications

3. Writing a Communications Plan Useful first step — even for small projects. At Blue Cross we write at least the basic outline of a plan whenever we are asked to produce a special event, brand a product, launch a new program, etc. Going through this process helps make sure all bases are covered.

4. The Four-Step Communications Process The four steps are sometimes given different names, but the names don’t really matter. The most common acronym is RACE : Research — Investigate and describe the situation and the need for communications. Analyze — Determine the target audience, goals and objectives and communications tactics. Communicate — Implement the tactics. Evaluate — Determine your level of success in achieving the plan’s goals and objectives.

5. Research Research is the key to the first and fourth steps of the four-step process. Pre-planning research and evaluation should be tied together from the beginning of your campaign planning. Research and evaluation are usually given less time, money and effort than goal-setting and the actual communications tactics. Many communicators mistakenly believe that research is inherently too expensive and difficult for small organizations, nonprofits, etc. Many communicators equate “research” with doing a scientific survey. In fact, there is a wide range of valid research techniques available to us — many of which are “free” (except for the time investment).

6. Types of Research: A Quick Refresher Formal vs. informal research Formal research uses the scientific method to ensure that the results can be extrapolated to a larger population — for example, a scientifically designed telephone survey. Objective and systematic data – “hard” data Generally uses random sampling Highly structured Can be repeated reliably Can be used to confirm informal research Informal research means that the results cannot be used to draw scientifically based conclusions. Soft data Open-ended, unstructured Can use to begin the research process Exploratory, probing research Use to identify what formal research needs to take place It’s important to note that “informal” does not mean “unreliable” or “invalid”!

7. Types of Research: A Quick Refresher Primary vs. secondary research These terms refer to the source of the research information. Primary research means that you and your coworkers conduct the research or examine the evidence personally, firsthand. New or original data You design and carry out research that’s specific to your current needs Secondary research means that you investigate secondhand evidence — e.g., the results of someone else’s studies or the report of someone else’s primary research. Secondhand, sometimes older data Uses what’s already available Again, both types are perfectly valid.

8. Types of Research: A Quick Refresher A brief list of research types (see your handout for more) Mail or telephone survey Focus groups Roleplaying Communications audit Website survey Public relations audit Man-on-the-street polls Advisory panels Readership study Database search

9. Types of Research: A “Quiz” Determine whether each of these research methods is formal or informal and primary or secondary. You conduct focus group interviews of several different possible target audiences. You hire a polling firm to develop a questionnaire about your new product and to poll a scientific sample of your target audience. You send an intern to the library to read through archives of the local newspaper and determine how many articles about your company have been positive, negative or neutral.

10. Types of Research: A “Quiz” (cont.) You e-mail each member of your Board of Directors to ask what they think the company’s strengths and weakness are and what challenges they will face in the coming year. You hire a PR agency to do a statistical content analysis of all media coverage of your organization in the previous three years. You ask your company receptionist to keep track of the types of phone calls she’s getting since your new product was launched. A trade publication does a Q&A-format article each month, interviewing an opinion leader in your industry. You read through a year’s worth of these interviews and jot down the topics these opinion leaders thing are important.

11. How Do You Know Which Method(s) to Use? Several factors come into play, and you have to use your judgment. How much time and money do you have? How important is this project to your organization in terms of: Potential impact on sales, finances, people Number of people it could affect The company’s financial investment Who’s in charge of the project (who you will answer to)

12. Set Goals and Evaluate Before you launch new projects: Draft a communications plan Set measurable objectives based on your research In an ideal world, repeat this research to measure the results Pre-planning research, goals and evaluation are tied closely together. Think ahead about ways to evaluate your results. For example, if you conduct a focus group to determine a certain audience’s attitudes during the research phase of your campaign, then design and implement a telephone survey based on the results of your focus groups, you could repeat one or both of these methods in the evaluation phase to determine whether you have met your objectives.

13. Goals and Objectives There is a difference between goals and objectives – they are not interchangeable. Most communicators are good at setting goals, but defining objectives is harder. A goal is a desired outcome of your plan. For example: To increase public use of mass transit. To raise awareness of green building practices in developing countries. Objectives are specific milestones that measure progress toward achieving one of your goals. They should consist of four elements (abbreviated LBPT): Level: Specify the expected level of accomplishment. Behavior: Address the desired result in terms of opinion change and/or behavioral outcome. Public: Designate the public or publics targeted. Time: Identify the time frame in which these accomplishments are to occur.

14. Goals and Objectives, cont. Examples of objectives for the following goal: To increase public use of mass transit. To increase by 10 percent the ridership of public transportation in the Alexandria area by workers earning less than $25,000 per year within the first six months of the communications program. By the end of the second year of the campaign, to have at least 20 percent of a randomly selected sample of public transportation riders in Alexandria identify one of our campaign tactics as their reason for using public transit.

15. Evaluation: Measuring Whether Objectives Are Met Evaluation consists of doing research to determine whether you’ve met these specific, measurable objectives. If you’ve planned effectively (developing the Research and Analysis portions of the four RACE steps), then you should have built-in areas for evaluation of your public relations success. Each objective should lend itself to evaluation, and in some cases you can actually repeat what you did in the first research step.

16. Research and Evaluation in Our Imperfect World Blue Cross annual reports consistently win awards, especially those related to graphic design. But it was hard in the past to evaluate results. (Management does not allow us to survey recipients.) For several years, we’ve been able to relate the results of our annual advertising effectiveness survey to the success of the annual report. We now align goals for the annual report to overall advertising/marketing goals, making it the “cornerstone” of our marketing communications for the year.

17. Evaluation, cont. Media clips are a valid measurement Take them a step farther by giving your client, boss or awards judges a context: i.e., “positive half-page article appeared on the front page of the business section of the top local daily newspaper.” Anecdotal evidence can be valuable Letters and e-mails from customers, attendees, management “ Testimony” from sales force, account managers, etc., that a program is gaining acceptance or helping to drive sales Other awards won for the same project Again, context is important.

18. Evaluation, cont. Benchmark and evaluate when you can. Pre- and post-testing and/or surveys Compare to previous year’s results/attendance/sales Compare to industry standards Tell judges what you learned. Don’t be afraid to admit it when not all the news is good You can learn from parts of a campaign that missed the mark Adjust goals for next year/next program Use any negative feedback to improve future efforts

19. Evaluation, cont. Some additional ways to evaluate your efforts: Specific measurement: Things that can be measured quantitatively, such as number of clippings, attendance at an event, percent answering a certain way on a survey. Semi-specific measurement: Items that an experienced professional can get a reading on, but that lack numerical measures, such as the balance of coverage in the media or the audience’s reaction to a speech. Acceptance on basis of judgment: Do complaints overall decrease? Does management believe efforts have been valuable? Recognizing value of input: Have your concepts led to marketing themes that have captured buyers’ enthusiasm? Has the stance of the organization recommended by PR staff been well received by its publics? Prevention: What didn’t happen as a result of public relations advice, such as issues that were blunted before they arose. Guidance: A general absence of issues and alarms may be testimonials to the PR department’s overall efficiency.

20. Set Goals and Evaluate, cont. Cutlip, Center and Broom identify three major categories to use in evaluating your PR campaign. Preparation: Basically, how well did we do in the research and analysis portions of RACE? Implementation: How well did we do with the “C” in RACE? Impact: This takes the overall campaign and our objectives into account

21. Set Goals and Evaluate, cont. Preparation Review our situation analysis and pre-planning research – how well did we understand the situation at the time? Was our message appropriate based on our research? Collect and review all material we produced for the campaign – check accuracy, quality of writing and design, etc.

22. Set Goals and Evaluate, cont. Implementation How many stories were placed? How many people were reached? How many positive stories were generated? Did we get our story out?

23. Set Goals and Evaluate, cont. Impact If benchmark research was done, do another round to see if there was any movement toward our objectives. Show changes in attitudes or behavior. Did sales increase? Did the bond issue pass? Etc.

24. Special Evaluation Methods Certain special evaluation methods are useful when you don’t have the time or money for scientific primary research. Collect and codify results. Checklist of media coverage Number of positive clips RSVPs to invitations Ticket sales Attendance records or sign-in sheets Petitions Letters or e-mails received for and against the issue

25. Special Evaluation Methods, cont. Measure the degree of achievement for each objective. Use before-and-after measurement devices Set benchmarks with some kind of quantifiable research Follow up after the campaign by re-doing the same or similar research

26. Special Evaluation Methods, cont. Accommodate positive as well as negative evaluations. Record what went right and what went wrong Hold a debriefing session with staff or your client Discuss performance on each item Use negatives to plan better for the next campaign

27. Special Evaluation Methods, cont. Collect proposals and projections growing out of the campaign. During the campaign, evaluate achievements so far and make adjustments for the remainder of the campaign Use campaign results to shape future efforts

28. Evaluation: Some Final Thoughts In general: Mix qualitative and quantitative measures. You should measure objectives, but you don’t have to do a scientific survey on everything. The most important thing to remember is to tie your evaluation in to your research. One way to ensure you don’t “forget” about evaluation is to think about how you will evaluate each objective while you are writing it. Thus, in the Analysis portion of your planning grid, when you are writing your LBPTs, ask yourself (and jot down) how you will evaluate the achievement of each behavioral outcome.

29. Q & A Feel free to contact me at Robin.Mayhall@bcbsla.com.