• Save
Evaluating Your Communications Efforts
Upcoming SlideShare
Loading in...5
×
 

Evaluating Your Communications Efforts

on

  • 1,115 views

A brief presentation on the "Evaluation" step of the four-step public relations planning process, with examples of how communicators can set measurable goals and objectives and then measure ...

A brief presentation on the "Evaluation" step of the four-step public relations planning process, with examples of how communicators can set measurable goals and objectives and then measure their success.

Statistics

Views

Total Views
1,115
Views on SlideShare
1,114
Embed Views
1

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 1

https://www.mturk.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Including a written plan in your award entry is not a make-or-break thing … we didn’t have a written plan, for example, for the 2006 Annual Report.
  • Obviously this is going to be a refresher for most people!
  • You conduct focus group interviews of several different possible target audiences. - Primary/informal You hire a polling firm to develop a questionnaire about your new product and to poll a scientific sample of your target audience. -- Primary/formal You send an intern to the library to read through archives of the local newspaper and determine how many articles about your company have been positive, negative or neutral. -- Primary/informal
  • You e-mail each member of your Board of Directors to ask what they think the company’s strengths and weakness are and what challenges they will face in the coming year. - Primary/informal You hire a PR agency to do a statistical content analysis of all media coverage of your organization in the previous three years. - Primary/formal You ask your company receptionist to keep track of the types of phone calls she’s getting since your new product was launched. - Primary/informal A trade publication does a Q&A-format article each month, interviewing an opinion leader in your industry. You read through a year’s worth of these interviews and jot down the topics these opinion leaders thing are important. - Secondary/informal
  • Several factors come into play, and you have to use your judgment. Realities of life: How much time and money do you have? What other resources? (Coworkers, temps, volunteers) How important is this project to your organization in terms of: Potential impact on sales, finances, people -- for example, is this a massive new product launch that could make or break the entire company? Is this your main advertising and PR campaign that will take up most of your communications budge for the year? Number of people it could affect -- will the plan have local, statewide, national impact? The company’s financial investment Who’s in charge of the project (who will you answer to) -- again, use judgment. It might suffice to give a report to a small nonprofit board of directors using mostly informal research -- they will understand and appreciate the time and budget constraints. On the other hand, if you are going to have to report on this campaign to the CEO, COO and CFO on behalf of your whole department, it might be worth the money and effort to hire a polling firm or do some more wide-reaching primary research.
  • Before you launch new projects: Draft a communications plan -- see PRSA’s website at www.prsa.org; also my own website at www.hieran.com/comet Set measurable goals For Lantern Awards and many similar competitions, measurable goals and results are very important. Think ahead about ways to evaluate your results.
  • Love those abbreviations!
  • How do we set such specific, measurable goals in our non-ideal world? We all know that not every project we work on each day is a full-fledged PR campaign. What about smaller campaigns, individual projects, one-time special events -- or the all-time favorite -- those projects we have to do because the senior VP says so? Blue Cross annual reports consistently win awards, especially those related to graphic design. But it was hard in the past to evaluate results. Management does not allow us to survey recipients, and even if they did, we weren’t exactly sure what to ask them. We probably wouldn’t stop doing an annual report based on survey results… So, for several years, we’ve been able to relate the results of our annual advertising effectiveness survey to the success of the annual report. We align goals for the annual report to overall advertising/marketing goals, making it the “cornerstone” of our marketing communications for the year. This ties everything neatly together … even the goals for next year’s annual report.
  • Importance of context: See our winning annual report entry Used words like “unprecedented” number of awards won to convey the impressiveness of the achievement
  • If these different ideas and methods are starting to get confusing, take a step back and remember that you don’t have to use all of them on any one project! These are all just tools in your PR toolbox -- tools that allow you to put your campaign in context and report successes meaningfully to clients, coworkers, bosses, boards, etc. -- as well as to tout your campaign’s success to an awards judging panel.

Evaluating Your Communications Efforts Evaluating Your Communications Efforts Presentation Transcript

  • Evaluating Your Communications Efforts Robin Mayhall, APR Corporate Communications Blue Cross and Blue Shield of Louisiana Originally Presented 2.11.09 | Updated 2.04.10
  • About Me
    • Baton Rouge native
    • Journalism degree with PR concentration from the University of Texas at Austin
    • PRSA member since 1991
      • Austin Chapter, 1991-2001
      • BR Chapter, 2001-present
    • Joined PRAL after I moved back to Baton Rouge in 2001
    • Accredited in Public Relations
    • Named a SPRF Senior Practitioner in 2004
    • Named 2008 Practitioner of the Year by PRAL-BR
    • Have been with Blue Cross since August 2003 as one of three writers in Corporate Communications
  • Writing a Communications Plan
    • Useful first step — even for small projects.
    • At Blue Cross we write at least the basic outline of a plan whenever we are asked to produce a special event, brand a product, launch a new program, etc.
    • Going through this process helps make sure all bases are covered.
  • The Four-Step Communications Process
    • The four steps are sometimes given different names, but the names don’t really matter.
    • The most common acronym is RACE :
      • Research — Investigate and describe the situation and the need for communications.
      • Analyze — Determine the target audience, goals and objectives and communications tactics.
      • Communicate — Implement the tactics.
      • Evaluate — Determine your level of success in achieving the plan’s goals and objectives.
  • Research
    • Research is the key to the first and fourth steps of the four-step process.
    • Pre-planning research and evaluation should be tied together from the beginning of your campaign planning.
    • Research and evaluation are usually given less time, money and effort than goal-setting and the actual communications tactics.
    • Many communicators mistakenly believe that research is inherently too expensive and difficult for small organizations, nonprofits, etc.
    • Many communicators equate “research” with doing a scientific survey.
    • In fact, there is a wide range of valid research techniques available to us — many of which are “free” (except for the time investment).
  • Types of Research: A Quick Refresher
    • Formal vs. informal research
      • Formal research uses the scientific method to ensure that the results can be extrapolated to a larger population — for example, a scientifically designed telephone survey.
        • Objective and systematic data – “hard” data
        • Generally uses random sampling
        • Highly structured
        • Can be repeated reliably
        • Can be used to confirm informal research
      • Informal research means that the results cannot be used to draw scientifically based conclusions.
        • Soft data
        • Open-ended, unstructured
        • Can use to begin the research process
        • Exploratory, probing research
        • Use to identify what formal research needs to take place
      • It’s important to note that “informal” does not mean “unreliable” or “invalid”!
  • Types of Research: A Quick Refresher
    • Primary vs. secondary research
      • These terms refer to the source of the research information.
      • Primary research means that you and your coworkers conduct the research or examine the evidence personally, firsthand.
        • New or original data
        • You design and carry out research that’s specific to your current needs
      • Secondary research means that you investigate secondhand evidence — e.g., the results of someone else’s studies or the report of someone else’s primary research.
        • Secondhand, sometimes older data
        • Uses what’s already available
      • Again, both types are perfectly valid.
  • Types of Research: A Quick Refresher
    • A brief list of research types (see your handout for more)
      • Mail or telephone survey
      • Focus groups
      • Roleplaying
      • Communications audit
      • Website survey
      • Public relations audit
      • Man-on-the-street polls
      • Advisory panels
      • Readership study
      • Database search
  • Types of Research: A “Quiz”
    • Determine whether each of these research methods is formal or informal and primary or secondary.
      • You conduct focus group interviews of several different possible target audiences.
      • You hire a polling firm to develop a questionnaire about your new product and to poll a scientific sample of your target audience.
      • You send an intern to the library to read through archives of the local newspaper and determine how many articles about your company have been positive, negative or neutral.
  • Types of Research: A “Quiz” (cont.)
    • You e-mail each member of your Board of Directors to ask what they think the company’s strengths and weakness are and what challenges they will face in the coming year.
    • You hire a PR agency to do a statistical content analysis of all media coverage of your organization in the previous three years.
    • You ask your company receptionist to keep track of the types of phone calls she’s getting since your new product was launched.
    • A trade publication does a Q&A-format article each month, interviewing an opinion leader in your industry. You read through a year’s worth of these interviews and jot down the topics these opinion leaders thing are important.
  • How Do You Know Which Method(s) to Use?
    • Several factors come into play, and you have to use your judgment.
    • How much time and money do you have?
    • How important is this project to your organization in terms of:
      • Potential impact on sales, finances, people
      • Number of people it could affect
      • The company’s financial investment
      • Who’s in charge of the project (who you will answer to)
  • Set Goals and Evaluate
    • Before you launch new projects:
      • Draft a communications plan
      • Set measurable objectives based on your research
      • In an ideal world, repeat this research to measure the results
    • Pre-planning research, goals and evaluation are tied closely together.
    • Think ahead about ways to evaluate your results.
    • For example, if you conduct a focus group to determine a certain audience’s attitudes during the research phase of your campaign, then design and implement a telephone survey based on the results of your focus groups, you could repeat one or both of these methods in the evaluation phase to determine whether you have met your objectives.
  • Goals and Objectives
    • There is a difference between goals and objectives – they are not interchangeable.
    • Most communicators are good at setting goals, but defining objectives is harder.
      • A goal is a desired outcome of your plan. For example:
        • To increase public use of mass transit.
        • To raise awareness of green building practices in developing countries.
      • Objectives are specific milestones that measure progress toward achieving one of your goals. They should consist of four elements (abbreviated LBPT):
        • Level: Specify the expected level of accomplishment.
        • Behavior: Address the desired result in terms of opinion change and/or behavioral outcome.
        • Public: Designate the public or publics targeted.
        • Time: Identify the time frame in which these accomplishments are to occur.
  • Goals and Objectives, cont.
    • Examples of objectives for the following goal:
    • To increase public use of mass transit.
      • To increase by 10 percent the ridership of public transportation in the Alexandria area by workers earning less than $25,000 per year within the first six months of the communications program.
      • By the end of the second year of the campaign, to have at least 20 percent of a randomly selected sample of public transportation riders in Alexandria identify one of our campaign tactics as their reason for using public transit.
  • Evaluation: Measuring Whether Objectives Are Met
    • Evaluation consists of doing research to determine whether you’ve met these specific, measurable objectives.
    • If you’ve planned effectively (developing the Research and Analysis portions of the four RACE steps), then you should have built-in areas for evaluation of your public relations success.
    • Each objective should lend itself to evaluation, and in some cases you can actually repeat what you did in the first research step.
  • Research and Evaluation in Our Imperfect World
    • Blue Cross annual reports consistently win awards, especially those related to graphic design.
    • But it was hard in the past to evaluate results. (Management does not allow us to survey recipients.)
    • For several years, we’ve been able to relate the results of our annual advertising effectiveness survey to the success of the annual report.
    • We now align goals for the annual report to overall advertising/marketing goals, making it the “cornerstone” of our marketing communications for the year.
  • Evaluation, cont.
    • Media clips are a valid measurement
      • Take them a step farther by giving your client, boss or awards judges a context: i.e., “positive half-page article appeared on the front page of the business section of the top local daily newspaper.”
    • Anecdotal evidence can be valuable
      • Letters and e-mails from customers, attendees, management
      • “ Testimony” from sales force, account managers, etc., that a program is gaining acceptance or helping to drive sales
      • Other awards won for the same project
      • Again, context is important.
  • Evaluation, cont.
    • Benchmark and evaluate when you can.
      • Pre- and post-testing and/or surveys
      • Compare to previous year’s results/attendance/sales
      • Compare to industry standards
    • Tell judges what you learned.
      • Don’t be afraid to admit it when not all the news is good
      • You can learn from parts of a campaign that missed the mark
      • Adjust goals for next year/next program
      • Use any negative feedback to improve future efforts
  • Evaluation, cont.
    • Some additional ways to evaluate your efforts:
      • Specific measurement: Things that can be measured quantitatively, such as number of clippings, attendance at an event, percent answering a certain way on a survey.
      • Semi-specific measurement: Items that an experienced professional can get a reading on, but that lack numerical measures, such as the balance of coverage in the media or the audience’s reaction to a speech.
      • Acceptance on basis of judgment: Do complaints overall decrease? Does management believe efforts have been valuable?
      • Recognizing value of input: Have your concepts led to marketing themes that have captured buyers’ enthusiasm? Has the stance of the organization recommended by PR staff been well received by its publics?
      • Prevention: What didn’t happen as a result of public relations advice, such as issues that were blunted before they arose.
      • Guidance: A general absence of issues and alarms may be testimonials to the PR department’s overall efficiency.
  • Set Goals and Evaluate, cont.
    • Cutlip, Center and Broom identify three major categories to use in evaluating your PR campaign.
        • Preparation: Basically, how well did we do in the research and analysis portions of RACE?
        • Implementation: How well did we do with the “C” in RACE?
        • Impact: This takes the overall campaign and our objectives into account
  • Set Goals and Evaluate, cont.
      • Preparation
        • Review our situation analysis and pre-planning research – how well did we understand the situation at the time?
        • Was our message appropriate based on our research?
        • Collect and review all material we produced for the campaign – check accuracy, quality of writing and design, etc.
  • Set Goals and Evaluate, cont.
      • Implementation
        • How many stories were placed?
        • How many people were reached?
        • How many positive stories were generated?
        • Did we get our story out?
  • Set Goals and Evaluate, cont.
      • Impact
        • If benchmark research was done, do another round to see if there was any movement toward our objectives.
        • Show changes in attitudes or behavior.
        • Did sales increase? Did the bond issue pass? Etc.
  • Special Evaluation Methods
    • Certain special evaluation methods are useful when you don’t have the time or money for scientific primary research.
        • Collect and codify results.
          • Checklist of media coverage
          • Number of positive clips
          • RSVPs to invitations
          • Ticket sales
          • Attendance records or sign-in sheets
          • Petitions
          • Letters or e-mails received for and against the issue
  • Special Evaluation Methods, cont.
      • Measure the degree of achievement for each objective.
        • Use before-and-after measurement devices
        • Set benchmarks with some kind of quantifiable research
        • Follow up after the campaign by re-doing the same or similar research
  • Special Evaluation Methods, cont.
      • Accommodate positive as well as negative evaluations.
        • Record what went right and what went wrong
        • Hold a debriefing session with staff or your client
        • Discuss performance on each item
        • Use negatives to plan better for the next campaign
  • Special Evaluation Methods, cont.
      • Collect proposals and projections growing out of the campaign.
        • During the campaign, evaluate achievements so far and make adjustments for the remainder of the campaign
        • Use campaign results to shape future efforts
  • Evaluation: Some Final Thoughts
    • In general:
    • Mix qualitative and quantitative measures. You should measure objectives, but you don’t have to do a scientific survey on everything. The most important thing to remember is to tie your evaluation in to your research.
    • One way to ensure you don’t “forget” about evaluation is to think about how you will evaluate each objective while you are writing it. Thus, in the Analysis portion of your planning grid, when you are writing your LBPTs, ask yourself (and jot down) how you will evaluate the achievement of each behavioral outcome.
  • Q & A
    • Feel free to contact me at Robin.Mayhall@bcbsla.com.