The 3 Rs of Impact Reporting

280 views

Published on

Professional development training on impact reporting developed for Louisiana Cooperative Extension Service professionals.

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
280
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • It’s that time of year for Louisiana Cooperative Extension Service professionals to complete those important IMPACT reports. Our educational program impact reports help to tell our Extension story to our stakeholders. This training is designed to help you improve these important reports. There is a book – “The Oz Principle” by Roger Connors, Tom Smith and Craig Hickman. “The Oz Principle” addresses how to be more personally accountable. One of the catch phrases of the book is about working “above the line”. To assist you in working “above the line” here is a short lesson on developing better impact reports. Let’s learn about the 3 R’s of Impact Reports.
  • Hello, I am Becky White, a specialist in the Organization Development and Evaluation Unit for the LSU AgCenter. My role is to assist Extension Professionals in program planning, implementation and evaluation. I also help colleagues in developing grant proposals to enhance their Extension programs.Welcome to this online training on the 3 R’s – Telling the Extension Story with Impact Reports. This training should take about 15 minutes to complete. Let’s get started.
  • There are three primary components to our Extension impact reports, therefore to develop an impact report involves a 3-step process. The steps in the process or the components in our impact reports are: relevance, response and results.Relevance – refers to the issue and why it is important to address.Response – refers to how your Extension program responded to the issue of importance.Results – refers to what difference the Extension program made to the lives of people.So keep in mind this simple formula when writing your impact statement:RELEVANCE + RESPONSE = RESULTS or IMPACT
  • The 1st “R” in telling your Extension story is developing the Relevance Section of your Impact Report.
  • In addition to addressing the reason why the issue was addressed you can also describe the situation surrounding the issue.This section of the impact report should be limited to 3-5 sentences. Our impact reporting system suggests that you keep this section from 100-150 words long.
  • The RELEVANCE section includes four to five sentences that provide—A definition of the importance of the issueA description of how the issue was identifiedAn explanation of how education helps to address the issueIdentification the target audienceclear program goal or goals
  • Let’s look at an example of a relevance section on an impact report.The 2011 Ag Summary reports the gross farm value of soybeans in XYZ Parish was $15.5 million. Data in the LA Ag Summary has shown a steady decrease in the acreage of soybeans produced in XYZ Parish over the last 5 years. Reducing input costs by educating producers on ways to improve efficiency can help them stay in business. One hundred twenty soybean producers in XYZ Parish were targeted for an educational outreach program. The goal of the program was to for XYZ Parish soybean farmers to adopt recommended farm practices to achieve sustainable management of their soybean resources and protect the environment.
  • The 2nd “R” in telling your Extension story is developing the Response Section of your Impact Report.
  • In this section you describe what you did in your educational program that addressed the issue featured in the relevance section. You should:briefly list and/or describe the activities, events and other teaching methods that you used. (examples are club meetings, field days, workshops, camps, seminars, trainings, demonstrations plots, fair/festival events, information thru news articles, social media, TV, and websites, etc.)Tell when and where the activities and events were conductedIdentify any partners and collaborators (include local, regional and state agencies, organizations, groups, and entities as appropriate) This section may be narrative or bulleted.Include any acknowledgements that are appropriate. This might include Extension specialists or special program presenters. Also recognize involved Extension volunteers and any funders for the program.Our Impact Reporting System suggests that you keep this section from 150-200 words long.
  • Let’s review an example of a Response Section. A multi-faceted educational program was conducted targeting 120 soybean producers in XYZ Parish from 2008 - 2011. The program included :Series of six presentations were given at each annual soybean field day held in northeast Louisiana on soybean production best practicesMonthly newspaper column in the ABC Gazette addressing current farm production issues. Readership is 3,000.Educational exhibit at the State Farm Bureau convention held in XYZ Parish in June 2010Field demonstrations at eight cooperating farms Virtual seminar in Spring 2009 on Farm Financial ManagementDevelopment of a soybean production website in 2008360 farm visits - 40 on nutrient management/water quality issuesAnnual pesticide safety seminars Partners in this educational endeavor included Farm Bureau, the Soybean Growers Association, the ABC Gazette and local aerial applicators.
  • The 3rd and final “R” in telling your Extension Story is developing your Results Section of your Impact Report. The results section of an impact report is the toughest and most critical piece! A strong results section requires that as an extension professional you conduct an excellent program evaluation.
  • For this section you must describe:how the effectiveness was evaluated andthe changes that occurred as a result of the educational program To do this:explain the evaluation or assessment strategy in 1-2 sentences.indicate the response rate in 1 sentence.list significant results in order of importance including both numbers & percentages in either a bulleted list or narrative format and finallysummarize the results. If feasible, include any economic benefit.Lets review each of these steps in more detail.
  • The beginning of the results section should start with your evaluation strategy. This should be 1-2 sentences long and explain how you expected to know if impact or outcomes were achieved. Here are some examples of evaluation strategies in different Extension program scenarios.Pre- and Post-tests were used to assess knowledge changes regarding fruit and vegetable consumption with 608 5th graders who completed the series.The physical activity logs of 587 5th graders were analyzed to determine physical activity patterns.A survey was conducted at field days to gather benchmark data regarding soybean production practices using a 20-item instrument.
  • You need one sentence about your response rate. Here are two examples.A total of 608 of 780 (78%) 5th graders who participated in the program completed both the Pre- and Post-test. 80% of the producers in attendance at the field days (n=124) completed the post-program survey.Note: If you are going to use percentages, be sure to use actual numbers, too.
  • When revealing your results of your program evaluation be sure to list them in the following order of importance:Begin with any changes in behavior - Documented behavior change. You might also include intent to change behavior. Next indicate any learning that occurred – knowledge gained, skills acquired, attitudes changedAlso remember to order the responses with the most impressive first. For example with behavior changes list the most significant or impressive change first.Mention of any special demographic information comes next followed byParticipants’ satisfaction with the program.Then you can close with what’s next for the program
  • 84% of the 608 5th grade students who completed the program tasted at least one new fruit and one new vegetable during the program.559 of 608 (92%) participants increased their number of minutes of physical activity by 30 minutes on 5 of 7 days.61% of 430 program participants scored higher on subsequent standardized tests in science mastery.
  • Over ½ (58%)of the students in the program indicated they would choose fresh fruit or vegetables over chips for a snack. 118 of 124 respondents (95%) indicated they intended to implement at least one new BMP.30% of those who participated in the SET program indicated they would choose a more science-oriented career as a result of participating in the program.
  • 492 out of 608 (81%) learned the number of servings of fruits and vegetables they should eat each day.54% of producers indicated that they understood proper spray conditions at a good or excellent level before the program and 93% said they understood proper spray conditions at a good or excellent level after the program.72% of students reported that they were less afraid of science after the SET program than they were before.
  • Your results section can also include a describe of yourprogram participants.You might feature:AgeGenderIncome levelsYears farming or as a 4-H memberArea (parish, region, state, etc.)Size of herd or acreageEducational level
  • Many agents ask participants their view of the program or put another way, their satisfaction level with the program. While some of this is important, be careful not to be “satisfied” with it as THE results of your program. Results should be about participant changes in knowledge, skills, attitude and aspirations, or better yet, about changes in human behavior or adoption of recommended practices. Some examples of reporting regarding program satisfaction are:100% of the respondents stated they were mostly or completely satisfied with the information being easy to understand.579 of 608 (95.24%) of respondents stated they were mostly or completely satisfied with the helpfulness of the information in making decisions about their own situation.
  • Results indicate that the Childhood Obesity Prevention Program in XYZ Parish met its objectives of increasing the consumption of fruits and vegetables and increasing physical activity in 5th grade students. The greatest behavior changes were associated with trying new fruits and vegetables and becoming more physically active. This program will be expanded next year to include 6th and 7th grade physical education teachers and students.
  • You may want to add sentences about how these results will be beneficial in the future.As we continue to address the childhood obesity issue in XYZ County, this program has helped to guide future program efforts. We now know that the program is effective and can be used with other youth to improve overall physical health and reduce the incidence of obesity-related chronic diseases later in life. This program will be expanded next year to include 6th and 7th grade students.
  • A survey was sent to 120 farmers participating in the program to measure programmatic impact using a 20-item instrument. Ninety-six (80%) of the producers completed the survey. Seventy-two (75%) of the 96 soybean producers surveyed adopted at least two of ten new management practices as a result of the program. Eighty respondents (83%) indicated they intended to implement at least one new BMP in the future.Eighty-nine (93%) said they understood proper spray conditions at a good or excellent level after the program.
  • Sixty (63%) of the respondents had participated in the Master Farmer program in previous years; eight participated during (2014)Eighty-seven (91%) of respondents stated they were mostly or completely satisfied with the timeliness of information. Results indicate that the Soybean Producers Extension Program in XYZ Parish met its goal of farmers adopting recommended farm practices to achieve sustainable management of their soybean resources and protect the environment. The greatest behavior change was farmers adopting recommended Extension soybean management practices. This results section was 160 words long.
  • The next two slides show a two-page Impact Report Worksheet for you to use in developing your program impact report. It walks you through each component of our Impact Report suggested format step by step.
  • The second page of the impact report worksheet features your results section.In addition to this worksheet, I have developed examples of impact reports for a typical FCS, 4-H and Agriculture program. You can obtain these samples from our Organization Development and Evaluation unit site within the LSU AgCenter’s intranet sharepoint website
  • Some additional tips in developing your Impact Report include:Proofread the reportCheck your statsDon’t use jargonFormat appropriatelyWrite for your target audienceSubmit Report regularly & use multiple modes
  • Well that’s it folks! Today we have covered the 3 R’s for developing Impact Reports. The 3 R’s are relevance, response and results. Each of these sections will comprise your complete Impact Report. It’s important to develop impact reports to demonstrate our accountability and to help people understand Extension.
  • Mastering Impact reports can be as easy as 1 - 2 – 3! Impact reporting is an important step in being accountable to our program stakeholders. I hope this brief training video helps you improveyour impact reports.
  • The 3 Rs of Impact Reporting

    1. 1. Cooperative Extension Professionals Working Above the Line: The 3 R’s – Telling the Extension Story with Impact Reports
    2. 2. Rebecca White Professor Organization Development & Evaluation
    3. 3. RELEVANCE RESPONSE RESULTS
    4. 4. The 3 R’s - Telling the Extension Story Step 1 – Developing Your Relevance Section
    5. 5. • The reason why and the situation • 3-5 sentences • Limit to 100-150 words
    6. 6.  Includes 4-5 sentences: 1. Importance of the issue 2. How issue was identified 3. How education can help 4. A target audience 5. Clearly stated program goal(s)
    7. 7. Relevance Example The 2011 Ag Summary reports the gross farm value of soybeans in XYZ Parish was $15.5 million. Data in the LA Ag Summary has shown a steady decrease in the acreage of soybeans produced in XYZ Parish over the last 5 years. Reducing input costs by educating producers on ways to improve efficiency can help them stay in business. One hundred twenty soybean producers in XYZ Parish were targeted for a four year educational outreach program. The goal of the program was to for XYZ Parish soybean farmers to adopt recommended farm practices to achieve sustainable management of their soybean resources and protect the environment. (105 words long)
    8. 8. The 3 R’s – Telling the Extension Story Step 2 – Developing Your Response Section
    9. 9.  Describe what you did  List educational methods used  Note when and where activities were conducted  Identify partners and collaborators  May be narrative or bulleted  Include acknowledgements  Limit to 150-200 word
    10. 10. A multi-faceted educational program was conducted targeting 120 soybean producers in XYZ Parish from 2008 - 2012. The program included :  Series of six presentations were given at each annual soybean field day held in northeast Louisiana on soybean production best practices  Monthly newspaper column in the ABC Gazette addressing current farm production issues. Readership is 3,000.  Educational exhibit at the State Farm Bureau convention held in XYZ Parish in June 2010  Field demonstrations at eight cooperating farms  Virtual seminar in Spring 2009 on Farm Financial Management  Development of a soybean production website in 2008  360 farm visits - 40 on nutrient management/water quality issues  Annual pesticide safety seminars Partners in this educational endeavor included Farm Bureau, the Soybean Growers Association, the ABC Gazette and local aerial applicators. (137 words)
    11. 11. The 3 R’s – Telling the Extension Story Step 3 – Developing Your Results Section
    12. 12.  Describe: 1. How effectiveness of the program was evaluated  Explain evaluation or assessment strategy (1-2 sentences)  Indicate response rate (1 sentence) 2. Changes that occurred as a result of the program  List significant results in order of importance, including both numbers & percentages 3. Summarize results
    13. 13.  Pre- and post-tests were used to assess science investigatory and critical thinking skills of 120 eighth grade students.  The physical activity logs of 587 fifth graders were analyzed to determine physical activity patterns.  A survey was conducted at field days to gather benchmark data regarding soybean production practices using a 20-item instrument.
    14. 14.  A total of 608 of 780 (78%) fifth graders who participated in the program completed both the pre- and post-test.  80% of the producers in attendance at the field days (n=124) completed the post-program survey.  Note: If you are going to use percentages, be sure to use actual numbers, too.
    15. 15.  Behavior - Documented behavior change, intent to change behavior  Learning - Knowledge gained, skills acquired, attitudes changed  Order responses  Demographics  Satisfaction  What’s next
    16. 16.  84% of the 608 fifth grade students who completed the program tasted at least one new fruit and one new vegetable during the program.  559 of 608 (92%) participants increased their number of minutes of physical activity by 30 minutes on 5 of 7 days.  Approximately three-fourths (76%) of soybean producers surveyed adopted at least two new management practices as a result of the program.
    17. 17.  Over ½ (58%)of the students in the program indicated they would choose fresh fruit or vegetables over chips for a snack.  118 of 124 respondents (95%) indicated they intended to implement at least one new BMP.  30% of those who participated in the SET program indicated they would choose a more science- oriented career as a result of participating in the program.
    18. 18.  492 out of 608 (81%) learned the number of servings of fruits and vegetables they should eat each day.  54% of producers indicated that they understood proper spray conditions at a good or excellent level before the program and 93% said they understood proper spray conditions at a good or excellent level after the program.  72% of students reported that they were less afraid of science after the SET program than they were before.
    19. 19.  Describe program participants  Age  Gender  Income levels  Years farming or as a 4-H member  Area (parish, region, state, etc.)  Size of herd or acreage  Educational level
    20. 20.  100% of the respondents stated they were mostly or completely satisfied with the information being easy to understand.  579 of 608 (95.24%) of respondents stated they were mostly or completely satisfied with the helpfulness of the information in making decisions about their own situation.
    21. 21. Results indicate that the Childhood Obesity Prevention Program in XYZ Parish met its objectives of increasing the consumption of fruits and vegetables and increasing physical activity in 5th grade students. The greatest behavior changes were associated with trying new fruits and vegetables and becoming more physically active. This program will be expanded next year to include 6th and 7th grade physical education teachers and students.
    22. 22.  Note how these results will be beneficial in the future  As we continue to address the childhood obesity issue in XYZ County, this program has helped to guide future program efforts. We now know that the program is effective and can be used with other youth to improve overall physical health and reduce the incidence of obesity-related chronic diseases later in life. This program will be expanded next year to include 6th and 7th grade students.
    23. 23. A survey was sent to 120 farmers participating in the program to measure programmatic impact using a 20-item instrument. Ninety-six (80%) of the producers completed the survey.  Seventy-two (75%) of the 96 soybean producers surveyed adopted at least two of ten new management practices as a result of the program.  Eighty respondents (83%) indicated they intended to implement at least one new BMP in the future.  Eighty-nine (93%) said they understood proper spray conditions at a good or excellent level after the program.
    24. 24.  Sixty (63%) of the respondents had participated in the Master Farmer program in previous years; eight participated during (2014)  Eighty-seven (91%) of respondents stated they were mostly or completely satisfied with the timeliness of information. Results indicate that the Soybean Producers Extension Program in XYZ Parish met its goal of farmers adopting recommended farm practices to achieve sustainable management of their soybean resources and protect the environment. The greatest behavior change was farmers adopting recommended Extension soybean management practices. (160 words long)
    25. 25.  Proofread the report  Check your stats  Don’t use jargon  Format appropriately  Write for your target audience  Submit impact report regularly  Share impact reports with stakeholders
    26. 26.  Write toward the Three R’s  Relevance  Response  Results
    27. 27. Mastering Impact Reports Can be as easy as 1 – 2 - 3
    28. 28. Rebecca White Professor Organization Development & Evaluation

    ×