Revisit IGA and bylaws to ensure reflection of current goals Revisit priority pollutant focus and associated behaviors to determine strategies - methodically Employ tools to measure embraced behavior - tracking and evaluate resource expenditures ?
When I think about education and outreach, I consider it across all Minimum Control Measures. IDDE, CSW, PCSW, GHPP
I also keep in my mind the 1999 text in the Federal Register regarding Maximum Extent Practicable. EPA envisioned an iterative process that would take 10-15 years to eliminate water quality impairments caused by MS4s In this frame of mind, education and outreach was never expected to be a permanent effort if MCMs were effective Illicit discharges were reported when found and citizens disposed of all waste properly Construction sites were all maintained properly and inspections were self-conducted to identify corrective actions Maintenance facilities were managed in pristine condition with pollution prevention integrated into all operations Watersheds were planned wisely, all developments included water quality treatment and permanent maintenance
The many unique differences between one program and another were fascinating, but frustrating at the same time Each partnership had its own approach and localized resources to approach education and outreach with One unifying theme – credit Mindy Fohn for illustrating this point for me – effectiveness is best defined by direct observations
If we aren’t clear where the end point might be, perhaps we could trend against categorical expenses incurred by other MS4s Trouble comparing due to expense categories and not tracking all things. Did start to get me thinking about budget amounts and the “knee of the curve” argument for CSO LTCPs exponential output of cost to generate an incremental benefit result put that idea into the parking lot for later use
One of these things is not like the other…stormwater is the only mandated effort required to change social and institutional behaviors.
Man’s impact upon the environment is nothing new. We have always been slow to acknowledge our need to embrace best practices in order to get the most out of the land. Hugh Bennett was the first to definitively communicate to the nation that the Dust Bowl was the result of man’s behavior The impact was wide, personal, and deadly – a consequence we may be to far from to appreciate the correlation to stormwater runoff
source: CNN 8/16/2012 http://www.cnn.com/2012/08/16/business/tobacco-health-warnings/index.html?hpt=hp_c2 Even measuring raised awareness isn’t a very good indicator of good behavior. America has one of the highest levels of national awareness about the health effects of smoking. And yet it continues to be the habit of choice for so many. Awareness does not equal embraced behavior.
Guns kill people and people create pollution The Causes of Pollution Conceptually, pollution is NOT the affect of pollutants. Pollution is the result of the ACT of introducing a pollutant into the environment. Therefore, MS4s should focus education on the pollutant generator. With proper behavior, pollution doesn’t occur and permit is not necessary.
Changing behavior is the underlying principle of any public education and outreach effort. Use a variety of assessment methods to evaluate the effectiveness. Are public education and outreach programs changing public awareness and behaviors?
US Stormwater education focuses on surveys and evaluations, but leave direct measurments off hoping that raised awareness will equate changed behaviors McKenzie-Mhor understands that sustainable behaviors are the result of a targeted outreach effort the starts with direct measurements and is supported by indirect assessments.
Keep It Clean Partnership Measuring Embraced Behavior To Evaluate Effectiveness of MS4 Education and Outreach StormCon 2012 Session P-35
Keep It Clean Partners Boulder County - 22,469 /44,000 City of Boulder – 97,385 Town of Erie – 18,135 City of Lafayette – 24,453 City of Longmont – 86,270 City of Louisville – 18,376 Town of Superior – 12,483
Audiences and Behaviors Adults School Children Construction Businesses Municipal employees and contractors
Partners for a Clean Environment Restaurants Vehicle Service Gas Stations Retail Facility Managers Landscapers Pressure Washers Municipal
Questions about Effectiveness Questions for the Consultant Are KICP shared programs effective? Is KICP program administration efficient? How can KICP improve water quality and measure it?
Recommendations for Effectiveness Update IGA and Bylaws Reinforce watershed approach Leverage brand awareness “Keep it Clean” Define priority pollutants and the behaviors that generate them Measure “Embraced Behavior”
Embraced Behavior Proposal “Be the expert about the best practices.” ~ Greg Davis – EPA Region 8 “Benefits can be monetized for MEP.” ~ John Sorensen, AMEC “Put the “Point” back into Non-Point Source pollution education.” ~ Jesse Poore, Felsburg Holt & Ullevig
The Trouble with “Effectiveness” “I don’t care if my program is effective. I just want to be in compliance.” ~ MS4 Operator If MS4 Education and Outreach Programs are supposed to accomplish “NPDES” (Pollutant Discharge Elimination) … Can we correlate Education to Water Quality?
The Trouble with “Effectiveness” “We are ‘flying blind’ or, probably, by the seat of our pants, when it comes to making decisions about how best to address water quality problems and allocate our limited resources for cleanup, pollution prevention and restoration.” ~ G. Tracy Mehan, former EPA Assistant Administrator
The Trouble with “Effectiveness” “Without assessing the effectiveness of the stormwater management program the permittee will not know which parts of the program need to be modified to protect and/or improve water quality and instead will essentially be operating blindly.” EPA 2010 Permit Improvement Guide
The Trouble with“Effectiveness” FR Vol 65 No. 235 Part II(H)(3)(a)(iii)
The Trouble with “Effectiveness” Peer Research KPCRC, Washington LCSC, South Carolina Mindy Fohn Synithia Williams MRSWMP, California Heidi Niggemeyer
The Trouble with “Effectiveness” MRSWMP - Co-permittees, evaluation structure provided, benchmarks defined KPCRC - State campaign, local economics, focus on observing behavior LCSC - University facilitated education, local input, targets for audience and pollutants
The Trouble with “Effectiveness” Literature Research 2001 Study of Six California MS4 Costs 2010 Tracked KICP Expenditures Category Average KICP Budget $2.5 Million $1.7 Million Management $375,000 $362,000 Education $120,000 $48,500 Proportion 4.0% 2.9% Per Person $0.46 $0.18 Per House $1.24 $0.48
What Gets Measured Matters Typical Permit Language: “Within [insert deadline, e.g., within the permit term], the permittee must assess changes in public awareness and behavior resulting from the implementation of the program such as using a statistically valid survey and modify the education/outreach program accordingly.”
What Gets Measured Matters Permits and guidance typically promote measuring percent change in behavior. BUT Does not correlate behavior to a loading rate Rarely identifies a valid start point or a satisfactory end point Ability to prioritize limited dollars is minimized
What Gets Measured Matters Permit Effectiveness ≠ Behavior Effectiveness Measuring evaluations and surveys against learning objectives ≠ Behavior Effectiveness
Evaluate Cost Effectiveness Recent webcasts provide helpful techniques WEF: Evaluating Stormwater Outreach EPA: Conducting Effective Stormwater Outreach Neither promoted direct measurements to supplement surveys and raised awareness Neither discussed pollutant loading associated with behaviors
Community Based Social Marketing Approach by McKenzie-Mohr and Smith Identify barriers and benefits Define tools to change behavior Pilot the approach Evaluate results Emphasize direct measurements Supported by surveys and focus groups
Important Question #2 “If you can’t establish a load associated with a behavior, how are you going to measure improvement in water quality?” Hint: Urban runoff loading based on behavior is uncharted territory
Behaviors Impact TMDLs # of Restaurants Screened: 38 Number Grease Bins Open: 5 (13.2%) Non-Conforming Bins: 1 (2.6%) No Bin Seen: 4 (10.5%) Surface Area Stain None/Covered: 20 (52.6%) Slight: 10 (26.3%) Significant: 4 (10.5%)
Behaviors Impact TMDLsNone/Covered = X Slight = X lbs./Yr. lbs./Yr . Significant = X Non-Conforming = X lbs./Yr. lbs./Yr.
Embraced Behavior Proposal Establish desired social, institutional behaviors Centralize “Embraced Behavior” conversation Discuss best methods for direct measurement Research pollutant loading associated with undesired behaviors International Stormwater Behavior Database