This document discusses the challenges of evaluating, monitoring, and verifying energy savings from energy efficiency programs as investment in these programs increases. It outlines differing views on the current state of EM&V and challenges such as estimating savings in the absence of a direct measurement and accounting for factors like free riders. It recommends improving transparency, increasing consistency between states and programs, and developing regional and national EM&V protocols and standards to enhance the credibility and reliability of energy savings measurements.

1. Alliance to Save EnergyApril 2010Scaling-Up EE Programs: The Measurement Challenge

2. Project ReportScaling-Up Energy Efficiency Programs: The Measurement ChallengeSupported by a U.S. Department of Energy State Energy Program–Special Projects grant through the Penna. Dept. of Environmental Protection.Additional support from the Kresge Foundation and the Sea Change Foundation.Written by Joe Loper, Steve Capanna, Rodney Sobin & Tom SimchakCompanion report: Energy Savings Credits: Are Potential Benefits Being Realized?

3. OverviewEvaluation, Monitoring & Verification (EM&V)Growing resources and requirements for energy efficiency (EE)Growing need for EM&V to determine if savings are real“Schools of thought” on current state-of-the-art EM&VChallenges to effective EM&VOvercoming challenges: Transparency and CredibilityToward national protocols and standardsRecommendations

4. Growing Focus on EEAmerican Recovery and Reinvestment Act (ARRA aka Stimulus Act) ~$25 billion for EEGrowing state requirements (over 20 states)Energy Efficiency Resource Standards (EERS)Renewable Electricity Standards (RES) that include EE componentEE performance-based utility compensation incentives (e.g., California $100s of millions at stake)Potential for federal EERS, RES, EE carbon offsets

5. EM&V: Why Measure?EM&V is needed to document and ensure energy savings and associated goals (e.g., emissions avoidance) 1. Are we getting our money’s worth from EE investment? 2. Are we achieving other objectives of EE?Environmental benefitsEnhanced electric grid reliabilityPositive economic and employment impacts

6. EM&V Schools of Thought Key Stakeholders Hold Differing Opinions on Current EM&V, including:EE EM&V today is imperfect but it is getting betterEE EM&V is unreliable and can’t be improvedEE EM&V is imperfect but so is EM&V in other areas (health, crime, safety, poverty reduction, education, defense); don’t hold EE to a higher standardEE EM&V is imperfect, but no good alternatives and we have to do more given growing investment in EE

7. Challenges to Effective EM&VFundamental EM&V challenge is how to estimate energy not consumed that would have been otherwise—the counterfactualCan’t use a meter to measure energy not usedNumerous additional challenges to estimating energy savings and to attributing or crediting savings to programs

8. Challenges to Effective EM&VBaseline adjustments.How much did EE measures affect energy use in contrast to other factors (weather, changed business hours or building occupancy or production levels)? Were savings “additional” (due to the program or measure) or “business-as-usual” improvements in performance?Interactions of EE measures.E.g., compact fluorescent lamps (CFLs) emit less heat than incandescent lamps—reduce air conditioning load in summer but may increase heating need in winterPersistence of measuresAre measures properly installed, operated, and maintained?

9. Challenges to Effective EM&VDetermining Impact(s) of “free riders”EE measures may be implemented even without the program; did the program cause savings above BAU?Determining “spillover effect”A program may induce EE by those not in the program (additional EE)Impact of BehaviorSome participants may do additional EE due to increased awareness and sensitivity; others may leave efficient lights, TV, etc. on longer

10. Challenges to Effective EM&VPrograms may interact or indirectly induce EEMultiple federal, state, local, utility, and retail marketing programs may interact—tax breaks, rebates, other subsidies, discounts, information—which programs get credit for savings?It is particularly difficult to attribute specific energy savings to information and education programsCost of EM&VCheaper to use “deemed” or assumed values than actually measure energy use for standardized projectsCustomized projects are more complicatedCost of measurement, sampling, surveying, etc. and tradeoffs with accuracy and precision of measurement

11. Challenges to Effective EM&VExisting state technical resource manuals vary greatly in assumptions and, thus, in savings calculated for a given measure or projectVarying assumptions, definitions, methodologiesInconsistent EM&V can harm credibility of savings and associated (environmental, grid impact) claimsInconsistent EM&V can devalue and reduce the credibility of trading programs for energy savings credits and carbon offsetsSee companion report: Energy Savings Credits: Are Potential Benefits Being Realized?

12. Challenges to Effective EM&V: ExampleDeemed values in state technical resource manuals can vary widely:15 W CFL in living room, rated lifetime 10,000 hrs.

13. Overcoming Challenges: Transparency and CredibilityProcess important for transparency and credibilityEERS compliance and performance-based compensation creates vested interestsStakeholders questioning/debating EM&V assumptions, data, and methods can enhance transparencyTransparency can enhance credibility of EE programs if questions and criticisms are addressed and EM&V improvedIf not, credibility of EE programs may be compromised

14. Overcoming Challenges:Toward regional & national protocolsEM&V Protocols and Processes May be Impacted/Improved by:Growing scope of EERS and EE in state-imposed RES requirementsPotential interstate trading of ESCs directly or as components of RES or as carbon credits and offsetsNortheast Energy Efficiency Partnerships (NEEP) EM&V Forum—work on regional protocol developmentNorth American Energy Standards Board (NAESB) —EE EM&V task force development of model EM&V business practicesNational Action Plan for Energy Efficiency (NAPEE) —EM&V studies and tech assistance to states

15. RecommendationsTo improve credibility and reliability of EM&V government and stakeholders should:Use processes for EM&V design and review that promote transparency and promote debate/consensus on methods, data, and assumptionsImprove EM&V methods, data, and assumptionsIncrease consistency of methods and assumptions between states, regions and program typesAssure evaluation practitioner competency and integrity.*Manage stakeholder expectations—recognize what EM&V can and can’t provide, and at what price*The Efficiency Valuation Organization and Association of Energy Engineers developed relevant training and credential based on the Int’l Performance M&V Protocol (IPMVP).

16. For more information, visit the Alliance to Save Energy websitehttp://www.ase.orgor contactRodney SobinSenior Policy ManagerAlliance to Save Energy202-530-2234rsobin@ase.org