MEEA Technical Webinar: A New Approach to Estimating Free Ridership in Upstream Lighting Programs


Published on

One of the most challenging evaluation questions for residential lighting energy efficiency programs in the U.S. is the identification and correction for net-to-gross (NTG) effects such as free ridership and spillover. Over the last twenty years, considerable effort and financial resources have been directed toward accurately measuring these effects. Furthermore, the correction for these NTG effects has direct, and sometimes, drastic impact on program savings.

APT and Opinion Dynamics discuss a new framework for the estimation of free ridership in upstream lighting programs grounded in sound, economically rational decision making on the part of retail partners. This approach, built on functional retail behavior, provides a clearer more insightful look into the elements comprising the retail sales environment thus providing program implementers with a more predictable outcome of end results – up front.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

MEEA Technical Webinar: A New Approach to Estimating Free Ridership in Upstream Lighting Programs

  1. 1. MEEA Technical Webinar Series:A New Approach to Estimating Free Ridership for Upstream Lighting Programs Presenters: Stan Mertz - Applied Proactive Technologies & Tami Buhr - Opinions Dynamics Thursday, October 18th, 2012
  2. 2. MEEA’s Role in the Midwest• Nonprofit serving 13 Midwest states• 10+ years serving states, energy offices, utilities and communities• Staff of 25 in Chicago• Actions – Designing & Administering Energy Efficiency Programs – Evaluating & Promoting Emerging Technologies – Regional Voice for DOE/EPA & ENERGY STAR – Coordinating Utility Program Efforts – Delivering Training & Workshops – Advancing Energy Efficiency Policy – Promoting Best Practices
  3. 3. REVENUE NEUTRAL MODEL A New Approach to Estimating FreeRidership for Upstream Lighting Programs October 2012
  4. 4. Agenda1. Challenges associated with estimating lighting program NTG2. Theoretical background underlying Revenue Neutral Sales Model3. Example of how the model works4. Questions Revenue Neutral Model 2
  5. 5. Lighting Program EvaluationChallenges and Traditional Methods Revenue Neutral Model 3
  6. 6. Estimating Lighting Program Free Ridership is Challenging• Programs usually delivered through an upstream markdown method• Customers purchase discount lighting, walk out the store and disappear • Often unaware of the discount • Note that this does not mean customer would have purchased bulbs at full price• Retailers are fully aware of their participation• Traditional methods use data from both customers and retailers to estimate free ridership Revenue Neutral Model 4
  7. 7. What Evaluators Really Want: Full Sales Data• Data that tracks sales of EE lighting with and without program pricing would provide the best estimate of program impact • Could see the actual lift in sales when program in effect• Depending on how long program has been running, measurement approaches could include: • Pre-program sales, sales when programs turn pricing on and off, sales of like products that are not discounted, sales in comparison areas that do not have programs• Unfortunately, retailers will not provide sales of EE lighting at regular price. Will only provide sales of products lighting programs discount. Revenue Neutral Model 5
  8. 8. Sales Data Alternatives for Estimating Free Ridership• Participant self-reports • Common method for estimating program free ridership for rebate programs • Difficult with upstream programs where customers purchase program incented product and disappear• Two survey methods commonly used: • General population telephone surveys • Call utility customers and ask detailed questions about past lighting purchases • Results are of questionable validity due to timing of survey , small nature of purchase, and difficulty identifying program purchasers • In-store customer interviews • Interview customers in store immediately after they make purchase decision • Greater confidence in self-report results • Usually make use of non-probability samples • Expensive and challenging to conduct Revenue Neutral Model 6
  9. 9. Sales Data Alternatives for Estimating Free Ridership (2)• Retailer Interviews • Conduct interviews with corporate or store level retailers and ask for estimate of program impact on sales ti t f g i t l • Are no more likely to give up the numbers in an interview than a request for data. • Store level staff often do not know sales • Corporate level do not know for a specific utility territory • At best, get rough estimates • May have vested interest in seeing programs continue• Modeling Techniques • Many require use of self-report data in addition to other data (e.g. multi- state model revealed preference models) model, • Suffer from the same validity problems Revenue Neutral Model 7
  10. 10. Another Look at Sales Data• Existing FR methods are challenging, expensive and produce questionable results• We keep going back and asking for complete sales data. • Maybe if we ask again, or ask nicer, or ask a different person, we’ll get it.• We have asked everyone for sales data including program implementers • Implementers also h l evaluators g t access t stores t conduct intercept I l t l help l t get to t to d ti t t interviews• We started talking about the challenges associated with our existing free ridership methods and alternatives p• Is there something we can do with the sales data we do have? • Have program sales data. Also have program prices, regular prices, and sales goals for each retailer for each product• Estimate non-program sales using data we do have and a model of retailer behavior Revenue Neutral Model 8
  11. 11. Revenue Neutral ModelThe Alternative Sales Data Method Revenue Neutral Model 9
  12. 12. Retailer Behavior• Retailers will only participate in utility lighting programs if their participation is revenue neutral • Their “top line sales” remain the same or increase; cannot decrease• But why are “top line sales” so important to a retailer?• How exactly do retailers factor topline sales into their decision to participate in utility programs? Revenue Neutral Model 10
  13. 13. What is Topline Sales?• “Top Line Sales” is a reference to the gross sales or revenues of a company.• The "top" reference relates to the fact that on a companys income statement, the first line at the top of the page is generally reserved for gross sales or revenue. • Program reimbursements for sales are not included in revenue• A company that increases its revenues is said to be "growing its top line", or "generating top-line growth".• This contrasts with net income (or net earnings per share), which is usually the bottom line of the companys income statement. Revenue Neutral Model 11
  14. 14. Topline Sales Impact Example• Retailer must sell a minimum of 254 additional units just to get back to $697 total sales dollars generated before program $697.00 $697.00 Incremental  Top Line  sales lift Sales $  Sales $ +254 units +254 units Generated $197.00 Regular Retail $6.97 R l R il $6 97 Discount $5.00 100 Units  100 Units  354 Units  Promo Retail   $1.97 sold sold sold Revenue Neutral Model 12
  15. 15. Gross Margin Impact Example• Retailer is made whole on the Gross Margin line after discounts have been reimbursed on first 100 units and all additional units provide incremental GM $ Growth. G th• Unfortunately, the rebate dollars are not able to be credited to Top Line Sales dollars. Incremental  Incremental Gross  Gross  Margin $  Margin lift  Generated on +254 units 100 Units  If only 100  354 Units  sold before  Units sold  sold Program during  Program Revenue Neutral Model 13
  16. 16. Retailer Decision Tree: Year 1 (SKU Level) Decline to participate OR Less than  Revise Strategy of Incentives to  gy prior to  i t meet Necessary Revenue Dollar  How many units can  Program level be sold in the  Promotional time  p period at the  reduced price? Same or Above  prior to Program Same Above Participate at Participate at Agreed Upon Incentive  Agreed Upon Incentive level  level and Allocation  and Negotiate Additional  Amount Allocation Amount Revenue Neutral Model 14
  17. 17. Retailer Decision Tree: Additional Years (SKU Level) Was the  No Discontinue  Previous Year  OR  promotion revenue  promotion revenue Participate with Revised Incentive neutral or better? Discontinue  OR Yes Less than  Revise Strategy of Incentives to  Previous Year meet Previous Year Revenue or  How many units can  Additional Promotion Opportunity be sold in the  Next Year? Same or Above  Previous Year Same Above Continue at Continue at  Current Incentives  Current Incentives  and  d and Negotiate  d Same Allocation Additional Allocation Revenue Neutral Model 15
  18. 18. Model Implementation Revenue Neutral Model 16
  19. 19. Required Data• Need program tracking data at sku level for: • Sales goals • MSRP • Incentive amount (and any changes in incentive amount over the promotional timeframe) • Program sales• Use the first three to estimate sales without program discount pricing• Use program sales to calculate free ridership Revenue Neutral Model 17
  20. 20. Model Implementation• Can estimate the program’s likely free ridership before the program year• Likely free ridership is what the program will receive if each sku meets its sales goal and the program does not allow sales to exceed goals• Actual free ridership is based on final program sales • Will be higher for skus that do not meet sales goal • Will be lower for those that exceed goals Revenue Neutral Model 18
  21. 21. Use in Actual Evaluations• Have used the model in two evaluations so far• Results are not yet publicly available• Used multiple methods for one evaluation and all came up with similar p p numbers• We are pursuing a patent on the model. Revenue Neutral Model 19
  22. 22. Model Benefits• Can calculate free ridership by • Bulb type (standard, specialty, fixtures, CFLs, LEDs) • Retailer • During special promotions compared to regular program pricing• None of the existing lighting free ridership methods provide this level of detail Revenue Neutral Model 20
  23. 23. Program Design• Can use results to identify changes to program implementation to minimize free ridership within existing program budgets.• Tailor the product mix by bulb type• Vary incentive levels by retailer and bulb type • Some retailer and bulb types have higher FR rates • Need greater incentives to encourage purchase by non-free riders• Maximize incentive dollars by matching incentive amount to bulb and retailer types Revenue Neutral Model 21
  24. 24. Questions or Comments? Tami Buhr Director of Survey Research Opinion Dynamics 617-301-4654 Stan Mertz Director of Retail Operations Applied Proactive Technologies 413-731-6546 413 731 6546 Revenue Neutral Model 22
  25. 25. The EE Story (Future)• Future: Finding a new portfolio – Lighting savings going down – Some program saturation – Need ‘new’ programs • Whole home (HPwES, air sealing, etc) • Systems work (HVAC systems, smart homes, etc) • Behavior programs (changing the customer habit) • Education • Building Energy Codes (adoption, training and compliance) – Challenges • Cost effectiveness (non-energy benefits not counted) • More complex (contractors, systems, etc)
  26. 26. Presenter Contact Information Stan Martz, Applied Proactive Technology – Tami Buhr, Opinion Dynamics –