Research to Inform Design of Residential Energy Use Behavior Change Pilot

Research to Inform Design of
Residential Energy Use Behavior
Change Pilot
Conservation Improvement Program
Discussion Hosted by the Minnesota Office
of Energy Security

July 21st, 2009

Ed Carroll
Mark Brown

Flow of Discussion
Why Consider Behavior Change?
Project Overview and Research Approach
Behavioral Change Interventions Overview
Literature R i
Lit t Review
Utility Experiences with Behavior Change Pilot Programs
Cost Effectiveness and Applicability to MN
Key Lessons Learned – Utility Manager Perspectives
Pilot Models to Consider
Q&A

Why Consider Behavior Change?
Valuable tool to help you meet your CIP program goals
Utility experiences show that interventions can lead to significant
energy savings:
2% to 7% savings for program participants in the first year
Traditional product-based prescriptive and custom incentive programs
product based
may be inadequate and provide diminishing returns:
“We realized we would be unable to meet our energy savings
targets for the residential sector with purely a product based
product-based
approach.” – Program Manager, Seattle City Light
“The lighting program we have here is so successful that we are
really going t run out of potential i th next f
ll i to t f t ti l in the t few years. It is by f
i b far
the program that results in the biggest savings. It is sort of like the
linchpin will be gone.” – Program Manager, SMUD
Cost effectiveness as low as 3¢ per kWh in first-year savings

Project Overview
OBJECTIVE: to provide the Minnesota Office of Energy Security (OES)
and utilities a solid plan for piloting residential energy use behavior
change programs as part of their CIP efforts
GOAL: to help Minnesota utilities better understand how to accelerate
energy savings resulting from changes in residential energy-use
behavior
ACTIVITIES: (completed Nov. 2008 to Apr. 2009):
Collected data and analysis from available published research
Interviewed experienced program managers, consultants, and
researchers
Identified b t
Id tifi d best practices and lessons learned f
ti dl l d from studies and
t di d
pilots aimed at addressing consumer behavior
Developed a report p
p p providing recommendations for utility p
g y pilot
programs applicable to Minnesota’s residential market

Information Resources
Interview Respondents Published Research Resources

Utilities/Administrators: ACEEE
• Austin Utilities (K. Lady)
EPRI
• Baltimore Gas & Electric (R. Kiselewich)
• BC Hydro (A. Korteland) Precourt Institute for Energy
• Connexus Energy, MN (B. Sayler) Efficiency (Stanford Univ.)
• City Utilities, MO (C. Shaefer)
• Energy Trust of Oregon (K. Youngblood) Environmental Change Institute (UK)
• Pacific Gas & Electric (J Medvitz)
(J. BEHAVE Program (Europe)
• Silicon Valley Power (L. Brown)
• SMUD, Sacramento (A. Crawford) CIEE (California Institute for Energy
and Environment)
Consultants/Vendors:
• Comverge/ComEd (K. Papadimitriu) Journal P bli ti
J l Publications:
• The Brattle Group (A. Faruqui) • Energy Efficiency
• Paragon Consulting (B. Jackson) • Energy Policy
• Positive Energy (A. Laskey) • Journal of Environmental Psychology
• Van Denburgh Consulting (E. Van Denburgh)
g g( g ) • Journal of Environmental Systems
y

Researchers:
• Energy Center of Wisconsin (I. Bensch)
• FSEC - Florida (D. Parker)
• Org. for Energy and the Environment – the
Netherlands (H. van Elburg)

Behavior Change Theory
Behavior Change Impact of Feedback
Decision Making
• Identifies cost of
Realize that there is a problem behavior or deviation
from peers

Habitual Behavior Realize relevance of behavior
to problem • Indicates the impact of
specific behavior
Realize
R li possibilities t
ibiliti to changes
influence problem
• Turning on/off lights
• Use of appliances Weigh motives:
• S tti the thermostat
Setting th th t t • Personal norms
• Social norms • Can frame behavior in
• Other motives (e.g., comfort) terms of cost ($) or
Challenges impact on the
Electricity: environment
• E bli product
Enabling d t Evaluate conflicting motive

• Low-involvement
• Intangible Take action • Repetitive prompts help
to form new persistent
• Dissatisfier
habits
• Low cost priority

Monitors
Categories of Interventions Reports
We see three distinct behavioral change program categories: Rates

In-home devices and displays providing feedback
• Real-time feedback on energy use and costs
• Devices interface with utility electric meter or through CT
clips installed at electric panel
• Examples: PowerCost Monitor, Kill-A-Watt, TED

Customized, regular feedback delivered to consumers
• Processed feedback via mailed reports or online interface
• Opportunity to incorporate comparative data/feedback
pp y p p
• Examples: Positive Energy, BC Hydro’s Team Power
Smart

Dynamic pricing / rate designs (e.g., smart metering)
D i i i t d i ( t t i )
• Protocols that allow for different rates to be charged
based on time of use
• Enabled by advanced metering infrastructure and two- two
way communication between the utility and customer

Monitors (Direct Feedback)
Pros:
PowerCost Monitor • Users able to receive real-time feedback from their meter via a
from Blue Line Innovations mobile monitor.
• Real-time feedback allows users to experiment and see the impact
of their behavior
• Multiple utilities have demonstrated the savings achieved by
customers using these devices
• NSTAR: 3% annual energy savings in ongoing pilot
• Hydro One: 6.5% annual savings in a 500-home pilot
Click to Launch Video
• Dominion: 6% saving in non-electric water heat homes
Cons:
• Opt in nature of programs (e.g., soliciting customers to install
Opt-in (e g
devices) leads to low adoption rates and limited scale
The Energy Detective
from Energy, Inc. • Low willingness to pay relative to device cost
• Significant drop-out rates among participants as the novelty of the
device wears off, monitors are put away, or batteries die
• Questions about persistence of savings and cost effectiveness of
the $130+ devices
• Data capture reliability and resolution raises concerns
Click to Launch Video #2 • Compatibility issues with meter/panel designs and interface

Reports (Indirect Feedback)
Pros:
• Opt-out (vs. opt-in) nature allows utilities to design and conduct
rigorous large-scale pilots and implementation for entire populations
in desired segments
• P id comparative f db k showing a customer’s performance
Provide ti feedback, h i t ’ f
relative to their neighbors; power of social norms
• Customized reports based on housing, demographic, and
psychographic factors to maximize appeal
• Can operate with or without in-home devices and AMI
• Cost effectiveness of savings achieved:
• 3¢ per kilowatt hour in first year (Positive Energy at SMUD)

Cons:
• Will not match the real-time and (unless coupled with AMI-enabled
technology) use-specific feedback that in-home devices provide
gy) p p
• Utilities must be careful in targeting and crafting their messaging in
order to minimize potential negative effects:
• Small minority of customers offended by comparative feedback
• C t
Customer may d id t i
decide to increase th i energy consumption
their ti

Rates (Dynamic Pricing)
Pros:
• Dynamic pricing provides direct monetary incentives for consumers
• Utilities are better able to match prices to energy production and/or
purchase costs
• Flexibility in rate design (e.g., time-of-use, real-time, critical-peak).
• Solutions typically require in-home displays that provide feedback:
• Real-time and cumulative cost/energy consumption info and
associated energy savings impacts
• Advantage of permanent installation/use

Cons:
• Costly infrastructure investment requiring substantial resources to
install meters and develop integrated IT platforms
• Programs costs are typically justified by returns from operational
efficiency and capacity (i e peak load) management and savings
(i.e.,
• Energy efficiency/conservation savings are typically secondary
benefits and not primary drivers

Literature Review

Major review studies: Key Findings
Feedback leads to energy
gy
savings:
• Direct: 5 to 15%
• Indirect: 0 to 10%
Characteristics of effective
feedback:
• Given frequently
• Involves interaction
• Involves appliance-specific
information
• Given over longer period
• Presented in a user-
friendly format
Concern with experimental
rigor of studies

Findings from
Literature Review
(C. Fischer, 2008)
Covers 5 review studies and
21 original studies across 10
countries
Concludes that feedback
stimulates energy savings –
i l i
‘usual savings of 5% to 12%’
Characteristics of effective
feedback:
• Given frequently
• Involves interaction
• Involves appliance-specific
pp p
information
• Given over longer period
• Presented in an
understandable and appealing
way

Findings from
Literature Review
(S. Darby, 2006)
Savings from direct feedback – average from 5-15%
Savings from indirect feedback (e.g., billing) - range from 0-10%.
High energy users may respond more than low users to direct
feedback
Persistence of energy savings created from feedback when individuals
develop new habits or invest in efficiency measures
Useful display features include instantaneous usage, expenditure, and
historic feedback
Indirect feedback can be most helpful for evaluating heating load and
the impact of investments in insulation/new major appliances
Direct feedback is better for understanding the impact of smaller end-
uses and the significance of moment to moment behavior

Findings from
Literature Review
(Abrahamse et al, 2005)

Reviews thirty-eight field studies from 1977 to 2004 aimed at
thirty eight
encouraging households to reduce energy consumption
Identifies that much of the research on energy conservation
interventions h l k d th appropriate experimental conditions (
i t ti has lacked the i t i t l diti (e.g.,
significant sample size, appropriate control groups) to validate findings
and draw definitive conclusions
The large majority of studies addressing feedback find it to be an
effective means to generate energy savings, with more frequent
feedback leading to greater effectiveness
Rewards for energy conservation may influence behavior, but the
effects are found to be short-lived
Using inter entions in combination is fo nd to ha e an impro ed effect
interventions found have improved

Utility Experiences with Behavior
Change Programs

Illustrative Case Studies
Direct feedback via display devices:
• Hydro One (Ontario, Canada)
• NSTAR (Massachusetts)
• Recent Findings Update:
• Dominion (Virginia)
• Seattle City Light
y g
• Energy Trust of Oregon
Indirect feedback
• Positive Energy/SMUD
• Update on savings validation (Summit Blue)
• BC Hydro

Case Study: Hydro One - PowerCost Monitor Pilot
Study Findings

6.5% aggregate reduction in
electricity (kWh) consumption
y( ) p
8% reduction in non-electrically
heated homes
5% reduction in non-electric
heat/hot water homes
Pilot Program Methodology
16% reduction in non-electric
Study period >1 year heat homes w/ electric hot water
400+ participants 1% reduction in electrically heated
Sample across wide variation of homes; load “completely overwhelms”
11% of homes have electric heat in area
climate and geography
“income and demographic factors
income
Impact measured based on historical
had no impact on the
comparison
responsiveness to the monitor”
PowerCost Monitor (Blue Line
60% of participants felt the monitor
Innovations) used by participants
made a difference in their homes
Source: Summary: The Impact of Real-Time
Feedback on Residential Electricity Consumption:
The Hydro One Pilot, March 2006

Case Study: NSTAR - PowerCost Monitor Pilot
Study Findings

2.9% savings for customers who
used the monitor (~$64/year)
66%-75% installation rate
33% of initial users stopped using
Pilot began May 2008 the monitor during the study period
3,100+ units sold 63% of participants indicate behavior
change
Media coverage (TV, print) coincided
with significant rise in sales 60% noticed savings in their bill

Offering Unit Price Adoption Rate
Direct install during energy audit Free 95% PCM
Offering previous audit customers free PCM Free 14% Retail
$9.99 6% Price:
Direct Mail Solicitation/ ~$140
$29.99 5%
Media Promotion
$49.99
$49 99 0.3%
0 3%

Source: 2008 BECC Conference Presentation:
Power Cost Monitor Pilot, David MacLellan, NSTAR,
November 2008

Recent In-Home Display Pilots: Dominion Virginia Power
Study Findings

6% kWh energy savings in homes
g gy without electric hot water
Free PowerCost Monitor, pre- 19% kWh savings in homes with
programmed with rate electric hot water
Enrolled 1 000 users from 4 600
1,000 4,600 Savings estimates based on weather-
g
solicitations normalized billing analysis comparing
historical consumption
13-month study; began Nov. 2007
53% of respondents reported
GoodCents used as vendor to technical difficulties
execute pilot
Battery life
30% response to mailed survey
soliciation, with pre-paid return and Sensor water damage
coupon i incentive f f
ti for free multipack
lti k Plan to structure full rollout with $25
of CFLs user payment for the meter; to
In process of completing post-study achieve “skin in the game”
survey Using Blue Line PCM monitors that
have AMI compatibility

Source: AESP Webinar Presentation: Managing an
In-Home Energy Display Pilot Project, July 2009

Recent In-Home Display Pilots: Seattle City Light
Study Findings

2-3% average electricity savings in
comparison with control group
Pilot Program Methodology No significant variation in savings
achieved across different monitors
Goal of 33 home energy monitors evaluated
installed
Somewhat disappointed with
S h t di i t d ith
Randomly chosen participants results to date;
Single family homes only Expected greater savings based
3 types of meters installed on manufacturer claims and
previously published studies
PowerCost Monitor
Difficulty in convincing customers to
The Energy Detective participate in the study
Cent-a-Meter Survey company found
8-month test period themselves having to sell hard
Logistics issues for electrical permits,
installation scheduling for panel
devices

Recent In-Home Display Pilots: Energy Trust of Oregon
Study Findings

Preliminary findings indicate
“monitors did not have a
Pilot P
Pil t Program Methodology
M th d l significant impact on energy use
for either cohort”
Over 350 monitors deployed: Six month response rates:
164 sold via Web site @ $29 99
$29.99 57% of Early Adopters
to “Early Adopters” (EA) 55% of HER cohort
201 installed as part of a Home 66% of EA and 64% of HER using
Energy Review (HER) device after 6 months
Savings evaluation involved control Two thirds of non-users report monitor
groups with random stratified sample no longer functional
with adequate regional and home
vintage representation
g p Lighting, space heating, and clothes
dryers most often attributed as
Surveys conducted within 3 weeks savings source
of installation and at 6 months after
While never significant, point
First installations in Jan of 2008 estimates of energy savings were
gy g
highest at 3 months, and declined
at 6-9 months

Case Study: Positive Energy @ SMUD
Study Findings (Ongoing)

2% savings achieved on average
for treatment group (~250kWh p.a.)
g gy
3¢ per kWh savings cost average
Program launched April 2008
Significantly higher savings among:
35,000 customer treatment group
• Higher energy consumers
(non-targeted)
• Greenergy (renewable energy)
• 25,000 homes receiving monthly
customers
• 10,000 homes receiving quarterly
• Monthly vs. quarterly recipients
55,000
55 000 customer control group
t t l
Indication of correlation of higher
Random sampling to create savings for lower income population
representative population
800 of 35,000 decided to opt out
Reports provide a ‘h ’ h
R t id ‘here’s how you
<1% of 35,000 opted to set personal
compare to your neighbors’
goal
message customized to the home
(type, size, location) Positive customer feedback
Customized energy savings tips Few very negative reactions
provided along with report
Source: Interviews with President of Positive
Energy and Program Manager at SMUD

Verification Analysis for Impact of Positive Energy at SMUD
Summit Blue Consulting - May 2009

“The estimate of annual savings from each of the three
methods ranged from 2.1% to 2.2% showing strong robustness
of results. The range around each of these estimates is tight,
providing good reliability and precision…The strength of these
estimates rests on the clean design of the experiment
and the very large sample sizes that were used. It is often
difficult to accurately assess a program savings of 2% from
billing analysis because of the wide range of variability in
customer bills, b t th l
t bill but the large scale of thi experiment allowed
l f this i t ll d
for accurate assessment of savings from this program.”

Source: Impact Evaluation of Positive Energy SMUD
Pilot Study, May 26, 2009

Behavioral Science Research – Normative Feedback
Seminal research published in 2007 by Nolan J. M.. Schultz, P. W.,
Nolan, J M Schultz P W
Cialdini, R. B., Goldstein, N. J., & Griskevicius, V.
Used doorhanger messages to test response to four conservation
messages among California residents:
(1) they could save money by conserving energy
(2) they could save the earth’s resources by conserving energy
(3) they could be socially responsible citizens by conserving energy
(4) the majority of their neighbors tried regularly to conserve energy
Only the social norming message produced significant savings

Source: "Normative Social Influence is Underdetected,"
J.M. Nolan, P.W. Schultz, R. B. Cialdini, N.J. Goldstein,
and V. Griskevicius, Personality and Social Psychology
Bulletin (July 2008).

Positive Energy – Home Electricity Report Example

Source: Positive Energy

Positive Energy – Home Electricity Report Example
Customized Tips
Driven By:

Housing
• Si
Size
• Age
• Fuel type
• Pool etc
Pool, etc.
Consumption
• Amount
• Pattern
Demographics
• Income
•A
Age
• Length of
residence
• DIY
• Green
Source: Positive Energy

Case Study: BC Hydro Behavior Change Market Test
Study Findings
Reduction target had significant
impact on recruitment success
5% target h d significant
had i ifi
1-Year pilot launched early 2007 freeridership problem
10% goal found to be optimal
Recruited employees of BC Hydro’s
largest customer
l t t Cash rewards more appealing than
prize draw rewards
Employees encouraged to eNewsletter drove online visits
participate:
More frequent visitors to online
• Commit to a given electricity tool hi
t l achieved higher electricity
d hi h l t i it
reduction target savings
• Use online tool to track/compare Reported behavior changes
consumption
• Turning o lights
u g off g ts
• Participants received cash rebate for • Changing laundry habits
achieving target (e.g. 5% electricity
rebate for achieving • Shorter showers
• Unplugging chargers
4 Different incentive rewards tested • Turning down the thermostat

Source: BC Hydro

BC Hydro – Team Power Smart

Online tools allow anyone in BC to enroll by committing to use 10%
less energy over one year
Track consumption
Compare consumption to similar households
Visibility to community rivalry and promotion of “Pride of Province”
Members benefits i l d special offers and opportunities t win
M b b fit include i l ff d t iti to i
prizes in drawings and contests
Program supported by a roster of Team Power Smart Leaders including
celebrity athletes and community leaders
Expected results among participants (~4% to 5% total savings):
17% become Achievers – average savings of 21%
24% become Savers – average savings of 4%
59% become Non-Achievers – no savings on average
Currently 74,000 members (4% of customers) enrolled toward goal of
210,000 by 2010

Source: BC Hydro

BC Hydro – Team Power Smart

Source: BC Hydro

Psychographic Segmentation
Targeting “Stumbling
Stumbling
Proponents” with
Team Power Smart

Cross references
utility-focused
categories (e.g., home
heating, appliances,
and lighting) with
emotive categories:
• Health+Wellness
• Food+Drink
• Life+Leisure
• Family+Friends
• Home+Garden
G
• Gadgets+Tech.

Uses survey data and
demographic/housing
parameters to target
customized messages
most likely to be
received positively to a
given audience
audience.

Source: BC Hydro

Cost Effectiveness
and Applicability to MN

Examining Program Cost Effectiveness
Home Energy Reports – Positive Energy @ SMUD (N=35,000)
250 kWh (~2%) first-year savings in non-targeted households
First-year
First year cost of conserved energy (i e assumes no persistence):
(i.e.,
3¢ per kWh (<$8 per household per year variable cost)
In-Home Direct Feedback – PowerCost Monitor
Device Cost: ~$140 (without installation)
Requires utility subsidy of ~$100+ to spur adoption (e.g. NSTAR)
Likely savings potential of 3% (NSTAR) to 7% (Hydro One)
Cost of conserved energy @ $100/household program cost:
Assumed Savings Persistence Horizon
1 Year
2 Years 5 Years 10 Years 20 Years
'first-year'
Savings scenario: (cost of conserved energy: $ per kWh)
3% - 330kWh $0.32 $0.16 $0.07 $0.04 $0.02
7% - 770 kWh $0.14 $0.07 $0.03 $0.02 $0.01
(Assumes: $100 program cost, 11,000 kWh average consumption, 5% discount rate)

Benchmarking First-Year Costs of Energy Savings

Source: Summit Blue Consulting, 2008

Importance of Opt-In vs. Opt-Out
NSTAR’s PCM pilot sought to evaluate customer willingness to pay
Findings indicate the utility would have to subsidize nearly $100 of the
device cost in order to reach a significant population (>1%)
Limited participation in opt-in programs has significant implications to
achievable program savings:
Even if device programs could yield 10% savings (as per literature), if only
5% participate a utility would be limited to a 0 5% population impact
participate, 0.5%
Conversely, a program like Positive Energy, saving only 2% among
participants (possibly all customers), could have 4X the population impact
NSTAR Pil t Fi di
Pilot Findings – C t
Customer Willingness t P
Willi to Pay
Offering Unit Price Adoption Rate
Direct install during energy audit Free 95%
Offering previous audit customers free PCM Free 14%
$9.99 6%
Direct Mail Solicitation/
$29.99 5%
Media Promotion
$49.99 0.3%

Source: NSTAR

Regional Energy Intensity
Intensity in West North Central States matches national average
Factors that can influence regional differences:
Climate – associated HVAC energy use
Age distribution of the housing stock – associated appliance and weatherization efficiency
Population’s attitude toward conservation
Amount of resources going toward EE and conservation programs

Source: Energy Information Administration

Residential Electricity End Use Consumption Drivers
Average % of Household kWh by Region
35%

30% Climate and fuel source
differences reflect relative
% of Electricity Consumption .

25% share of electricity
consumption by region
20%

15%

10%

5%

0%
Air Space HVAC Kitchen Water Home Laundry Other Other End
Lighting
Conditioning Heating Appliances Appliances Heating Electronics Appliances Equipment Uses
U.S. Avg. 16.0% 10.1% 5.0% 26.7% 9.1% 8.8% 7.2% 6.7% 2.5% 7.7%
West North Central 14.7% 8.2% 6.1% 29.2% 7.7% 8.6% 7.0% 8.1% 2.1% 8.3%
New England 6.6% 6.6% 4.5% 32.6% 8.0% 13.2% 11.3% 8.6% 4.6% 4.1%
South Atlantic 21.4% 10.4% 4.2% 23.2% 12.4% 6.8% 5.7% 6.1% 2.4% 7.3%


Residential Electricity End Use Consumption Drivers
Average Household kWh by Region
4,000

3,500

3,000
Annual Kilowatt Hours

2,500
t

2,000 Uniformity exists in uses
involving frequent behavioral
1,500 interaction
A

1,000

500

-
Air Space HVAC Kitchen Water Home Laundry Other Other End
Lighting
Conditioning Heating Appliances Appliances Heating Electronics Appliances Equipment Uses
U.S. Avg. 1,837 1,159 574 3,065 1,045 1,010 827 769 287 884
West North Central 1,689 942 701 3,356 885 988 805 931 241 954
New England 491 491 334 2,423 595 981 840 639 342 305
South Atlantic 3,150 1,531 618 3,415 1,825 1,001 839 898 353 1,075


Evaluation of Miscellaneous Electric Loads
(
(MELs))


Residential Electricity Consumption
by End Use
y


Average Household Consumption by MEL


MELs in the Context of Total Energy Use


Water Heater Fuel Source by Region
100%

90%
Fuel
F l Oil
80% Gas

70%
Gas
ds

Gas
% of Household

60% Gas

50%
Gas
40%
o

Electric
30% 63%
Electric
20%
39% Electric Electric
29% Electric
10% 26%
20%
0%
U.S.
US West
W t Midwest
Mid t South
S th Northeast
N th t


Water Heater Fuel Source
by Population Density
y p y
100%
Other
90%
Fuel Oil
80%
Gas
70%
Gas Gas
ds
% of Household

60% Gas

50%

40%
o

Electric
30% 63%

20% Electric Electric
35% 35% Electric
10% 26%

0%
Cities
Citi Town
T Suburbs
S b b Rural
R l


Average Household Energy Spending
usehold
by End Use and Region - (All Fuel Sources)

$3,000
Average Annual Energy Spending per Hou

$2,388
$2 388
$2,500

$1,885 All Other
$2,000 $1,823 $1,788 $739
$1,634
All Other All Other
$1,500 All Other Refrig.
$647 All Other $596 $635 Water Heat
Refrig. $642 Refrig.
Refrig. A/C
$1,000
, Water Heat Refrig.
Refrig Water Heat
E

Water H t
W t Heat
Water Heat A/C
A/C
$500 A/C Heating
A/C
Heating Heating
Heating Heating
$0
Total West Midwest South Northeast

10%: $190 $160 $180 $180 $240
5%:
5% $95 $80 $90 $90 $120
2%: $38 $33 $36 $36 $28


Key Lessons Learned –
Utility Manger Perspectives

Program Manager Perspectives: Key Lessons Learned
Motivation is the essential ingredient
Upfront customer input is invaluable
• “Don’t design a project within your own four walls.”

Taking an iterative approach ensures consistency
with goals and avoids technical issues
• “Know your goals at the outset.”

A cross functional pilot team helps to ensure success
It is important to be sensitive to customer satisfaction
impacts
Leveraging peer utility experience improves likelihood
of success

Program Manager Perspectives: Key Lessons Learned
Pre pilot surveys establish a baseline for analysis
Pre-pilot
Incorporate a control group
Novelty of the feedback will wear off
Meter interface can present barriers
p
IHDs can be hampered by low installation rates
Solution
S l ti must b well suited t th customer
t be ll it d to the t
population
• “There probably isn t going to be a silver bullet ”
There isn’t bullet.

Tailoring messaging to specific segments can ensure
messages resonate with your audience

Program Models to Consider
Model 2: Model 3:
Model 1:
Indirect/Comparative Hybrid Approach –
Program Models In-Home Energy Use
Feedback on Home Comparative and Direct
Monitor
Energy Use Feedback
Participants receive regular
p g
Participants receive regular comparative feedback
Participants receive a
reports in the mail that will reports and energy tips.
monitor that provides real-
compare their energy use Participants will be
time feedback on home
Program Basics with neighbors in similar encouraged to make use of
energy use in order to track
homes. Targeted energy real-time power monitors
a d e pe e t t t e
and experiment with their
saving ti will also b
i tips ill l be that
th t can be purchased or
b h d
energy use behavior
communicated. borrowed for several
months at a time.
Customer Engagement Opt-out (reports)
Opt-in Opt-out
Method Opt-in (in-home device)
2% 2%+
Average in total customer Average in total customer
5%
Targeted participant population; targeted population; targeted
(mid of 3% to 7% range)
household savings segments would have segments would have
Valid among self-selected
(as % of total kWh) significantly higher savings significantly higher savings
participant population
(e.g., in the 5% to 10% (e.g., in the 5% to 10%
range) range)
Real-time feedback for Cost effective approach with Hybrid approach maximizes
Big Advantage
participants broader reach savings potential
Significantly
Significantl higher cost per Requires
Req ires integration with
ith Greater comple it /
complexity/
Big Disadvantage
kWh saved system data resource requirements


Program Considerations: Model 3
Points of Emphasis

• Give customers the ability to compare energy-use with their neighbors
Program Objective
• Provide opportunity for the utilization of in-home monitors, possibly on a temporary basis

• Broad reach of the opt-out home energy report across geographic, housing, demographic strata
Target Customer
• Use data from indirect feedback program to identify customer segments with the greatest
Market
potential to benefit from direct feedback
• Need internal IT system for report generation or contract third-party services
• Detailed data on houses and homeowners may need to be obtained from third-party/proprietary
Program Logistics
sources
• Consider subsidized purchase for feedback devices or model to provide on a temporary basis
Customer • Utilize energy use reports as a platform for education about conservation ideas and promotion of
Education the direct feedback program
Enhancements • Raise awareness and promote associated devices to aid in customer behavior changes
Trade Ally Plan • Evaluate need for technical/installation assistance for feedback devices
Savings and Goals • Anticipated savings of 2% in indirect feedback population; additional savings from device group
Assumptions • Ongoing measurement is necessary to establish baselines for long-term savings persistence
• If a temporary device lending program is ruled out, subsidies for customer device purchases
Marketing and
would be necessary, promoted through the indirect feedback reports
Incentive Strategy
• Evaluate the incorporation of customer goal setting and commitments as a motivator
Quality Control • Having adequate pilot scale, duration, and measurement systems will ensure accurate cost
Plan effectiveness quantification
Program Budget
• Evaluate available internal resources, third-party service costs, and need for device subsidies
Considerations


Example: Behavior Change Pilot Program Plan - Model 3
Critical Success Factors
Process Step Inputs Actions Outputs (Application of Lessons
Learned)
 Identify required program pilot team with
cross functional (operational, finance,  Project team
 Available internal technical, customer service) capabilities to  Project plan
Identify resources address all aspects of program execution and  Define pilot
 A diverse pilot team helps to
p p
Team/  Potential business case assessment program outcome
ensure success
Objectives implementation  Define project timeline and specific pilot measures
partners learning objectives (e.g., quantify savings  Pilot program
potential and $/kWh for program) budget
 Quantify resource and budget requirements
 Review work of peer utilities; engage in  Determination of
 Taking an iterative approach
dialog
di l program partner
t
to piloting solutions ensures
 Engage program partners (if engagement
consistency with goals
 Identification of necessary/desired)  Identified
Prepare for  Leveraging the experience
potential program  Develop IT integration plan to enable challenges to
Customer of peer utilities improves
partners (e.g., generation of home energy use reports report generation
Engagement chances of success
Positive Energy)  Develop list of items on which to collect  Identified device
 Validating the functionality of
customer input preferences
new technology can avoid
 Obtain real-time feedback devices and test  Customer input
headaches down the road
internally objectives
 Solicit customer engagement
 Collect feedback from a focus group (or  Identified customer  Upfront customer input
survey) concerns with provides invaluable
 Collect feedback on key aspects of program reports guidance for successful
 Small customer (e.g.,
marketing and execution:  Key themes to program design
Collect focus group)
o Receptivity to comparative feedback incorporate in  Ensure the solution is well
Customer population
o Desired report information elements, customer targeting suited to customer
Input  Customer input
format/graphics and messaging population
objectives
o Attitudes toward conservation  Identified barriers  Interfacing with meters for
o Interest in real-time feedback devices to user acceptance
p in-home devices can
o Interest in device distribution/rental of device present barriers
arrangements


Process Step Inputs Actions Outputs Applicable Lessons Learned

 Establish desired customer segments on which to
determine program i
d t i impactt
 Available data on  Calculate required program sample size (in each  Necessary program
customer energy population) to allow for adequate treatment and  Incorporating a control group
use and precision/confidence in program outcomes control group size that representative of the
segmentation measurement*  Identified customer underlying population and
Define
parameters:  Establish a control group of (at least) similar size segment sufficiently large allows for
Parameters
o Level of for comparison that is representative of the representation the necessary precision and
for C t
f Customer
energy use treatment group desired in pilot confidence to draw
Comparison
o Age  Develop customer education plans to maximize group conclusions about specific
o Income awareness and satisfaction  Customer education sub-segments of the
o Home  Determine means/parameters to group customer plan population
size/type/age homes for energy use comparisons (e.g., 100  Program budget
homes of similar size in neighborhood)
 Determine program budget
*Note: See Appendix 1 for discussion of sample size determination. Control and treatment groups should be defined to observe impact of indirect feedback.
The selection bias of device user population requires historical data comparison to evaluate savings.
 Develop energy use reports to communicate
 Template for home  Motivation is the essential
customer energy use in comparison to neighbors
energy use report
gy p ingredient
g
Develop and historical consumption
 Customer  Means to determine  Look beyond traditional
Energy  Develop/obtain comprehensive lists of energy
segmentation customized savings customer segmentation
Report savings measures to potentially recommend
data tips to include (may models to find messages that
Content  Establish means to select customized energy
come from program resonate with particular
savings tips for customers based on known
partner) groups
segmentation parameters
 Id if plan f d i l di /
Identify l for device lending/rental program
l
Develop  Device
(e.g. distribution through mail, library checkout,
Real-Time preferences  Real-time feedback gives
etc.)
Feedback  Identified barriers  Device lending users the opportunity to
 Purchase adequate number of devices to support
Device to user program resources experiment in finding energy
pilot
Distribution acceptance of saving behaviors
 Develop necessary customer education materials
Model device
to facilitate device lending program


Process Step Inputs Actions Outputs Lessons Learned

 Define survey to capture:
o Home characteristics (e.g., appliances)  Baseline profile of
o Demographics customer
 Customer focus
o E Energy use b h i / tt
behaviors/patterns characteristics and
h t i ti d
group feedback
o Attitudes toward conservation attitudes  Pre-pilot surveys can
 Example surveys
o History of participation in utility energy  Confirmation that establish baselines for
Conduct Pre- from past
efficiency programs (e.g., rebates, etc.) treatment and control analysis
Pilot Survey programs and
 Select pilot treatment and control groups (likely samples represent
other utilities
random/stratified sample) the underlying
 Collect feedback from customers across treatment
treatment, population
control, and total customer populations
 Selected  Distribute customer education materials describing
treatment program/reports
population  Regularly generate and distribute home energy use
 Resources to reports to treatment group customers
 Pilot program
g
support report o More frequent feedback has been shown to
participation
generation and lead to greater energy savings
 Addressed customer
distribution  Promote opportunities for participants to obtain real-  Ensure pilot execution
Execute Pilot concerns
 Device time feedback devices to aid in their efforts to save allows for measurement
Study  Demand for real-time
distribution/ energy of cost effectiveness
feedback devices
collection model  Facilitate distribution and collection of real-time
 Motivated and
 Resource to field feedback devices
educated participants
customer calls,  Assist/respond to customer questions/issues with
questions, issues device installation/operation
 Customer  Consider offerings customer the opportunity to
communications establish an energy reduction goal
 Develop survey instruments to evaluate:
oPerceptions of home energy use reports/devices  Ability to adjust
oImpact on motivation savings for
oBehavior changes made concurrent efficiency
Collect  Be sensitive to
 Pilot program oInvestments made program participation
Participant program’s impact on
participation oParticipation in other utility energy efficiency  Survey data/feedback
Feedback customer satisfaction
programs (e.g., rebates/incentives) – Important on participant
for savings adjustments/avoid double-counting experience and
oConservation attitudes satisfaction
 Collect feedback from pilot treatment/control groups


Process Step Inputs Actions Outputs Lessons Learned

 Measurement of
 Opt-out nature of
participant energy
program allows for
 Energy savings
results t be more
lt to b
consumption data  Obtain measures of actual consumption over  Determination of
Evaluate reasonably extended to
 Quantification of treatment period for treatment, control (if any), and program cost
Program potential for savings in
pilot program population (sample) effectiveness ($ per
Results/Savin entire population
costs  Compare to normalized historical consumption and kWh of savings)
gs Cost  Specific customer
 Data from control group data to determine impact of the feedback  Determination of
Effectiveness segments (e.g., higher
participant intervention on energy conservation differences across
energy users) are likely
feedback survey segments (e.g.,
to see different levels of
savings for high
savings
energy users)
 Data on device use
Conduct  Execute customer surveys and data collection to  [Limited data exists on
 Pilot program pattern
ongoing determine persistence of energy savings and customer persistence of savings
participation  Data on savings
monitoring involvement from utility programs]
persistence


Note on Sample Size Determination
Opt-in device program inherently prohibit simple control group
determination due to the self-selected nature of the treatment group
Opt-out programs lend themselves to easier control group definition
Avoids problems that can come from using historical consumption
data beyond the need for weather normalization
Economic conditions
Media messaging
Individual household factors:
Tenant changes
Occupancy
Renovations
Alternative approaches to evaluation of savings
Confidence interval around the mean
Confidence interval around the % change from prior period
Linear regression and differenced linear fixed effects models
g

Source: Author’s calculations

Note on Sample Size Determination
The required sample size for a study aimed at verifying savings
performance is a function of several parameters:
Hypothesized magnitude of energy saving to detect (μ0-μ1)
Standard deviation of energy consumption across households (σ)
Desired confidence (1-α) and power (1-β): tolerance for making a wrong conclusion
Sample size to test the difference in two population means
Rule of thumb for 95% Confidence, 80% Power:

2z1 / 2  z1  
2
16
n n
  0  1 
2
   1 
2

 0   
     
Hypothesized Annual Energy Savings (to Test)
1% 2% 5% 10%
100 kWh
100 kWh 200 kWh
200 kWh 500 kWh
500 kWh 1000 kWh
1000 kWh
1000 kWh 1,600 400 64 16
Std. Dev. of
2000 kWh 6,400 1,600 256 64
Annual
3000 kWh 14,400 3,600 576 144
Energy
Consumption
Cons mption 4000 kWh
4000 kWh 25,600 6,400 1,024 256
5000 kWh 40,000 10,000 1,600 400

Source: Author’s calculations

Thank you!
y

Questions?
Ed Carroll: ecarroll@franklinenergy.com
608-310-6910

Mark Brown: mbrown@franklinenergy.com
612-237-8268

Behavior Change Through Rate Design
30
Studies have Average Customer
shown that as 25
much as a 6%

Cents / kWh
Rate D
energy savings can
i 20

k
be achieved from
15
inclining block Rate C
rates that take 10
advantage of price Rate B Existing
5
elasticity in Flat Rate
consumer demand. Rate A
0
0
200 400 600 800 1,000 1,200 1,400 1,600 1,800 2,000
1 000 1 200 1 400 1 600 1 800 2 000
kWh / Month

Avg Percent Change in Usage
Price Elasticity Rate A Rate B Rate C Rate D

Short Run Mean -5.9% -2.2% -1.0% -0.5%
Std Dev 2.0% 0.8% 0.3% 0.2%
Long Run
g Mean -18.4% -6.7% -3.1% -0.7%
Std Dev 6.5% 2.4% 1.1% 0.4%

Inclining Block Rate Bill Impacts
I li i bl k rate would b d i
Inclining block t ld be designed so th t only th hi h t users of
d that l the highest f
electricity would see billing increases.

Simulated Distribution of Bill Impacts

30%
Tier 1 Original Break-
20%
Cutoff even Point
10%
Change in Monthly Bill

0%
-10%
-20%
n

-30% Break-even Point
w/Price Elasticity
-40%
-50% No Price Elasticity
-60% With Price Elasticity
-70%
100
200
300
400
500
600
700
800
900
1,000
1,100
1,200
1,300
1,400
1,500
1,600
1,700
1,800
1,900
2,000
Customer Size (kWh/month)

Research to Inform Design of Residential Energy Use Behavior Change Pilot

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Research to Inform Design of Residential Energy Use Behavior Change Pilot

Similar to Research to Inform Design of Residential Energy Use Behavior Change Pilot (20)

More from University of Minnesota

More from University of Minnesota (20)

Recently uploaded

Recently uploaded (20)

Research to Inform Design of Residential Energy Use Behavior Change Pilot