2. How to Design a Gas Program Impact
H t D i
G P
I
t
Evaluation
Jonathan B. Maxwell, Energy & Resource Solutions (ERS)
Kathryn Parlin, West Hill Energy & Computing
January 19, 2011
aesp.org
3. Agenda
• Examine results for 13 gas evaluations
–
–
–
–
Realization rates
Variation of realization rates
Net-to-gross
Non-gas impact
• Ramifications on evaluation designs
aesp.org
4. Key Points
• Gas evaluation less mature than electric
• Gas realization rates somewhat lower than electric
• Evaluated project savings varies widely from
reported—even in large custom programs
• Account for interaction with other fuels
• Account for non-energy benefits & costs
aesp.org
5. Gas Programs Examined
Program or Portfolio or Targeted
Measure Type
No.
Programs
Sample
Size
Residential single family new construction
1
25
Multifamily retrofit
1
6
C/I new construction & retrofit
t ti
t fit
2
48
C/I performance contracting
1
6
g
Commercial retrocommissioning
4
34
Industrial
1
29
Agriculture
1
30
Specialized ( i i
S
i li d (pipe insulation, bid program)
l ti
)
2
73
TOTAL
13
251
aesp.org
6. Program & Evaluation Types
• Applicants typically estimated site-specific savings
site specific
– Expectation of good estimates
• Evaluators estimated site-specific savings for all
site specific
– Majority “enhanced” level of evaluation engineering rigor
• Many p g
y program types & administrators; many
yp
;
y
evaluation engineering firms
– Actor bias unlikely
aesp.org
7. Savings Realization Rates
• 0.68 median portfolio realization rate
.08
.72
.08
.91
.21
.92
.33
.93
.53
.98
.64
1.07
.68
• Lower RR than for similar electric portfolios
– 5 of 13 portfolios had both electric & gas RRs
– 0.70 median elec RR for the 5
– 0.53 median gas RR for the 5
aesp.org
8. Error Ratio - Definition
• Measures variation of realization rates (Stratified
ratio estimation)
ti
ti ti )
– Lower is better
– Higher requires larger sample to get high precision
– Unrelated to magnitude of realization rate
• 0 4 to 1 0 typical in EE evaluation
0.4 1.0
• 0.4 to 0.6 typical for electric EE programs with
site specific
site-specific analysis
aesp.org
9. Error Ratio - Illustration
From California Evaluation Framework Ch 13 June 2004
Framework,
13,
2004.
aesp.org
10. Error Ratio - Results
• Median 1.04
• Shown with 6 outlier projects removed
aesp.org
11. Error Ratio - Observations
• Why are error ratios so poor?
–
–
–
–
–
Less mature programs
Difficult for applicants (& evaluators) to measure
Baseline less clear (11% projects with 0 RR)
(
p j
)
Fuel switching
Inherently difficult-to-predict measures (RCx)
• What to do?
– Intensively study gas measures
– Do not increase sample sizes at expense of rigor per
site
aesp.org
12. Net to Gross
Net-to-Gross
• Similar methods used as with electric
– Mostly enhanced for these portfolios
• 0 85 median NTG factor
0.85
• 0.31 to 1.09 range for NTG factor
• More consistent than realization rate
aesp.org
13. Gas Measure Impact on
Electricity, Steam, Oil
• Results from 3 portfolio evaluations
Electric Impacts
(
(kWh/ MMBtugas)
Other Energy
Impacts
(MMbtu/ MMBtugas)
(
Commercial New Construction
1.8
0.00
C/I Retrofit
15.1
0.02
Manufacturing--Ag-Food
Man fact ring Ag Food
18.5
18 5
na
Portfolio Type
yp
• Infrequent, but significant when it occurs
• Customers add ~22% in utility bill savings
22%
• Should include in benefit-cost calculations
aesp.org
14. Impact on Non-Energy Costs
Non Energy
• Results from 4 program evaluations
Group
Delivery Program
C/I
New Construction
Existing Facilities
g
Non Energy
Non-Energy Impact
(/ MMBtugas)
$0.00
Loan Fund
1-4 Res.
Multifamily
•
•
•
•
$
$9.46
$1.22
ENERGY STAR Homes
$0.91
Multifamily Building (Existing)
$0.06
$0 06
Infrequent but significant when it occurs
Labor
L b & water most common savings
t
t
i
Customer save up to $0.75 for every $1 gas saved
Should include in benefit-cost calculations
aesp.org
15. Summary - Evaluation Planning
• Gas programs are less mature than electric
• Expect large error ratios--evaluated savings varies
widely from reported
• Don’t sacrifice digging deeper for more sites
– Invest in enhanced M&V until programs mature
– Allow time for vigorous feedback with program staff
–A
Account for interaction with other fuels
tf i t
ti
ith th f l
– Account for non-energy benefits & costs
aesp.org
16. Thank You
Thank you to clients & prime contractors (CA)
NYSERDA
KEMA
CPUC
Itron
SBW
Jon Maxwell, ERS
Maxwell
jmaxwell@ers-inc.com
Kathryn Parlin, West Hill Energy & Computing
kathryn@westhillenergy.com
aesp.org
