Training Module on Electricity Market Regulation - SESSION 6 - Efficiency Assessments

Experience you can trust.
http://www.leonardo-energy.org/training-module-electricity-market-regulation-session-6
Training on Regulation
A webinar for the European Copper Institute
Webinar 6: Efficiency Assessments
Dr. Konstantin Petrov / Dr. Daniel Grote
11.1.2009

11/01/2010 1
Agenda
a) TOTEX versus OPEX benchmarking
3. Application of efficiency results
a) Overview
2. Methods for efficiency assessments
c) Data Envelopment Analysis
b) Performance indicators
b) Efficiency convergence speed
1. Why measure efficiency?
c) Supporting schemes
e) Virtual network models
d) Parametric Approaches

11/01/2010 2
• Regulation is needed in areas where
competition does not work (e.g. natural
monopolies - transmission, distribution
networks) to limit excessive pricing and
to set incentives for efficient
performance
• Regulators apply benchmarking to
assess efficiency of regulated
companies for the purposes of
incentive regulation
Major Reasons
Cap regulation
Actual Cost
Current price level
Current price + Inflation
Current price + Inflation – productivity growth
Efficiency gains
time
Influenced by company
Influenced by company
Set by regulator

11/01/2010 33
Definition of efficiency
Efficiency =
Outputs
Inputs
+ “Correction for
Environment”
Distribution Company
e.g. # employees, fuel,
operational costs,
Input Factors
e.g. # customers,
delivered energy (kWh),
peak load (kW)
Output Factors
e.g. firm size, network topology,
climate, topography, terrain, task
complexity
Environmental Factors

11/01/2010 4
• Technological change (frontier shift): change in production technology within the
sector
• Efficiency change (catch-up): change in efficiency of production
− Change in the scale of production (scale efficiency)
− Pure technical efficiency change
• Allocative efficiency
− Input mix allocative efficiency: producing same outputs with different mix of
inputs
− Output mix allocative efficiency: producing different level of outputs with same
mix of inputs
• Changes in operating environment
Reasons for efficiency changes

11/01/2010 5
Efficiency assessment and price control
Efficiency
Assessment
Efficiency
Scores
Efficiency
Improvement
Targets
Integration in
Price
Control
Allowed Revenue (Tariffs)Efficiency
Interface
Benchmarking
– Approach
– Sample
– Model Orientation
– Data Collection
– Data Validation
Conversion
– Convergence Time
– Convergence Profile
– Inefficiency Caps
– Efficiency Bands
Integration
– Chargeable Basis
– Capex Treatment
– Revenue Requirements
– Regulatory Formula

11/01/2010 6
Practical Relevance of Benchmarking and the X-factor
• Reflects the regulatory view for anticipated efficiency improvement
• Ensures ex-ante sharing of the anticipated efficiency gains between customers and
regulated companies
• The X-factor is not a confirmation but rather indication of the anticipated efficiency
improvement
• In some regulatory regimes the X-factor has a dual function:
− Efficiency improvement
− Revenue profiling

11/01/2010
Overview (1)
Benchmarking Methods
Partial Methods Total Methods
Non-parametric Parametric
Reference
Networks
(Virtual
Networks)
Index Methods
Data
Envelopment
Analysis
(DEA)
Stochastic
Frontier
Analysis
(SFA)
Ordinary
Least
Squares
(OLS)
Corrected
Ordinary
Least
Squares
(COLS)
Total Factor
Productivity
(TFP)
Uni-
dimensional
ratios
Performance
Indicators
Linear
programming
Econometrics
Engineering
Models
7
Total methods can be based on the average performance or the efficient frontier of
comparable companies

11/01/2010 88
Overview (2)
• Efficiency performance assessment (benchmarking) applied in various forms
• Methods differ in the standard of comparison
• No consensus among regulators as to which methodology is best
• Sometimes different methods applied simultaneously for cross-checks
• Frontier methods preferred by regulators, in particular DEA and SFA
– Parametric (econometric) models (Germany, UK)
– DEA analysis (Norway, the Netherlands, Germany, several countries in CEE)
– Reference network models (Spain, Sweden, Chile, Argentina)

11/01/2010 99
Overview (3)
Efficiency Score
A B C D E
Measures of relative
inefficiencies towards
best performance
Conversion
(definition of
efficiency increase
targets)
Companies

11/01/2010 10
Performance Indicators
• Uni-dimensional ratios:
– Comparison of single performance indicators between firms
– Fails to account for the relationships between different input and output factors
• Productivity (Managerial) Indicators
– GWh/Employee
– OPEX/GWh
– OPEX/Employee
– GWh/Line Length
• Financial indicators
– Debt/Equity Ratio
– Return on Investment (ROI)
– Return on Capital Employed (ROCE)
• Partial methods produce simple, easy to calculate straightforward indicators of performance
… but do not recognize trade-offs between different improvement possibilities or areas
• Can only be used as a rough indication

11/01/2010 11
Index methods – Total Factor Productivity (TFP)
• Total factor productivity (TFP) is a measure of the physical output of a regulated company
produced by a given quantity of inputs
• With multiple inputs (Y) and outputs (X), outputs are usually weighted by their revenue
shares (sR) and inputs are weighted by their cost shares (sC)
• Weights can be either static or dynamic (different weights used for each period)
• Extensively used in the US for utility regulation (both energy and telecoms)
• Data requirements can be harsh
• TFP does not provide any information about ‘infra-marginal’ efficiency improvement
possibilities; for this we need more articulated benchmarking techniques (frontier-based
methodologies)
• More suitable for an assessment of company performance over time than comparisons
between regulated companies
∑
∑
=
=
= n
j
jj
C
m
i
ii
R
Xs
Ys
TFP
1
1
Input factors
Output factors

11/01/2010 12
Frontier methods
• Frontier methods are based on the concept that all companies should be able to operate at
an optimal efficiency level that is determined by other efficient companies in the same
sample
• These efficient companies are usually referred to as the “peer firms” and determine the
“efficiency frontier”
• The “efficiency frontier” is formed from the observed performance of the companies in the
analyzed sample, as determined by the relationships between the inputs and outputs of the
sampled units
• The companies that form the efficiency frontier use the minimum quantity of inputs to
produce the same quantity of outputs (input oriented model)
• The “efficiency frontier” is used as a reference against which the comparative performance
of all other companies (that do not lie on the frontier) is measured
• The distance to the efficiency frontier provides a measure for the inefficiency

11/01/2010 13
Data Envelopment Analysis (DEA) (1)
Output 1 Input 1
Input 2
Data Envelope
A
B
C
D
E
most efficient
companies
F
F’
Inefficiency
Input minimisation
Inefficiency
Output 2
Data Envelope
A
B
C
D
E
F
most efficient
companies
F’
Output maximisation
G
G’

11/01/2010 14
Outputs
Inputs
A
B
C
constant returns
to scale frontier
variable returns to
scale frontier
FF’
• Variable returns to scale account for short-run scale inefficiencies
• In the long run, firms should optimally adjust their size so that constant returns to scale are
achieved

11/01/2010 15
• DEA is a non-parametric approach to calculate the relative Input-Output efficiency of a
regulated company
• DEA benchmarks an individual company in relation to the best-practice (most efficient)
companies
• Companies that are able to produce a given output at minimum cost or a maximum output
with a given input define the best-practice frontier that envelops all data points
• Inefficiency is determined by the distance between the observed company and the best-
practice frontier
• Calculation of inefficiency is done via a series of linear programming (mathematical
software needed)
• The programs will output a series of efficiency scores, which may be normalized, ranked,
and split according to a number of components (scale, purely technical, allocative etc.)

11/01/2010 16
• Advantages:
– Multi-dimensional method covering multiple inputs and outputs
– Establishes peer companies
– It does not require functional relationships between input and output factors
– Distinguishes between different types of inefficiency (scale, productive, allocative,
purely technical) in the presence of input (or output) price data
• Disadvantages:
– The results could be influenced by random errors, measurement error or extreme
events
– Results depend on the selection of input and output factors
– Companies exhibiting “extreme” parameters will be classified as efficient “by default”
– Provides no information about statistical significance of the results
– Small samples and a high number of input or/and output variables can result in an
over-specification of the model and “made-up” results for efficiency scores (number of
efficient firms increases with the number of input and output variables)

11/01/2010 17
Parametric / Econometric approaches – Regression analysis
Corrected OLS (COLS)
Ordinary Least Square (OLS)
Most efficient
observation
Input (Costs)
Output
Stochastic Frontier Analysis (SFA)

11/01/2010 18
Ordinary Least Squares (OLS)
• Regression analysis: Mathematical relationship (functional form) that describes the
relationship between a dependent variable and one or more independent variables
• Used to determine the values of parameters that cause the function to best fit a set of data
observations
• The OLS regression line cuts across the observations by minimising the sum of the
squares of the distance (residual) between the line itself and each of the observations
• Fit a line so that, at each point, the (regression) line is close to the corresponding observed
values, while minimising the sum of squared deviations from the line over all the
observable values in the sample
• Efficiency frontier is based on the average cost function
• OLS compares the (in)efficiency of an individual company with the average efficiency level

11/01/2010 19
Corrected Ordinary Least Squares (COLS)
• Estimation of production or cost functions via Ordinary Least Squares
• Use of regression residuals to characterise relative distances between observations in the
sample
• Corrects the regression line by subtracting the largest negative residual (for a cost
function) from the OLS fit (shift the regression line to (unique) best-practice observation)
• Measures the relative inefficiency of all other companies (points) from the line passing
through the largest negative residual (the most efficient company)
• Allows to assess the significance of each network cost driver
• No measurement of stochastic errors
• Requires large data volume in order to create a robust regression relationship
• Very dependent on data quality and, in particular, sensitive to outliers (the company
defining the frontier could just be an outlier!)

11/01/2010 20
Stochastic Frontier Analysis (SFA)
• Uses same premises as COLS, but treats best practice as a “stochastic” process (a mix of
true efficiency and “random noise” effects)
• Several statistical assumptions behind the errors
• SFA requires a large sample size to be statistically relevant
• In the presence of patchy and/or too small samples, COLS is relatively more reliable than
SFA (SFA cannot be drawn as a “frontier” line as COLS)
• Less sensitive to inputs and/or outputs as DEA / COLS
• Allows to assess the significance of each network cost driver
• Considers stochastic errors explicitly
• Complex and statistically demanding
• Requires large data sets in order to create a robust regression relationship
• Genuine inefficiency could be allocated to stochastic elements: scores might be too
generous (too high)

11/01/2010 21
Virtual network models
• Artificially construct an efficient (engineering-designed) reference network according to
commonly accepted planning principles and taking into account technical and geographical
constraints
• The regulated firm’s relative (in)efficiency is estimated by the firm’s performance in relation
to the virtual network
• Virtual network models are not dependent on obtaining and analyzing data of “real”
companies
• Does not require a significant set of comparable companies as benchmarks
• Very complicated and difficult to specify
• Model sensitive to changes in inputs
• Reasons for the deviation from reference network might be beyond control of the company

11/01/2010 22
TOTEX versus OPEX benchmarking
• Building Block Approach
− Implemented as linked (coupled) cap regulation
− Explicit projection of capex for the upcoming regulatory period
− Separate checks and inclusion of investments
− Sometimes formalised efficiency analysis based on controllable opex
• TOTEX Approach
– Implemented as unlinked (decoupled) cap or yardstick regulation
– Inclusion of (historic) capital cost into efficiency assessment modelling (total cost
analysis)
– Standardisation of capital costs for benchmarking purposes
– (Planned) investment for the regulatory period not taken into account for the annual
allowed revenue

• Building blocks (UK, Australia, Central and Eastern Europe) supported by:
– Efficiency carry over schemes
– Sliding scale schemes
• Total cost approach (Germany, Norway, the Netherlands, Austria) supported by:
– Quantity terms (pre-specified cost drivers) incorporated in price control formulas
– Explicit investment allowances
– Inefficiency caps
TOTEX versus OPEX benchmarking
23

11/01/2010 24
Efficiency convergence speed
Allowed revenue
Initial level Initial one-off cut
1 2 3 4 5 Regulatory period
Proportional decrease
• The X-factor prescribes the rate of change in the company’s prices or revenues, reflecting
the expected transition from the existing price level towards the efficient price level
• Regulator to decide whether existing price level serve as starting point for the regulatory
formula or whether one-off cut of the initial price
• Advantage of initial one-off cut,
prices can be brought to more
realistic levels at once
• Large one-off adjustments quickly
eliminate inefficiencies at the
beginning, but decrease incentives
for further efficiency improvements
by the company

11/01/2010 25
Supporting Schemes
• Inefficiency caps (Austria, Germany)
– Germany: Minimum (individual)
efficiency score – 60 %
– Austria: Max. (individual) efficiency
increase – 5.45 %
• Sliding scale (Norway, 1997-2001 and
2002-2006)
– Base return with dead band plus
caps/ collars
• Efficiency bands (Norway, 1997-2001)
Germany
(2009-2013)
Norway
(1997-2001)
KAb,0
Year 1 Year 10
KAdnb,t Permanently
Controllable
costs (base year)
KAvnb,0 Temporary non-
controllable costs
Max. 60% of total costs after deducting of
(permanently) non-controllable cost
(proportionally over 10 years)
-
non-controllable costs
-
15%
Profit floor level
(tariff increase)
Profit cap level
(tariff reduction))
Dead band
8,3%
2 %
Profit floor level
(tariff increase)
Profit cap level
(tariff reduction))
Dead band

Summary
• There are several benchmarking techniques and no consensus amongst regulators as to
which methodology is best
• Data quality and model specification are fundamental for successful and defensible
outcomes
• Benchmarking is an indication and not a confirmation of efficiency position
• Integration of benchmarking results should take into account its imperfections and the
specifics of the price control design

Experience you can trust.
End of webinar 6
KEMA Consulting GmbH
Kurt-Schumacher-Str. 8, 53113 Bonn
Tel. +49 (228) 44 690 00
Fax +49 (228) 44 690 99
Dr. Konstantin Petrov
Managing Consultant
Mobil +49 173 515 1946
E-mail: konstantin.petrov@kema.com

Training Module on Electricity Market Regulation - SESSION 6 - Efficiency Assessments

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Training Module on Electricity Market Regulation - SESSION 6 - Efficiency Assessments

Similar to Training Module on Electricity Market Regulation - SESSION 6 - Efficiency Assessments (20)

More from Leonardo ENERGY

More from Leonardo ENERGY (20)

Recently uploaded

Recently uploaded (20)

Training Module on Electricity Market Regulation - SESSION 6 - Efficiency Assessments