tracking procedures ERA

Emma Hooper, Daniel Lewis The Application of Selective Editing to
and Claire Dobbins the ONS Monthly Business Survey
SMB 68 3/111
The Application of Selective Editing to the ONS
Monthly Business Survey
Emma Hooper, Daniel Lewis and Claire Dobbins
Methodology Directorate, ONS
Summary
This is an updated version of a paper by Emma Hooper and Daniel Lewis, originally
presented at the Q2010 conference in Helsinki.
When running surveys it is important to detect and correct errors in the response data
in order to maintain the quality of results. The UK’s Office for National Statistics
(ONS) is undertaking improvements to its data editing strategy across business
surveys through the introduction of selective editing.
The selective editing approach that the ONS has taken with its short-term business
surveys is to use an estimate-related score function to create item scores for key
variables. These item scores are combined into a unit score, thresholds are then
constructed for the unit score to determine whether a unit will be manually edited or
not. Various quality indicators such as bias, change rates and savings are used
throughout this process in order to choose an appropriate selective editing approach.
Two existing ONS short-term business surveys have recently been combined into a
new survey called the Monthly Business Survey (MBS). The development of MBS
was an opportunity to improve the survey process using selective editing. This survey
is presented as an example of how selective editing has been implemented at ONS,
and various quality indicators are presented showing the impact of selective editing
on the published estimates and the savings that can be achieved over the traditional
edit rule approach.
1. Introduction
For any survey it is important to detect and correct errors in the response data in order
to maintain the quality of results. The UK’s Office for National Statistics (ONS) has
been undertaking improvements to its data editing strategy across business surveys
through the introduction of a new selective editing approach for its short-term
business surveys. The recent development of the Monthly Business Survey was an
opportunity to improve the survey process by applying the new ONS selective editing
methodology to this survey.
Data editing is an expensive part of the survey process in terms of processing time,
operating costs, and burden on respondents. ONS is hoping to reduce these costs
through focussing editing efforts onto those records with the largest impact on the
published estimates, while still maintaining the quality of those estimates.
The selective editing approach that ONS has taken with its short-term business
surveys is to use an estimate-related score function (Hedlin 2003) to create item
scores for key variables. These item scores are combined into a unit score, thresholds

SMB 68 3/112
are then constructed for the unit score to determine whether a unit will be manually
edited or not. Various quality indicators such as bias, change rates and savings are
used throughout this process in order to choose appropriate threshold values.
This paper outlines the process undertaken in order to apply selective editing to the
ONS Monthly Business Survey (MBS). Section 2 introduces MBS, while section 3
explains the current use of selective editing at ONS. Section 4 outlines the selective
editing methodology specifically for MBS, and section 5 presents some quality
indicators of the impact of selective editing. An explanation of the system used for
implementation and some of its current limitations is provided in section 6, followed
by some post implementation plans in section 7.
2. Monthly Business Survey
MBS was launched in January 2010 with its first publication “Turnover and Orders in
Production and Service Industries”, released in March 2010 (ONS 2010). It brings
together existing short-term surveys that cover different sectors of the economy. The
existing surveys that have moved to MBS are the Monthly Inquiry into the
Distribution of Services Sector (MIDSS) and the Monthly Production Inquiry (MPI);
these surveys are now processed as one survey. Other short term surveys – the Retail
Sales Inquiry (RSI) and the Construction Statistics surveys, have been rebranded as
MBS as well, but are processed separately for now. The benefits of bringing these
surveys together include: harmonised methodology and consistent processes; a
reduction in burden of businesses through less frequent collection of some variables
and dropping some questions entirely; and a reduced number of surveys and systems
to maintain in the long term.
As part of the aim to harmonise the methodology, the existing editing processes were
reviewed and it was decided that MBS would be a good candidate for using the new
selective editing methodology. References to MBS in the remaining paper do not
include the RSI or Construction Statistics surveys.
The MBS questionnaire is sent out to approximately 30,000 businesses every month.
The data feed into the Index of Services, the Index of Production and consequently
the output measure of Gross Domestic Product. The main variables collected are
Turnover, Total number of employees and Employees by male / female, part-time /
full-time. The variables are collected monthly, except for the employment variables,
which are only collected quarterly for a sub-sample of the units. Additional variables
are collected on some specific questionnaires sent to a small number of industries.
3. Selective editing at ONS
Selective editing was implemented in MIDSS in August 2001; however, this was a
slightly different approach to the selective editing methods discussed in this paper
(Underwood 2001). The historic selective editing method for MIDSS was for
returned questionnaires to pass sequentially through: automatic editing; edit rules;
and selective editing score calculation. All edit rule failures that involved missing or
inconsistent values in the key variables were checked, as were any edit rule failures
incurred by new respondents to the survey. If a return still had edit failures remaining
after these checks, then this return would pass through to the score calculation stage.
Only those returns with an estimate-related item score greater than a set threshold

SMB 68 3/113
would have their remaining edit rule failures checked. Following the MIDSS
implementation, the same selective editing method was implemented for MPI.
In 2008 a project was set up at ONS to fundamentally review the editing processes for
ONS business surveys. A result of this project was the development of a new
selective editing methodology for ONS short-term business surveys through
collaboration between the ONS Methodology Directorate and Pedro Luis do
Nascimento Silva of Southampton University.
RSI was used to develop the new ONS selective editing method; using the key
variables Total turnover and Total employment (Silva 2009). How this methodology
has been applied to MBS is described in section 4.
4. Outline of selective editing methodology for MBS
In order to complete the analysis of the current micro editing approach and specify a
selective editing methodology for MBS, 24 periods of edited and unedited data were
used for the period January 2008 to December 2009. The large number of periods
used should ensure the robustness of the results.
4.1 Key variables and domains
The first step in developing a selective editing methodology for MBS was to identify
key variables and domains. Following discussions with the MBS results team, they
identified four key variables – Turnover, Export turnover, New orders (all collected
monthly), and Total employment (collected quarterly). Export turnover and New
orders are only collected from units that were in industries previously covered by
MPI. Additionally, some industries that were previously in MIDSS have Commission
and Sales collected instead of Turnover. In this situation Commission and Sales were
summed to give a Turnover value. This is also the case for some industries which
collect Sales and Invoices; these were summed to give Total turnover.
The key domains identified are UK National Accounts industry input/output groups;
these are roughly equivalent to 2 or 3 digit UK Standard Industrial Classifications
2007 (SIC07). SIC07 is consistent with the latest version of NACE (the European
industrial activity classification) at the 4 digit level. For MBS there are around 80
input/output groups in scope.
4.2 Item and unit score functions
Estimate-related score functions focus on large contributions to the key survey
estimates, in this case Turnover, Export turnover, New orders, and Total employment
by input/output group. An item score is calculated for each key variable, for each
unit. The item score used is
1
ˆ
100
ˆ
t t t
i ij ijt
ij t
jd
a z y
score
T −
−
= × (1)

SMB 68 3/114
1
is the sample design weight for unit at time
is the unedited value for variable , unit at time
ˆ is a predicted value for variable , unit at time
ˆ is the previous perio
t
ij
t
ij
t
ij
t
jd
a i t
z j i t
y j i t
T −
d's total estimate for variable in domain .j d
The sample design weight is used because calibration weights tend not to be available
until later in the survey process. The process for selective editing should be that the
item scores get calculated after batch take-on, so that editing can begin as soon as the
first batch of data is returned and processed.
The predicted values used for MBS are previous edited value where available (this is
the value from the previous month, except for Total employment where it is the value
from the previous quarter). If this is not available then a corresponding register value
for the current period is used for the variables Turnover and Total employment. For
Export turnover and New orders, a register value is not available, so a pseudo-
imputed value is used instead. The predicted value is thus defined
1
1
* 1
if available for unit
, for Turnover or Employment
, for E
ˆ _ if is not available
if is not available
t
ij
t t t
ij i ij
t t
ij ij
iy
y Selection x y
y y
−
−
−
=
xport turnover or New orders.
⎧
⎪⎪
⎨
⎪
⎪⎩
(2)
1
*
is the edited value for variable in the previous period for unit
_ is the corresponding register value for Turnover or Employment for unit
t
ij
t
i
t
ij
y j i
Selection x i
y
−
is the current period pseudo-imputed value for Exports or New orders for unit .i
The pseudo-imputed value used for Export turnover and New orders when there is no
available previous value, is defined as:
* 1
_t t t
ij SIC iy CLink Selection to−
= × (3)
1
1
1
11
1
_
t
impclass tn
ij
t
it i
SIC t
impclass
y
Selection to
CLink
n
−
−
−
=−
−
=
∑
(4)
1
_ is the current period register value for Turnover for unit
is the constructed value link (a trimmed ratio of averages) from the previous period
t
i
t
SIC
Selection to i
CLink −
1
for the 4-digit SIC07 corresponding to this period for unit
is the number of units left in the imputation class after trimming, in the previous
per
t
impclass
i
n −
iod.
We are not able to use 1
_ t
iSelection to −
in the calculation of *
ˆ t
ijy because the
corresponding register turnover value is not be held in the processing system for an
enterprise that was entering the survey for the first time this period. We are currently
restricted to only using variables that are available in this system.

SMB 68 3/115
We then want a single selective editing score for each unit; we call this the unit score.
Only those enterprises with a unit score above a preset threshold are selected for
manual micro editing. The size of the item scores give an indication as to which
variables have contributed most to the unit score and caused it to fail editing.
However, the principle used at ONS is that all variables on the questionnaire should
be validated if the questionnaire has been selected for manual micro editing. Those
enterprises that have a unit score below the threshold will not have their returned
questionnaires edited any further, unless they are later identified in macro editing.
For the original RSI selective editing study we combined the item scores into a unit
score using the unified global score function for selective editing presented in (Hedlin
2008). The method is based on the Minkowski distance function and is defined as:
1
1
( )
p
t t
i ij
j
u score
λ
λ
−
=
⎛ ⎞
= ⎜ ⎟
⎝ ⎠
∑ (5)
is the number of item scores for unit
1 is equivalent to sum of the item scores (or, equivalently, the mean)
2 is equivalent to the Euclidean distance
Large values of would be roughly equ
p i
λ
λ
λ
=
=
ivalent to the maximum item score.
It was found in the RSI study that there was no strong evidence to support choosing
one value for lambda over the others that were tested. Limitations with the processing
system meant that we were restricted to using the mean or maximum of scores. For
RSI the decision was made to use the average of scores. For MBS we first used the
mean of scores and found this to perform satisfactorily, so have not tested the
maximum of scores. Choosing the mean of scores means that each key variable has
an equal influence on the unit score. The maximum number of item scores that a unit
in the MBS could have is four – in an end of quarter month where Turnover, Total
employment, Export turnover and New orders were collected.
4.3 Setting of thresholds
We were concerned with three main quality indicators when we set the threshold for a
domain. These are Relative Bias (RB), Absolute Relative Bias (ARB) and Savings,
defined in formulae 6-8. The bias measures are indicators of the bias that would
remain in the domain estimate if the enterprises with a unit score below the threshold
were not edited. We make the assumption that the historic domain estimates after
micro and macro editing are the ‘true’ figures. Note that in practice MBS data were
already subject to a more conservative selective editing method, as described in
section 3. Therefore, the bias measures are actually measuring the additional bias in
introducing this new selective editing method.
We were interested in the relative bias because it gives us an indicator as to whether
there is any systematic up or down movement in the level of bias. The absolute
relative bias gives us a measure of the overall bias in the domain estimate as well as
being an indicator of the potential bias in future survey rounds. The savings are
measured by the relative change in the number of units that are currently edited and
the number that will be edited under selective editing for the given threshold and
domain.

SMB 68 3/116
( )ˆ ˆ( )
t
t t t t t t
jd ij ij ij i d jd
i s
RB w z y I u c T
∈
= × − × <∑ (6)
( )ˆ ˆ| |
t
t t t t t t
jd ij ij ij i d jd
i s
ARB w z y I u c T
∈
= × − × <∑ (7)
is the sample at time
is the estimation weight for variable , unit at time
is equal to 1 if the unit score for unit at time is less than threshold for domain , else it is equ
t
t
ij
s t
w j i t
I i t c d al to 0.
t t
t d d
d t
d
trad select
Savings
trad
−
= (8)
is the number of units failing at least one traditional edit rule at time in domain
is the number of units with a unit score above the threshold at time in domain .
t
d
t
d
trad t d
select t d
The decision was made with the business area survey owners that we would aim to
keep the estimated absolute relative bias for a domain estimate below 1%. With the
bias at this level, there should be negligible difference between the domain level
estimate based on selective editing and the domain level estimate based on the current
MBS edit rules. Some bias outliers would be allowed over the periods of data, as
these would be picked up at the macro editing stage if they had an unacceptable effect
on the domain level estimates. A range of thresholds were looked at for each domain,
and a threshold was selected that balanced the need to control the level of bias while
also aiming to make positive savings in the number of units that would be edited.
Examples of the types of graphs used to help inform the choice of thresholds follow
in graphs 1 and 2.
Each box plot in graphs 1 and 2 contains quality indicators for a number of periods.
23 periods of data are contained in each box plot. There is not a measure for the first
period of the data, as we do not have previous values to use as the predicted value, so
it is excluded from the analysis. The ten thresholds in the graphs were set to different
values for the different domains. If none of the initial ten thresholds tried proved to
have an acceptable level of bias and savings then a different group of threshold levels
were tried. We can see in graph 2 that the median level of savings does not increase
rapidly after the fourth threshold, and that the fourth threshold in graph 1 has an
acceptable level of absolute relative bias which is below 1% of the domain level
estimate. There does not appear to be many additional savings to be made by
increasing the level of absolute relative bias, so in this situation the fourth threshold
seems to be a suitable level for this domain and variable. We would then need to
check the bias graphs for all of the other key variables to make sure they were at an
acceptable level. We would also want to look at the choice of threshold on the overall
level of savings for the survey, in addition to the domain level savings, and see
whether the increased level of bias between thresholds 3 and 4 was worth the extra
savings achieved.

SMB 68 3/117
Graph 1. Absolute relative bias for variable j, domain d, for 10 different
thresholds
1 2 3 4 5 6 7 8 9 10
0. 000
0. 500
1. 000
1. 500
2. 000
A
b
s
B
i
a
s
1
cut of f
Graph 2. Savings for domain d, for 10 different thresholds
1 2 3 4 5 6 7 8 9 10
0. 00
25. 00
50. 00
75. 00
100. 00
R
e
l
S
a
v
i
n
g
s
_
S
c
o
r
e
1
cut of f
For some of the domains the number of units in a given month can be a lot smaller
than other domains, for example there could be less than 50 units. For these smaller
domains there may not be many savings to be made over the current number of units
being edited, in which case we are more concerned with controlling the estimated
bias and editing a sensible percentage of units in the cell than making savings.
5. Quality indicators
As mentioned in section 4, we are interested in overall level impact as well as domain
level impact. There are a number of quality indicators we could use to see how well
the selective editing methodology and choice of parameters is performing in
comparison to the current MBS editing strategy.
For MBS, the initial investigation showed median overall savings (see formula 9) of
approximately 46% in the months January, February, April, May, July, August,
October and November. The median overall savings in the months March, June,
September and December were approximately 3%. On these months, the employment
variables are also collected. It would not be possible to achieve many savings in these
months whilst maintaining the given level of accuracy for the Total employment
variable, as well as the other key variables which are collected on a monthly basis.

SMB 68 3/118
t t
t
t
trad select
OverallSavings
trad
−
= (9)
is the number of units failing at least one traditional edit rule at time for the chosen
thresholds
is the number of units with a unit score above the threshold at time f
t
t
trad t
select t or the chosen
thresholds.
A comparison between the estimated absolute relative bias left in the domain
estimate after selective editing and the absolute relative bias left in the domain
estimate after the current micro editing showed large improvements across MBS
domains. We took the final macro edited values to be the true estimates. This means
that many errors that are left after the current micro editing and which are being
captured at the macro editing stage will instead be captured at the selective editing
stage. In practice this could lead to savings in the survey area because many errors
that are currently identified at the macro editing stage would already be checked and
cleared. The method and parameters we are using for the MBS selective editing
methodology are clearly performing well at identifying influential errors at the micro
editing stage.
Further useful quality indicators are edit failure rates and edit change rates. These are
defined as follows:
100
number of units being micro edited
Edit failure rate
total number of responding units
= × (10)
100
number of units with value changes as a result of micro editing
total number of responding units
Edit change rate= × (11)
These give us a further indication of how well micro editing is performing, whether it
is the selective editing approach or the current edit rule approach. One would hope
that the edit failure rate is not too high, as all these units need to be checked which is
a resource expense. Furthermore for efficiency reasons we would hope that the edit
change rate was similar to the edit failure rate, because if time is spent checking a unit
we would want to find an error in need of correction.
Graphs 3, 4, 5 and 6 give us evidence that the new selective editing methodology is
more efficient than the previous edit rule based method used for micro editing.

SMB 68 3/119
Graph 3. Edit failure rates and edit change rates after using edit rules to
identify units to edit from the old MPI domains
0
5
10
15
20
25
30
35
40
45
200804
200806
200808
200810
200812
200902
200904
200906
200908
200910
200912
%
Edit failure rate Edit change rate
Graph 4. Edit failure rates and edit change rates after using selective editing to
identify units to edit from the old MPI domains
-5
5
15
25
35
45
200804
200806
200808
200810
200812
200902
200904
200906
200908
200910
200912
%
Graph 5: Edit failure rates and edit change rates after using edit rules to
identify units to edit from the old MIDSS domains
0
5
10
15
20
25
30
35
200804
200806
200808
200810
200812
200902
200904
200906
200908
200910
200912
%

SMB 68 3/1110
Graph 6: Edit failure rates and edit change rates after using selective editing to
identify units to edit from the old MIDSS domains
0
5
10
15
20
25
30
35
200804200805200806200807200808200809200810200811200812200901200902200903200904200905200906200907200908200909200910200911200912
%
The edit failure rate is smaller in Graph 4 than Graph 3, which reflects the savings we
have made. Also, the gap between the edit failure rate and the edit change rate has
decreased showing that the number of units changing as a result of micro editing is
closer to the number of units that have been identified for editing.
In Graph 5, we can see that there were more edit failures in 2008 than there were in
2009. This was due to efficiencies that were implemented into MIDSS, such as
telephone data entry (TDE) which reads back answers to respondents and gives them
the opportunity to correct errors. When comparing Graphs 5 and 6, we can see that in
the months January, February, April, May, July, August, October and November, we
have consistently achieved a lower edit failure rate under selective editing. However,
for March, June, September and December in 2009, we have a slighter higher edit
failure rate under selective editing. This is due to incorporating the Total employment
variable into the selective editing process, which has been discussed previously in this
paper (see section 5).
6. Editing process, implementation and system limitations
MBS will still have some edit rule checks for valid dates or comments in the
comment box, followed by automatic editing for the “thousand pounds error” and
component checks. The thousand pounds error identifies where it is likely that the
respondent did not round the returned figure to the closest thousand as required. If
this error is identified then the automatic editing will divide the returned figure by
1000. Component checks are for employment questions where the female/male/part-
time/full-time splits do not add up to the given total number of employees. Selective
editing will be carried out after automatic editing, and macro editing will then take
place before publication. The continued use of edit rule checks for date validation
will impact on the level of savings presented in the previous section.
ONS uses an in-house built system called Common Software to store and process
data for most short-term business surveys. A module has been developed to carry out
selective editing, as specified in section 4, in the Common Software system. This
module does have some limitations on what Methodology Directorate can specify for
the selective editing methodology. These current limitations include:
• being restricted to specifying at most five key variables for which
item scores will be calculated;

SMB 68 3/1111
• being restricted to only using variables already available in Common
Software for use in calculating the predicted values;
• being restricted to combining item scores into a unit score through
either a maximum function or an average function;
• being unable to use current edit rules to calculate other types of
selective editing score.
7. Future work and post implementation
The new selective editing methodology and associated thresholds were implemented
into the Monthly Business Survey in August 2010. ONS plans to continue testing and,
where appropriate, implementing selective editing methods for other short-term and
some annual business surveys. This will lead to more efficient editing, resulting in a
better quality editing process.
Post implementation, ONS will want to know how well the new selective editing
methodology is performing. As there will be no micro edited data for the enterprises
that have a score below the thresholds we will not have information to monitor the
performance and review the thresholds. Evidence can be gathered though by selecting
a small sample of those units that are not selected for micro editing; enterprises in this
sample will then be edited. This will enable us to calculate a measure of the estimated
bias that is left in the survey estimates by those units that are not selected for editing.
We can then monitor this bias or adjust the thresholds accordingly.
References
Hedlin D (2003) ‘Score Functions to Reduce Business Survey Editing at the UK
Office for National Statistics’, Journal of Official Statistics 19, pp 177-199.
Hedlin D (2008) ‘Local and global score functions in selective editing’, UNECE
Work Session on Statistical Data Editing, Vienna, 21-23 April 2008.
ONS (2010) ‘Changes to the survey outputs from MBS (formerly ETOD and DST)’,
available at: http://www.statistics.gov.uk/articles/nojournal/Changes-survey-outputs-
MBS.pdf
Silva P L N (2009) ‘Investigating selective editing ideas towards improving editing in
the UK Retail Sales Inquiry’, European Establishment Statistics Workshop,
Stockholm, 7-9 September 2009.
Underwood C (2001) ‘Implementing selective editing in a monthly business survey’,
Economic Trends 577.

SMB 68 3/1112
Tracking Procedures on the Employment, Retention
and Advancement Survey
Kathryn Ashton and Martina Portanti
ONS, Social Survey Division
Summary
Longitudinal surveys are subject to attrition throughout their waves. This is because it
can be difficult to locate respondents and ensure they are able to be contacted at each
wave and individuals may be more likely to refuse an interview over subsequent
waves of a survey. Locating respondents between waves of a longitudinal study is
particularly complex in countries where a central population register does not exist.
This is a problem for the United Kingdom. In recent years, the linkage of individual
level data from administrative sources with details from longitudinal survey
respondents has become increasingly common to help locate respondents at the
present location.
This research presents the results of using both prospective and reactive tracking
methods to locate individuals participating in Wave 3 of the Employment, Retention
and Advancement (ERA) survey carried out by the Office for National Statistics
(ONS). At Wave 2 of ERA more than 15 per cent of the sample could not be located.
This was predicted to increase at Wave 3, which was carried out three years later. In
order to improve the accuracy of contact details, ONS used a number of different
tracking methods, including a telephone unit (TU) ‘keep-in-touch’ exercise and
linkage to several administrative sources in an attempt to increase contact rates. We
report on the results of these methods in terms of improvements to contact and
response rates, and changes in the composition of the respondent group. We conclude
with an assessment of the tracking procedures used and of their potential for a more
extensive use as part of ONS longitudinal surveys’ tracking strategy.
1. Introduction
In the context of longitudinal surveys, the term attrition is normally used to refer to
the loss of survey participants over time. Although attrition may occur for a number
of different reasons, researchers tend to be particularly concerned with attrition due to
non-response, i.e. loss of sample members at follow-up because they cannot be
contacted or they refuse to continue participation (Watson and Wooden, 2004).
Attrition due to non-response not only may decrease the power of longitudinal data
analysis but it may also be selective, thus impacting on the generalisability of results
to the target population.
The process that leads to attrition due to non-response can be divided into three
conditional processes: failure to locate respondents; failure to contact respondents,
having locating them; and failure to obtain cooperation from respondents, having
contacting them (Lepkowski and Couper , 2002).
In particular, failure to locate respondents may be a major contributor to attrition in a
longitudinal study. This was the case for the third Wave of the Employment

Kathryn Ashton and Martina Portanti Tracking Procedures on the Employment,
Retention and Advancement Survey
SMB 68 3/1113
Retention and Advancement survey (ERA), carried out by the Office for National
Statistics between December 2008 and March 2010.
This paper describes the methods that were used on the ERA in order to track sample
members. It illustrates the outcome of these methods in terms of improvements to
contact and response rates and changes in the composition of the respondent group.
2. Employment Retention and Advancement Survey (Wave 3)
The Employment Retention and Advancement (ERA) programme was designed to
test a method to improve job retention and advancement among low-income
individuals and consisted of a combination of job coaching and financial incentives
which were offered to participants once they were working. It was implemented
between 2003 and 2004 in six pilot areas across the UK.
The ERA programme was implemented using a random assignment process.
Individuals who volunteered to be included in the programme were randomly
assigned either to the ERA programme or to a control group, which did not have
access to the ERA services.
As part of the ERA policy evaluation, a large-scale longitudinal survey with a sample
from both the ERA programme and control group members was carried out by ONS.
The survey has so far followed the respondents for 5 years through 3 waves of
interviews. Wave 1 was carried out between December 2004 and March 2006; Wave
2 from December 2005 to March 2006; and Wave 3 from December 2008 to February
2010. The ERA survey followed a mixed-mode design. All sample members were
attempted to be contacted and interviewed by the Telephone Unit. Non-contacts and
circumstantial refusals were then reissued to face-to-face interviewers to attempt
conversion.
At Wave 2, around 15% of the sample issued to field could not be located. This
percentage was expected to increase even further at Wave 3 for a number of reasons.
Firstly, the time gap between the Wave 2 and 3 interviews was much longer than
between the Wave 2 and Wave 1 interviews (3 years versus one year). Secondly,
between Wave 2 and 3, funds were not available to carry out any keep-in-touch
exercise to try and maintain respondents’ contact details up-to-date. Finally, the Wave
3 sample also included non respondents at previous waves. This means that the Wave
3 survey could have been the first time some sample members were contacted since
their enrolment into the ERA programme five years before.
In Wave 3, both prospective and reactive techniques were used in an attempt to locate
sample members.
3. Prospective tracking
Forward-tracing or prospective techniques are tracking methods that try to ensure that
up-to-date contact details for all sample members are available by the start of the
fieldwork period. Information is normally gained from the respondent themselves,
either as recorded at the latest interview or by updating the contact details before the
next wave occurs (Burgess, 1989; Couper and Ofstedal, 2009). These methods are
relatively inexpensive and have proved successful, as the most useful source of
information for tracking is often the participants themselves (Ribisl et al, 1996).

SMB 68 3/1114
On ERA two forward-tracing methods were used. Postal and telephone keep in touch
exercises were carried out before the start of the mainstage fieldwork to try and
update sample members’ contact details.
3.1. Postal ‘Keep-in Touch’ Exercise (KITE)
Between May and June 2008, all Wave 3 sample members (5,688 individuals) were
sent a pre-contact letter. The letter included a leaflet setting out the purpose of the
survey, a postcard so that sample members could inform ONS of any changes to their
contact details, and an ERA/ONS labelled key ring as an unconditional incentive.
Around 7% of the ERA sample returned the postcard to ONS with updated contact
details.
3.2. Telephone Unit KITE
Each sample member was also contacted for a keep-in-touch interview by the ONS
Telephone Unit (TU) around 3 months before their main stage interview. The aim of
this short questionnaire was to ask respondents about their most up-to-date contact
details, and also inform respondents they were to be contacted in 3 months time for
the main stage interview.
Overall, the TU KITE, which was administered from September 2008 to September
2009 achieved a response rate of 52.4 per cent and 54 per cent contact rate. Table 4.1.
shows the type and amount of information which was collected during the TU KITE.
Table 3.1. Information Collected at the TU KITE
TU KITE outcome TU KITE Responders Total Sample Size
Update some information 35.1% 18.4%
Address Update at TU KITE 14.1% 7.4%
Telephone number Update at TU
KITE 16.2% 8.5%
Both telephone number and
address update at TU KITE 4.8% 2.5%
No new details update at TU KITE 64.9% 34.0%
Total TU KITE respondents 100.0% 52.4%
Total TU KITE non respondents 47.6%
Around 35 per cent of those individuals who completed a TU KITE interview
updated at least one element of their contact information (address and/or phone
number): 14.1 per cent updated their address details; 16.2 per cent gave details of a
new telephone number; and 4.8 respondents updated both their telephone and address.
This means that the TU KITE collected new contact details for around 18.4% of the
sample that was later issued to the mainstage and confirmed existing details for 34%
of the sample.

SMB 68 3/1115
The outcome of the TU KITE was a strong predictor of the outcome at the mainstage
interview, as shown in table 3.2.
Table 3.2. TU KITE and mainstage outcomes
TU KITE outcome
Respondent Non respondent
Response rate at mainstage 78% 20%
Proportion of cases reissued to
field at mainstage 21% 70%
Contact rate among cases
reissued to field 83% 51%
Response rate among cases
reissued to field 59% 32%
Around 78% of those individuals who were successfully contacted for the TU KITE
exercise responded to the mainstage interview. Among TU KITE non-respondents,
only around 20% of the sample were respondents at the mainstage interview. TU
KITE non-respondents were also less likely to be interviewed in the Telephone Unit
and they had to be reissued to a face-to-face interviewer more often. Around 70% of
the cases who did not respond to the TU KITE had to be reissued to a face-to-face
interviewer against 20.5% of the TU KITE responding cases.
Once in field, TU KITE non-productive cases were again less likely to be contacted
and interviewed. Only 51% of TU KITE non-responding cases who were re-issued to
the face-to-face field force were successfully contacted and only 32% successfully
interviewed. In the group of TU KITE respondents who were re-issued to face-to-face
interviewers, around 83% of cases were successfully contacted (with around 59%
being interviewed).
Overall, cases who did respond at the TU KITE proved to be less costly to be
contacted during the mainstage, as they were more often interviewed by the telephone
unit and when reissued to field, they were more likely to be contacted.
4. Reactive techniques
On completion of the ERA main stage field period, the survey achieved a response
rate of 62.9 per cent and a non-contact rate of 22.3 per cent. The main reason for non-
contacts was due to inaccurate and out-of-date contact details. In an effort to improve
response, additional tracking was attempted after the end of the mainstage field
period. Three administrative sources were used to try and update contact details for
1,010 sample members who did not respond to the mainstage ERA interview, in the
vast majority of cases because they had moved since the last time they had been
contacted and their current whereabouts were unknown.

SMB 68 3/1116
4.1. Use of administrative sources
Administrative data sources are increasingly being used to track respondents between
waves of longitudinal surveys. Access to administrative records in the UK is
regulated by national legislation. Government departments have put in place specific
data sharing agreements which often require explicit consent to be collected from
individuals before their data can be shared across departments.
ERA participants provided, back in 2001, consent for data held by Inland Revenue
(now HMRC) and DWP, to be accessed and used as part of the ERA evaluation
project. On this basis, in 2010 ONS sought access to HMRC tax credit records and
DWP benefit records to update ERA respondents contact details. HMRC refused
access to their data as they deemed the consent for data sharing being too old; DWP,
as joint data controller and sponsor of the scheme, could instead provide access to
their benefit records.
DWP provided ONS with updated contact details, including addresses and telephone
numbers, for all ERA sample members who were present on the DWP benefit record
database. The data was passed confidentially and securely across the Government
Secure Intranet. Once the updates had been received, it was simple to match the new
contact details to the ERA sample using an encrypted national insurance number.
Additional contact details were then linked to the sample members using the Royal
Mail National Change of Address (NCOA) Register, which is sourced from the Royal
Mail Redirection Service. The NCOA register not only identifies that an individual
has moved from their original address, but also offers details of where that individual
has moved to. Access to the database is provided by a number of commercial
organisations, which provide online facilities to search and match addresses against
the NCOA database.
New telephone numbers were also attached to sample members using information
sourced from the British Telecom Operator Services Information System (OSIS). The
OSIS database is updated on a daily basis and it contains more than 26 million
residential and business names, addresses, and telephone, although it does not include
ex-directory and mobile telephone numbers. Access to the database was obtained
through one of OSIS licenced commercial users.
Finally, the Electoral Register was searched for those cases where a new address
could not be found either in the DWP records or in the NCOA register. ONS holds a
copy of the full electoral register, listing names and addresses of all individuals in the
UK who are entitled to vote. The search of the Electoral Register proved to be
extremely complex and resource-intensive for several reasons. Firstly, the only
information available on the electoral register for matching purposes were
individuals’ names and address. This made it difficult to try and identify a correct
match for people with common names and individuals whose name may have been
misspelt. Secondly, cases had to be searched one by one, as no facility was available
to automatically link a list of names and/or addresses to the register. Finally, the
register is actually a collection of separate databases organised by each Local
Authority (LA) within the UK. Each LA database had to be searched separately, as no
facility existed to search a name or address across several LA databases.
For these reasons, and because ERA face to face fieldwork did not follow movers out
of the areas of previous residence, ERA sample members were searched only on the
electoral register of the Local Authority of their latest address. Firstly, the database of
the LA where the respondent last lived in was searched to check whether the
respondent was still resident at the most recent address held by ONS. If different

SMB 68 3/1117
names were found at the address, then a search by name and surname was performed
on the LA database. If one match was found, than the new electoral register address
was assigned as an alternative address for that case. If more than one match was
found or there was no match, no alternative address was assigned.
4.2. Quality of new contact information
After updating their contact details, cases were included in one of three mop-up
exercises which took place in July 2009, October 2009 to March 2010 and January to
March 2010. The mop-up exercises took place after the end of the mainstage
fieldwork period as a final effort to try and obtain interviews from cases who were
still uncooperative. The first mop-up exercise was implemented solely in the TU
using updated telephone numbers from the administrative sources. The second and
third mop-ups were designed in the same way as the main stage; all cases were sent to
the TU with updated telephone numbers and all TU non-contacts were then re-issued
to the face-to-face interviewers.
If an alternative address, different from that held by ONS and used during the
mainstage fieldwork, was found in one of the administrative sources, this was used
either as a primary or a secondary address during the mop-up fieldwork. If a case had
been coded as a non-contact at mainstage because the address details were incorrect,
then the address obtained from the administrative sources was used as the primary
address and interviewers were instructed to call at that address only. For all other
mainstage outcome codes (e.g. non-contact because nobody was at the address), the
case was issued to the mop-up fieldwork with both the mainstage address and the
alternative address obtained from the administrative sources. Interviewers were
instructed to attempt contact first at the mainstage address, and if unsuccessful, then
attempt contact at the new address, if in the same local area.
The quality of the additional contact details proved complex to analyse. If a full
interview was obtained during the mop-up fieldwork, respondents up-to-date contact
details were recorded at the end of the interview. This allowed us to check whether
the contact details obtained from the administrative sources proved to be correct in
field. If however the case was not productive, it was often impossible to establish
whether interviewers had actually used the new contact details and whether these had
proved to be correct or not.
Table 4.1 summarises the administrative sources uses, how many contact details were
obtained through each of the sources and how many of these could be confirmed with
certainty whether they were correct or not at the end of the mop-up exercise

SMB 68 3/1118
Table 4.1. Contact details updates by administrative source used
New addresses
New telephone
numbers
Administrative
source Found
Confirmed
incorrect
Confirmed
correct Found
Confirmed
correct
DWP benefit
records
588
(100%)
109
(18.5%) 88 (15%)
345
(100%)
60
(17.4%)
NCOA
36
(100%) 6 (16.7%) 4 (11.1%) Not applicable
Electoral
register
49
(100%)
10
(20.4%) 8 (16.3%) Not applicable
BT OSIS Not applicable
26
(100%) 8 (30.8%)
All sources
673
(100%)
125
(18.6%)
100
(14.9%)
371
(100%) 68
5. Response rates and respondents characteristics
The three mop-up periods proved successful in improving both response and contact
rates. The final response rate increased of five percentage points, from 62.9 per cent
to 67.9 per cent, and the non-contact rate was reduced from 22.3 per cent to 17.8 per
cent.
Although an increase in response rates is beneficial for the statistical power of the
analysis, it does not provide any indication about whether the additional tracking
helped and reduced the non-response bias. Non-response may be selective, in that the
individuals who do not take part in survey may differ from those who remain in the
sample. This is a concern for analysts as it impacts on the generalisability of results to
the entire target population.
For longitudinal studies in particular, much research has shown how characteristics of
individuals lost at follow-ups because they cannot be located tend to differ from those
of respondents (Hawkes and Plewis, 2006; Grey et al, 1996). Additional tracking
efforts may help to reduce attrition bias by ensuring hard-to-reach group can be
located.
In order to investigate whether the additional tracking carried out at the end of the
ERA mainstage field period helped and reduced potential attrition bias, we used
pooled T-tests to compare the characteristics of those individuals who responded to
ERA during the main stage with those of the individuals who were instead
interviewed during the mop-up periods.
As shown in table 5.1, the proportion of single respondents interviewed during the
mop-up exercises was significantly higher than the proportion recorded at the main
stage. This agrees with the literature that single people are more likely not to be

SMB 68 3/1119
located or contacted at a follow-up of a longitudinal survey (eg. Grey et al, 1996), and
supports the assumption that single respondents may require more tracking effort to
be located.
Table 5.1. Marital status of respondents from the main stage and mop fieldwork
periods
Marital status Main stage Mop-up
Single *** 45.2% 60.1%
Married and Living with Partner 10.5% 10.4%
Civil Partner 0.7% 0.3%
Married and Separated 6.9% 7.2%
Divorced *** 34.6% 20.4%
Widowed 2.0% 1.6%
Total 100.0% 100.0%
*** Significance level of less than 1%
The difference in proportions of divorcee respondents in the main stage sample and in
the mop-up sample also shows a significant difference. The proportion of divorced
respondents at the main stage was 34.6 per cent, whereas only 20.4 per cent of the
ERX respondents were divorced.
Mop-up respondents were more likely to rent their home compared with mainstage
respondents (table 5.2.). Overall 79.5 per cent of respondents at ERX indicated they
rented, compared to 66% among mainstage respondents. This results also supports
survey attrition literature, which finds that people who rent their home are more likely
to move and change their contact details (Watson and Wooden, 2004), thus being
more difficult to locate.
Table 5.2. Tenure of respondents from the main stage and ERX fieldwork
periods
Tenure Main stage Mop-up
Own it outright 4.4% 1.0%
Buying it with the help of a mortgage *** 28.6% 18.6%
Pay part rent and part mortgage (shared) 0.6% 0%
Rent it *** 66.3% 79.5%
Live here rent-free * 0.1% 1.0%
Total 100% 100%
*** Significance level of less than 1%;
* Significance level of more than 1% but less than 5%

SMB 68 3/1120
Our results support the theory that job instability appears related to non-contactability
(Hawkes and Plewis, 2006; Branden et al, 1995). The proportion of respondents with
seasonal, temporary or casual jobs increased significantly from 3.8 per cent at the
main stage interview to 8.1 per cent after the mop-up exercises (table 5.3). Without
the use of the reactive tracking procedures we can assume that this category of
individuals would have been under-represented after the main stage fieldwork period.
Table 5.3. Nature of job of respondents from the main stage and ERX fieldwork
periods
Job Main Stage Mop-up
Permanent job 91.2% 86.9%
Seasonal, temporary or casual job 3.8% ***8.1%
Job done under contract or for a limited time 4.0% 3.2%
Some other type of job 1.0% 1.8%
Total 100% 100%
*** Significance level of less than 1%
There was no identifiable or significant difference between the number of male and
female respondents before or after the reactive tracking procedures were used. This
may have been a result of the majority of the ERA W3 sample being female, leaving
little room to test the effect of tracking procedures on gender Also, analysis showed
that there were no significant differences between mainstage and mop-up respondents
in the proportion of respondents who had children under the age of 19.
6. Conclusions
Survey organisations have devised and implemented a number of systems to locate,
contact and ensure continued cooperation from panel members in an effort to reduce
attrition over the life of their longitudinal studies. In spite of the growing array of
methods available, there is little systematic evidence about which ones works best for
different attrition problems (Couper and Ofstedfal 2006; Fumagalli, Laurie and Lynn
2009) and their relative cost-effectiveness.
At Wave 3 of the ERA survey, it was expected that a significant proportion of sample
members could not be located because of out-of-date contact details. For this reason,
much effort was needed to carry out extensive tracking both before and during
fieldwork. The survey did not include any experimental design to test the efficacy of
the tracking methods used and therefore only limited conclusions can be drawn from
the ERA experience for wider application.
Keep-in-touch exercises are extremely popular tracking methods and our results show
how they can be strong predictors of the mainstage outcome. This highlights the
potential of attempting additional tracking on KITE non-responding cases before the
mainstage fieldwork commences. Although this was not the case for ERA, other

SMB 68 3/1121
larger scale ONS longitudinal surveys like the Wealth and Assets Survey carry out
administrative data linkage to find new contact details for respondents who could not
be located at KITE before the start of the mainstage fieldwork.
The use of administrative sources to update contact details helped to improve both the
level of response and the representativeness of the responding sample. Administrative
data sources are increasingly being used to track respondents between waves of
longitudinal surveys. A number of ONS surveys already make use of services
provided by commercial organisations which allow updated addresses to be obtained
from the NCOA register and telephone numbers from the BT OSIS database. Other
administrative data sources are used less extensively on an ad-hoc basis as ONS often
does not have an automatic and continuous right to access such records. An
administrative data source which is not affected by access issue for ONS is the full
electoral register. Although this would appear as an ideal data source for a more
extensive use as part of surveys tracking efforts, it is currently used only sporadically
for sample checks. The main obstacles to a wider use of the register for surveys
processing is the lack of sophisticated searching and matching functionalities, which
makes the use of the register extremely resource-intensive.
References
Branden L, Gritz R M and Pergamit M R (1995) ‘The Effect of Interview Length on
Attrition in the National Longitudinal Survey of Youth’. No. NLS 95-28, US
Department of Labour.
Couper M and Ofstedal M (2006) ‘Keeping in Contact with Mobile Sample
Members’. Draft prepared for presentation at: MOLS 2006: Methodology of
Longitudinal Surveys, University of Essex, Colchester, UK. July 2006.
Gray R, Campanelli P, Deepchand K and Prescott-Clarke P (1996) ‘Exploring Survey
Non-Response: The Effect of Attrition on a Follow-up of the 1984-85 Health and Life
Style Survey’. The Statistician, 45, pp 163-183.
Fitzgerald J, Gottschalk P and Moffitt R (1998) ‘An Analysis of Sample Attrition in
Panel Data: the Michigan Panel Study of Income Dynamics’ Journal of Human
Resources, 33, pp 251-299.
Fumagalli L, Laurie H and Lynn P. (2009) ‘Methods to Reduce Attrition in
Longitudinal Surveys: An Experiment’ ESRC Survey Design and Measurement and
Initiative Programme on Understanding Non-Response and Non-Response Bias.
European Survey Research Association Conference Warsaw June 29th-July 3rd 2009.
Hawkes D, Plewis I (2006) ‘Modelling Non-Response in the National Child
Development Study’ Journal of Royal Statistics Society A, 169, Part 3, pp 479-491.
Laurie H, Smith R and Scott, L (1999) ‘Strategies for Reducing Non-Response in a
Longitudinal Panel Survey’ Journal of Official Statistics, 15:2, pp269-2.
Lepkowski J and Couper M (2002) ‘Non-Response in the Second Wave of
Longitudinal Household Surveys’ in Groves R, Dillman D, Eltinge J, Little R (2002),
‘Survey Non Response’, Wiley Series in Survey Methodology.
McAllister R, Goe S and Edgar B (1973) ‘Tracking Respondents in Longitudinal
Surveys: Some Preliminary Considerations’, The Public Opinion Quarterly, 47:3, pp
413-416.

SMB 68 3/1122
Nicoletti C and Peracchi F (2002) ‘A Cross-Country Comparison of Survey Non-
Participation in the ECHP’, ISER Working Papers No. 2002-32, Colchester:
University of Essex.
Uhrig S (2008) ‘The Nature and Causes of Attrition in the British Household Panel
Survey’, ISER Working Paper Series No.2008-5.
Watson N and Wooden M (2004) ‘Sample Attrition in the HILDA Survey’,
Australian Journal of Labour Economics Vol. 7, No 2, pp 293-308.

SMB 68 3/1123
Model-Based Small Area Estimates of Households in
Poverty
Philip Clarke, Kevin McGrath, Nargis Rahman, Denise Britz do N. Silva and
Alan Taylor
All authors from Methodology, ONS, except Denise Silva who is from IBGE,
Brazil
Summary
The Office for National Statistics (ONS) has published household mean income
estimates for local areas since 2004. Model-based estimates at small areas are now
accepted as a part of established statistical outputs from ONS. These are used by
many different users across central and local government, academics, research
organisations and charities. Users have expressed the need for estimates targeted at
more specific poverty measures, particularly those related to the poverty line defined
as 60% of national median income. To meet this need, ONS implemented small area
estimation methods based on a logistic mixed model of the Family Resources Survey
(FRS) and data from administrative sources to estimate the proportion of households
with income under the poverty threshold. The models are designed with a unit level
response and area level covariates and allow for varying sampling and estimation
geographies by using both fixed and random effects.
This paper presents the work for the 2007/08 estimates. A discussion of the statistical
model used, model diagnostics and model validation is included. Details of planned
further development are also given. These estimates were published as experimental
statistics in November 2010 and are fully documented by a Regional Trends Article1
,
User Guide2
, Validation Report3
and Technical Report4
.
1. Introduction
Income information is needed at the small area level in order to help identify deprived
and disadvantaged communities and to support work on social exclusion. This
requirement was previously identified by Census User Groups who made a strong
case for a question on income to be included in both the 2001 and 2011 Censuses.
Although this need was recognised by the government, concerns were also expressed
about the sensitivity of an income question, potential impacts on response rates and
data privacy issues. As a result, a question on income has not been included in these
Censuses. Instead, alternative methods for obtaining data on income at the small area
level were identified and implemented leading to the use of small area estimation
methods to produce local area income estimates.
ONS has published these household mean income model-based estimates at middle
1
Regional Trends Article
2
User Guide
3
Validation Report
4
Technical Report

Philip Clarke et al Model-Based Small Area Estimates of
Households in Poverty
SMB 68 3/1124
layer super output area5
(MSOA) based on data from the FRS6
and Households
Below Average Income (HBAI)7
statistics for 2004/05 and 2007/08. This followed
previous publication at ward level in 1998/99 and 2001/02. Four measures of mean
income have been published each time – household total gross income, household net
income, equivalised household income before housing costs (BHC) and equivalised
household income after housing costs (AHC).
Users have welcomed the publication of model-based estimates of mean income and,
as noted above, have also expressed the need for estimates of more specific poverty
measures, such as the HBAI indicator of those below 60 per cent UK median
household weekly income. There are two measures of poverty currently used by the
Department for Work and Pensions (DWP), one based on income before housing
costs are taken into consideration and the other after housing costs. These statistics
are not available at local area level. DWP strongly supported the request from users
for producing small area poverty estimates.
ONS has investigated the possibility of generating such estimates. The analysis was
based on proportions of households (rather than persons) as poverty is considered to
be a household characteristic shared by all individuals in a household. This approach
also integrates with the methodology developed for the estimation of mean household
income. This paper documents the work undertaken to produce 2007/08 MSOA-level
estimates of the proportion of households in poverty for England and Wales,
calculated based on equivalised household income after housing costs and produced
using the same methodology developed by ONS to provide small area estimates of
mean income (Longhurst et al, 2004 and 2005).
The development of the 2007/08 poverty estimates calculated on an AHC basis was
conducted following initial work modelling both income variables (AHC and BHC)
for time periods 2004/05, 2006/07, and 2007/08. The results of this initial work to
develop separate small area estimation models for all six target variables (percentage
of households below threshold income based on AHC and BHC for three time
periods) indicated that 2004/05 models had a lower explanatory power (measured by
between area variability explained) than models for 2007/08. The quality of the
model-based estimates (measured in terms of confidence intervals and associated
distinguishability - defined as non-overlapping confidence intervals between low
poverty and high poverty areas) were much better for AHC in all time periods. In
addition, the models indicated some instability regarding the selected covariates as
only a few were consistent across the models.
A decision was taken to investigate model stability revising the variable selection
procedure but concentrating efforts on the model to obtain poverty estimates for
2007/08 using AHC data, since the aim was to publish estimates for the most recently
available survey data and AHC estimates had better properties than the BHC
estimates. This paper, therefore, presents results for 2007/08 MSOA-level estimates
of the proportion of households below 60% national median income after housing
costs. The other datasets were either used at the model selection stage or at the
validation stage.
The paper is structured as follows. Section 2 describes the general small area
5
Super Output Areas (SOAs) is a geographic hierarchy designed to improve the reporting of small area
statistics in England and Wales. There are 3 layers in the hierarchy and 7194 middle layer super output areas
(MSOAs).
6
FRS Information Page
7
HBAI Information Page

SMB 68 3/1125
estimation methodology used by ONS to deal with income related variables and
Section 3 describes its application to the problem of estimating proportions of
households in poverty. Section 4 explains the model selection procedure and the fitted
model is described in Section 5, with the model-based estimates being presented in
Section 6. An assessment of the quality of the estimates and their validity is given in
Section 7, with a final summary and description of future work presented in Section
8.
2. Small area modelling
The principal reasoning behind the need for small area estimation is that whilst
surveys are designed to provide reliable estimates at national and sometimes regional
levels, they are not typically designed to provide estimates at lower geographical
levels (local authorities, wards, etc.). With the exception of the Labour Force Survey,
most of the principal national household surveys in UK have clustered designs. This
means that the sample is not distributed totally randomly across the nation but that
certain areas are first selected as primary sampling units (PSUs) and then households
are selected for interview from these. The PSUs in the FRS are postcode sectors.8,9
The PSUs are stratified by 27 regions and also by three other variables derived from
the 2001 Census of Population. Stratifying ensures that the proportions of the sample
falling into each group reflect those of the population. Within each region the
postcode sectors are ranked and grouped into eight equal bands using the proportion
of households where the household reference person (HRP) is in National Statistics
Socio-economic Classification (NS-SEC) 1 to 3. Within each of these eight bands,
the PSUs are ranked by the proportion of economically active adults aged 16-74 and
formed into two further bands, resulting in 16 bands for each region. These are then
ranked according to the proportion of economically active men aged 16-74 who are
unemployed. This set of stratifiers is chosen to have a maximum effectiveness on the
accuracy of two key variables: household income and housing costs (Department of
Work and Pensions (2009) Family Resources Survey United Kingdom 2007-8).
The problem for estimation at the small area level is that, irrespective of the total
sample size, with clustering like this the inevitable result for areas such as MSOAs is
that the vast majority will contain no sample respondents at all. Hence no direct
survey estimates are possible. Also, where there are estimates for particular MSOAs,
the sample sizes would be so small that the precision of the estimates would be low.
Note that MSOAs and PSUs are of similar size in terms of number of households.
Following some preliminary studies into small area estimation, ONS established the
Small Area Estimation Programme (SAEP) in April 1998. SAEP methodology (ONS
2003, 2005) involves combining survey data (in this case income related variables)
with other data that are available at the small area level and modelling their
relationship. The small area level is usually an area for which direct survey estimates
cannot be reliably produced. The area-level relationship between the survey variable
and auxiliary variables (covariates) is estimated by regressing individual survey
responses on area-level values of the covariates.
The basic aim of the SAEP methodology is the construction of a statistical model
relating the observed value of the survey variable of interest (measured at individual,
8
A postcode is associated with each address in the UK and they are assigned by the Royal
Mail. They are also a key means of providing locational references for statistical data. There
are approximately 1.78 million postcode units in the UK.
9
A postcode sector is formed by a set of contiguous postcode units. There are roughly 11,600
in the UK.

SMB 68 3/1126
household, or address-level) to the covariates that relate to the small area in which the
survey observation is located. These covariates are generally average values or
proportions relating to all individuals or households in the area, based on
administrative or census data which have coverage in all areas being modelled. Once
the covariates have been selected and the model has been fitted, the model parameters
can be applied to the appropriate covariate values for each and every area, hence
obtaining estimates of the target variable for all small areas.
While the model is constructed only on responses from sampled areas, the
relationships identified by the model are assumed to apply nationally. As
administrative and census covariates are known for all areas, not just those sampled,
the fitted model can be used to obtain estimates and confidence intervals for every
area. This is the basis of the synthetic estimation ONS has already used to produce
the estimates of average (mean) income for MSOA for 2004/05 and 2007/08 that
were published as experimental statistics.
3. Modelling the Poverty Indicator
This section describes how the general SAEP methodology has been applied to the
specific problem of estimating poverty at the MSOA-level. The datasets (both survey
and covariate) used in the modelling process are also described. The standard
approach to measuring low income or poverty has been to look at how many people,
households or families have an income that falls below some threshold. The
threshold is commonly set at a particular fraction of mean or median income,
calculated across the whole population. Sixty percent of the median is currently the
most widely used threshold and is the definition used by the UK government as one
measure of progress on its target to reduce the number of children in low income
households. The European Statistical Office (Eurostat) and the countries belonging to
the Organisation for Economic Co-operation and Development (OECD) also use
‘below 60% of the median’ as their low income definition. The Households Below
Average Income (HBAI) report (DWP, HBAI Team, 2009), published annually by
the Department of Work and Pensions (DWP), is the principal source of information
on the size and characteristics of the low income population in Great Britain.
3.1 The Datasets
3.1.1. The Survey Data
The survey data used in this modelling exercise comes from the HBAI datasets that
are prepared by DWP using data from the 2007/8 Family Resources Survey (FRS)
(Sullivan et al, 2009). FRS was chosen as the source for survey data for this study
since it is the survey with the largest sample that includes suitable questions on
income.
The target parameter to be estimated is the proportion of households below 60% of
national median income based on net weekly household equivalised income after
housing costs. As the SAEP methodology uses household level responses, the survey
variable to be modelled is a binary variable that indicates if the household has net
weekly equivalised income after housing costs below a threshold defined as 60% of
national median income. The threshold value for 2007/08 is £199 pw and corresponds
to 60% of national median AHC equivalised net income as published by DWP10
.
10
DWP Poverty Theshold Information 2007/08

SMB 68 3/1127
Equivalised income means that the household income values have been adjusted to
take into account varying household size and composition. Equivalisation is needed
in order to make sensible income comparisons between households.
Over recent years, DWP have moved to use the OECD equivalisation scale
throughout the main body of their HBAI publication rather than the alternative
McClements equivalisation scale11
. This change occurred from the 2005/06
publication and was prompted by the UK Government’s 2004 Spending Review,
which stated that future child poverty measurements will report incomes Before
Housing Costs and equivalised using the OECD scale.
FRS uses a geographically stratified clustered probability sample drawn from the
Royal Mail’s small users Postcode Address File (PAF). The survey selects 1,848
postcode sectors with a probability of selection that is proportional to size. Each
sector is known as a Primary Sampling Unit (PSU). Within each PSU, a sample of
addresses is selected. In 2007/08, 24 addresses were selected per PSU. More
information on the FRS methodology is contained within the FRS technical report
(Sullivan et al (2009)).
FRS aims to interview all adults in a selected household. A household is defined as
fully co-operating when it meets this requirement. In addition, to count as fully
cooperating, there must be less than 13 'don't know' or 'refusal' answers to monetary
amount questions in the benefit unit schedule (i.e. excluding the assets section of the
questionnaire). Proxy interviews are accepted only under restricted circumstances. In
2007/08, for those households classed as fully co-operating, proxy responses were
obtained for 16% of adults. In 2007/08 the final achieved sample size (for Great
Britain) was 24,982 households after removal of ineligible households, those partially
co-operating, refusals and non-contacts.
The requirement for this project was to produce MSOA-level estimates of proportion
of households in poverty for England and Wales. For 2007/08 this resulted in a
survey data file that contained 18,822 households from 1,524 primary sampling units.
The additional sample loss was primarily due to unusable income data for responding
households. The final survey data file for England and Wales contained cases in
3,083 different MSOAs out of a total of 7,194.
3.1.2. The Covariate Data Sets
The small area estimation methodology requires covariate data to be available on a
geography compatible with MSOAs. A range of data sources were used in the
modelling process that included variables considered to be related to the propensity of
a household having income below a threshold. They were: Census 2001 data, DWP
benefit claimant counts (August 2007), Valuation Office Agency Council Tax
Bandings (2007), Her Majesty’s Revenue and Customs (HMRC), Child Tax Credit
and Working Tax Credit (2006), Communities and Local Government (CLG),
Change of ownership by dwelling price (2007) and regional indicators.
The covariates used for modelling poverty were the same for England and Wales.
3.2 The Statistical Model
Binary response models that take into account the fact that each individual household
belongs to a specific area were developed for England and Wales. These models take
11
For further information regarding the differences between the OECD and McClements equivalisation
scale please see this link DWP

SMB 68 3/1128
as the response variable whether the households are in poverty or not (1 if the
household is in poverty and 0 otherwise) and the area level covariates are the
explanatory variables. The models relate this survey variable of interest (measured at
household level) to the covariates that correspond to the small area in which the
household is located. Once fitted, the models can be used to produce estimates of the
target variable at the small area level, i.e. the models can be used to produce MSOA-
level estimates of the proportion of households in poverty (i.e. percentage of
households with net equivalised income AHC below 60% of national median income)
and confidence intervals for the estimates calculated appropriately.
The sampling area in the survey is the PSU but the estimation area is the MSOA. As
FRS uses a clustered sample design, PSUs and MSOAs can cross cut each other. This
means that the area level variation in the model has to be measured using the PSU (as
this is the area where the data are modelled). The model assumes that variation for
MSOAs is similar to variation for PSUs as PSUs and MSOAs are of similar size in
terms of households. This allows for the use of the variance associated with PSUs in
error calculations relating to MSOAs. Inference from the model also takes account of
the effect of clustering within the data. This assumption was assessed (Heady et al,
2003) using LFS data which showed that within PSU variability was similar to
MSOAs variability.
The underlying model is a two-level model given by:
(1)
id id idy eπ= +
( ) j
T
did ulogit ++= βXαπ
where
idy is the survey variable for household i for MSOA d, so idy is the poverty indicator
for household i in MSOA d;
j is the sampling area (primary sampling unit);
idπ is the expected probability of idy ;
dX is a vector of values for MSOA d of a set of covariates;
ju is the area level residual for primary sampling unit j (sampling area), assumed to
have expectation 0 and variance 2
uσ ;
and ide is within area residual for household i in MSOA d with expectation 0.
Models are fitted on the sample data and using covariates in areas for which a sample
is present. As covariates are available for all areas, a synthetic estimator of the
proportion of households with an income level below 60% of the national median can
be produced for all areas from the fitted model parameters, given by
is distributed as Binomial (1, )id idy π

SMB 68 3/1129
( )
( )βX
βX
ˆˆexp
ˆˆexp
ˆ
T
d
T
dsynth
d
++
+
=
α
α
π
1
(2)
where αˆ is the estimate of α and βˆ is the vector of estimated coefficients for the
covariates.
The 95% confidence interval for an area prediction in the logit scale is given by
( ) ( )2ˆ ˆˆ 1.96 ( )T T
d u d dVarα σ β+ ± +X X Xβ (3)
Under the SAEP methodology the area level variance, 2
uσ , is added to the standard
error term to provide an approximate 95 per cent confidence limits. The anti logit of
these limits are taken to produce a 95 per cent confidence interval for ˆsynth
dπ .
4. Developing the Models
The previous sections introduced the statistical model and the datasets used to
produce model-based estimates of proportion of households below 60% of median
income. This section describes the model fitting procedures. During the initial
development of these models automatic variable selection methods were used. This
analysis showed that these were inadequate to produce valid and stable models and
therefore a more “controlled” model selection approach was implemented. This
strategy is described in detail in the following.
The first method applied was to select significant covariates for inclusion in model
(1) by applying an automatic stepwise selection procedure in a single level model
approach by excluding the random effect term for area. All of the appropriate
covariates (those expressed as percentages or proportions) were transformed onto the
logit scale and both the transformed and original covariates were considered for
inclusion in the model. In this case, the automatic model selection procedure was
carried out using SPSS options for fitting single level logistic regression because
these automatic selection routines are not available when fitting multi-level models in
most of the statistical packages usually used to fit multilevel models, such as SAS,
STATA and MLwiN. The selected variables were then used to fit a multilevel model
for the binary response and non-significant variables were removed. With these
significant covariates, interaction terms were created, tested for significance and
included in the model where appropriate. Interaction terms were again selected using
an automatic stepwise procedure on a single level model.
The model was fitted using the statistical software SPSS (the initial model selection
stage) and STATA (for the multilevel modelling) with postcode sectors at the higher
level and households at the lower level, as outlined in Section 3.2. Regional indicator
terms were always included in the model (whether significant or not) so as to control
for region differences and to reduce the amount of calibration that would be necessary
for benchmarking the model-based estimates to the published HBAI estimates at
regional level. This procedure was used to develop initial small area estimation
models for the 3 time periods (2004/05, 2006/07 and 2007/08) separately. Two issues
were found with this approach. Firstly, the explanatory power for the 2004/05 was
lower than for the other the two years and, secondly; the models included different

SMB 68 3/1130
sets of variables. Based on this, a decision was taken to use a controlled approach for
model selection in which the relation between the response variable and the
covariates in each auxiliary source was analysed separately.
Explanatory analysis indicated that no unique covariate in the available auxiliary
datasets showed a high correlation with the response variable, whereas the covariates
are highly correlated to each other. This is not an optimal case as this implies that the
covariate set all convey similar information about the response variable and none of
them present a well defined relationship. Therefore, the controlled variable selection
procedure was an attempt to identify if it would be possible to find a set of variables
that would do reasonably well for the 3 time periods in order to develop more stable
models (in terms of selected covariates).
One of the problems with an automatic stepwise selection procedure is that it can
only be implemented on a single level model as the procedure is not available for
multilevel models. In order to validate the “automatic selected” model, a more
controlled procedure based on the multilevel model fitting was carried out. The full
set of covariates was first separated into two groups: untransformed covariates and
logit transformed covariates. Then, within these two groups, covariates from different
datasets (e.g. Census, DWP, HMRC, etc.) were fitted separately. Automatic selection
was carried out separately within each dataset followed by a forward selection (using
multilevel model fitting) that started with Census covariates and was followed by
adding the significant variables from each auxiliary data set. This controlled
procedure results in more stable models and ensures variable for specific domains of
the covariate sets are included in the final model.
This procedure was carried out for the 2004/05 and 2007/08 data twice, one run
including only untransformed proportions and the other considering only the logit
transformed covariates. Although the selected variables were not the same for every
model, all models included at least one Census covariate representing the same
underlying dimensions/area characteristics: socio-economic classification or social
grade, nationality/ethnicity of the resident population, labour market status, dwelling
characteristics plus the proportion of the population in working age.
The conclusion from this work was that the models presented a type of stability in
relation to which information was conveyed by the selected Census covariates. The
same argument was valid for DWP covariates. Therefore, in order to choose the final
model, the variables identified as good predictors for the 2007/08 poverty outcome
were all included in a model and a controlled backward elimination carried out. The
decision to use a covariate on its original scale (as a proportion) or as a (logit)
transformed covariate was based on exploratory analysis and on a measure of
explanatory power obtained by fitting a model including each of the variables at a
time. A measure of model adequacy was used to compare competing models. This
was calculated as the percentage of between area variability explained by the
covariates in the model calculated as:
Percentage between area variability explained =
2
2
(full model)
1 100
(null model)
u
u
σ
σ
⎛ ⎞
− ×⎜ ⎟
⎝ ⎠
. (4)
In addition, the percentage of between area variability explained by one covariate in
the presence of all covariates was also calculated as:

SMB 68 3/1131
Percentage between area variability explained by one covariate =
2
2
(full model)
1 100
(model excluding 1 covariate)
u
u
σ
σ
⎛ ⎞
− ×⎜ ⎟
⎝ ⎠
. (5)
The chosen model and corresponding adequacy measures are presented in Section 5.
5. The Fitted Model
The final model for the proportion of households below income threshold for 2007/08
is given below and in Table 1 which contains a key to the labels of the covariates.
The covariates/auxiliary data have been grouped by source. This model contained
main effects and three interaction terms. The key predictors are listed below:
• proportion of household reference persons aged 16-74 whose NS-SEC is
managerial and professional
• proportion of household spaces that are detached, semi detached or terraced
• proportion of persons aged 16 to 59
• proportion of households that contain one person
• proportion of persons aged 16-74 that are full-time students
• proportion of persons claiming Disability Living Allowance: Mobility Award
Higher
• proportion of persons aged over 60 claiming Pension Credit: Guaranteed
Element Only
• families out of work receiving Child Tax Credit
• the Government Office Region indicators

SMB 68 3/1132
Full details of fitted model coefficients are as follows:
( )id
ˆlogit π =
+ 1.6436 x Constant
+ 0.1058 x Lnphrpman
- 0.0989 x Lnphouse
- 5.026 x p16_59
+ 0.5129 x lnphhtype1
+ 0.1494 x Lnpftstud
- 0.3250 x Lndlamah
+ 0.3027 x Lnpcgeo
+ 0.0007 x Famoutct
+ 0.1140 x lnphrpman x lndlamah
- 2.7671 x p16_59 x lnphhtype1
- 0.2521 x lnphhtype1 x lndlamah
- 0.0307 x Northest
- 0.1876 x Northwst
- 0.2532 x York
- 0.0758 x Eastmid
- 0.2026 x Westmid
- 0.2560 x Eastern
- 0.3466 x London
- 0.2741 x Southest
- 0.2781 x Southwest

SMB 68 3/1133
Table 1. Estimated coefficients and key to covariates included in the model for
households in poverty 2007/08, AHC
Covariate
Name
Label Source
Lnphrpman Logit transformation of proportion of HRPs
aged 16-74 whose NS-SEC is managerial
and professional
Census
Lnphouse Logit transformation of proportion of
household spaces that are detached, semi
detached or terraced Census
P16_59 Proportion of persons aged 16 to 59 Census
Lnphhtype1 Logit transformation of proportion of
households that contain one person
Census
Lnpftstud Logit transformation of proportion of
persons aged 16-74 that are full time
students
Census
Lndlamah Logit transformation of the proportion of
persons claiming Disability Living
Allowance: Mobility Award Higher
DWP
Lnpcgeo Logit transformation of the proportion of
persons aged over 60 claiming Pension
Credit: Guarantee Element Only
DWP
Famoutct Families out of work receiving Child Tax
Credit
HMRC
Northest Dummy variable indicating MSOA is in
North-East region
Country/regional
indicators
Northwst Dummy variable indicating MSOA is in
North-West region
Country/regional
indicators
York Dummy variable indicating MSOA is in
Yorkshire and the Humber region
Country/regional
indicators
Eastmid Dummy variable indicating MSOA is in
East Midlands region
Country/regional
indicators
Westmid Dummy variable indicating MSOA is in
West Midlands region
Country/regional
indicators
East Dummy variable indicating MSOA is in
East of England region
Country/regional
indicators
London Dummy variable indicating MSOA is in
London region
Country/regional
indicators

SMB 68 3/1134
Covariate
Name
Label Source
Southest Dummy variable indicating MSOA is in
South-East GOR
Country/regional
indicators
Southwst Dummy variable indicating MSOA is in
South-West region
Country/regional
indicators
Lnphrpman
x lndlamah
Interaction between lnphrpman and
lndlamah
P16_59 x
lnphhtype1
Interaction between p16_59 and lnphhtype1
Lnphhtype1
x Lndlamah
Interaction between lnphhtype1 and
Lndlamah
With no covariates included in the model, the estimated standard residual area
variance 2
ˆuσ is 0.24 (0.025) compared with 0.025 (0.018) obtained when the
significant covariates are included in the model. These covariates together, therefore,
account for 89.5% of the total between area variance.
To understand the decomposition of the between area variance, the model can be
fitted by including each covariate on its own. The covariates that account for most of
the between area variability are persons claiming pensions credit (lnpcgeo - DWP)
and families out of work receiving tax credit (famoutct - HMRC), with each
accounting for 65% and 64%, respectively, of the between area variability. The
covariate indexing persons claiming Disability Living Allowance (lnphrpman -
HMRC) accounts for 58% of the between area variability, while the regional
indicators account for 13%. Further details of the final model are given in the
Technical Report12
.
6. Guidance on Use
It is important to provide guidance on the use of the estimates as there are some
limitations that users need to be aware of. The main limitation of estimates for small
areas, those estimated directly from responses to surveys or model-based, is that they
are subject to variability. Confidence intervals associated with the model-based
estimates for each MSOA are estimated in order to make the precision of the
estimates clear.
These MSOA level estimates can be aggregated to provide poverty estimates for
larger geographical areas such as local authority level but their confidence intervals
cannot be computed. The model-based estimates for different MSOAs are correlated
because they are obtained from a single model in which the set of regression
coefficients is estimated using data from all MSOAs. Therefore, to make
comparisons between broader areas, the standard error of the aggregated estimate has
to incorporate not only the variability of each estimate but also the correlation
12
Technical Report

SMB 68 3/1135
between estimates. Currently, ONS does not have estimates of the required
correlations. Hence, it is not possible to assess the precision of the aggregated
estimates, though this issue will be considered in future work.
The model-based methodology has been developed to ensure that the MSOA
estimates are constrained to direct survey estimates from the HBAI for GORs in
England and the estimate for Wales. For example, the model-based estimates for the
MSOAs in Wales when aggregated correspond to the HBAI estimate of the
proportion of households in poverty for Wales. However, the model-based estimates
will not be consistent with HBAI estimates of proportion of households in poverty for
other geographies (for example, for LAs).
In common with any ranking procedure based on estimates, when ordering MSOAs
by poverty level, care must be exercised in interpreting these ranks. One needs to take
into account the variability of the estimates when using these figures. For example,
the confidence interval around the lowest ranked MSOA (lowest level of poverty)
suggests that the estimate lies among the group of MSOAs with the lowest poverty
levels rather than being the MSOA with the very lowest poverty level.
In order to compare model-based estimates of two particular MSOAs, it is necessary
to account for the correlation between the estimates. None overlapping confidence
intervals may be taken as an indication that the MSOA estimates are statistically
different. However, this evidence must be used with caution as this constitutes a naive
procedure to account for the uncertainty when estimating the difference between two
cross-sectional estimates.
Although these model-based estimates can be used to rank MSOAs by proportion of
households in poverty, they cannot be used to make any inferences on the distribution
of poverty across the MSOAs. The estimation procedure tends to shrink estimates
towards the average level of poverty for the whole population, so model-based
estimates at each end of the scale tend to be over or under-estimated. This is a natural
outcome of using a model-based approach as the predictions put less weight on
outlying observations and so they appear to “shrink” towards the average level.
Nevertheless, estimates can be used to make certain inferences; for example, the
proportion of households in poverty in MSOA A is greater than the proportion of
households in poverty in MSOA B (if the appropriate confidence intervals do not
overlap). Given that model-based estimates are subject to limitations, some examples
of appropriate and inappropriate uses for the estimates are provided in the associated
User Guide13
.
7. Model Diagnostics & Validation
A number of diagnostic checks have been used to assess the appropriateness of the
models developed for producing MSOA-level estimates of poverty. The diagnostic
checks employed are those developed by ONS for small area estimation (Heady et al
(2003) and Brown et al (2001)). This analysis showed that in general the models were
well specified and the assumptions are sound. This provides confidence in the
accuracy of the estimates and the associated confidence intervals. In addition, the
methodology used to produce the model-based estimates has undergone internal and
13
User Guide

SMB 68 3/1136
external review. Full details of this analysis can be found in the associated Technical
Report.
Analysis to compare the model–based estimates with other sources of poverty data
was carried out to establish the plausibility of the model-based estimates. For
example, as detailed in the Validation Report14
, the small area estimates of mean
income show a consistent relationship to the poverty estimates. Areas with high levels
of household poverty are in areas with low levels of household income, and the two
indicators show a strong negative relationship. The rank of the Index of Multiple
Deprivation (income component) in England shows a positive relation to the rank of
the model-based estimates of households in poverty. Both of these processes have
ensured that the methodology and its application are valid, the models developed are
the best possible for the data available and the model-based estimates are plausible.
8. Summary and Further Work
Previous work on developing models for both BHC and AHC income variables for
the three time periods 2004/05, 2006/07 and 2007/08 showed that the best estimates
could be obtained for the AHC model for 2007/08. As a result, a more detailed study
of a model for this period was undertaken and presented in this paper. The model
chosen explains 89.5% of total between area variability (compared with the intercept
only null model) and distinguishability between MSOAs defined as non-overlapping
confidence intervals between low poverty and high poverty areas was 24%. In
conclusion, the analysis shows that the AHC model proposed is well specified,
performing well in terms of explanatory power, estimate precision and
distinguishability between areas. It also performs well in other time periods. It will
now be important to consult with users on the value of these now experimental
statistics.
Further work, on the 2007/08 AHC model, will include a more detailed examination
of some specific areas comparing the poverty estimates with the mean income
estimates. The BHC based models will also be further investigated, possibly by
pooling two or three years of FRS data to see if this helps improve model
performance for 2004/05 and 2006/07. A Local Authority level model will also be
investigated as well as other alternatives modelling approaches such as the use of
composite estimators. Pooling several years FRS data may also be useful for
modelling specific household types, for example, households with children or
pensioner households. Lastly, an investigation of modern methods of model selection,
such as Bayesian Model Averaging will be carried out together with attempts to
reduce the level of correlation in the predictors using principal components analysis.
References
Brown G, Chambers R, Heady P and Heasman D (2001) ‘Evaluation of Small Area
Estimation Methods – An Application to Unemployment Estimates from the UK
LFS’, Proceedings of Statistics Canada Symposium in 2001.
Heady P, Clarke P, Brown G, Ellis K, Heasman D, Hennell S, Longhurst J and
Mitchell B (2003) ‘Small Area Estimation Project Report’, Model-Based Small Area
14
Validation Report

SMB 68 3/1137
Estimation Series No.2, ONS Publication.
http://www.statistics.gov.uk/methods_quality/downloads/small_area_est_report/saep1
_Prelims&ch1&2_v2.pdf
Department for Work and Pensions, HBAI team (2009) ‘Households Below Average
Income 1994/5 – 2007/08’.http://research.dwp.gov.uk/asd/hbai/hbai2008/contents.asp
Department of Work and Pensions (2009) ‘Family Resources Survey United
Kingdom 2007-8’.
http://research.dwp.gov.uk/asd/frs/2007_08/frs_2007_08_report.pdf
Longhurst J, Cruddas M, Goldring S and Mitchell B (2004) ‘Model-based Estimates
of Income for Wards, 1998/99: Technical Report’, Published in Model-Based Small
Area Estimation Series, ONS Publication.
Longhurst J, Cruddas M and Goldring S (2004) ‘Model-based Estimates of Income
for Wards, 2001/02: Technical Report’, Published in Model-Based Small Area
Estimation Series, ONS Publication.
Office for National Statistics (2003) http://www.statistics.gov.uk/geography
Office for National Statistics (2005) ‘Super Output Area’.
http://www.statistics.gov.uk/geography/soa.asp
Sullivan J, Gibb P, Chung R, Snow J, Beckhelling J, Vekaria R and Lord C (2009)
‘Family Resources Survey: United Kingdom, 2007-08’.
http://research.dwp.gov.uk/asd/frs/2007_08/frs_2007_08_report.pdf
Valuation Office Agency (2006) ‘2004 Council Tax data’.
http://www.neighbourhood.statistics.gov.uk

SMB 68 3/1138
Appendix 1: Percentage of households (OECD) with income (AHC) < 60% of
median (2007-2008 MSOA estimates (calibrated))
Source: Office for National Statistics
© Crown copyright. All rights reserved 100019153, 2010

SMB 68 3/1139
Pruning the Labour Force Survey: Removing
Respondents Aged 75 and Above from the Waves 2-5
Interviews
Karl Ashworth, Matthew Greenaway and Paul Smith
Methodology Directorate, Office for National Statistics
Summary
This article describes a change introduced to the UK Labour Force Survey (LFS) in
Quarter 3 of 2010, where a longitudinal sample component was excluded in order to
reduce the cost of the survey. This article details the work undertaken to evaluate the
impact of this exclusion and to ensure key labour market statistics are not adversely
affected. The paper first describes the LFS data collection and estimation procedures
prior to the design change. We then describe the weighting adjustments to the
estimation methodology under the new data collection procedures and show the
results of simulations derived from using the new weighting on existing data.
1. Description of the Labour Force Survey
The Labour Force Survey (LFS) is the UK’s largest continuous social survey. It
provides an up-to-date source of labour market statistics, which are widely used both
within and outside of Government. In recent years, budget pressures within ONS
have required the Office to introduce a number of savings, and given the large sample
size of the LFS it was identified as one potential source of savings. In particular, as
the principal focus of the survey is to produce labour market estimates which are
normally based on either working age or economically active individuals, it was
decided to explore the possibility of reducing the amount of interviewing of older
people in the survey. However, it was important first to establish that any such
change would not have an adverse effect on the precision of key estimates.
Since 1992, the LFS has operated a rotating panel survey design. Each calendar
quarter a new cohort of addresses is added to the sample and interviewed over five
successive quarters before being replaced by another new cohort. Consequently, in
any given calendar quarter, the LFS has a sample that is constructed of five cohorts of
respondents, each cohort is either on its first, second, third, fourth or fifth and final
interview (see Figure 1).

tracking procedures ERA

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (13)

Similar to tracking procedures ERA

Similar to tracking procedures ERA (20)

tracking procedures ERA