2021 Census Outputs and Dissemination:
Introduction
2021 Census outputs roadshows
Spring 2018
Aims for today
• Outline of 2021 Census outputs strategy as detailed in
our ‘Initial View on 2021 Census Output Content Design’
consultation launched at the end of February 2018
• Provide a demonstration of our current prototype of a
flexible dissemination system
• Update on our plans for enhanced census outputs
• Answer your questions and get feedback on our plans
for outputs
Session outline
13:30 – 13:45 Introduction to 2021 Census Outputs
13:45 – 14:15 Outputs Consultation
14:15 – 15:30 Flexible Dissemination System
Includes:
 Presentation
 Coffee break
 Try out the flexible table builder prototype
 Visit market stalls to find out more about other
2021 Census outputs
15:30 – 16:00 Enhanced Census Outputs
16:00 – 16:15 Wrap-up and questions
Next steps
Vision for 2021 Census Outputs
Flexible Timely Accessible
Analysis Programme
• Make the best use of census data, alongside
administrative and survey data to describe the
society we live in
• Add value by helping users understand and
interpret census data
• Statistical bulletins, interactive visualisations etc
• Currently being developed
Future User Engagement
Type of
Engagement
User
Research
Public
Consultation
Working
Groups
???
UKCC / UK
Harmonised
Working
Group
Conferences
User Groups
ONS staff
Any questions:
census.outputs@ons.gov.uk
Initial View on 2021 Census Output Content
Design: Public consultation
2021 Census outputs roadshows
Spring 2018
Overview
• Initial high-level plans for 2021 Census output content
design and dissemination (England and Wales only)
 Outline our vision for 2021 Census outputs
 Our first step towards understanding what users
require from 2021 Census outputs
 Proposal at present
 Feedback will help inform our future research and
development of these outputs
Consultation open:
28th February – 23rd May 2018
https://consultations.ons.gov.uk/census/initial-view-
on-the-2021-census-output-design
Flexible Dissemination System
• Web-based interactive dissemination system
 Define own queries
 Select geography, population base and variables
required
• Pre-determined univariate tables
More details in next session
Flexible Timely Accessible
Enhanced Census Outputs
• 2021 Census-based outputs where census
data and administrative data are linked at the
record level to enable the production of
multivariate, small area outputs
More details in a later session
UK Harmonisation and UK Statistics
• Working with NRS and NISRA to harmonise
wherever possible
 Statistical disclosure control
 Dissemination approach
 Metadata
• Exploring how we provide UK data to users
Microdata
Public 1% sample - individuals
Accessed via ONS website
Safeguarded 2x5% sample – individuals
Accessed via UKDS
Secure 10% sample – individuals and household
Accessed via ONS SRS at ONS Titchfield,
Newport, London
2011:
• Requirements for safeguarded household file
• Review sample design and size
• Microdata Working Group
Origin-Destination
• Review statistical disclosure control methodology
• Aim to make more data publically available
• Consider number of tables produced
• Origin-destination Working Group
Migration Residence
Workplace Student
Accessing sensitive census data
• Provide access to sensitive census data in
secure and controlled research environments
• Access granted in accordance with ONS
Approved Researcher Scheme
• Continue to review locations and access
arrangements
• Longitudinal Study – 1% sample
Output categories and population bases
Output categories
• Topics - research is ongoing
• Recommendations to inform White Paper in
2018
Population bases
• Continue to use 2011 population bases
Small populations
• Continue to identify value and dissemination
Geography
Output Areas and Workplace Zones
• Continue to be building blocks for census geographies
• Keep maintenance to within 5%
Grids
• 2011 – 1km square grid for Eurostat using postcode-level
population estimates
• Explore user requirement for grids
Civil parishes
• 2011 – 10% parishes without data
• Identify methodology to make more data available for
parishes
Interpreting 2021 Census results
Metadata
• Will continue to produce metadata
• Will consider best way to provide this based on
the dissemination system
Quality Assurance
• National Statistics Accreditation
• Population estimates
Evaluation
• Strength of impact (positive, negative or none) of
our intended design on:
Accessibility
Flexibility
Timeliness of outputs for users
• Evidence used to identify the most important
concerns for users and prioritise research on
these aspects
Public good
• Work being undertaken to “serve the public
good” 
 using census data for public policy decision making
 using census data for public service planning and
delivery
 using census data for allocation of public resources
 using census data for decisions which are likely to
significantly benefit the UK economy, society or
quality of life of people in the UK
Example response to consultation
As a private sector transport provider we worked with
local authorities [insert name of LA] in our region to
identify where a new bus route should be placed.
We required access to safeguarded and secure origin-
destination data as we needed to know where residents
lived and worked and how many cars they owned.
We were unable to easily access the safeguarded and
secure origin-destination data as we are based in the
North West and the VML is in Newport, London, Titchfield.
This caused us 12 month delay in the project to enable
time to travel to these locations and ££ in extra transport
costs to travel to these locations.
Once we did access the data, because it was 5 years after
census day, it was out of date.
Consultation open:
28th February – 23rd May 2018
https://consultations.ons.gov.uk/census/initial-view-
on-the-2021-census-output-design
Any questions:
census.outputs@ons.gov.uk
Flexible Dissemination System
2021 Census outputs roadshows
Spring 2018
Presentation Overview
• Our Aims for 2021 Census Outputs
• How we propose meeting those aims
• The benefits
• The trade-offs
• Where we are - the prototype
• What’s next ?
• Hands On Session
• Other Activities
Aims: Designed based on User Feedback
Flexible Timely Accessible
Proposal: Fast & Flexible Dissemination
Statistical disclosure control:
Aims to ensure that statistical outputs provide
as much utility to users, whilst maintaining the
confidentiality of individuals, households, etc.
Automated
checks
Statistical Disclosure Control
Automated
checks
The type of automated SDC checks we are looking at are:
•Maximum number of cells
•Maximum number of variables
•Small values in totals
•Percentage of cells that are populated
•Percentage of cases in one cell/row/column
•Real/fake 1s and attribute disclosures
Checks and parameters have not been finalised yet.
Record swapping and cell key information is on the display
Is this possible?
• Prototype developed by ONS with Sensible Code Co.
• User interface – for demonstration purposes only
• Automated Statistical Disclosure Control embedded.
• 60m record artificial database grossed up from a 2011
Census Microdata Sample
• 8 levels of Output Area geography hierarchy (181,408
OAs)
• 80 variables incl. person and household characteristics
Here is an example….
Number of areas for which data
will be released
Pass or Fail? - Same Query for Different Areas
Number of areas for which
data will be released
?
??
?
?
So… what are the benefits?
Users can
define their
own data
query
All standard
outputs
available from
one place
More detail
for some
small areas
Wider benefits for
data
dissemination
across the GSS
Multivariate data
earlier than before
The benefits come with some trade offs
• The protection added to reduce the potential for
“differencing” between similar tables is given by the
“cell-key method”.
• Small inconsistencies will exist in the totals
between different tables
• These inconsistencies can occur in two
ways………
Summing of Individual Cells
Ethnicity by occupation at local authority level
LA=1 Occupation
Ethnicity: 1 2 3 4 5 6 7 8 9
White 3885 6014 4325 5726 5202 4172 4394 4282 6784 44784
Mixed 27 55 64 44 47 53 70 34 70 464
Asian 198 357 134 168 171 135 292 233 294 1982
Black 23 93 42 49 34 76 44 38 97 496
Chinese 23 40 11 6 25 15 23 15 33 191
4156 6559 4576 5993 5479 4451 4823 4602 7278 47917
Ethnicity by occupation at Output Area level aggregated to local authority level
LA=1 Occupation
Ethnicity: 1 2 3 4 5 6 7 8 9
White 3886 6012 4324 5730 5205 4171 4391 4276 6784 44779
Mixed 27 57 66 43 48 52 68 35 70 466
Asian 196 359 133 166 172 137 292 231 291 1977
Black 23 93 41 48 33 76 44 41 94 493
Chinese 22 40 11 5 25 15 24 15 32 189
4154 6561 4575 5992 5483 4451 4819 4598 7271 47904
E.g. Local Authority total
White ethnic group
category:
Total for row at local
authority level is 44,784
Total for row in Output Area
aggregation is 44,779
Difference of 5
vs Sum of aggregated output areas
Tables Sharing a Common Variable
Age by marital status at local authority level
LA=1
Single Married Separated Divorced Widowed Total:
0-15 12176 0 0 0 0 12176
16-24 8313 207 11 13 8 8552
25-34 5249 2439 241 199 20 8148
35-44 2886 4468 403 914 44 8715
45-54 1517 5355 432 1521 160 8985
55-64 613 5312 223 1257 426 7831
65-74 308 3769 105 659 930 5771
75+ 274 2030 38 251 2357 4950
Total: 31336 23580 1453 4814 3945 65128
Age by ethnic group at local authority level
LA=1
N.A. White Mixed Asian Black Chinese Total:
0-15 89 10146 420 1179 233 107 12174
16-24 568 6946 186 679 106 67 8552
25-34 23 6895 145 822 158 106 8149
35-45 3 7810 87 581 178 57 8716
45-54 3 8378 83 393 96 34 8987
55-64 0 7518 37 227 29 20 7831
65-74 0 5612 15 111 26 8 5772
75+ 0 4856 13 54 22 4 4949
Total: 686 58161 986 4046 848 403 65130
0-15 Age category:
total for row by ethnic
group is 12174
total for row by marital
status is 12176
Difference of 2
E.g. Age by ethnic group
vs Age by marital status
Aims: Recap
Flexible Timely Accessible
Access to more data in some small areas
What’s next?
• User research:
o Increasing understanding of user needs and priorities
• Statistical Disclosure Control:
o Further assurance of SDC methods
o Refining disclosure control parameters
• Next phase of system development
Hands On Session
• Flexible Dissemination System Prototype
o Test out the tool yourself on the laptops provided
o Try out a User Persona to access Census data
o Please complete our mini questionnaire afterwards
o Opportunity to provide feedback in a group discussion in 45 mins
• Please remember: It’s a prototype…..
o The data are artificial
o It’s demonstrative not representative
o The SDC methods and parameters are a work in progress
o Pass rates are illustrative not actual
Other Activities
• User Journey Activity
o Plot your own User Journey
o How would you expect to access, create and download your own
Census table?
• Statistical Disclosure Control Display
o Find out more about the methodology behind Flexible
Dissemination System
Enhanced Census Outputs
Administrative Data Census Team
2021 Census outputs roadshows
Spring 2018
Enhanced Census Outputs are…
Census based outputs where census
and administrative data are linked at
record level to enable the production of
multivariate, small area outputs
How do Enhanced Census Outputs differ
from Research Outputs?
Research Outputs Enhanced Census Outputs
Base Statistical Population
Dataset
2011 Census (to test quality)
and 2021 Census
Priorities Replicating as many census
topics as possible
Replacing some census
questions and producing new
topics
ECO topics
Number of rooms
Income
• Intend to recommend number of bedrooms question
be asked on the 2021 Census
• Continue research into using VOA data (and
potentially other sources) for 2021 Census outputs on
number of rooms (subject to statistical quality of the
admin data)
• High user requirement
• Admin Data Census Research Outputs already
produced – methodology to be applied to ECOs
• Awaiting self-assessment data (expected later this year)
ECO topics
Size of property
• Eurostat requirement currently met using English
Housing Survey
• Current research involves looking at the size of property
variable in the VOA data
• May provide more relevant data on occupancy rating
(research ongoing)
Possible other topics to consider in
medium-long term
Improving existing census topics (opportunistic use
of existing data)
Using VOA data to provide more detailed
breakdown of property type
Improving qualifications
statistics using data from
DfE, HESA etc
Next Steps
• To continue research and development for ECOs into:
1. Number of Rooms
2. Income
3. Size of Property
• Understanding data sources and user needs(See ‘Upcoming
Events’ slide)
• Developing and testing the methods and processes
• Assessing statistical quality of Enhanced Census Outputs
against National Statistics requirements
• Harmonisation
Our questions to you…
• Is there anything more on these topics
that you want to know?
• Are there any surprises in what we’re
covering?
• Statistical quality vs. geography?
National vs. experimental statistics?
How you can respond…
• Consultation
• Email:
Admin.Data.Census.Project@ons.gsi.gov.uk
• Come and have a chat
Upcoming Events
• Conference: Integrated data - Combining
data to answer society's biggest questions
9th – 10th July 2018
University of Surrey
• Transforming Labour Market Statistics by
Integrating Survey and Non-Survey Data
17th May 2018
Amba Hotel Charing Cross, London
2021 Census Outputs and Dissemination:
Closing
2021 Census outputs roadshows
Spring 2018
Vision for 2021 Census Outputs
Flexible Timely Accessible
2021 Census Outputs – What next?
2018
• Initial view on 2021 Census Output Content Design
2019
• Census Rehearsal
2020
• Finalise plans for 2021 Census outputs
2021
2022
• Begin publishing outputs
• Census Day
Questions
Please get in touch with any further feedback or questions:
census.outputs@ons.gov.uk
2021
Microdata products
Census 2021 | Outputs and Dissemination team | census.outputs@ons.gov.uk
What are census microdata?• Samples of anonymous unit-
record data from both
individuals and households
• Variables and derived
variables created using
responses to census
questions
• Census microdata products
have been published since the
1991 Census
Successes and lessons in 2011:
Some users of our microdata:
2021: Priorities for microdata
• Repeat the successes of 2011 products
• Products more timely and accessible
• New variables to reflect new questions
• Update product design where necessary
• Review disclosure control methodology
• Identify if UK-wide products are
possible
• Assess the possible use of admin data
• Possibility of safeguarded household
file
2011 public-access file:
What went well:
• Most comprehensive products to date
• Nested sampling methodology across
files
• First ever public-access file produced
Lessons learned:
• Products need to be considered earlier
• Accessibility concerns must be
Other census microdata:
You can also access historical microdata:
• Similar products for past censuses
• Files made from 1981, 1971 and 1961
data
• Similar file types and access
2011 Microdata products:
How are we engaging with users?
• Public consultation with users
• Working group with users and census
offices of Scotland and Northern
Ireland
In 2011 we made three microdata
products, with different levels of detail
and security, which we plan to replicate
in 2021:
Advantages over aggregate
data:• More complexity
• More flexibility
• More variables
• Comparability
across censuses
• All data in one file
• More multivariates
• Better represents
small populations
• Useful in linkage
1% sample, available to everyone online
Public-access
file:
5% sample, available with registration
Safeguarded
files:
10% sample, available in secure settings
Secure files:
First Name Last Name DOB County Occupation Ethnicity
John Smith 07/07/1993 Hampshire Researcher White British
Sensitive input (census responses)
A random 1% sample of records is selected, identifiable
information is removed, disclosure control methods are
applied, derived variables are added, all variables are
coded, and a codebook is created to help users interpret
data.
caseno age region occg ethnicityew fmspuk11
####### 24 10 21 01 2
Microdata output (safeguarded file)
(Removed sensitive
data)
(New
variable)
Click to edit Master title style
55
What are origin-destination data?
· Provide information relating to the flows
between two locations, and the characteristics
(gender, age, ethnic group) of the individuals
who make these journeys
· Derived from several census questions, relating
to place of current residence, place of residence
one year prior to census day, main place of
work and second residence if applicable
Diagram showing the flows between location of usual residence and place of work using 2011
origin-destination data for Portsmouth. Image available from
https://www.nomisweb.co.uk/census/2011/WU03UK/chart/1132462339
Types of origin-destination data
There are two main types of origin-destination
data:
1) Residential migration in the year before the
census (both internally and from abroad)
2) Travel to work by those (aged 16 and over) in
employment the week before the census and
their mode of transport. In 2011, this was
available for both main and second residence, if a
respondent resides there for more than 30 days a
year
Why are they important?
Census information on migration and commuting patterns means we can
assess:
• movement patterns in the population
• how and if the characteristics (age, gender, ethnicity and socio-economic
status) of these populations are changing
These data are of vital importance to local and central government in order to
strategically plan infrastructure for health, education, transport and housing
based on population need.
3 ways to access 2011 data
What are our plans for 2021?
We’re currently developing our plans for the 2021
Census origin-destination data. We’re doing this by:
· assessing the most appropriate access
arrangements with consideration of confidentiality
· assessing the number of tables to be produced and
the variable groupings in tables
· assessing Statistical Disclosure Control methods in
order to make more data available publicly
· researching what went well in 2011 and what
improvements are needed for 2021
· establishing a working group with external
stakeholders to advise on the creation and
dissemination of origin-destination data
For 2021, our key aim is to ensure that origin-
destination data are available within two years of
census day (sooner than 2011).
If you want more information or have any
suggestions please contact us at
census.outputs@ons.gov.uk.
Public
Availability:
- Available under “Open
Government License”
- Available via the NOMIS
website
Users:
Local authorities,
transport planners,
academics, students,
general public and local
interest/community
groups
Key facts:
- There were 20 publicly
available tables
- Since 2011, public
origin-destination tables
have been downloaded
138,000 times
Safeguarded
Availability:
- Only available to data
analysts under certain
terms and conditions
- Safeguarded tables were
made available via the UK
Data Service
Users:
Central and local
government, researchers,
academics and higher
level students
Key facts:
- Since 2011,
safeguarded tables have
been downloaded 3,900
times
Secure
Availability:
- Tables only available via
the ONS Secure Research
Service
- The secure data provide
multivariate flows at low
levels of geography
Users:
Researchers engaged in
large-scale social research
Researchers must be
completing research
deemed for the “public
good”
Key facts:
-Users must become
“Approved Researchers”
before gaining access to
the data
-Since 2011, there have
been 24 unique projects
using origin-destination
data
Image showing travel to work flows using census origin-destination. Blue lines represent
flows coming in and red lines show flows out from the selected location, to work
elsewhere. Image available from http://blog.datashine.org.uk/2014/07/datashine-travel-
to-work-flows/
Origin-destination data
Census 2021 | Outputs and Dissemination team | census.outputs@ons.gov.uk
National Statistics for Census 2021
Census 2021 | Outputs and Dissemination team | census.outputs@ons.gov.uk
The Outputs and Dissemination team is working in collaboration with teams throughout ONS who are involved in delivering
a successful census. Our aim is to ensure we successfully retain the National Statistics accreditation for the 2021 Census.
Innovations such as the development of new questions, the Statistical Disclosure Control methodology and the flexible
dissemination system will be of particular interest to the awarding body, the Office for Statistics Regulation. We’ll provide
testable evidence to demonstrate that we meet the Code of Practice for Statistics.
The Code of Practice is… a framework for producers of official statistics with the detailed practices they must commit to
when producing and releasing official statistics. The Code is based on three pillars and the following principles:
TRUSTWORTHINESS
Confidence in the people
and organisations that
produce statistics and data
QUALITY
Data and methods that
produce assured statistics
VALUE
Statistics that support
society’s needs for
information
Suitable data sources
Sound methods
Assured quality
Relevance to users
Accessibility
Clarity and insight
Innovation and improvement
Efficiency and proportionality
Honesty and integrity
Independent decision making and
leadership
Orderly release
Transparent processes and management
Professional capability
Data governance
The interests of users of statistics sits at the heart of the Code and we’re committed to involving users in the development of
our outputs. Help us to deliver census outputs by informing us how you use our data and take part in the consultation.
Click to edit Master title style
Targeted Record Swapping
Disclosure Checks
Cell Key Perturbation
Impact
Pairs of similar households from different geographical locations are swapped
Key features:
• This was the main source of protection used in the 2011 Census and will be the main source of protection for 2021
• Risky households are more likely to be swapped than non-risky households but all households have a chance of being
swapped
• Households are only swapped with other households of the same size so the number of persons and households for
each area are preserved
• Households are swapped within Local Authorities, however in rare cases, if a household has very unusual
characteristics then it could be swapped with a match in a nearby Local Authority. There are no households in Cornwall
swapped to Birmingham for example
Small inconsistencies can occur from the cell key method
when cells are constructed or aggregated in different
ways
Small changes are made to cells consistently to protect against
differencing
The criteria determining what level of detail is safe to
publish
Key features:
• The checks will be applied automatically speeding up the time between census day and publication as there will be
no need for ONS staff to assess every table as was the case in 2011
• The checks will be applied to each geography individually allowing more detail to be published for the areas that
pass the checks, unlike in 2011 where only the minimum amount of detail that passed for the whole country could be
published
• Key tables, for example univariates, will be published for all geographic areas
Key features:
• This method is adapted from the method originally developed by Australian Bureau of Statistics for protecting
their census tables
• Perturbation is applied consistently so that cells will always be perturbed in the same way; the same cell in a
different table will always have the same value
• To allow small cells to appear in census tables, zero value cells are also perturbed using a similar method which
will result in a consistent perturbation of zeros
• The method was originally developed to protect against disclosure from differencing when similar tables are
compared
Key features:
• The cell key method will be applied as a “light touch” so inconsistencies that occur are likely to be small
• Larger tables and tables that contain a large number of small cells will be more affected
What is Statistical
Disclosure Control (SDC)?
The application of methods to
protect individuals,
households and their
characteristics from
identification in published
tables (and microdata).
Why do we need SDC?
The Office for National
Statistics (ONS) has legal and
ethical obligations to protect
the data that we collect.
Failure to do so could result in
reduced response rates
affecting the quality of the
data.
In order to protect the data
and include small cells in the
published tables, we need to
modify the tables or
underlying microdata by
adding a sufficient layer of
uncertainty as to whether a
disclosure is real or not.
Assign each record a random number (rkey) For each cell, sum rkey and apply a function to get a
cell key
Use a look up table to get a perturbation value Apply the perturbation value to
the cell
Record rkey
r1 54
r2 4
r3 93
…
rN 26
Record rkey
r2 4
r4 61
r56 7
r72 90
Sum = 162
e.g. take the last two digits →
ckey = 62
Age by
sex
Male Female
0-15 . .
16-24 . 4
25-34 . .
…
Age by
sex
Male Female
0-15 . .
16-24 . 5
25-34 . .
…
1 2 3 … 61 62 63 … 99
1 +1
2 +1 -1
3 +1
4 -1 +1
5 -1 -1
…
Cell
Key
C
e
l
l
V
a
l
u
e
The same marginal totals in different
tables
Marginal totals can be created by
summing over the other variables in the
table. If you create a table of variable x
by y and compare it with a similar table
of variable x by z, you may see
differences between the values in the
marginal total of variable x.
Variables that are grouped differently in
different tables
Perturbation is applied to each cell
independently and grouping categories in
different ways will result in different
record keys and different perturbation
values being applied. This can also affect
the zeros perturbation.
Aggregating smaller geographies into a
larger area
Tables at different levels of geography
are perturbed independently so if you
sum all the Output Area (OA) level tables
within a Middle Super Output Area
(MSOA), the values may not match the
values in the perturbed MSOA table.
Aggregated
OAs
T
o
t
a
l
+
3
T
o
t
a
l
-
3
T
o
t
a
l
-
4
T
o
t
a
l
+
2T
o
t
a
l
-
1
Perturbation
total = -3
MSOA table
T
o
t
a
l
+
2
Perturbation
total = +2
Difference
= 5
Five year age
groups
Ten year age
groups
Ag
e
10-
14
Ag
e
15-
19
+
0
+
1
Age
10-
19
-
1
Difference
= 2
Ag
e
0-4
Ag
e
5-9
+
1
+
0
Age
0-9
+
0
Difference
= 1
p
v
a
l
u
e
p
v
a
l
u
e
For each record in the sample, look in the
surrounding areas for a similar household of the
same size
3
Table: Pet ownership by weekly household
expenditure
Why is this table
not safe to
publish?
Identity disclosure
Lots of 1s in a
table could mean
that respondent
could be identified
Attribute
Disclosures
When all values in a
row fall into one
column, you can
learn something
new about a
respondent. For
example, you can
work out a
household’s
expenditure by
knowing which pet
they have
Swap the households4
Select a sample of households to swap with
risky households more likely to be selected
2Identify risky households, for example unique on
some characteristic
1
Difference
= 3
x x +1 x x
y .. .. +1 .. ..
y .. .. .. .. ..
y .. .. -1 .. ..
y .. .. .. .. ..
y .. .. +1 .. ..
x x -2 x x
z .. .. .. .. ..
z .. .. -1 .. ..
z .. .. .. .. ..
z .. .. -1 .. ..
z .. .. .. .. ..
Using the 2021
method:If this area is safe
then it will be
published
If this area is not
safe then it won’t
be published
Using the 2011
method:This area, although
safe, could not be
published…
… because this
area is too
disclosive to
publish
Output
Area 1 £0
- £499
£500
- £999
£1,000
- £1,499
£1,500+
Dog 4 40 11 3
Cat 2 34 5 2
Fish 3 15 2 0
Horse 0 2 3 1
Other 1 10 3 0
Mixed 0 5 2 2
None 5 137 18 0
Output
Area 2 £0
- £499
£500
- £999
£1,000
- £1,499
£1,500+
Dog 0 52 0 0
Cat 0 36 0 0
Fish 0 1 0 0
Horse 0 0 0 1
Other 0 1 0 0
Mixed 0 0 0 0
None 1 203 0 0
Statistical Disclosure Control team
SDC.Queries@ons.gov.uk
Flexible Table Builder: Statistical Disclosure
Control Proposed method for Census 2021
1 2
3 4
1 2 3
Click to edit Master title style

2021 Census Outputs and Dissemination: Introduction

  • 1.
    2021 Census Outputsand Dissemination: Introduction 2021 Census outputs roadshows Spring 2018
  • 2.
    Aims for today •Outline of 2021 Census outputs strategy as detailed in our ‘Initial View on 2021 Census Output Content Design’ consultation launched at the end of February 2018 • Provide a demonstration of our current prototype of a flexible dissemination system • Update on our plans for enhanced census outputs • Answer your questions and get feedback on our plans for outputs
  • 3.
    Session outline 13:30 –13:45 Introduction to 2021 Census Outputs 13:45 – 14:15 Outputs Consultation 14:15 – 15:30 Flexible Dissemination System Includes:  Presentation  Coffee break  Try out the flexible table builder prototype  Visit market stalls to find out more about other 2021 Census outputs 15:30 – 16:00 Enhanced Census Outputs 16:00 – 16:15 Wrap-up and questions Next steps
  • 4.
    Vision for 2021Census Outputs Flexible Timely Accessible
  • 5.
    Analysis Programme • Makethe best use of census data, alongside administrative and survey data to describe the society we live in • Add value by helping users understand and interpret census data • Statistical bulletins, interactive visualisations etc • Currently being developed
  • 6.
    Future User Engagement Typeof Engagement User Research Public Consultation Working Groups ??? UKCC / UK Harmonised Working Group Conferences User Groups ONS staff
  • 7.
  • 8.
    Initial View on2021 Census Output Content Design: Public consultation 2021 Census outputs roadshows Spring 2018
  • 9.
    Overview • Initial high-levelplans for 2021 Census output content design and dissemination (England and Wales only)  Outline our vision for 2021 Census outputs  Our first step towards understanding what users require from 2021 Census outputs  Proposal at present  Feedback will help inform our future research and development of these outputs
  • 10.
    Consultation open: 28th February– 23rd May 2018 https://consultations.ons.gov.uk/census/initial-view- on-the-2021-census-output-design
  • 11.
    Flexible Dissemination System •Web-based interactive dissemination system  Define own queries  Select geography, population base and variables required • Pre-determined univariate tables More details in next session Flexible Timely Accessible
  • 12.
    Enhanced Census Outputs •2021 Census-based outputs where census data and administrative data are linked at the record level to enable the production of multivariate, small area outputs More details in a later session
  • 13.
    UK Harmonisation andUK Statistics • Working with NRS and NISRA to harmonise wherever possible  Statistical disclosure control  Dissemination approach  Metadata • Exploring how we provide UK data to users
  • 14.
    Microdata Public 1% sample- individuals Accessed via ONS website Safeguarded 2x5% sample – individuals Accessed via UKDS Secure 10% sample – individuals and household Accessed via ONS SRS at ONS Titchfield, Newport, London 2011: • Requirements for safeguarded household file • Review sample design and size • Microdata Working Group
  • 15.
    Origin-Destination • Review statisticaldisclosure control methodology • Aim to make more data publically available • Consider number of tables produced • Origin-destination Working Group Migration Residence Workplace Student
  • 16.
    Accessing sensitive censusdata • Provide access to sensitive census data in secure and controlled research environments • Access granted in accordance with ONS Approved Researcher Scheme • Continue to review locations and access arrangements • Longitudinal Study – 1% sample
  • 17.
    Output categories andpopulation bases Output categories • Topics - research is ongoing • Recommendations to inform White Paper in 2018 Population bases • Continue to use 2011 population bases Small populations • Continue to identify value and dissemination
  • 18.
    Geography Output Areas andWorkplace Zones • Continue to be building blocks for census geographies • Keep maintenance to within 5% Grids • 2011 – 1km square grid for Eurostat using postcode-level population estimates • Explore user requirement for grids Civil parishes • 2011 – 10% parishes without data • Identify methodology to make more data available for parishes
  • 19.
    Interpreting 2021 Censusresults Metadata • Will continue to produce metadata • Will consider best way to provide this based on the dissemination system Quality Assurance • National Statistics Accreditation • Population estimates
  • 20.
    Evaluation • Strength ofimpact (positive, negative or none) of our intended design on: Accessibility Flexibility Timeliness of outputs for users • Evidence used to identify the most important concerns for users and prioritise research on these aspects
  • 21.
    Public good • Workbeing undertaken to “serve the public good”   using census data for public policy decision making  using census data for public service planning and delivery  using census data for allocation of public resources  using census data for decisions which are likely to significantly benefit the UK economy, society or quality of life of people in the UK
  • 22.
    Example response toconsultation As a private sector transport provider we worked with local authorities [insert name of LA] in our region to identify where a new bus route should be placed. We required access to safeguarded and secure origin- destination data as we needed to know where residents lived and worked and how many cars they owned. We were unable to easily access the safeguarded and secure origin-destination data as we are based in the North West and the VML is in Newport, London, Titchfield. This caused us 12 month delay in the project to enable time to travel to these locations and ££ in extra transport costs to travel to these locations. Once we did access the data, because it was 5 years after census day, it was out of date.
  • 23.
    Consultation open: 28th February– 23rd May 2018 https://consultations.ons.gov.uk/census/initial-view- on-the-2021-census-output-design Any questions: census.outputs@ons.gov.uk
  • 24.
    Flexible Dissemination System 2021Census outputs roadshows Spring 2018
  • 25.
    Presentation Overview • OurAims for 2021 Census Outputs • How we propose meeting those aims • The benefits • The trade-offs • Where we are - the prototype • What’s next ? • Hands On Session • Other Activities
  • 26.
    Aims: Designed basedon User Feedback Flexible Timely Accessible
  • 27.
    Proposal: Fast &Flexible Dissemination Statistical disclosure control: Aims to ensure that statistical outputs provide as much utility to users, whilst maintaining the confidentiality of individuals, households, etc. Automated checks
  • 28.
    Statistical Disclosure Control Automated checks Thetype of automated SDC checks we are looking at are: •Maximum number of cells •Maximum number of variables •Small values in totals •Percentage of cells that are populated •Percentage of cases in one cell/row/column •Real/fake 1s and attribute disclosures Checks and parameters have not been finalised yet. Record swapping and cell key information is on the display
  • 29.
    Is this possible? •Prototype developed by ONS with Sensible Code Co. • User interface – for demonstration purposes only • Automated Statistical Disclosure Control embedded. • 60m record artificial database grossed up from a 2011 Census Microdata Sample • 8 levels of Output Area geography hierarchy (181,408 OAs) • 80 variables incl. person and household characteristics
  • 30.
    Here is anexample…. Number of areas for which data will be released
  • 31.
    Pass or Fail?- Same Query for Different Areas Number of areas for which data will be released
  • 32.
    ? ?? ? ? So… what arethe benefits? Users can define their own data query All standard outputs available from one place More detail for some small areas Wider benefits for data dissemination across the GSS Multivariate data earlier than before
  • 33.
    The benefits comewith some trade offs • The protection added to reduce the potential for “differencing” between similar tables is given by the “cell-key method”. • Small inconsistencies will exist in the totals between different tables • These inconsistencies can occur in two ways………
  • 34.
    Summing of IndividualCells Ethnicity by occupation at local authority level LA=1 Occupation Ethnicity: 1 2 3 4 5 6 7 8 9 White 3885 6014 4325 5726 5202 4172 4394 4282 6784 44784 Mixed 27 55 64 44 47 53 70 34 70 464 Asian 198 357 134 168 171 135 292 233 294 1982 Black 23 93 42 49 34 76 44 38 97 496 Chinese 23 40 11 6 25 15 23 15 33 191 4156 6559 4576 5993 5479 4451 4823 4602 7278 47917 Ethnicity by occupation at Output Area level aggregated to local authority level LA=1 Occupation Ethnicity: 1 2 3 4 5 6 7 8 9 White 3886 6012 4324 5730 5205 4171 4391 4276 6784 44779 Mixed 27 57 66 43 48 52 68 35 70 466 Asian 196 359 133 166 172 137 292 231 291 1977 Black 23 93 41 48 33 76 44 41 94 493 Chinese 22 40 11 5 25 15 24 15 32 189 4154 6561 4575 5992 5483 4451 4819 4598 7271 47904 E.g. Local Authority total White ethnic group category: Total for row at local authority level is 44,784 Total for row in Output Area aggregation is 44,779 Difference of 5 vs Sum of aggregated output areas
  • 35.
    Tables Sharing aCommon Variable Age by marital status at local authority level LA=1 Single Married Separated Divorced Widowed Total: 0-15 12176 0 0 0 0 12176 16-24 8313 207 11 13 8 8552 25-34 5249 2439 241 199 20 8148 35-44 2886 4468 403 914 44 8715 45-54 1517 5355 432 1521 160 8985 55-64 613 5312 223 1257 426 7831 65-74 308 3769 105 659 930 5771 75+ 274 2030 38 251 2357 4950 Total: 31336 23580 1453 4814 3945 65128 Age by ethnic group at local authority level LA=1 N.A. White Mixed Asian Black Chinese Total: 0-15 89 10146 420 1179 233 107 12174 16-24 568 6946 186 679 106 67 8552 25-34 23 6895 145 822 158 106 8149 35-45 3 7810 87 581 178 57 8716 45-54 3 8378 83 393 96 34 8987 55-64 0 7518 37 227 29 20 7831 65-74 0 5612 15 111 26 8 5772 75+ 0 4856 13 54 22 4 4949 Total: 686 58161 986 4046 848 403 65130 0-15 Age category: total for row by ethnic group is 12174 total for row by marital status is 12176 Difference of 2 E.g. Age by ethnic group vs Age by marital status
  • 36.
    Aims: Recap Flexible TimelyAccessible Access to more data in some small areas
  • 37.
    What’s next? • Userresearch: o Increasing understanding of user needs and priorities • Statistical Disclosure Control: o Further assurance of SDC methods o Refining disclosure control parameters • Next phase of system development
  • 38.
    Hands On Session •Flexible Dissemination System Prototype o Test out the tool yourself on the laptops provided o Try out a User Persona to access Census data o Please complete our mini questionnaire afterwards o Opportunity to provide feedback in a group discussion in 45 mins • Please remember: It’s a prototype….. o The data are artificial o It’s demonstrative not representative o The SDC methods and parameters are a work in progress o Pass rates are illustrative not actual
  • 39.
    Other Activities • UserJourney Activity o Plot your own User Journey o How would you expect to access, create and download your own Census table? • Statistical Disclosure Control Display o Find out more about the methodology behind Flexible Dissemination System
  • 40.
    Enhanced Census Outputs AdministrativeData Census Team 2021 Census outputs roadshows Spring 2018
  • 41.
    Enhanced Census Outputsare… Census based outputs where census and administrative data are linked at record level to enable the production of multivariate, small area outputs
  • 42.
    How do EnhancedCensus Outputs differ from Research Outputs? Research Outputs Enhanced Census Outputs Base Statistical Population Dataset 2011 Census (to test quality) and 2021 Census Priorities Replicating as many census topics as possible Replacing some census questions and producing new topics
  • 43.
    ECO topics Number ofrooms Income • Intend to recommend number of bedrooms question be asked on the 2021 Census • Continue research into using VOA data (and potentially other sources) for 2021 Census outputs on number of rooms (subject to statistical quality of the admin data) • High user requirement • Admin Data Census Research Outputs already produced – methodology to be applied to ECOs • Awaiting self-assessment data (expected later this year)
  • 44.
    ECO topics Size ofproperty • Eurostat requirement currently met using English Housing Survey • Current research involves looking at the size of property variable in the VOA data • May provide more relevant data on occupancy rating (research ongoing)
  • 45.
    Possible other topicsto consider in medium-long term Improving existing census topics (opportunistic use of existing data) Using VOA data to provide more detailed breakdown of property type Improving qualifications statistics using data from DfE, HESA etc
  • 46.
    Next Steps • Tocontinue research and development for ECOs into: 1. Number of Rooms 2. Income 3. Size of Property • Understanding data sources and user needs(See ‘Upcoming Events’ slide) • Developing and testing the methods and processes • Assessing statistical quality of Enhanced Census Outputs against National Statistics requirements • Harmonisation
  • 47.
    Our questions toyou… • Is there anything more on these topics that you want to know? • Are there any surprises in what we’re covering? • Statistical quality vs. geography? National vs. experimental statistics?
  • 48.
    How you canrespond… • Consultation • Email: Admin.Data.Census.Project@ons.gsi.gov.uk • Come and have a chat
  • 49.
    Upcoming Events • Conference:Integrated data - Combining data to answer society's biggest questions 9th – 10th July 2018 University of Surrey • Transforming Labour Market Statistics by Integrating Survey and Non-Survey Data 17th May 2018 Amba Hotel Charing Cross, London
  • 50.
    2021 Census Outputsand Dissemination: Closing 2021 Census outputs roadshows Spring 2018
  • 51.
    Vision for 2021Census Outputs Flexible Timely Accessible
  • 52.
    2021 Census Outputs– What next? 2018 • Initial view on 2021 Census Output Content Design 2019 • Census Rehearsal 2020 • Finalise plans for 2021 Census outputs 2021 2022 • Begin publishing outputs • Census Day
  • 53.
    Questions Please get intouch with any further feedback or questions: census.outputs@ons.gov.uk
  • 54.
    2021 Microdata products Census 2021| Outputs and Dissemination team | census.outputs@ons.gov.uk What are census microdata?• Samples of anonymous unit- record data from both individuals and households • Variables and derived variables created using responses to census questions • Census microdata products have been published since the 1991 Census Successes and lessons in 2011: Some users of our microdata: 2021: Priorities for microdata • Repeat the successes of 2011 products • Products more timely and accessible • New variables to reflect new questions • Update product design where necessary • Review disclosure control methodology • Identify if UK-wide products are possible • Assess the possible use of admin data • Possibility of safeguarded household file 2011 public-access file: What went well: • Most comprehensive products to date • Nested sampling methodology across files • First ever public-access file produced Lessons learned: • Products need to be considered earlier • Accessibility concerns must be Other census microdata: You can also access historical microdata: • Similar products for past censuses • Files made from 1981, 1971 and 1961 data • Similar file types and access 2011 Microdata products: How are we engaging with users? • Public consultation with users • Working group with users and census offices of Scotland and Northern Ireland In 2011 we made three microdata products, with different levels of detail and security, which we plan to replicate in 2021: Advantages over aggregate data:• More complexity • More flexibility • More variables • Comparability across censuses • All data in one file • More multivariates • Better represents small populations • Useful in linkage 1% sample, available to everyone online Public-access file: 5% sample, available with registration Safeguarded files: 10% sample, available in secure settings Secure files: First Name Last Name DOB County Occupation Ethnicity John Smith 07/07/1993 Hampshire Researcher White British Sensitive input (census responses) A random 1% sample of records is selected, identifiable information is removed, disclosure control methods are applied, derived variables are added, all variables are coded, and a codebook is created to help users interpret data. caseno age region occg ethnicityew fmspuk11 ####### 24 10 21 01 2 Microdata output (safeguarded file) (Removed sensitive data) (New variable)
  • 55.
    Click to editMaster title style 55 What are origin-destination data? · Provide information relating to the flows between two locations, and the characteristics (gender, age, ethnic group) of the individuals who make these journeys · Derived from several census questions, relating to place of current residence, place of residence one year prior to census day, main place of work and second residence if applicable Diagram showing the flows between location of usual residence and place of work using 2011 origin-destination data for Portsmouth. Image available from https://www.nomisweb.co.uk/census/2011/WU03UK/chart/1132462339 Types of origin-destination data There are two main types of origin-destination data: 1) Residential migration in the year before the census (both internally and from abroad) 2) Travel to work by those (aged 16 and over) in employment the week before the census and their mode of transport. In 2011, this was available for both main and second residence, if a respondent resides there for more than 30 days a year Why are they important? Census information on migration and commuting patterns means we can assess: • movement patterns in the population • how and if the characteristics (age, gender, ethnicity and socio-economic status) of these populations are changing These data are of vital importance to local and central government in order to strategically plan infrastructure for health, education, transport and housing based on population need. 3 ways to access 2011 data What are our plans for 2021? We’re currently developing our plans for the 2021 Census origin-destination data. We’re doing this by: · assessing the most appropriate access arrangements with consideration of confidentiality · assessing the number of tables to be produced and the variable groupings in tables · assessing Statistical Disclosure Control methods in order to make more data available publicly · researching what went well in 2011 and what improvements are needed for 2021 · establishing a working group with external stakeholders to advise on the creation and dissemination of origin-destination data For 2021, our key aim is to ensure that origin- destination data are available within two years of census day (sooner than 2011). If you want more information or have any suggestions please contact us at census.outputs@ons.gov.uk. Public Availability: - Available under “Open Government License” - Available via the NOMIS website Users: Local authorities, transport planners, academics, students, general public and local interest/community groups Key facts: - There were 20 publicly available tables - Since 2011, public origin-destination tables have been downloaded 138,000 times Safeguarded Availability: - Only available to data analysts under certain terms and conditions - Safeguarded tables were made available via the UK Data Service Users: Central and local government, researchers, academics and higher level students Key facts: - Since 2011, safeguarded tables have been downloaded 3,900 times Secure Availability: - Tables only available via the ONS Secure Research Service - The secure data provide multivariate flows at low levels of geography Users: Researchers engaged in large-scale social research Researchers must be completing research deemed for the “public good” Key facts: -Users must become “Approved Researchers” before gaining access to the data -Since 2011, there have been 24 unique projects using origin-destination data Image showing travel to work flows using census origin-destination. Blue lines represent flows coming in and red lines show flows out from the selected location, to work elsewhere. Image available from http://blog.datashine.org.uk/2014/07/datashine-travel- to-work-flows/ Origin-destination data Census 2021 | Outputs and Dissemination team | census.outputs@ons.gov.uk
  • 56.
    National Statistics forCensus 2021 Census 2021 | Outputs and Dissemination team | census.outputs@ons.gov.uk The Outputs and Dissemination team is working in collaboration with teams throughout ONS who are involved in delivering a successful census. Our aim is to ensure we successfully retain the National Statistics accreditation for the 2021 Census. Innovations such as the development of new questions, the Statistical Disclosure Control methodology and the flexible dissemination system will be of particular interest to the awarding body, the Office for Statistics Regulation. We’ll provide testable evidence to demonstrate that we meet the Code of Practice for Statistics. The Code of Practice is… a framework for producers of official statistics with the detailed practices they must commit to when producing and releasing official statistics. The Code is based on three pillars and the following principles: TRUSTWORTHINESS Confidence in the people and organisations that produce statistics and data QUALITY Data and methods that produce assured statistics VALUE Statistics that support society’s needs for information Suitable data sources Sound methods Assured quality Relevance to users Accessibility Clarity and insight Innovation and improvement Efficiency and proportionality Honesty and integrity Independent decision making and leadership Orderly release Transparent processes and management Professional capability Data governance The interests of users of statistics sits at the heart of the Code and we’re committed to involving users in the development of our outputs. Help us to deliver census outputs by informing us how you use our data and take part in the consultation.
  • 57.
    Click to editMaster title style Targeted Record Swapping Disclosure Checks Cell Key Perturbation Impact Pairs of similar households from different geographical locations are swapped Key features: • This was the main source of protection used in the 2011 Census and will be the main source of protection for 2021 • Risky households are more likely to be swapped than non-risky households but all households have a chance of being swapped • Households are only swapped with other households of the same size so the number of persons and households for each area are preserved • Households are swapped within Local Authorities, however in rare cases, if a household has very unusual characteristics then it could be swapped with a match in a nearby Local Authority. There are no households in Cornwall swapped to Birmingham for example Small inconsistencies can occur from the cell key method when cells are constructed or aggregated in different ways Small changes are made to cells consistently to protect against differencing The criteria determining what level of detail is safe to publish Key features: • The checks will be applied automatically speeding up the time between census day and publication as there will be no need for ONS staff to assess every table as was the case in 2011 • The checks will be applied to each geography individually allowing more detail to be published for the areas that pass the checks, unlike in 2011 where only the minimum amount of detail that passed for the whole country could be published • Key tables, for example univariates, will be published for all geographic areas Key features: • This method is adapted from the method originally developed by Australian Bureau of Statistics for protecting their census tables • Perturbation is applied consistently so that cells will always be perturbed in the same way; the same cell in a different table will always have the same value • To allow small cells to appear in census tables, zero value cells are also perturbed using a similar method which will result in a consistent perturbation of zeros • The method was originally developed to protect against disclosure from differencing when similar tables are compared Key features: • The cell key method will be applied as a “light touch” so inconsistencies that occur are likely to be small • Larger tables and tables that contain a large number of small cells will be more affected What is Statistical Disclosure Control (SDC)? The application of methods to protect individuals, households and their characteristics from identification in published tables (and microdata). Why do we need SDC? The Office for National Statistics (ONS) has legal and ethical obligations to protect the data that we collect. Failure to do so could result in reduced response rates affecting the quality of the data. In order to protect the data and include small cells in the published tables, we need to modify the tables or underlying microdata by adding a sufficient layer of uncertainty as to whether a disclosure is real or not. Assign each record a random number (rkey) For each cell, sum rkey and apply a function to get a cell key Use a look up table to get a perturbation value Apply the perturbation value to the cell Record rkey r1 54 r2 4 r3 93 … rN 26 Record rkey r2 4 r4 61 r56 7 r72 90 Sum = 162 e.g. take the last two digits → ckey = 62 Age by sex Male Female 0-15 . . 16-24 . 4 25-34 . . … Age by sex Male Female 0-15 . . 16-24 . 5 25-34 . . … 1 2 3 … 61 62 63 … 99 1 +1 2 +1 -1 3 +1 4 -1 +1 5 -1 -1 … Cell Key C e l l V a l u e The same marginal totals in different tables Marginal totals can be created by summing over the other variables in the table. If you create a table of variable x by y and compare it with a similar table of variable x by z, you may see differences between the values in the marginal total of variable x. Variables that are grouped differently in different tables Perturbation is applied to each cell independently and grouping categories in different ways will result in different record keys and different perturbation values being applied. This can also affect the zeros perturbation. Aggregating smaller geographies into a larger area Tables at different levels of geography are perturbed independently so if you sum all the Output Area (OA) level tables within a Middle Super Output Area (MSOA), the values may not match the values in the perturbed MSOA table. Aggregated OAs T o t a l + 3 T o t a l - 3 T o t a l - 4 T o t a l + 2T o t a l - 1 Perturbation total = -3 MSOA table T o t a l + 2 Perturbation total = +2 Difference = 5 Five year age groups Ten year age groups Ag e 10- 14 Ag e 15- 19 + 0 + 1 Age 10- 19 - 1 Difference = 2 Ag e 0-4 Ag e 5-9 + 1 + 0 Age 0-9 + 0 Difference = 1 p v a l u e p v a l u e For each record in the sample, look in the surrounding areas for a similar household of the same size 3 Table: Pet ownership by weekly household expenditure Why is this table not safe to publish? Identity disclosure Lots of 1s in a table could mean that respondent could be identified Attribute Disclosures When all values in a row fall into one column, you can learn something new about a respondent. For example, you can work out a household’s expenditure by knowing which pet they have Swap the households4 Select a sample of households to swap with risky households more likely to be selected 2Identify risky households, for example unique on some characteristic 1 Difference = 3 x x +1 x x y .. .. +1 .. .. y .. .. .. .. .. y .. .. -1 .. .. y .. .. .. .. .. y .. .. +1 .. .. x x -2 x x z .. .. .. .. .. z .. .. -1 .. .. z .. .. .. .. .. z .. .. -1 .. .. z .. .. .. .. .. Using the 2021 method:If this area is safe then it will be published If this area is not safe then it won’t be published Using the 2011 method:This area, although safe, could not be published… … because this area is too disclosive to publish Output Area 1 £0 - £499 £500 - £999 £1,000 - £1,499 £1,500+ Dog 4 40 11 3 Cat 2 34 5 2 Fish 3 15 2 0 Horse 0 2 3 1 Other 1 10 3 0 Mixed 0 5 2 2 None 5 137 18 0 Output Area 2 £0 - £499 £500 - £999 £1,000 - £1,499 £1,500+ Dog 0 52 0 0 Cat 0 36 0 0 Fish 0 1 0 0 Horse 0 0 0 1 Other 0 1 0 0 Mixed 0 0 0 0 None 1 203 0 0 Statistical Disclosure Control team SDC.Queries@ons.gov.uk Flexible Table Builder: Statistical Disclosure Control Proposed method for Census 2021 1 2 3 4 1 2 3
  • 58.
    Click to editMaster title style