As an organisation who work extensively with the chief statistical organisations on both sides of the Tasman, our demographer and Census expert, Glenn Capuano, shares his experience working with data from the Australian Bureau of Statistics and Stats New Zealand.
2. About .id
.id is a company of demographers, housing analysts, forecasters and Census
data experts who provide tools and consulting services for Local Government.
.id stands for ‘informed decisions’. We provide government and community
decision makers with readily accessible, easy to use information about their
changing places.
.id believes by making demographic information accessible and promoting
evidence-based decision making, we are contributing to a fairer and more
sustainable society.
.id online community profiles are the industry standard for Census
dissemination in Australia. We have over 300 local government clients covering
more than 85% of the Australian population. We now have these resources for
NZ local government.
3. In 2010, .id moved into the New Zealand market.
We thought due to the similarities between our countries,
we could easily move our socio-demographic suite of
planning tools across the Tasman.
However we encountered
some difficulties….
4. 1. The approach to geographies is different
between Australia and New Zealand.
In Australia, we offer the ability to do custom geography (not
available with ABS).
Data is ordered at the smallest geographic level, and aggregated
up, splitting separately for each Census period, the CDs or SA1s, to
match the client’s defined boundaries.
In Australia we have multiple census years (initially 3 Census
periods, this built to 4, 5 and now 6 periods. We now present 25
years of demographic change (2016 back to 1991).
5. 1. The approach to geographies is different
between Australia and New Zealand
(continued).
StatsNZ provide different services to ABS:
StatsNZ release data for Census Area Units, allowing users a suburb level
experience.
StatsNZ each Census released data on current boundaries, back-
adjusting two census periods.
We focused on our simple yet effective way of presenting and analyzing
data, called “Dominant-Emerging Analysis”.
Understanding changing populations helps councils to improve their
analysis.
Also personalized service and demographic assistance to all our LG
clients.
6. 2. Differences in questions
Data in the NZ Census which would be useful in the
Australian Census?
Smoking rates.
Sources of Income
Pre-calculated time series
7. 2. Differences in questions
Data in the Australian Census which would be useful
in the New Zealand Census?
mortgage repayments
more detailed ancestry
information
8. 3. Differences in confidentiality – the StatsNZ approach
StatsNZ treatment of confidentiality
2013 - Eight confidentiality rules applied.
http://archive.stats.govt.nz/Census/2013-
census/methodology/confidentiality-how-applied.aspx
All numbers are rounded to multiples of 3.
Depending on the complexity of the table (number of crosstab
variables) and the average cell size, cells with less than 6
respondents may be suppressed. Suppressed cells are marked
with a “C”.
This can cause problems in databases not set up to work with
strings
While it’s clear which cells are suppressed, calculating percentages
becomes a problem.
9. 3. Confidentiality: Difference in approach
Both ABS and Stats NZ take confidentiality very seriously, and protect
individual data very well. But they do it quite differently.
ABS treatment of confidentiality
Pre 2006 - Small cells with a 1 or 2 in them were randomly adjusted to 0 or
3
2006-2011 - “Perturbation” applied, making small adjustments to all cells,
but additivity retained, just as many cells adjusted down as up
2016 - “Perturbation” continued to be applied, but without “additivity”
adjustments.
10. 4. Changes to the Australian 2016 Census
Removal of Additivity
The removal of additivity caused issues with our 2016 Census data
because small numbers were now set to zero, (similar to the Stats NZ
confidentiality model but without the convenient “C” designation so you
could identify adjusted cells).
Notification of the additivity change was too late to allow adjustment of
custom orders. When we got our data order for 2016 Census, we were
“missing” almost 2 million people, or 10% of population, due to the new
rules.
Had to reorder a number of our datasets
Flexibility was impacted.
11. 5. Changes to the 2016 Australian Census
… and changes to Journey to Work coding
In 2016, the ABS decided to impute work destinations for those
who didn’t state one, or it couldn’t be coded. Done by comparing to
other similar records.
This change meant 2016 data and 2011 data can’t be directly
compared on a place of work basis.
Change not well publicised, so many people make the mistake of
comparing – leading to incorrect conclusions about the magnitude
of increase in working population.
We sourced experimental imputed data from 2011 from ABS
unpublished data. A huge job as it wasn’t available at Local
Government level and needed to be concorded.
12. 6. To the future: The Australian 2021 Census
submissions process
ABS has just been through a comprehensive topic consultation process.
As representatives for almost 300 Local Government Areas in Australia,
.id was in a unique position to advocate for topics on behalf of our clients.
We canvassed our clients’ views as well as gathering information gleaned
during training and presentations across Australia
We collated this into 7 submissions, of which 5 suggested new topics, one
for retention of a mooted topic removal, and one for removal of a topic.
13. 6. The Australian 2021 Census submissions process
(continued)
Overall the ABS received 450 submissions. They make interesting
reading (315 published)
https://consult.abs.gov.au/census/census-
topics/consultation/published_select_respondent
There is clearly a strong level of public interest in the Census despite
the detail required when submitting. Submissions required the
following detail
• The need for data at a national level
• The impact on respondent burden
• The importance of cross-tab and small area information,
• The lack of alternative sources
14. 7. The Australian 2021 submissions process - results
How did we do? – as a private sector organisation we don’t get automatic access to
stakeholder meetings etc..
Three of our seven proposals were supported, and will be
investigated as new topics Including some of the new Census topics.
A further two topics were not supported in terms of the proposed change
or expansion, but will be retained in current form.
15. 7. The Australian 2021 submissions process - Changes
to questions
More detail on disability and need for assistance
the ABS are going to review this question
Volunteering
ABS are retaining this topic without improvement.
Journey to Education
Where people go to education and how they get there - to be tested for
2021.
Long-term or chronic health conditions
ABS is looking at how to ask these questions in the future.
16. 7. The Australian 2021 submissions process - Changes
to questions
Method of travel to work
Our suggestion of including “ride sharing service” as a category was not supported.
Number of motor vehicles per household
The ABS have confirmed the removal of this question in the 2021 Census. This will
impact Local Government clients, who use this data in their planning processes,
particularly for new developments, and working out developer contributions.
Domestic work
An underutilized topic – we suggested “drop it” to make way for more useful topics.
The ABS didn’t agree, and are keeping the question on domestic work which has
been in the Census since 2006.
17. 7. The Australian 2021 submissions process - Changes
to questions
Other new or changed questions include:
Sexual orientation and gender identity
Shared care of children within families
Aboriginal/Torres Strait Islander cultural identity
Defence force service.
18. 8. Where to now in New Zealand?
The delay in 2018 Census data in New Zealand has caused some
issues for our clients.
Increased demand for our forecasts, which provide more detail,
nuanced information independent of Census results.
We are ready to work with the new geographic areas from 2018
Census.
And we can have the data on the sites soon after release
Including some of the new Census topics.
We will work with our clients to get the best result in future.
19. Bringing together in depth analysis:
demographic
economic
housing
population forecasting
Our main value-add in Australia is the ability to do custom geography.
The Australian Census has never been released at meaningful local geography – they used these things called Statistical Local Areas, which no-one understood
So we put together a profile, based on actual localities, suburbs, districts, which were workshopped with council. This proved extremely popular.
The biggest selling point for this was that we could do multiple census years by cutting the geographic areas. Initially 3 Census periods, this built to 4, 5 and now 6 periods (2016 back to 1991).
The ABS always treated Census as a snapshot and released very little at time series due to changes in geographic areas making this not supported.
We ordered all the data at the smallest geographic level, and aggregated up, splitting separately for each Census period, the CDs or SA1s, to match current user defined boundaries.
Difficulty with porting this to New Zealand:
Statistics NZ already released data for Area Units, which gave users a suburb level experience already.
Statistics NZ each Census released data on current boundaries, back-adjusting the old Census data to match – this was our big value-add in Australia.
What we still had though was a very user friendly, simple yet effective way of presenting and analyzing data, called “Dominant-Emerging Analysis”.
And our expertise in understanding changing populations – all our sites come with access to this expertise and training sessions to improve the capacity of councils to do their own analysis.
As a result we have still been fairly successful in New Zealand with the Community Profile.
Precalculated time series – while this is our major value-add, it’s a great service provided by Stats NZ which Australia doesn’t really have except for quite wide regions.
We have largely had to make sure that the tables we present don’t have C’s in them. Mostly they are single variable tables which don’t have this rule applied, so that’s OK. Otherwise calculating percentages is an issue.
But this method of doing confidentiality meant we couldn’t order data at the smallest level and aggregate up for output, the way we did in Australia. Any custom geographic areas required a special order each time.
The move to the 2013 Census following the Christchurch Earthquake worked OK for us. It’s a huge job updating all the new data, so it suited us to have the censuses staggered.
The delay to the 2018 results has caused a few issues, however. Glenn – do you think that these internal issues are of interest to our audience?
ABS Pre 2006
ABS 2006-2011
ABS 2016
ABS
Small cells with a 1 or 2 in them were randomly adjusted to 0 or 3. All other data remained the same, and could be added.
“Perturbation” applied, making small adjustments to all cells, but additivity retained, just as many cells adjusted down as up.
Tables could be added to produce totals close to actual totals, though adding small cells not recommended.
“Perturbation” continued to be applied, but without “additivity” adjustments. Meaning that small cells were set to zero, and tables no longer had internal consistency, adding to totals. Totals were provided but percentages could no longer be calculated for sub-categories as they don’t add to 100%.
Notification on additivity removal very late, around the time of data release. We had typically built up larger geographies from smaller for our clients, and also larger categories based on smaller categories. Eg. we ordered single year of age data and aggregated to a range of age groups.
When we got this particular table from the 2017 Census, we discovered the totals were “missing” almost 2 million people, or 10% of population, due to the new rules.
Had to reorder a number of our datasets and switch to only ordering the categories we actually display – removing flexibility for our clients. Also many clients have been moved onto fairly standard geographic areas – fortunately the ABS now supports these in 2016 to a level not supported before.
In 2016, the ABS decide to impute work destinations for those who didn’t state one, or it couldn’t be coded.
Much of the interest from our clients is in time series – how are their commuting patterns changing over time?
This change means that 2016 data and 2011 data can’t be directly compared on a place of work basis.
But the ABS hasn’t publicised this and many people make the mistake of comparing – leading to incorrect conclusions about the magnitude of increase in working population.
We sourced experimental imputed data from 2011 from ABS unpublished data, but this was a huge job as it wasn’t available at Local Government level and needed to be concorded.
There is clearly a strong level of public interest in the Census and the data which can be obtained from it. Some of our clients also put in submissions directly.
Submissions had to be quite detailed, outlining the need for data at a national level, impact on respondent burden, importance of cross-tab and small area information, and lack of alternative sources etc.
In November 2018, ABS published its response, including which topics are planned for inclusion or at least further exploration.
3 of our 7 proposals were supported, and will be investigated as new topics.
A further two topics were not supported in terms of the proposed change or expansion, but will be retained in current form.
More detail on disability and need for assistance
Supported and the ABS are going to review this question
Volunteering
ABS are retaining this topic without the improvement in detail we suggested. Describe the potential improvement
Journey to Education
One of the most exciting new topics under consideration – where people go to education and how they get there is going to be tested for 2021.
Long-term or chronic health conditions
Suggested by several of our respondents and many other submitters, this is supported and ABS is looking at how to ask these questions in the future.
Method of travel to work
Our suggestion of including “ride sharing service” as a category was not supported.
Number of motor vehicles per household
We strongly supported retention of this topic which was planned to be dropped. Unfortunately the ABS have confirmed that this will be removed from the 2021 Census. This is a major blow to our Local Government clients, who use this data in their planning processes, particularly for new developments, and working out developer contributions.
Domestic work
This very underutilized topic was our suggestion to drop, to make way for more useful topics. The ABS didn’t agree, and are keeping the question on domestic work which has been in the Census since 2006.
Other new or changed questions include:
Sexual orientation and gender identity (will be very interested to see how this can work on a self-responded form).
Shared care of children within families
Aboriginal/Torres Strait Islander cultural identity – more information on this perhaps similar to Iwi information collected on the Maori population in NZ?
Defence force service.
DEMOGRAPHIC
Community profiling
Demographic analysis
Social narratives.
ECONOMIC
Economic profiling
Strategic industry analysis
Segment profiling (including tourism analysis)
Strategic context assessment, economic impact & benefit-cost analysis
Economic modelling
Employment projections
Activity centre analysis.
HOUSING
Demographic profiling
Dwelling and household analysis
Suburb lifecycle analysis
FORECASTING
Independent population forecast modelling and analysis
Strategic population insights
Demand analysis for services and infrastructure