This document discusses how data from new sources is fueling the datafication of society and how official statistics are adapting to incorporate these new data sources. It notes that a city of 1 million people will generate over 200 million gigabytes of data per day by 2020, with much of it coming from smart buildings, factories, vehicles and other IoT devices. It then outlines three ways official statistics are beginning to use new data sources: 1) basing statistics directly on new sources like social media and sensor data, 2) using new sources to improve existing statistics, and 3) answering more complex policy questions by leveraging large, diverse new data sources.

2. Datafication of Society
A city of 1M people will generate 200M GB of data per day by 2020
SMART BUILDINGS
55M
GB/day SMART FACTORIES
50M
GB/day
PUBLIC SAFETY
SYSTEMS
50M
GB/day
SMART VEHICLES
40M
GB/day
SMART AIRPLANES
4M
GB/day SOCIAL MEDIA
+ OTHER
2M
GB/day
EXPLOSION OF DATAEXPLOSION OF DATA
Source: CISCO, INTEL

3. Where is the Revolution for
Official Statistics?

4. 1. Official statistics based on new data sources

5. 2. Making better statistics based on new sources
Quetelet, 1796-1874

6. 3. Answering more complex relevant policy questions

7. Surveys
and
census
Admin
sources
Big
Data
Primary to Secondary to New Datasources
7
Targeted
data
collection,
pre-
determined
questions
and
indicators
Structured data,
collected by
government
not for NSI purposes
Not necessarily
collected for
NSI purposes,
data in high volume,
velocity, variety
2000BC 20th Century 21st century

8. 1. Statistics Based on New Data Sources
Official stats
based on BD
Experimental
stats
Future of
official stats
Consumer
price index
Traffic
Intensity
Dot maps
Social tension indicator
Sensor data
Pheno
menon

9. 2. Better Statistics Based on New Data Sources
− Daas et al. (2016) Profiling of Twitter Users: a big data selectivity study. Discussion paper 201606, Statistics Netherlands
− Puts et al. (2016) Using huge amounts of road sensor data for official statistics. European Conference on Quality in Official Statistics 2016, Madrid, Spain.
− Van den Brakel, J. et al. (2016) Social media as a data source for official statistics; the Dutch Consumer Confidence Index. Discussion paper 201601,
− Bart Buelens et al. (2015) Predictive inference for non probability samples: a simulation study . Statistics Netherlands Discussion Paper 201513.
− Martijn Tennekes and Edwin de Jonge (2015) Color Schemes for Tree-Structured Data, IEEE Transactions on Visualization and Computer Graphics V. 20, issue 12
− Ton de Waal et al. (2017) Multi-source statistics: Basic Situations and Methods. Discussion Paper 201712, Statistics Netherlands
Month|year
Socialmediasentiment

10. 3. Answering Complex Policy Questions
Can we predict migrant streams?
How do we match offer and
demand in the labour market?
How do we tackle the energy transition?
How do we manage mobility and
pollution?
How do we make our businesses
smarter?

11. Information Based Economy
Input
•Labour
•Capital
•Raw material
•Data
Economic
Production
function
Output
•Goods
•Services
•Information

16. 16