Statbel and big data

http://statbel.fgov.be
Marc Debusschere
Coordinator Administrative & Big Data
Statbel and big data
Data Science Meetup Leuven, 27 June 2018

Overview
❑ Context
Statbel, big data and the third data revolution in statistics
❑ Big data and Statbel
Projects, accomplishments and problems
❑ European collaboration
❑ Big data and official statistics
Provisional conclusions and way forward
❑ Big data, Statbel and data science

Statbel =
❑ Statistics Belgium
❑ The institute formerly known as Nationaal Instituut voor de
Statistiek (NIS) / Institut national de Statistique (INS)
❑ Administratively part of the FPS (‘ministry’) Economy
❑ Member of the European Statistical System (ESS) =
Eurostat + 32 EU & EFTA national statistical institutes + associated statistics producers

Big data
❑ = data impossible to process in a ‘normal’ way
‘normal’ is relative …
❑ 3 v’s: volume, velocity, variety
❑ Result of societal and technological changes
Satellites, cameras and sensors, internet and e-mail, social media, mobile
phones and tablets, e-business, e-government, machine-to-machine
(internet of things, IoT)
❑ Result: data explosion, data deluge

Big data and statistics
❑ Big data = ‘digital footprint’
❑ Containing valuable information, statistically exploitable
(but also commercially …)
❑ Resulting in the third data revolution in statistics
After surveys (>1846) and administrative data (>2000), now: big data!
❑ Possible data sources – list far from exhaustive!
❖ Scanner data, electronical payments, credit card data
❖ Webscraping for job vacancies, enterprise characteristics
❖ Traffic cameras and detection loops
❖ Smart meters (electricity, gas, water)
❖ Last but not least: mobile phone data!

The future of statistics … 
… big data!
Instant statistics based on big data, complemented and/or
validated by administrative data and small and specific ad hoc
surveys.
Also known as: smart statistics …

Big data and Statbel: Big Data Team
❑ Start at the end of 2015
❑ Restricted group, operating informally and ad hoc
❑ Focus on mobile phone data, webscraping job vacancies
❑ Tasks:
❖Reflection on strategy, priorities
❖External contacts concerning big data, with data owners, potential users,
federal and regional authorities, academia, EU, international
organisations, …
❖Analysing big datasets and connecting them to statistical ones – see
below

Big data and Statbel: in production
❑ For consumer price index (CPI)
❖Scanner data supermarkets and retail chains
❖Webscraping prices (e.g. airplane tickets, webshops)

Big data and Statbel: not planned (at present)
❑ Social media, internet search results, text analytics, …
=‘high-hanging fruits’, access and interpretation very problematic!
❑ Smart meters
Political decision of regions (2012) not to deploy
=> no data (about to change in Flanders)
❑ Traffic cameras, traffic loops
Regional competence and data

Big data and Statbel: projects
❑ Mobile phone data
❖Project Statbel-Proximus-Eurostat
❖Border Region Data Collection (BRDC)
❖City data from LFS and Big Data
❑ Webscraping
❖Job vacancies
❑ Satellite data and aerial photography
❖Deep Solaris

Big data in production
❑ Scanner data for CPI
❖ Based on agreement with data owners, facilitated by political pressure
❖ Legal basis (HICP regulation) but cooperative model
❖ Being expanded gradually with new supermarket and retail chains
❖ Extremely smooth and cost-efficient after initial set-up
❑ Webscraping prices for CPI
❖ Collecting prices on webshops (e.g. airplane tickets)
❖ For efficiency but also out of necessity: e-commerce fast expanding!
❖ Legal issues possible

Big data almost in production
❑ Webscraping job vacancies: about to go in production …
❖ Methodological and practical issues
❖ Stand-alone results not sufficient, need to combine with existing Job
vacancy survey (JVS) and ‘administrative’ data from regional
employment agencies (VDAB, FOREM, Actiris)
❖ Linked to European project (ESSnet Big Data, see below): https://
webgate.ec.europa.eu/fpfis/mwikis/essnetbigdata/index.php/WP1_Webscraping_job_vacancies

Project Statbel, Proximus, Eurostat
❑ Start December 2015, first results April 2016
❑ Step by step approach:
❖ First: actual present population
❖ Basis for: resident population (via place of residence), workplace,
‘usual environment’, tourism, labour migration, migration, time use, …
❑ Innovative:
❖ First collaboration NSI/operator in EU => ‘real’ data
❖ No ‘call detail records’ but network signals: 10 x more frequent!
❖ Combining mobile phone data with statistical datasets
❑ Via geo-coupling of aggregates: no privacy issues!

Project Statbel, Proximus, Eurostat 
Results and next steps
❑ Results
❖ 10 publications - see De Meersman e.a., Debusschere e.a., Demunter e.a., Reis e.a., Seynaeve e.a.,
Wirthmann e.a., all at https://webgate.ec.europa.eu/fpfis/mwikis/essnetbigdata/index.php/WP5_Documentation
❑ Our objectives:
❖ Further exploration for statistical ánd commercial applications
❖ Concrete use cases with a view to statistical production lines
❑ Unfortunately …
❖ March 2017: data blocked for everyone
❖ In the meantime: new initiatives via MIT/Univ. Newcastle, Eurostat

Some results
Population density per km²:
Mobile phone data (left) versus Census 2011 (right)

Some results, continued
Weekday cells identified as ‘residential’, ‘commuting’ or ‘work’, with geographical
representation

Some results, continued
‘Residential’, ‘work’ and ‘commuting’-cells in the Brussels-Leuven area

Other projects
❑ Border Region Data Collection (BRDC)
❖Grant EC DG Regio, 1 year, July 2017 - July 2018
❖Cross-border living place-workplace mobility through
combining Labour force survey (LFS), administrative data and
mobile phone data
❖With CBS Netherlands (lead), Destatis Germany, Insee
France, GUS Poland, SURS Slovenia

Other projects, continued
❑ Deep Solaris
❖Grant Eurostat, 1 year, kickoff 19 Febr. 2018
❖Detecting solar pannels on the basis of satellite data
and aerial photography
❖Via machine learning
❖With CBS Netherlands (lead), Destatis Germany,
IT.NRW (Düsseldorf, DE) and BISS (Heerlen, NL)

Other projects, continued
❑ City data from LFS and Big Data
❖Grant EC DG Regio, 1 year, Jan. – Dec. 2018
❖Mapping metropolitan areas on the basis of Labour force
survey (LFS) and mobile phone data
❖With CBS Netherlands (lead), Destatis Germany, Insee
France, Statistics Austria

From exploration to exploitation 
Developing use cases
❑ Scanner data and webscraping for CPI
❑ Webscraping job vacancies
❑ Validation living place and workplace Census
❑ Matrix living place-workplace
❑ … (population, migration, tourism, mobility, transport, time use,
environment, agriculture, …)

Big data in the ESS (European Statistical System) 
Initiatives and projects

Big data and official statistics 
Provisional conclusions
❑ Size and complexity of datasets
❖ Less problematic than anticipated (because of pre-processing, at
least some structure)
❖ Focus consequently less on IT infrastructure and software

Provisional conclusions, continued
❑ Biggest obstacle: access!
❖ Data owned by private enterprises: profit-oriented
❖ Fail to see any advantage in collaborating, on the contrary
(mistakenly!)
❖ Imposing legal obligation seems unavoidable …
❑ Link to privacy issues: fear of reputational damage

Provisional conclusions, continued
❑ Additional challenge: methodology
❖ All ancient headaches are still there …
… with a lot of new ones added!

The next stage: smart statistics
❑ Monitoring systems which are:
❖ integrated
❖ flexible
❖ multi-source
❖ real-time and highly detailed
❑ Some examples:
❖ continuous tracking of air quality
❖ highly granular actual present population (time, location, characteristics)
❖ smart farming statistics

For discussion: 
big data, Statbel and data science
❑ Statbel owns numerous geocoded datasets (population,
employment, income, lodgings, …)
❑ and might gain access to big data sources …
❑ … but lacks data science, capability to analyse big data
❑ Two possible solutions:
❖ collaboration with academia, researchers
❖ hiring …

Questions?
Comments?

Statbel and big data

More Related Content

What's hot

Similar to Statbel and big data

More from Data Science Leuven

Recently uploaded

Statbel and big data