Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Applications &
Implications of
Big Data for
Official Statistics
Emmanuel Letouzé
Director & co-Founder
Data-Pop Alliance
D...
1. The Emergence of Big Data &
The Statistical Tragedy
Framing and surfacing of the issue
2. Big Data and Official Statist...
1. The Emergence of Big Data
vs. The Statistical Tragedy
Framing and surfacing of the issue
Hal Varian’s nowcasting, GDP
and light emissions paper.…
Line shows returns for “Big Data” on Google Trends between 2007 a...
*Source:OxfamInternational,citingCreditSuisse,Jan.2014
“Data is the new oil”
Google Flu Trend: rise and fall
Hope or Hype?
2. Big Data & Official Statistics
It’s complicated. Or complex.
What is Big Data?
2010-12: the 3 Vs of big data
i. Exhaust
ii. Web
iii. Sensing
Crumbs
Capacities
Communities
What is Big Data?
Now: the 3 Vs of Big Data
Movement of an individual in
Rwanda over 4 years using CDRs (Source J. Blumenstock, 2010)
The new data ecosystem
1. Early warning
1. Real time awareness
1. Real-time feedback
Source: Letouzé, 2012
“What can it be used for?”
—Taxonomy o...
1. Descriptive
-e.g. maps, clouds..
1. Predictive:
-forecasting
-inference
1. Prescriptive
-causal inference Source: Letou...
National
Statistical
Institutes carry
out surveys
Telefonica team
used their data to
‘predict’ SELs from
Cell Phone Usage
...
Promoting a people-centered Big Data
Revolution
Counting people?
Sample bias correction
Then:
blending of hypothesis based vs. supervised
machine learning methods to model bias
Source: Letouzé, 2014, based on primary and secondary sources
What & how much do we know?
….and does it matter?
Poverty pr...
Are official statistics ever more than shadows in
the cave? If so what are they good for?
“Official statistics assumes a key role in ensuring
democracy and fostering social
progress…[should] ”provide society with...
(2) Official
Statistics as
systems—not
reducible to
producing (1)
(1) Official
Statistics as
data—entirely
defined as
prod...
Official
Statistics
(1) Ensure that
societies benefit from
“knowledge of itself”
(according to some
political and
technica...
(2) Ensure that societies
benefit from the
presence of a
deliberative public
space
It’s complex—and it’s political
(1) Ens...
Source: Letouzé, 2014, based on primary and secondary sources
(How) can data reduce poverty?
Poverty prevalence 1990-2030
...
…by that Gary King means: it’s about the analytics
Jonathan Glemmie
The Guardian, Oct 3, 2013
3. The case of the SDGs
A story of fish and fishermen
SDGs adopted by
the OWG
Big data examples What is monitored How is
monitored
Country(ies) Year Advantages of using
big dat...
SDG achievement & Big Data
Thank You
www.datapopalliance.org
eletouze@datapopalliance.org
@datapopalliance
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé
Upcoming SlideShare
Loading in …5
×

Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé

1,880 views

Published on

Presentation by Emmanuel Letouzé, Director & co-Founder Data-Pop Alliance. Deparment for International Development, London, February 26, 2015.

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

Applications & Implications of Big Data for Official Statistics - Emmanuel Letouzé

  1. 1. Applications & Implications of Big Data for Official Statistics Emmanuel Letouzé Director & co-Founder Data-Pop Alliance DfID, London, February 26, 2015
  2. 2. 1. The Emergence of Big Data & The Statistical Tragedy Framing and surfacing of the issue 2. Big Data and Official Statistics: Substitute, Complement, or “It’s complicated”? 3. The Case of the SDGs A story of fish and fishermen
  3. 3. 1. The Emergence of Big Data vs. The Statistical Tragedy Framing and surfacing of the issue
  4. 4. Hal Varian’s nowcasting, GDP and light emissions paper.… Line shows returns for “Big Data” on Google Trends between 2007 and 2014; 100=maximum value “We are at the beginning of what I call The Industrial Revolution of Data.” Joe Hellerstein , November 19, 2008 Context: the Big Data rush
  5. 5. *Source:OxfamInternational,citingCreditSuisse,Jan.2014 “Data is the new oil”
  6. 6. Google Flu Trend: rise and fall Hope or Hype?
  7. 7. 2. Big Data & Official Statistics It’s complicated. Or complex.
  8. 8. What is Big Data? 2010-12: the 3 Vs of big data
  9. 9. i. Exhaust ii. Web iii. Sensing Crumbs Capacities Communities What is Big Data? Now: the 3 Vs of Big Data
  10. 10. Movement of an individual in Rwanda over 4 years using CDRs (Source J. Blumenstock, 2010)
  11. 11. The new data ecosystem
  12. 12. 1. Early warning 1. Real time awareness 1. Real-time feedback Source: Letouzé, 2012 “What can it be used for?” —Taxonomy of applications (1)
  13. 13. 1. Descriptive -e.g. maps, clouds.. 1. Predictive: -forecasting -inference 1. Prescriptive -causal inference Source: Letouzé, Vinck and Meier, 2013 “What can it be used for?” —Taxonomy of applications (2)
  14. 14. National Statistical Institutes carry out surveys Telefonica team used their data to ‘predict’ SELs from Cell Phone Usage Predict the present (SELs for non- surveyed regions) and monitor the future (track changes over time) Survey from “a major city in Latin America” Source: “Prediction of Socio-Economic Levels Using Cell- Phone Records” (Telefonica research, 2011) ‘Predicting’ socioeconomic levels?
  15. 15. Promoting a people-centered Big Data Revolution
  16. 16. Counting people?
  17. 17. Sample bias correction Then: blending of hypothesis based vs. supervised machine learning methods to model bias
  18. 18. Source: Letouzé, 2014, based on primary and secondary sources What & how much do we know? ….and does it matter? Poverty prevalence 1990-2030 Fragile States vs. Non-Fragile States
  19. 19. Are official statistics ever more than shadows in the cave? If so what are they good for?
  20. 20. “Official statistics assumes a key role in ensuring democracy and fostering social progress…[should] ”provide society with knowledge of itself” Enrico Giovanni—former President of Istat Co-Chair of IEG on the Data Revolution “Knowledge is power; statistics is democracy” Former President of Statistics Finland
  21. 21. (2) Official Statistics as systems—not reducible to producing (1) (1) Official Statistics as data—entirely defined as product of (2)* * According to Fundamental Principles of Official Statistics What is/are official statistics?
  22. 22. Official Statistics (1) Ensure that societies benefit from “knowledge of itself” (according to some political and technical standards) (2) Ensure that societies benefit from the presence of a deliberative public space What is/are the purpose(s) of official statistics?
  23. 23. (2) Ensure that societies benefit from the presence of a deliberative public space It’s complex—and it’s political (1) Ensure that societies benefit from “knowledge of itself” (according to some political and technical standards) Official Statistics Big Data
  24. 24. Source: Letouzé, 2014, based on primary and secondary sources (How) can data reduce poverty? Poverty prevalence 1990-2030 Fragile States vs. Non-Fragile States
  25. 25. …by that Gary King means: it’s about the analytics Jonathan Glemmie The Guardian, Oct 3, 2013
  26. 26. 3. The case of the SDGs A story of fish and fishermen
  27. 27. SDGs adopted by the OWG Big data examples What is monitored How is monitored Country(ies) Year Advantages of using big data 1. Poverty eradication Satellite data to estimate povertyi Poverty Satellite images, night-lights Global map 2009 International comparable data, which can be updated more frequently Estimating poverty maps with cell- phone recordsii Poverty Cell phone records Cote d’Ivoire 2013-4 Internet-based data to estimate consumer price index and poverty ratesiii Price indexes Online prices at retailers websites Argentina 2013 Cheaper data available at higher frequencies Cell-phone records to predict socio- economic levelsiv Socio-economic levels Cell phone records City in Latin America 2011 Data available more regularly and cheaper than official data; informal economy better reflected 2. End hunger, achieve food security and improved nutrition, and promote sustainable agriculture Mining Indonesian Tweets to understand food price crisesv Food price crises Tweets Indonesia 2014 Uses indicators derived from mobile phone data as a proxy for food security indicatorsvi Food security Cell phone data and airtime credit purchases A country in Central Africa 2014 Use of remote-sensing data for drought assessment and monitoring Drought Remote sensing Afghanistan, India, Pakistanvii 2004 Chinaviii 2008 3. Health Internet-based data to identify influenza breakoutsix Influenza Google search queries US 2009 Real-time data; captures disease cases not officially recorded; data available earlier than official data Data from online searches to monitor influenza epidemicsx Influenza Online searches data China 2013 Detecting influenza epidemics using twitterxi Influenza Twitter Japan 2011 Monitoring influenza outbreaks using twitterxii Influenza Twitter US 2013 Systems to monitor the activity of influenza-like-illness with the aid of volunteers via the internetxiii,xiv Influenza Voluntary reporting through the internet Belgium, Italy, Netherlands, Portugal, United Kingdom, United States ongoi ng Cell-phone data to model malaria spreadxv Malaria Cell-phone data Kenya 2012 Using social and news media to Cholera Social and news Haiti 2012 SDG monitoring & Big Data
  28. 28. SDG achievement & Big Data
  29. 29. Thank You www.datapopalliance.org eletouze@datapopalliance.org @datapopalliance

×