Presentation by UN Global Pulse at the Strata Big Data conference in New York, October 2012. http://strataconf.com/stratany2012/public/schedule/detail/24956
"Big Data for Development: Opportunities & Challenges” - UN Global PulseUN Global Pulse
Presentation from UN Global Pulse event to launch a new white paper "BIg Data for Development: Challenges and Opportunities" on July 10, 2012 event at UN Headquarters.
Details, and webcast, of the event can be found at: http://unglobalpulse.org/bd4dwebcast
Big Data, Social Networks & Human Behavior (Jukka-Pekka Onnela)UN Global Pulse
Presentation by Jukka-Pekka Onnela, Assistant Professor of Biostatistics at Harvard University's School of Public Health. Presented at roundtable on "BIg Data for Development" hosted by Global Pulse, an innovation initiative of the United Nations (www.unglobalpulse.org).
Big Data for Development: Opportunities and Challenges, Summary SlidedeckUN Global Pulse
Summary points from UN Global Pulse White Paper "Big Data for Development: Opportunities & Challenges." See: http://www.unglobalpulse.org/BigDataforDevelopment
In emerging markets, eight out of ten small businesses cannot access the loans they need to grow. USAID’s Development Credit Authority (DCA) uses risk-sharing agreements to mobilize local private capital to fill this financing gap. The goal of this collaboration between UN Global Pulse and USAID is to explore how big data could support the work of USAID’s Development Credit Authority. Kenya has become an established tech leader in Africa in recent years – generating greater volumes of digital data as a result. The goal of this study is to explore what new sources of digital data, and methods for analysis, could be helpful in answering the question: “What barriers to accessing loans do small businesses in Kenya face?” Accordingly, this report paints a picture of the big data landscape in Kenya, shows preliminary findings, and lays the groundwork for further investigation.
By analyzing CDRs from mobile phone networks, researchers were able to:
1. Map population migration patterns during disasters like the 2010 Haiti earthquake, providing more accurate estimates of displacement than other methods.
2. Study regional travel patterns in Kenya to map the spread of malaria and identify hotspots for prevention efforts. Analyzing CDRs also showed how "imported" malaria infections spread to other areas.
3. Measure the effectiveness of government mandates in reducing mobility during the 2009 H1N1 outbreak in Mexico, allowing a better response to the epidemic.
Global Pulse is playing a leading role in helping UN and other development partners adopt more agile processes powered by Big Data to meet the challenges of driving sustainable development in a Post-2015 world. Our initiative has been closely involved in shaping the discussion of a Post-2015 development “data revolution.”
Over the past year, we have focused our efforts on advocating for the responsible use of Big Data, building partnerships for access to real-time data sources, cutting edge data mining tools and data science expertise. At the country level, we continued to expand our network of Pulse Labs to strengthen national and regional capacity for using Big Data. We are pleased to have begun operating our first regional innovation hub in the vibrant East African technology scene with the opening of Pulse Lab Kampala in late 2013. In 2013, our portfolio of innovation projects involved more than 25 partner organizations including UNICEF, UN Development Programme (UNDP), World Food Programme (WFP) and World Health Organisation (WHO).
The Annual Report 2013 summarizes this activity and explains how the UN's data science labs operate and innovate.
Data privacy and security in ICT4D - Meeting Report UN Global Pulse
On May 8th, 2015 UN Global Pulse hosted a workshop on data privacy and security in technology-enabled development projects and programmes, as part of a series of events about the Nine Principles for Digital Development. This report summarizes the presentations and discussions from the workshop. http://unglobalpulse.org/blog/improving-privacy-and-data-security-ict4d-projects
The goal of this project was to determine the relationship between privacy risk and data utility when using aggregated mobile data for policy planning and crisis response. The project assessed these factors for transportation planning and pandemic control using simulated mobile call data. Experts in these domains evaluated the utility of various aggregation levels for their work. Re-identification risk was also measured for each data set. Results showed that while aggregation reduced risk, it also reduced utility, and this relationship varied by context and purpose. The project aims to help develop evidence-based standards for using mobile data proportionately based on balancing privacy risk and social benefits. Further research is needed applying this methodology to more scenarios and experts to better understand how data aggregation can enable use of mobile data for public
"Big Data for Development: Opportunities & Challenges” - UN Global PulseUN Global Pulse
Presentation from UN Global Pulse event to launch a new white paper "BIg Data for Development: Challenges and Opportunities" on July 10, 2012 event at UN Headquarters.
Details, and webcast, of the event can be found at: http://unglobalpulse.org/bd4dwebcast
Big Data, Social Networks & Human Behavior (Jukka-Pekka Onnela)UN Global Pulse
Presentation by Jukka-Pekka Onnela, Assistant Professor of Biostatistics at Harvard University's School of Public Health. Presented at roundtable on "BIg Data for Development" hosted by Global Pulse, an innovation initiative of the United Nations (www.unglobalpulse.org).
Big Data for Development: Opportunities and Challenges, Summary SlidedeckUN Global Pulse
Summary points from UN Global Pulse White Paper "Big Data for Development: Opportunities & Challenges." See: http://www.unglobalpulse.org/BigDataforDevelopment
In emerging markets, eight out of ten small businesses cannot access the loans they need to grow. USAID’s Development Credit Authority (DCA) uses risk-sharing agreements to mobilize local private capital to fill this financing gap. The goal of this collaboration between UN Global Pulse and USAID is to explore how big data could support the work of USAID’s Development Credit Authority. Kenya has become an established tech leader in Africa in recent years – generating greater volumes of digital data as a result. The goal of this study is to explore what new sources of digital data, and methods for analysis, could be helpful in answering the question: “What barriers to accessing loans do small businesses in Kenya face?” Accordingly, this report paints a picture of the big data landscape in Kenya, shows preliminary findings, and lays the groundwork for further investigation.
By analyzing CDRs from mobile phone networks, researchers were able to:
1. Map population migration patterns during disasters like the 2010 Haiti earthquake, providing more accurate estimates of displacement than other methods.
2. Study regional travel patterns in Kenya to map the spread of malaria and identify hotspots for prevention efforts. Analyzing CDRs also showed how "imported" malaria infections spread to other areas.
3. Measure the effectiveness of government mandates in reducing mobility during the 2009 H1N1 outbreak in Mexico, allowing a better response to the epidemic.
Global Pulse is playing a leading role in helping UN and other development partners adopt more agile processes powered by Big Data to meet the challenges of driving sustainable development in a Post-2015 world. Our initiative has been closely involved in shaping the discussion of a Post-2015 development “data revolution.”
Over the past year, we have focused our efforts on advocating for the responsible use of Big Data, building partnerships for access to real-time data sources, cutting edge data mining tools and data science expertise. At the country level, we continued to expand our network of Pulse Labs to strengthen national and regional capacity for using Big Data. We are pleased to have begun operating our first regional innovation hub in the vibrant East African technology scene with the opening of Pulse Lab Kampala in late 2013. In 2013, our portfolio of innovation projects involved more than 25 partner organizations including UNICEF, UN Development Programme (UNDP), World Food Programme (WFP) and World Health Organisation (WHO).
The Annual Report 2013 summarizes this activity and explains how the UN's data science labs operate and innovate.
Data privacy and security in ICT4D - Meeting Report UN Global Pulse
On May 8th, 2015 UN Global Pulse hosted a workshop on data privacy and security in technology-enabled development projects and programmes, as part of a series of events about the Nine Principles for Digital Development. This report summarizes the presentations and discussions from the workshop. http://unglobalpulse.org/blog/improving-privacy-and-data-security-ict4d-projects
The goal of this project was to determine the relationship between privacy risk and data utility when using aggregated mobile data for policy planning and crisis response. The project assessed these factors for transportation planning and pandemic control using simulated mobile call data. Experts in these domains evaluated the utility of various aggregation levels for their work. Re-identification risk was also measured for each data set. Results showed that while aggregation reduced risk, it also reduced utility, and this relationship varied by context and purpose. The project aims to help develop evidence-based standards for using mobile data proportionately based on balancing privacy risk and social benefits. Further research is needed applying this methodology to more scenarios and experts to better understand how data aggregation can enable use of mobile data for public
Analysing Social Media Conversations to Understand Public Perceptions of Sani...UN Global Pulse
The United Nations Millennium Campaign and the Water Supply and Sanitation Collaborative Council partnered to deliver a comprehensive advocacy and communication drive on sanitation. Their efforts were in support of the UN Deputy Secretary General’s Call to Action on Sanitation to increase the number of people with access to better sanitation. Global Pulse provided an analysis of social media in order to provide insight on the baseline of public engagement, and explore ways to monitor a new sanitation campaign. Using a custom keyword taxonomy, English language tweets from January 2011 to December 2013 were extracted, sorted into categories and analysed.
Cite as: UN Global Pulse, 'Analysing Social Media Conversations to Understand Public Perceptions of Sanitation', Global Pulse Project Series, no.5, 2014.
Using Financial Transaction Data To Measure Economic Resilience To Natural Di...UN Global Pulse
This project explored how financial transaction data can be analysed to better understand the economic resilience of people affected by natural disasters. The project used the Mexican state of Baja California Sur as a case study to assess the impact of Hurricane Odile on livelihoods and economic activities over a period of six months in 2014. The project measured daily Point of Sale transactions and ATM withdrawals at high geospatial resolution to gain insight into the way people prepare for and recover from disaster.
The study revealed that people spent 50% more than usual on items such as food and gasoline in preparation for the hurricane and that recovery time ranged from 2 to 40 days depending on characteristics such as gender or income. Findings suggest that insights from transaction data could be used to target emergency response and to estimate economic loss at local level in the wake of a disaster.
Experimenting with Big Data and AI to Support Peace and SecurityUN Global Pulse
UN Global Pulse is working with partners to explore how data from social media and radio shows can inform peace and security efforts in Africa. The methodology, case studies, and tools developed as part of these efforts are detailed in this report.
Big Data for Development and Humanitarian Action: Towards Responsible Governa...UN Global Pulse
This report presents a summary of the main topics discussed by the PAG in general, which were mainly summarized during the
2015 PAG meeting. It also describes some of the outcomes that came out of the PAG meeting of 23-24 October 2015.
Understanding Immunisation Awareness and Sentiment with Social Media - Projec...UN Global Pulse
This multi-country study aims to track and analyse online conversations related to immunisation on social media and mainstream media in India, Kenya, Nigeria and Pakistan. Findings from the study showed that in social media, Nigerian and Pakistani politicians are active and influential in the vaccination debate and the political dimension is often referred to when discussing the failure to eradicate diseases such as polio. However, in Kenya, religious and ideological aspects were more frequently discussed. Twitter activity is primarily driven by sharing of news stories in all countries whereas Facebook focuses on the 'distrust' and 'ideals' categorisation.
Cite as: UN Global Pulse, “Understanding Immunisation Awareness and Sentiment Through Social and Mainstream Media”, Global Pulse Project Series no. 19, 2015.
Using Mobile Phone Activity for Disaster Management During Floods - Project O...UN Global Pulse
Natural disasters affect hundreds of millions of people worldwide every year. Emergency response efforts depend on the availability of timely information, such the movement and communication behaviours of affected populations. As such, analysis of Call Detail Records (CDRs) collected by mobile phone operators reveal new, real-time insights about human behaviour during such critical events. In this study, mobile phone activity data was combined with remote sensing data to understand how people communicated during severe flooding in the Mexican state of Tabasco in 2009, in order to explore ways that mobile data can be used to improve disaster response. By comparing the mobile data with official population census data, the representativeness of the research was validated.
Cite as: "Using Mobile Phone Activity For Disaster Management During Floods", Global Pulse Project Series no. 2, 2014
The document discusses how emerging technologies are enabling human sensor networks that can passively collect location-based data from mobile populations, transforming people into sensors and providing organizations with real-time insights without traditional infrastructure; it also examines how personal data collection on mobile devices can facilitate a personal census that gives individuals insights into their habits while also allowing communities to monitor collective behaviors and respond to changes.
This interview summarizes the work of UN Global Pulse, an initiative that explores how big data and real-time analytics can help with sustainable development efforts. The director, Robert Kirkpatrick, discusses Global Pulse's mission to accelerate the use of data science to protect populations from shocks. They derive data from sources like social media, mobile phone metadata, and other digital traces to gain insights into issues like food security, public health, and economic trends. Kirkpatrick highlights challenges like building analytical capacity, maintaining responsible data partnerships, and addressing issues of data access and privacy at scale. He provides examples of projects in Indonesia that use social media to study food prices and vulnerabilities.
‘The State of Mobile Data for Social Good’ report is a collaboration between UN Global Pulse and the GSMA, the global mobile telecommunications industry association. The report, which identifies over 200 projects or studies leveraging mobile data for social good, aims to survey the landscape today, assess the current barriers to scale, and make recommendations for a way forward. It details some of the main challenges with using mobile data for social good and provides a set of actions that (i) can spur investment and use, (ii) ensure cohesion of efforts and of customer privacy and data protection frameworks and (iii) build technical capacity.
The document discusses the importance of data in driving development and outlines a "social contract" needed to realize data's full potential. It argues that value, trust, and equity are needed for data systems to enable use and reuse of data for different purposes. It also discusses the "right tools" needed for a successful implementation of the social contract, including infrastructure policies, laws and regulations, economic policies, and institutions. Overall, the document advocates for capitalizing on data to improve lives and develop underprivileged areas through better use of resources and returns for individuals.
Proceedings from International Conference on Data Innovation For Policy MakersUN Global Pulse
The conference discussed the need to make data more accessible through open data initiatives. Indonesia has launched an open data portal with 700 datasets from 24 agencies. Open data is valuable for both outsiders and policymakers within government. It was noted that while official statistics are important, they have limitations and new data sources can supplement them. A success story on forest monitoring called Global Forest Watch was highlighted, which provides open access to satellite data on deforestation to help manage forests. Collaboration between stakeholders to share data through initiatives like Indonesia's One Map portal were discussed as ways to create "data ecosystems" where evidence is more accessible for policymaking.
Crowdsourcing High- Frequency Food Price Data in Rural Indonesia - Project Ov...UN Global Pulse
A feasibility study conducted by Pulse Lab Jakarta, UN World Food Programme, UN Food and Agriculture Organisation, Premise used crowdsourcing to track commodity prices in near real-time in areas where the availability of other data sources was limited. High-resolution and high frequency food price trends were derived from reports generated by “citizen reporters”.
Cite as: UN Global Pulse, “Feasibility Study: Crowdsourcing High- Frequency Food Price Data in Rural Indonesia”, Global Pulse Project Series no. 17, 2015.
Analysing Large-Scale News Media Content for Early Warning of Conflict - Proj...UN Global Pulse
A feasibility study conducted by Global Pulse with UNDP explored how data mining of large-scale online news data could complement existing tools for conflict analysis and early warning. Analyzing news media archives from before and after Tunisia's 2011 revolution showed that tracking changes in tone and sentiment over time offered insights into emerging conflicts. Mining digital content was found to have considerable potential for conflict prevention if further explored.
Using Machine Learning to Analyse Radio Content in Uganda UN Global Pulse
The document describes a project that uses machine learning to analyze radio content in Uganda for development and humanitarian purposes. It details the development of a Radio Content Analysis Tool that can automatically analyze hundreds of hours of radio broadcasts daily and extract text to identify discussions on predefined topics. Several pilot studies were conducted using this tool to understand how radio data could provide insights on issues like refugee perceptions, disaster impacts, health services, and disease outbreaks. The document outlines the automated and human analysis processes used and discusses opportunities and challenges around using talk radio as a source of big data to inform development goals.
This primer - or "Big Data 101" specifically for the international development and humanitarian communities - explains the concepts behind using Big Data for social good in easy-to-understand language. Published by the United Nations' Global Pulse initiative, which is exploring how new, digital data sources and real-time analytics technologies can help policymakers understand human well-being and emerging vulnerabilities in real-time. www.unglobalpulse.org
UN Global Pulse's 2016 annual report summarizes the organization's work to promote the use of big data for development and humanitarian purposes. In 2016, Global Pulse intensified efforts to leverage new data sources to support achieving the UN Sustainable Development Goals. It collaborated with UN agencies on 20 innovation projects using data from sources like social media, mobile phones, and satellite imagery. Global Pulse also worked to build an enabling environment for data innovation, strengthen partnerships, and accelerate adoption of ethical data use policies. The organization continued delivering capacity building and acting as a hub for stakeholders through its Pulse Labs in New York, Indonesia, and Uganda.
Social Media and Forced Displacement: Big Data Analytics and Machine Learning...UN Global Pulse
This white paper summarizes a project using social media data and machine learning to understand perspectives related to the Europe refugee emergency. The project conducted ten mini-studies analyzing Twitter data to monitor interactions between refugees and service providers, and understand host community sentiment toward refugees. Initial results were inconclusive for monitoring refugee interactions but revealed that a small number of tweets connected refugees to terrorist attacks in local Twitter communities. The paper outlines the methodology used and lessons learned to inform humanitarian decision-making and response through social media analysis.
A Guide to Data Innovation for Development - From idea to proof-of-conceptUN Global Pulse
‘A Guide to Data Innovation for Development - From idea to proof-of-concept,’ provides step-by-step guidance for development practitioners to leverage new sources of data. It is a result of a collaboration of UNDP and UN Global Pulse with support from UN Volunteers.
The publication builds on successful case trials of six UNDP offices and on the expertise of data innovators from UNDP and UN Global Pulse who managed the design and development of those projects.
The guide is structured into three sections - (I) Explore the Problem & System, (II) Assemble the Team and (III) Create the Workplan. Each of the sections comprises of a series of tools for completing the steps needed to initiate and design a data innovation project, to engage the right partners and to make sure that adequate privacy and protection mechanisms are applied.
People are becoming human sensor networks as mobile devices equipped with sensors passively collect location and environmental data during daily activities. This large network of "low-quality sensors" distributed across a wide area can provide useful real-time information with minimal infrastructure. Examples include bikes equipped to monitor pollution levels and traffic, wearable devices that track environmental conditions, and asthma inhalers that contribute to air quality mapping. The passive collection of this ambient data maximizes the potential of personal technologies while generating insights that can improve products, services, and communities.
The document discusses how new technologies are enabling the analysis of real-time data generated from mobile phones and other digital sources. This data, known as "digital exhaust," contains signals that can reveal information about human behavior, economic activity, and emerging vulnerabilities. The United Nations has launched an initiative called Global Pulse that aims to harness real-time data analysis to better protect vulnerable populations and inform crisis response. Several examples are provided that demonstrate how real-time analysis of communication data from mobile phones has provided insights into human behavior and economic conditions in different parts of the world.
When the Global Pulse initiative was launched by the UN Secretary-General in late 2009, its mission to use real-time and other non- traditional data sources in development and humanitarian action was groundbreaking. 2014 was a landmark year for embracing the importance of data analysis in achieving sustainable development. Throughout the year, the "Post-2015 data revolution" agenda was taken-up in governments, public sector and civil society organisations.
Over the past year, Pulse Labs in New York, Jakarta and Indonesia have supported the growth of a thriving community of practice, redefined the data innovation landscape and demonstrated how real-time data can play a role in supporting decision-makers and shaping public service delivery. With 25 joint data innovation projects implemented over the year, in partnership with 25 UN & Govt innovation project partners, 30 private sector collaborators and academics from 26 institutions, Global Pulse is contrbuting to a body of evidence that demonstrates how big data analysis can complement traditional approaches to development planning and monitoring.
Global Pulse's Annual Report 2014 highlights big data innovation projects carried out over the past year, and new milestones in the evolution of a "big data for development" ecosystem.
Analysing Social Media Conversations to Understand Public Perceptions of Sani...UN Global Pulse
The United Nations Millennium Campaign and the Water Supply and Sanitation Collaborative Council partnered to deliver a comprehensive advocacy and communication drive on sanitation. Their efforts were in support of the UN Deputy Secretary General’s Call to Action on Sanitation to increase the number of people with access to better sanitation. Global Pulse provided an analysis of social media in order to provide insight on the baseline of public engagement, and explore ways to monitor a new sanitation campaign. Using a custom keyword taxonomy, English language tweets from January 2011 to December 2013 were extracted, sorted into categories and analysed.
Cite as: UN Global Pulse, 'Analysing Social Media Conversations to Understand Public Perceptions of Sanitation', Global Pulse Project Series, no.5, 2014.
Using Financial Transaction Data To Measure Economic Resilience To Natural Di...UN Global Pulse
This project explored how financial transaction data can be analysed to better understand the economic resilience of people affected by natural disasters. The project used the Mexican state of Baja California Sur as a case study to assess the impact of Hurricane Odile on livelihoods and economic activities over a period of six months in 2014. The project measured daily Point of Sale transactions and ATM withdrawals at high geospatial resolution to gain insight into the way people prepare for and recover from disaster.
The study revealed that people spent 50% more than usual on items such as food and gasoline in preparation for the hurricane and that recovery time ranged from 2 to 40 days depending on characteristics such as gender or income. Findings suggest that insights from transaction data could be used to target emergency response and to estimate economic loss at local level in the wake of a disaster.
Experimenting with Big Data and AI to Support Peace and SecurityUN Global Pulse
UN Global Pulse is working with partners to explore how data from social media and radio shows can inform peace and security efforts in Africa. The methodology, case studies, and tools developed as part of these efforts are detailed in this report.
Big Data for Development and Humanitarian Action: Towards Responsible Governa...UN Global Pulse
This report presents a summary of the main topics discussed by the PAG in general, which were mainly summarized during the
2015 PAG meeting. It also describes some of the outcomes that came out of the PAG meeting of 23-24 October 2015.
Understanding Immunisation Awareness and Sentiment with Social Media - Projec...UN Global Pulse
This multi-country study aims to track and analyse online conversations related to immunisation on social media and mainstream media in India, Kenya, Nigeria and Pakistan. Findings from the study showed that in social media, Nigerian and Pakistani politicians are active and influential in the vaccination debate and the political dimension is often referred to when discussing the failure to eradicate diseases such as polio. However, in Kenya, religious and ideological aspects were more frequently discussed. Twitter activity is primarily driven by sharing of news stories in all countries whereas Facebook focuses on the 'distrust' and 'ideals' categorisation.
Cite as: UN Global Pulse, “Understanding Immunisation Awareness and Sentiment Through Social and Mainstream Media”, Global Pulse Project Series no. 19, 2015.
Using Mobile Phone Activity for Disaster Management During Floods - Project O...UN Global Pulse
Natural disasters affect hundreds of millions of people worldwide every year. Emergency response efforts depend on the availability of timely information, such the movement and communication behaviours of affected populations. As such, analysis of Call Detail Records (CDRs) collected by mobile phone operators reveal new, real-time insights about human behaviour during such critical events. In this study, mobile phone activity data was combined with remote sensing data to understand how people communicated during severe flooding in the Mexican state of Tabasco in 2009, in order to explore ways that mobile data can be used to improve disaster response. By comparing the mobile data with official population census data, the representativeness of the research was validated.
Cite as: "Using Mobile Phone Activity For Disaster Management During Floods", Global Pulse Project Series no. 2, 2014
The document discusses how emerging technologies are enabling human sensor networks that can passively collect location-based data from mobile populations, transforming people into sensors and providing organizations with real-time insights without traditional infrastructure; it also examines how personal data collection on mobile devices can facilitate a personal census that gives individuals insights into their habits while also allowing communities to monitor collective behaviors and respond to changes.
This interview summarizes the work of UN Global Pulse, an initiative that explores how big data and real-time analytics can help with sustainable development efforts. The director, Robert Kirkpatrick, discusses Global Pulse's mission to accelerate the use of data science to protect populations from shocks. They derive data from sources like social media, mobile phone metadata, and other digital traces to gain insights into issues like food security, public health, and economic trends. Kirkpatrick highlights challenges like building analytical capacity, maintaining responsible data partnerships, and addressing issues of data access and privacy at scale. He provides examples of projects in Indonesia that use social media to study food prices and vulnerabilities.
‘The State of Mobile Data for Social Good’ report is a collaboration between UN Global Pulse and the GSMA, the global mobile telecommunications industry association. The report, which identifies over 200 projects or studies leveraging mobile data for social good, aims to survey the landscape today, assess the current barriers to scale, and make recommendations for a way forward. It details some of the main challenges with using mobile data for social good and provides a set of actions that (i) can spur investment and use, (ii) ensure cohesion of efforts and of customer privacy and data protection frameworks and (iii) build technical capacity.
The document discusses the importance of data in driving development and outlines a "social contract" needed to realize data's full potential. It argues that value, trust, and equity are needed for data systems to enable use and reuse of data for different purposes. It also discusses the "right tools" needed for a successful implementation of the social contract, including infrastructure policies, laws and regulations, economic policies, and institutions. Overall, the document advocates for capitalizing on data to improve lives and develop underprivileged areas through better use of resources and returns for individuals.
Proceedings from International Conference on Data Innovation For Policy MakersUN Global Pulse
The conference discussed the need to make data more accessible through open data initiatives. Indonesia has launched an open data portal with 700 datasets from 24 agencies. Open data is valuable for both outsiders and policymakers within government. It was noted that while official statistics are important, they have limitations and new data sources can supplement them. A success story on forest monitoring called Global Forest Watch was highlighted, which provides open access to satellite data on deforestation to help manage forests. Collaboration between stakeholders to share data through initiatives like Indonesia's One Map portal were discussed as ways to create "data ecosystems" where evidence is more accessible for policymaking.
Crowdsourcing High- Frequency Food Price Data in Rural Indonesia - Project Ov...UN Global Pulse
A feasibility study conducted by Pulse Lab Jakarta, UN World Food Programme, UN Food and Agriculture Organisation, Premise used crowdsourcing to track commodity prices in near real-time in areas where the availability of other data sources was limited. High-resolution and high frequency food price trends were derived from reports generated by “citizen reporters”.
Cite as: UN Global Pulse, “Feasibility Study: Crowdsourcing High- Frequency Food Price Data in Rural Indonesia”, Global Pulse Project Series no. 17, 2015.
Analysing Large-Scale News Media Content for Early Warning of Conflict - Proj...UN Global Pulse
A feasibility study conducted by Global Pulse with UNDP explored how data mining of large-scale online news data could complement existing tools for conflict analysis and early warning. Analyzing news media archives from before and after Tunisia's 2011 revolution showed that tracking changes in tone and sentiment over time offered insights into emerging conflicts. Mining digital content was found to have considerable potential for conflict prevention if further explored.
Using Machine Learning to Analyse Radio Content in Uganda UN Global Pulse
The document describes a project that uses machine learning to analyze radio content in Uganda for development and humanitarian purposes. It details the development of a Radio Content Analysis Tool that can automatically analyze hundreds of hours of radio broadcasts daily and extract text to identify discussions on predefined topics. Several pilot studies were conducted using this tool to understand how radio data could provide insights on issues like refugee perceptions, disaster impacts, health services, and disease outbreaks. The document outlines the automated and human analysis processes used and discusses opportunities and challenges around using talk radio as a source of big data to inform development goals.
This primer - or "Big Data 101" specifically for the international development and humanitarian communities - explains the concepts behind using Big Data for social good in easy-to-understand language. Published by the United Nations' Global Pulse initiative, which is exploring how new, digital data sources and real-time analytics technologies can help policymakers understand human well-being and emerging vulnerabilities in real-time. www.unglobalpulse.org
UN Global Pulse's 2016 annual report summarizes the organization's work to promote the use of big data for development and humanitarian purposes. In 2016, Global Pulse intensified efforts to leverage new data sources to support achieving the UN Sustainable Development Goals. It collaborated with UN agencies on 20 innovation projects using data from sources like social media, mobile phones, and satellite imagery. Global Pulse also worked to build an enabling environment for data innovation, strengthen partnerships, and accelerate adoption of ethical data use policies. The organization continued delivering capacity building and acting as a hub for stakeholders through its Pulse Labs in New York, Indonesia, and Uganda.
Social Media and Forced Displacement: Big Data Analytics and Machine Learning...UN Global Pulse
This white paper summarizes a project using social media data and machine learning to understand perspectives related to the Europe refugee emergency. The project conducted ten mini-studies analyzing Twitter data to monitor interactions between refugees and service providers, and understand host community sentiment toward refugees. Initial results were inconclusive for monitoring refugee interactions but revealed that a small number of tweets connected refugees to terrorist attacks in local Twitter communities. The paper outlines the methodology used and lessons learned to inform humanitarian decision-making and response through social media analysis.
A Guide to Data Innovation for Development - From idea to proof-of-conceptUN Global Pulse
‘A Guide to Data Innovation for Development - From idea to proof-of-concept,’ provides step-by-step guidance for development practitioners to leverage new sources of data. It is a result of a collaboration of UNDP and UN Global Pulse with support from UN Volunteers.
The publication builds on successful case trials of six UNDP offices and on the expertise of data innovators from UNDP and UN Global Pulse who managed the design and development of those projects.
The guide is structured into three sections - (I) Explore the Problem & System, (II) Assemble the Team and (III) Create the Workplan. Each of the sections comprises of a series of tools for completing the steps needed to initiate and design a data innovation project, to engage the right partners and to make sure that adequate privacy and protection mechanisms are applied.
People are becoming human sensor networks as mobile devices equipped with sensors passively collect location and environmental data during daily activities. This large network of "low-quality sensors" distributed across a wide area can provide useful real-time information with minimal infrastructure. Examples include bikes equipped to monitor pollution levels and traffic, wearable devices that track environmental conditions, and asthma inhalers that contribute to air quality mapping. The passive collection of this ambient data maximizes the potential of personal technologies while generating insights that can improve products, services, and communities.
The document discusses how new technologies are enabling the analysis of real-time data generated from mobile phones and other digital sources. This data, known as "digital exhaust," contains signals that can reveal information about human behavior, economic activity, and emerging vulnerabilities. The United Nations has launched an initiative called Global Pulse that aims to harness real-time data analysis to better protect vulnerable populations and inform crisis response. Several examples are provided that demonstrate how real-time analysis of communication data from mobile phones has provided insights into human behavior and economic conditions in different parts of the world.
When the Global Pulse initiative was launched by the UN Secretary-General in late 2009, its mission to use real-time and other non- traditional data sources in development and humanitarian action was groundbreaking. 2014 was a landmark year for embracing the importance of data analysis in achieving sustainable development. Throughout the year, the "Post-2015 data revolution" agenda was taken-up in governments, public sector and civil society organisations.
Over the past year, Pulse Labs in New York, Jakarta and Indonesia have supported the growth of a thriving community of practice, redefined the data innovation landscape and demonstrated how real-time data can play a role in supporting decision-makers and shaping public service delivery. With 25 joint data innovation projects implemented over the year, in partnership with 25 UN & Govt innovation project partners, 30 private sector collaborators and academics from 26 institutions, Global Pulse is contrbuting to a body of evidence that demonstrates how big data analysis can complement traditional approaches to development planning and monitoring.
Global Pulse's Annual Report 2014 highlights big data innovation projects carried out over the past year, and new milestones in the evolution of a "big data for development" ecosystem.
Presentation from the official launch event for Pulse Lab Jakarta, held in Indonesia on 1 October 2012. Presentation includes a background on "Big Data for Development," a showcase of Pulse Lab Jakarta's initial social media analysis research results, and roadmap for the Lab. http://www.unglobalpulse.org/PLJLaunch
This is developed to provide real-time analytic from collected customers' online activities data in e-commerce and finance.
It's delivering very adaptive use scenarios to mktg managers and campaign planners with clear & useful customer insights thru basic & advanced analysis.
GAUL: Microfinance Data What is it, how can I get it and what can it tell me?UN Global Pulse
The document discusses microfinance data and its sources. Microfinance serves over 140 million poor clients globally through business investments and basic needs loans. Microfinance data comes from internal financial institution systems and external reporting, though it can be difficult to obtain. Aggregate microfinance data helps understand industry trends and identify supply and demand gaps. Risk indicators and outside data provide context to interpret credit growth and savings trends. The Microfinance Information Exchange is a nonprofit that partners with industry leaders to collect, analyze and disseminate microfinance data.
Translator Gator is a new language game to support research initiatives in Indonesia. Players can earn phone credit by translating words between English and six common Indonesian languages. The database of keywords generated by the game will be used by researchers on topics ranging from computational social science to public policy.
This report summarizes the 2015 achievements of Pulse Lab Kampala and provides a glimpse into the long-term projects and agenda in the field of big data innovation for development and humanitarian action.
Integrating big data into the monitoring and evaluation of development progra...UN Global Pulse
This report provides guidelines for evaluators, evaluation and programme managers, policy makers
and funding agencies on how to take advantage of the rapidly emerging field of big data in the design
and implementation of systems for monitoring and evaluating development programmes.
The report is organized in two parts. Part I: Development evaluation in the age of big data reviews the data revolution and discusses the promise, and challenges this offers for strengthening development monitoring and evaluation. Part II: Guidelines for integrating big data into the monitoring and evaluation frameworks of development programmes focuses on what a big data inclusive M&E system would look like.
This document discusses tipping points in privacy and provides examples from Israel. It defines a privacy tipping point as an event that changes how individuals and organizations perceive privacy and leads to more responsibility. Two examples from Israel are discussed: a 2006 leak of the population registry and a 2012 hack of Israeli credit cards. The document argues that privacy stories can create tipping points if properly communicated through influencers and media to spur discussion, regulation and public action.
News and scandals concerning large scale privacy breaches have the potential to influence privacy awareness and may even cause behavioral change. They could be looked at as tipping points that might help make a substantial change the way in which privacy is perceived by individuals and organizations. Communicating privacy stories is very important for creating tipping points.
1) Social media uses internet technologies to transform one-way broadcast media into two-way dialogues, allowing people to be both content consumers and producers.
2) Creating social justice involves establishing a society based on principles of equality, solidarity, and human rights that recognizes everyone's dignity.
3) New media foundations like blogs, social networks, mobiles, and broadband have increased access and lowered costs, allowing two-way communication, citizen news generation and dissemination, and people as content producers.
Maximizing the Value of Real-Time: The Importance of AuthenticityJulie Epstein
An overview of how authenticity lends credibility to brands and creates consumer confidence in the framework of today’s real-time digital world. It includes social media usage, best practice case studies, and suggested strategies for implementation.
This presentation was prepared for the IPG Media Lab in February 2011.
Daniel Austin from GRIN Technologies gave a presentation on how big data and the internet of things are driving changes in the nature of money. He argued that within the next 5-10 years, digital transactions will dominate globally and the evolution of digital currencies will be driven by big data and metadata. Further in the future, money may take on more autonomous and self-aware properties as it converges with human and monetary evolution. By giving money sufficient autonomy and awareness, it could potentially preserve and transfer wealth across very long timescales.
1) The webinar discusses how associations can meet rising expectations for mobile engagement at events by developing mobile apps and websites.
2) A panelist explains how her association developed a basic mobile app to provide schedules, maps, and networking to better engage attendees, speakers, exhibitors, and sponsors.
3) Future expectations for mobile include seamless check-ins, interactive notes, surveys and analytics to improve events over time.
The document summarizes an event on wireless sensor technology business opportunities held at Letterkenny Institute of Technology. The agenda included presentations from industry speakers, an ideation exercise to brainstorm and develop business concepts, and teams presenting value propositions to receive feedback. The document provides details on the speakers, ideation process, example ideas and opportunities developed, and next steps for participants.
This document discusses emerging trends in media and technology. It notes that paradigms are shifting as broadcast media integrates with social media to create personal media. It also discusses how Moore's Law is driving increases in speed, connectivity, mobility and lowering costs. Finally, it suggests next generation thinkers value technology, expression, peer production and critical thinking over stored knowledge.
Predictive Analytics in the Cloud - The Art of the PossibleAnthony Quartararo
This is the presentation I did last week at the USGIF Technology Day event. The presentation was in the Ignite format and talked about how to model human behavior like hurricanes and weather patterns to help build and calibrate a predictive model of where events will happen and what other impacts they will have on neighboring geographies.
Digital Transformation Major tech trends through the customer lens and relati...Larry Smith
Digital Transformation
Major tech trends through the customer lens and relationships to the Insurance Industry
7 core technology trends: Mobility – Data – Social - Bots – Intelligence – Visualization – Things
This document summarizes Philip Kotler's presentation on the new role of marketing in the global digital economy. The presentation covered four main topics: 1) how globalization, technology, and other forces are disrupting businesses, 2) the need for entrepreneurship and innovation to adapt to these changes, 3) the importance of strong branding and reputation, and 4) a panel discussion on marketing in Russia. Kotler discussed how marketing must evolve to address new challenges and opportunities in today's fast-paced global environment.
Dionne Lew gave a presentation on community engagement at the Libmark Community Engagement Forum. She discussed how community engagement has evolved from a discipline focused on face-to-face interactions to incorporate new digital tools and practices. However, she stressed that the core principles of why engagement is done, who it aims to reach, and how to effectively plan strategies remain important. Lew provided examples of how platforms like Pinterest, Wikipedia, blogs and Flickr have enabled new forms of open collaboration and learning.
Dionne Lew gave a presentation on community engagement at the Libmark Community Engagement Forum. She discussed how community engagement has evolved from a discipline focused on face-to-face interactions to incorporate new digital tools and practices. However, she stressed that the core principles of why engagement is done, who it aims to reach, and how to effectively plan strategies remain important. Lew provided examples of how new models of engagement have emerged online through platforms like Wikipedia, Pinterest, and blogging.
Crisis Information Management: A Primer, presentation by Sanjana Hattotuwa, Special Advisor, ICT4Peace Foundation. Prepared for ISCRAM Summer School 2011 - http://www.iscram.org/live/summerschool2011.
Viva La Revolution: Why Universal Banking is under siege and what needs to be...Aman Narain
The world is experiencing incredible change both politically and technologically where industries are being re-imagined almost daily. Universal Banking is one business model that has been a very profitable business model. However with the changes in customer needs, regulation, technology and the emergence of new competitors this industry is most vulnerable to disruption. Disruption not just from TechGiants like Google and Apple but more so more small starts and an ecosystem of new financial services providers who instead of a head on challenge may be thin slicing the industry into oblivion. All is not lost however there are actions Universal Banks can and must take and this presentation shares some thoughts around these.
co-production @Justice_Vic - online collaboration and citizen engagementPatrick McCormick
The document discusses the Department of Justice Victoria's efforts to engage citizens online through collaboration and participation. It notes that citizens now expect more online services and the ability to participate in government. The department aims to take an agile, collaborative approach using tools like social media to be more transparent and responsive to citizens. The goal is to better serve citizens and help them help themselves through co-production of information and services.
The document describes a workshop called the H(app)thon at Social Media Week. It includes an introduction by John C. Havens and presentations from various speakers on topics related to well-being, measuring life satisfaction beyond GDP, big data and trusted social networks. Attendees are divided into groups to develop solutions to improve well-being in a hypothetical place called Eudaimonia, considering how to reach people without modern technology and ideas for implementing and scaling the solutions globally.
Crowdsourcing and smartphone technology 2013 03 18Luigi Ceccaroni
This document discusses how crowdsourcing and smartphone technology can be used for environmental monitoring. It notes that while citizens can help with monitoring, issues around quality, coverage and sustainability must be addressed. The convergence of cheap sensors and social networks allows citizens to both consume and generate environmental data. However, there are concerns about "bad smart" technologies that limit autonomy through social engineering rather than enhance decision making. The document advocates for "good smart" crowdsourcing applications that promote problem solving and participation in environmental science.
This document provides an overview of crisis information management and the role of social media. It discusses how social media has transformed from one-way communication to enable many-to-many dialogues. Citizens can now generate and share content, becoming producers rather than just consumers of information. New technologies have made it possible to report from closed countries and share first-hand accounts from events. However, challenges remain around verifying information and addressing biases. The document also examines how crisis mapping can both complement and collaborate with UN systems to strengthen situational awareness.
Overview of mobile capabilities and behavior change concepts, resources. Presented to the Wireless Innovation Council, May 31, 2012. Dr. Phil Hendrix, immr
Similar to UN Global Pulse: Big Data for a Better World (Strata Conf NYC) (20)
Step 2: Due Diligence Questionnaire for Prospective PartnersUN Global Pulse
UN Global Pulse has developed a two-part Due Diligence Tool for Working with Prospective Technology Partners. The questionnaire should be filled out by the prospective partner prior to any commitment to collaborate.
Step 1: Due Diligence Checklist for Prospective Partners UN Global Pulse
UN Global Pulse has developed a two-part Due Diligence Tool for Working with Prospective Technology Partners. The checklist should be completed by the UN organization and encourages research about the corporate and social nature of the prospective partner, including their data related practices, prior to any commitment to collaborate.
Using Data and New Technology for Peacemaking, Preventive Diplomacy, and Peac...UN Global Pulse
This guide offers an overview of e-analytics in the context of peacemaking and preventive diplomacy. It presents a summary of e-analytics tools as well as examples from the peace and security field. It includes a data project planning matrix that aims to help facilitate and motivate data-driven analysis. Part of the guide is a glossary on basic terminology related to new technologies.
In 2016-2017, Pulse Lab Kampala worked with various UN agencies and development partners in Uganda and the region to test, explore and develop 17 innovation projects. The Lab also furthered the development of tools and technologies that leverage data sources from radio content, social media, mobile phones and satellite imagery, and created technology toolkits. These toolkits can enhance decision-making by providing real-time situational awareness for project and policy implementation.
The 2018 Annual Report details exploratory research conducted by the Pulse Labs and presents solutions that were mainstreamed with partners.
It summarized the adoption of the first UN Principles for Personal Data Protection and Privacy, and showcases Global Pulse's contributions to develop standards and national strategies for the ethical and privacy protective use of big data and artificial intelligence.
Finally, the report highlights Global Pulse's engagement with the data innovation ecosystem through capacity building, collaborative research, and responsible data partnerships.
Risks, Harms and Benefits Assessment Tool (Updated as of Jan 2019)UN Global Pulse
The Data Innovation Risk Assessment Tool is an initial assessment of potential risks for data use that includes seven guiding checkpoints to understand: the "Data Type" involved in the data analytics process, the "Risks and Harms" of data use, the mode and legitimacy of "Data Access", the "Data Use", the adequacy of "Data Security", the adequate level of "Communication and Transparency" and the due diligence on engagement of "Third Parties". The Assessment contains guiding comments for each checkpoint and its questions are grounded in the key international data privacy and data protection principles and concepts such as Purpose Specification, Purpose Compatibility, Data Minimization, Consent Legitimacy, Lawfulness and Fairness of data access and use.
2015 was an eventful year for Pulse Lab Jakarta. The broader data innovation ecosystem within which the Lab operates has grown from a specialist network to include a broader range of public, social, and private sector actors who are interested in exploring insights from new data sources as well as learning how data innovation can complement existing datasets and operations. This report provides an overview of the work of Pulse Lab Jakarta in 2015, including the foundation blocks that will lead to an impactful 2016.
Embracing Innovation: How a Social Lab can Support the Innovation Agenda in S...UN Global Pulse
Pulse Lab Jakarta extended their support to UNDP Sri Lanka through a scoping mission to assess Sri Lanka's readiness to establish an Innovation Lab. This report presents the findings and outlines the suggested approaches for creating an innovation lab, and how to expand it in the years following its inception.
This toolkit provides the methodology for focusing the data-gathering power of existing communities, increasing their capacity to work together and building awareness of the potential of the data created by this work. It aims to help citizens identify and articulate their own problems using the supplementing data in their communities.
Navigating the Terrain: A Toolkit for Conceptualising Service Design ProjectsUN Global Pulse
Pulse Lab Jakarta participated in a service design initiative to develop a citizen-centric public transportation service in Makassar, Indonesia. Following the initiative, which was undertaken along with United Nations Development Programme (UNDP) and Bursa Pengetahuan Kawasan Timur Indonesia (BaKTI), we chronicled our learnings on taking an idea from a design sprint to a ready-to-test prototype. Contextualised to help inform stakeholders working with or within the public sector, this resulting toolkit is useful for developing and delivering similar services.
Banking on Fintech: Financial inclusion for micro enterprises in IndonesiaUN Global Pulse
The Banking on Fintech: Financial Inclusion for Micro Enterprises
in Indonesia research was conducted by Pulse Lab Jakarta,
with the support of the Department of Foreign Affairs and Trade
(DFAT) Australia and the Indonesia Fintech Association (AFTECH). It presents successful practices from early adopters and attempts to translate them into opportunities for other unbanked populations.
Pulse Lab Jakarta, in collaboration with the Government of Indonesia, developed ‘Haze Gazer,’ a crisis analysis tool that provides real-time situational information from various data sources to enhance disaster management efforts. The prototype uses advanced data analysis of sources including: satellite imagery, information on population density and distribution from government databases, citizen-generated data and real-time data from social media. The capability afforded by the tool can
enhance disaster risk management efforts to protect vulnerable populations as well as the environment.
Cite as: UN Global Pulse, “Haze Gazer: A crisis analysis tool,” Tool Series, no. 2, 2016.
Building Proxy Indicators of National Wellbeing with Postal Data - Project Ov...UN Global Pulse
This study investigated using data from international postal flows and other global networks as proxy indicators for national socioeconomic metrics. Electronic postal records from 2010-2014 involving 187 countries were analyzed. Connectivity measures from these networks were strongly correlated with indicators like GDP, HDI, and poverty rate. Combining these network data into a multiplex model further improved correlations and generated multidimensional connectivity indicators. This demonstrated new approaches for approximating standard socioeconomic benchmarks in a global, real-time manner using alternative data sources like postal and digital network flows.
Sex Disaggregation of Social Media Posts - Tool OverviewUN Global Pulse
Global Pulse collaborated with Data2X and the University of Leiden to develop and prototype a tool to infer the sex of users. The tool automates the process of looking up public information from Twitter profiles, in particular the user name and profile picture. Using open source software, the tool analyses user names from a built-in database of predefined names (from sources such as official statistics) that contain gender information.
Cite as: UN Global Pulse, 'Sex-Disaggregation of Social Media Posts,' Big Data Tools Series, no. 3, 2016
Using Big data Analytics for Improved Public Transport UN Global Pulse
Pulse Lab Jakarta collaborated with Jakarta Smart City on a project to enhance transport planning and operational decision-making through real-time data analytics. Using data from TransJakarta – the city’s rapid bus transit system – buses and passenger stations, the project mapped origin-destination trends and identified bottleneck locations, information which can be used to identify whether new routes are needed. The project also explored the possibility of using real-time data to determine passenger-waiting times in order to enhance the efficiency of the bus dispatching system.
Cite as: UN Global Pulse, ‘Using Big Data Analytics for Improved
Public Transport,’ Project Series, no. 25, 2017.
Pulse Lab Jakarta developed Translator Gator, a people-powered language game that creates dictionaries for recognising sustainable development-related conversations in Indonesia. The game builds taxonomies, i.e. sets of relevant keywords, by incentivising players to translate words from English into different Indonesian languages, including Bahasa Indonesia, Jawa, Sunda, Minang, Bugis and Melayu.
Cite as: UN Global Pulse, 'Translator Gator: Crowdsourcing
Translation of Development Keywords in Indonesia’, Tool
Series no. 4, 2017.
Big Data for Financial Inclusion, Examining the Customer Journey - Project Ov...UN Global Pulse
Pulse Lab Jakarta collaborated with the UNCDF Shaping Inclusive Finance Transformations (SHIFT) programme to undertake an
analysis of financial services usage, particularly among women in the ASEAN region. The project analysed customer savings and loan data from four Financial Service Providers (FSPs) in Cambodia to understand the factors that affect savings and loans mobilisation, as well as how usage of these products explains economic issues in Cambodia.
Cite as: UN Global Pulse, 'Big Data for Financial Inclusion, Examining The Customer Journey', Project Series, no. 27, 2017.
Understanding Perceptions of Migrants and Refugees with Social Media - Projec...UN Global Pulse
This project used data from Twitter to monitor protection issues and the safe access to asylum of migrants and refugees in Europe. In collaboration with the UN High Commissioner for Refugees (UNHCR), Global Pulse created taxonomies that were used to explore interactions among refugees and between them and service providers, as well as xenophobic sentiment of host communities towards the displaced populations. Specifically, the study focused on how refugees and migrants were perceived in reaction to a series of terrorist attacks that took place in Europe in 2016. The results were used to develop a standardized information product to improve UNHCR’s ability to monitor and analyse relevant social media feeds in near real-time.
Cite as: UN Global Pulse, “Understanding Movement and Perceptions of Migrants and Refugees with Social Media,” Project Series, no. 28, 2017.
Using vessel data to study rescue patterns in the mediterranean - Project Ove...UN Global Pulse
Despite policy and media attention and a significant increase in search and rescue efforts, the number of deaths of refugees and
migrants crossing the Mediterranean Sea hit record numbers in 2016. UN Global Pulse worked with the UN High Commissioner for Refugees (UNHCR) on a project that analyzed new big data sources to provide a better understanding of the context of search and rescue operations. The project used vessel location data (AIS) to determine the route of rescue ships from Italy and Malta to rescue zones and back, and combined it with broadcast warning data of distress calls from ships stranded at sea. The insights were used to construct narratives of individual rescues and gain a better understanding of collective rescue activities in the region.
Cite as: UN Global Pulse, “Using Big Data to Study Rescue Patterns in the Mediterranean” Project Series, no. 29, 2017.
Using vessel data to study rescue patterns in the mediterranean - Project Ove...
UN Global Pulse: Big Data for a Better World (Strata Conf NYC)
1. “BEYOND TARGETED ADS
BIG DATA FOR A BETTER WORLD”
Robert Kirkpatrick
Director, UN Global Pulse
O’Reilly Strata Conference | New York | October 2012
www.unglobalpulse.org @unglobalpulse
6. PRIVATE SECTOR
Monitor operations…in real time
Track market trends…in real time
Get customer feedback…in real time
GLOBAL DEVELOPMENT
Unemployment?
Food Security?
Public Health?
Education?
Migration?
Disaster Relief?
7. BIG DATA IN REAL TIME: 3 OPPORTUNITIES
1. Better early warning: Earlier detection of
anomalies, trends and events allows earlier
response.
2. Real-time awareness: A more accurate and up-to-
date picture of population needs supports more
effective planning and implementation
3. Real-time feedback: Understanding sooner where
needs are changing -- or are not being met --
allows for rapid, adaptive course correction
25
October
2012
|
www.unglobalpulse.org
8. BIG DATA IS A HUMAN RIGHTS ISSUE
• Never analyze personally identifiable information
• Never analyze confidential data
• Never seek to re-identify individuals
25
October
2012
|
www.unglobalpulse.org
10. How Mobile Phone Carriers See the World
Call Detail Records (CDRs)
• Caller ID (hashed phone #)
• Caller Tower Location
• Receiver ID (hashed phone #)
• Receiver Tower Location
• Call Start Time
• Call Duration
Airtime Expense Records
• Caller ID (hashed phone #)
• Caller Tower Location
• Amount of Purchase
• Time of Purchase
• Balance at Time of Purchase
25
October
2012
|
www.unglobalpulse.org
11. Modeling Behaviors in Mobile Data
Consumption variables
• Number of calls, call duration, SMS/MMS/voice
• Size, frequency, total number of airtime purchases
• Handset Type and Features
Social variables
• Degree of the social network
• Weight of the contacts, frequency of communication
Mobility variables
• Diameter of mobility and social network
• Radius of gyration
• Mobility Patterns
Source:
Telefonica
Research,
2011
25
October
2012
|
www.unglobalpulse.org
13. SIZE AND FREQUENCY PREDICT HOUSEHOLD INCOME
Higher household
income
Average size of purchase
Lower household
income
Average # of purchases / month
25
October
2012
|
www.unglobalpulse.org
14. CALLING PATTERNS AND ECONOMIC OPPORTUNITY
Lower Higher
socioeconomic level socioeconomic level
25
October
2012
|
www.unglobalpulse.org
15. MEN AND WOMEN USE THEIR PHONES DIFFERENTLY
Men: Women:
• Fewer calls • More calls
• Shorter calls • Longer calls
• Smaller social network • Larger social network
• More work-related calls • More personal calls
25
October
2012
|
www.unglobalpulse.org
16. Tracking population movement to predict cholera
Source: Linus Bengtsson et. al., PLoS Medicine, 2011
25
October
2012
|
www.unglobalpulse.org
17. A mobility index to evaluate H1N1 response in Mexico City
Source: Telefonica Research, 2011
See: http://www.unglobalpulse.org/publicpolicyandcellphonedata
25
October
2012
|
www.unglobalpulse.org
18. TWITTER PREDICTS SPREAD OF INFLUENZA
r2 = .958
“You Are What You Tweet: Analyzing Twitter for Public Health. M. J. Paul and M. Dredze, 2011.”
http://www.cs.jhu.edu/%7Empaul/files/2011.icwsm.twitter_health.pdf
25
October
2012
|
www.unglobalpulse.org
19. Rumi Chunara et. al., American Journal of Tropical Medicine and Hygiene, 2012 86:39-45
25
October
2012
|
www.unglobalpulse.org
20. GOOGLE SEARCHES FOR SYMPTOMS PREDICT DENGUE
25
October
2012
|
www.unglobalpulse.org
21. 2010 VS. 2011: INDONESIAN TWEETS ABOUT HIV
See: http://www.unglobalpulse.org/WorldAIDSDay-Part2
25
October
2012
|
www.unglobalpulse.org
22. GLOBAL PULSE RESEARCH 2011
PROOF OF CONCEPT
STUDIES
Online at: http://www.unglobalpulse.org/applyingbigdatatodevelopment
33. DIGITAL SERVICES AS HUMAN SENSOR NETWORKS:
Observing fluctuations in well-being…in real-time
COPING STRATEGIES DIGITAL “SMOKE SIGNALS”
• Buy cheaper foods • Depletion of airtime credit
• Work longer hours • Smaller mobile airtime
• Reduce energy use purchases
• Draw down savings • Failure to repay microloans via
• Sell assets mobile financial services
• Borrow from relatives • Changes in calling patterns
• Inbound money transfers
• Web searches for jobs, health
• Sales of livestock via mobile
trading network
• “Venting” on social media
25
October
2012
|
www.unglobalpulse.org
34. Photo: Ministry of Foreign Affairs, Iceland
25
October
2012
|
www.unglobalpulse.org
36. Integrating real-time data into an institution
• This data may be less
accurate that official
sources.
• But it’s faster.
• And it’s cheaper to
collect.
• How can we leverage USGS Twitter Earthquake Detector
the speed to change (TED)
the outcome?
25
October
2012
|
www.unglobalpulse.org
37. THE PROBLEM WITH TELESCOPES…
…AND MACROSCOPES
There’s a universe of data that we can’t see.
25
October
2012
|
www.unglobalpulse.org
40. A
global
real-time
public/private
data commons?
41. EXAMPLE R&D PROJECT:
Mobile Networks as Drought Sensors in the Sahel
Proposal
• Obtain 2011-2012 mobile CDRs and airtime purchases.
• Derive mobility, consumption, and social variables.
• Correlate variables with precipitation levels, survey
data.
• Identify signatures of drought impacts in 2011.
• Identify signatures of aid impact in 2012.
• Develop and evaluate prototype during next drought.
• Release open source “appliance” through GSMA.
25
October
2012
|
www.unglobalpulse.org
43. Joint Research | Rapid Prototyping | Capacity Building
25
October
2012
|
www.unglobalpulse.org
44. Pulse Lab Network
Pulse Lab Jakarta…………October 2012
Pulse Lab Kampala……….January 2013
Other locations…………...???
25
October
2012
|
www.unglobalpulse.org
45. PULSE LABS
R&D INNOVATION STRATEGY
1. Partner with governments to establish Pulse Labs
2. Build world-class teams of data scientists,
engineers, and policy experts
3. Partner with private sector for real-time data and
cutting edge technology
4. Work with UN agencies and academia to conduct
research around challenges in
5. Build open source prototypes of tools to automatic
real-time monitoring
6. Support broad adoption of useful innovations
7. Share everything we learn and build
25
October
2012
|
www.unglobalpulse.org
46. SO HOW DO I GET INVOLVED?
ARE YOU..
• A company with powerful data you think
could make the world a better place?
• A technology provider with screaming fast
computing or killer analytics?
• A whiz data scientist interested in hard
problems, positive impact, and global scale?
• A big data privacy expert who understands
that we cannot help unless we also protect?
25
October
2012
|
www.unglobalpulse.org
48. Research Tool 1
Crimson Hexagon: ForSight
25
October
2012
|
www.unglobalpulse.org
49. Food Prices: What a real crisis looks like
23
July
-‐
2
Aug
‘tempeh’
and
‘tofu’
hot
debate
14
-‐
21
Aug
Ramadhan
/
Idul
Fitri
25
October
2012
|
www.unglobalpulse.org
50. Comparing Crises
Tweets
about
food
18
Mar
-‐
7
Apr
23
July
-‐
2
Aug
Fuel
subsidy
cut
plan
and
‘tempeh’
and
‘tofu’
hot
debate
protests
against
it
during
soybean
shortage
51. Research Tool 2
SAS Social Media Analytics and SAS Text Miner
25
October
2012
|
www.unglobalpulse.org
52. Analytic Workflow
1)
Over
200,000
new
5)
Explore
results
and
correlate
3)
Capture
senKment
with
official
staKsKcs
to
official
Indonesian
language
and
mood
for
Bahasa
BPS
staKsKcs
:
Consumer
Price
documents
per
day
Index
(CPI)
for
12
common
foods
Global Pulse Sentiment, Topic &
Internet Relevance Mood & Geography Interactive
Conversation Filter Influence Categories
Dashboard
2)
Extract
conversaKons
4)
Detect
locaKon,
price,
about
rice,
cooking
oil,
availability,
specific
fuel,
employment,
etc.
govt.
programs,
etc.
anxious,
confident,
confused,
hosKle,
sad,
happy
(-”-) ;-) ((+_+)) :@ :( :)
53.
54. What’s the deal with Indonesians and eggs?
For every 5000 more tweets about eggs…
…we see a 2-3% decrease in food CPI?
55. The Signals Are Getting Stronger
è Big increase in volume of relevant conversations over 18 months
40000
35000
minyak
(oil)
ketahanan
pangan
(food
security)
30000
budidaya
(culKvaKon)
25000
telur
(eggs)
20000
15000
10000
5000
0
Indonesians are increasingly using social media to discuss basic needs
25
October
2012
|
www.unglobalpulse.org
56. So Are the Temporal Correlations
è Listening to social conversations provides insight on official data
2.5
2
1.5
1
0.5
0
-‐0.5
-‐1
-‐1.5
Social
Media
Food
Index
-‐2
BAPPENAS
Food
Price
Index
-‐2.5
25
October
2012
|
www.unglobalpulse.org
57. Next up for Pulse Lab Jakarta research:
1 year of anonymized Indonesian CDRs
• 4 largest carriers
• 170 million subscribers
• 200 billion call records
• 80 terabytes of data
25
October
2012
|
www.unglobalpulse.org
59. ROBERT KIRKPATRICK
Director
UN Global Pulse
www.unglobalpulse.org
kirkpatrick@un.org
+1 (650) 796-5709
Image credit: Aaron Koblin
24 hours of AT&T phone calls and Internet
traffic flowing through New York City