Session 51. What happens when massive computing power brings together an ever-growing cross-section of the world’s information in realtime, from news media to social media, books to academic literature, the world’s libraries to the web itself, machine translates all of that material as it arrives, and applies a vast array of algorithms to identify the events and emotions, actors and narratives and their myriad connections that define the planet to create a living silicon replica of global society? The GDELT Project (http://gdeltproject.org/), supported by Alphabet’s Jigsaw (formerly Google Ideas), is one of the largest open data initiatives in the world focusing on cataloging and modeling global human society, offering a first glimpse at what this emerging “big data” understanding of society looks like. Historically GDELT focused on the textual world, mass machine translating everything it monitored globally in 65 languages, using sentiment mining to assess 4,500 different emotions, geocoding mentions of location down to the level of a hilltop, identifying millions of topics and hundreds of event classes and making it all available as an open dataset available through BigQuery and GCS. Yet, imagery and video are increasingly the way global events and narratives are communicated and so we needed a way to go past the text to catalog world events through the eyes of all of that visual material – enter Cloud Vision. Over the past 17 months we have cataloged more than a quarter billion news images from every corner of the globe, using Cloud Vision to literally watch the world go by each day. From mapping what “land vehicles” look like in every corner of the world to realtime assessment of air and land pollution from the backgrounds of those images to live alerts of flash flooding, earthquakes and other natural disasters with ground truthed estimates of damage severity, to mapping how “happy” or “sad” the news imagery of the world is to differing violence norms to peering into the world’s visual narratives (what are the images picked by each country’s news outlets to discuss Donald Trump?) Together with early experiments using Cloud Speech and Cloud Natural Language, this talk will dive into how we are using Google Cloud and especially its AI APIs to catalog our world in realtime

5. Datasets
• NEWS: Worldwide local news coverage in 100 languages (65 live
translated) – online news preserved via Internet Archive
• TELEVISION: Collaboration with the Internet Archive to process
more than 150 television stations across the US, updating daily
• ACADEMIC LITERATURE: 21 billion words covering 70 years
(JSTOR/DTIC/CORE/CITESEER/IA)
• BOOKS: Collaboration with Internet Archive and HathiTrust to
process 3.5 million books 1800-2015
• HUMAN RIGHTS: Half century of worldwide human rights reports
• IMAGERY: Quarter billion global news images processed via deep
learning: objects/activities, OCR, logos, facial sentiment, geolocation

7. Global Event Database Global Knowledge Graph

8. Greece
France
Germany
Italy
United Kingdom

9. Burundi - 12/13/2015Instability
Tone
Media Attention
Topics

10. • Red: Actual Ukraine
• Green: Avg Turkey
(2/19/1999-
4/20/1999) and
Lebanon
(3/24/2007-
5/23/2007)
• (r=0.49)

12. US Ebola News Coverage
Number American television news broadcasts per week mentioning "ebola"
• March 2014 WHO
announcement
• First American infections
• Eric Duncan arrives in Dallas
Average “tone” of English language media coverage of “ebola”
• Steady ascent towards more
and more positive coverage
as “Western medicine
miracles to the rescue”
theme dominates coverage

14. Carbon Capture & Sequestration
• English coverage of CCS 2010-2015
• 32,000 websites, 250,000 people, 140,000 organizations,
50,000 locations
• Green cluster (center): senior American policymakers
• Green cluster (lower): “cap and trade” politicians
• Red cluster (bottom): American lawmakers on Congressional
energy committee or sponsoring energy-related legislation
• Purple cluster (top right): climate skeptics
• Yellow (upper left): Australian politicians
• Pink (upper center): British politicians
• Periphery of all clusters: journalists and financial analysts who
feature prominently in coverage or who write much of the
coverage – Karolin Schaps (Reuters) and Alex Morales
(Bloomberg News London) are attached to British political
cluster; Tom Friedman is attached to American political
cluster

18. Cloud Vision API

19. Geolocation

20. Human Faces

21. Violent Imagery

23. Search Fields
• TEXT: EXIF/IPCC/Etc Metadata + OCR + Caption
• TAG: 10,000 algorithmically assigned tags
• WEBTAG: > 2.5 million tags over 1 month
• WEBCOUNT: popularity

30. Popularity

31. Cloud Vision API For Video

32. Cloud Speech API
• More than 80 languages supported
• ISIS videos: transcripts of multiple Arabic
speakers screaming over each other with
gunfire and explosions
• Global broadcast television and radio
transcriptioning

33. Cloud Natural Language API
• More than 30 million English articles
processed to date.
• Parse tree highly robust even to non-native
speakers and translated English.
• Heading towards using parse tree to allow
true free-form fact extraction and Q&A, with
goal to allow conversational Q&A via Google
Home.

34. BigQuery Datasets
• https://bigquery.cloud.google.com/dataset/gdelt-bq:gdeltv2
– gdelt-bq:gdeltv2.academicliteraturegkg
– gdelt-bq:gdeltv2.cloudvision
– gdelt-bq:gdeltv2.events
– gdelt-bq:gdeltv2.gkg
– gdelt-bq:gdeltv2.iatvgkg

36. kalev.leetaru5@gmail.com
http://kalevleetaru.com
http://gdeltproject.org/
36