Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT Project
Session 51. What happens when massive computing power brings together an ever-growing cross-section of the world’s information in realtime, from news media to social media, books to academic literature, the world’s libraries to the web itself, machine translates all of that material as it arrives, and applies a vast array of algorithms to identify the events and emotions, actors and narratives and their myriad connections that define the planet to create a living silicon replica of global society? The GDELT Project (http://gdeltproject.org/), supported by Alphabet’s Jigsaw (formerly Google Ideas), is one of the largest open data initiatives in the world focusing on cataloging and modeling global human society, offering a first glimpse at what this emerging “big data” understanding of society looks like. Historically GDELT focused on the textual world, mass machine translating everything it monitored globally in 65 languages, using sentiment mining to assess 4,500 different emotions, geocoding mentions of location down to the level of a hilltop, identifying millions of topics and hundreds of event classes and making it all available as an open dataset available through BigQuery and GCS. Yet, imagery and video are increasingly the way global events and narratives are communicated and so we needed a way to go past the text to catalog world events through the eyes of all of that visual material – enter Cloud Vision. Over the past 17 months we have cataloged more than a quarter billion news images from every corner of the globe, using Cloud Vision to literally watch the world go by each day. From mapping what “land vehicles” look like in every corner of the world to realtime assessment of air and land pollution from the backgrounds of those images to live alerts of flash flooding, earthquakes and other natural disasters with ground truthed estimates of damage severity, to mapping how “happy” or “sad” the news imagery of the world is to differing violence norms to peering into the world’s visual narratives (what are the images picked by each country’s news outlets to discuss Donald Trump?) Together with early experiments using Cloud Speech and Cloud Natural Language, this talk will dive into how we are using Google Cloud and especially its AI APIs to catalog our world in realtime
Recommended
More Related Content
Similar to Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT Project
Similar to Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT Project (20)
More from DevFest DC
More from DevFest DC (12)
Recently uploaded
Recently uploaded (20)
Using Cloud Vision To Watch The World’s News Imagery In Realtime: The GDELT Project
- 5. Datasets • NEWS: Worldwide local news coverage in 100 languages (65 live translated) – online news preserved via Internet Archive • TELEVISION: Collaboration with the Internet Archive to process more than 150 television stations across the US, updating daily • ACADEMIC LITERATURE: 21 billion words covering 70 years (JSTOR/DTIC/CORE/CITESEER/IA) • BOOKS: Collaboration with Internet Archive and HathiTrust to process 3.5 million books 1800-2015 • HUMAN RIGHTS: Half century of worldwide human rights reports • IMAGERY: Quarter billion global news images processed via deep learning: objects/activities, OCR, logos, facial sentiment, geolocation
- 10. • Red: Actual Ukraine • Green: Avg Turkey (2/19/1999- 4/20/1999) and Lebanon (3/24/2007- 5/23/2007) • (r=0.49)
- 12. US Ebola News Coverage Number American television news broadcasts per week mentioning "ebola" • March 2014 WHO announcement • First American infections • Eric Duncan arrives in Dallas Average “tone” of English language media coverage of “ebola” • Steady ascent towards more and more positive coverage as “Western medicine miracles to the rescue” theme dominates coverage
- 14. Carbon Capture & Sequestration • English coverage of CCS 2010-2015 • 32,000 websites, 250,000 people, 140,000 organizations, 50,000 locations • Green cluster (center): senior American policymakers • Green cluster (lower): “cap and trade” politicians • Red cluster (bottom): American lawmakers on Congressional energy committee or sponsoring energy-related legislation • Purple cluster (top right): climate skeptics • Yellow (upper left): Australian politicians • Pink (upper center): British politicians • Periphery of all clusters: journalists and financial analysts who feature prominently in coverage or who write much of the coverage – Karolin Schaps (Reuters) and Alex Morales (Bloomberg News London) are attached to British political cluster; Tom Friedman is attached to American political cluster
- 18. Cloud Vision API
- 19. Geolocation
- 20. Human Faces
- 21. Violent Imagery
- 23. Search Fields • TEXT: EXIF/IPCC/Etc Metadata + OCR + Caption • TAG: 10,000 algorithmically assigned tags • WEBTAG: > 2.5 million tags over 1 month • WEBCOUNT: popularity
- 30. Popularity
- 32. Cloud Speech API • More than 80 languages supported • ISIS videos: transcripts of multiple Arabic speakers screaming over each other with gunfire and explosions • Global broadcast television and radio transcriptioning
- 33. Cloud Natural Language API • More than 30 million English articles processed to date. • Parse tree highly robust even to non-native speakers and translated English. • Heading towards using parse tree to allow true free-form fact extraction and Q&A, with goal to allow conversational Q&A via Google Home.
- 34. BigQuery Datasets • https://bigquery.cloud.google.com/dataset/gdelt-bq:gdeltv2 – gdelt-bq:gdeltv2.academicliteraturegkg – gdelt-bq:gdeltv2.cloudvision – gdelt-bq:gdeltv2.events – gdelt-bq:gdeltv2.gkg – gdelt-bq:gdeltv2.iatvgkg