SlideShare a Scribd company logo
1 of 8
Datasets
• NEWS: Worldwide local news coverage in 100 languages (65 live
translated) – online news preserved via Internet Archive
• TELEVISION: Collaboration with the Internet Archive to process
more than 100 television stations across the US, updating daily
• ACADEMIC LITERATURE: 21 billion words covering 70 years
(JSTOR/DTIC/CORE/CITESEER/IA)
• BOOKS: Collaboration with Internet Archive and HathiTrust to
process 3.5 million books 1800-2015
• HUMAN RIGHTS: Half century of worldwide human rights reports
• IMAGERY: Large fraction of global news imagery processed via deep
learning: objects/activities, OCR, logos, facial sentiment, geolocation
Preserving Online News
• World’s largest initiative to preserve online news – partnership with the Internet
Archive
• Only program to focus on worldwide local news in local languages
• 1.5-2% of news articles disappear within 2 weeks
• 5% disappear within a month
• Up to 14% gone after 2 months – half with 404 and half ranging from sustained
500’s to domain removal (popular in some areas of the world)
• Of GDELT-relevant coverage, 140,000 articles published today will be gone in 2
months
• 14 million GDELT monitored articles disappeared over a 6 month period
representing 2x the total output of the New York Times over the last half century
• Nepal 2015 Earthquake: preserving coverage of sudden-onset natural disasters
requires “always on” preservation – GDELT preserved 667,000 articles – 225,000
non-English, with top being Nepali
kalev.leetaru5@gmail.com
http://kalevleetaru.com
http://gdeltproject.org/
http://blog.gdeltproject.org/

More Related Content

Viewers also liked

High School Diploma
High School DiplomaHigh School Diploma
High School Diploma
Kady Eby
 
Certificate_34
Certificate_34Certificate_34
Certificate_34
Adeel Khan
 

Viewers also liked (9)

High School Diploma
High School DiplomaHigh School Diploma
High School Diploma
 
Certificate_34
Certificate_34Certificate_34
Certificate_34
 
Broussard, Meredith: Save News Apps!
Broussard, Meredith: Save News Apps!Broussard, Meredith: Save News Apps!
Broussard, Meredith: Save News Apps!
 
Samsung Techwin SCO-2040R Data Sheet
Samsung Techwin SCO-2040R Data SheetSamsung Techwin SCO-2040R Data Sheet
Samsung Techwin SCO-2040R Data Sheet
 
U2 f22-trinomio al cuadrado
U2 f22-trinomio al cuadradoU2 f22-trinomio al cuadrado
U2 f22-trinomio al cuadrado
 
Modul hardware komputer
Modul hardware komputerModul hardware komputer
Modul hardware komputer
 
0005886 2015 (1)
0005886   2015 (1)0005886   2015 (1)
0005886 2015 (1)
 
Experiencia tránsito 2014 2015 refinitivo
Experiencia tránsito 2014 2015 refinitivoExperiencia tránsito 2014 2015 refinitivo
Experiencia tránsito 2014 2015 refinitivo
 
Solução para clientes SAP
Solução para clientes SAPSolução para clientes SAP
Solução para clientes SAP
 

More from Reynolds Journalism Institute (RJI)

Archie Thornton: Content and the Fourth Industrial Revolution
Archie Thornton: Content and the Fourth Industrial RevolutionArchie Thornton: Content and the Fourth Industrial Revolution
Archie Thornton: Content and the Fourth Industrial Revolution
Reynolds Journalism Institute (RJI)
 
Uzo Iweala: Who speaks for Africa
Uzo Iweala: Who speaks for AfricaUzo Iweala: Who speaks for Africa
Uzo Iweala: Who speaks for Africa
Reynolds Journalism Institute (RJI)
 
Sarah Hill: The uncanny valley of VR distribution
Sarah Hill: The uncanny valley of VR distributionSarah Hill: The uncanny valley of VR distribution
Sarah Hill: The uncanny valley of VR distribution
Reynolds Journalism Institute (RJI)
 
Alejandro González: The story is your mothership
Alejandro González: The story is your mothershipAlejandro González: The story is your mothership
Alejandro González: The story is your mothership
Reynolds Journalism Institute (RJI)
 
Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...
Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...
Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...
Reynolds Journalism Institute (RJI)
 

More from Reynolds Journalism Institute (RJI) (20)

Art Holliday, BJ ’76, alumni speaker
Art Holliday, BJ ’76, alumni speakerArt Holliday, BJ ’76, alumni speaker
Art Holliday, BJ ’76, alumni speaker
 
Christopher Guess presents Push
Christopher Guess presents PushChristopher Guess presents Push
Christopher Guess presents Push
 
Hurley Symposium 2017 fake news survey
Hurley Symposium 2017 fake news surveyHurley Symposium 2017 fake news survey
Hurley Symposium 2017 fake news survey
 
Archie Thornton: Content and the Fourth Industrial Revolution
Archie Thornton: Content and the Fourth Industrial RevolutionArchie Thornton: Content and the Fourth Industrial Revolution
Archie Thornton: Content and the Fourth Industrial Revolution
 
Victor Hernandez: 50 things we learned at RJI-Distribution
Victor Hernandez: 50 things we learned at RJI-DistributionVictor Hernandez: 50 things we learned at RJI-Distribution
Victor Hernandez: 50 things we learned at RJI-Distribution
 
Kaizar Campwala: Distribution in service of brand loyalty
Kaizar Campwala: Distribution in service of brand loyaltyKaizar Campwala: Distribution in service of brand loyalty
Kaizar Campwala: Distribution in service of brand loyalty
 
Zahra rasool presentation
Zahra rasool presentationZahra rasool presentation
Zahra rasool presentation
 
Uzo Iweala: Who speaks for Africa
Uzo Iweala: Who speaks for AfricaUzo Iweala: Who speaks for Africa
Uzo Iweala: Who speaks for Africa
 
Adam Falk: Subscribe now or forever hold your audience?
Adam Falk: Subscribe now or forever hold your audience?Adam Falk: Subscribe now or forever hold your audience?
Adam Falk: Subscribe now or forever hold your audience?
 
Ben Norskov and Mohini Duta: Playing news
Ben Norskov and Mohini Duta: Playing newsBen Norskov and Mohini Duta: Playing news
Ben Norskov and Mohini Duta: Playing news
 
Katherine Bell: Beyond the funnel
Katherine Bell: Beyond the funnelKatherine Bell: Beyond the funnel
Katherine Bell: Beyond the funnel
 
John Rampton: Advanced content promotion — how the big boys are doing it
John Rampton: Advanced content promotion — how the big boys are doing itJohn Rampton: Advanced content promotion — how the big boys are doing it
John Rampton: Advanced content promotion — how the big boys are doing it
 
Sarah Hill: The uncanny valley of VR distribution
Sarah Hill: The uncanny valley of VR distributionSarah Hill: The uncanny valley of VR distribution
Sarah Hill: The uncanny valley of VR distribution
 
Alejandro González: The story is your mothership
Alejandro González: The story is your mothershipAlejandro González: The story is your mothership
Alejandro González: The story is your mothership
 
Kari Paul: Brand loyalty as a distribution strategy
Kari Paul: Brand loyalty as a distribution strategyKari Paul: Brand loyalty as a distribution strategy
Kari Paul: Brand loyalty as a distribution strategy
 
Welsh, Ben: The framework fix: how to build better archives by helping news n...
Welsh, Ben: The framework fix: how to build better archives by helping news n...Welsh, Ben: The framework fix: how to build better archives by helping news n...
Welsh, Ben: The framework fix: how to build better archives by helping news n...
 
Skinner, Katherine: Alignment and Reciprocity
Skinner, Katherine: Alignment and ReciprocitySkinner, Katherine: Alignment and Reciprocity
Skinner, Katherine: Alignment and Reciprocity
 
Leetaru, Kalev: The GDELT Project
Leetaru, Kalev: The GDELT ProjectLeetaru, Kalev: The GDELT Project
Leetaru, Kalev: The GDELT Project
 
Zwaard, Kate: Technology and Community: Why we need partners, collaborators a...
Zwaard, Kate: Technology and Community: Why we need partners, collaborators a...Zwaard, Kate: Technology and Community: Why we need partners, collaborators a...
Zwaard, Kate: Technology and Community: Why we need partners, collaborators a...
 
Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...
Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...
Younger, Jennifer: lighntning talk, Digital Preservation: Aggregated, Collabo...
 

Recently uploaded

₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...
₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...
₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...
Diya Sharma
 
Powerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost Lover
Powerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost LoverPowerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost Lover
Powerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost Lover
PsychicRuben LoveSpells
 
{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...
{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...
{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...
hyt3577
 

Recently uploaded (20)

06052024_First India Newspaper Jaipur.pdf
06052024_First India Newspaper Jaipur.pdf06052024_First India Newspaper Jaipur.pdf
06052024_First India Newspaper Jaipur.pdf
 
Nurturing Families, Empowering Lives: TDP's Vision for Family Welfare in Andh...
Nurturing Families, Empowering Lives: TDP's Vision for Family Welfare in Andh...Nurturing Families, Empowering Lives: TDP's Vision for Family Welfare in Andh...
Nurturing Families, Empowering Lives: TDP's Vision for Family Welfare in Andh...
 
₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...
₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...
₹5.5k {Cash Payment} Independent Greater Noida Call Girls In [Delhi INAYA] 🔝|...
 
Enjoy Night ≽ 8448380779 ≼ Call Girls In Gurgaon Sector 47 (Gurgaon)
Enjoy Night ≽ 8448380779 ≼ Call Girls In Gurgaon Sector 47 (Gurgaon)Enjoy Night ≽ 8448380779 ≼ Call Girls In Gurgaon Sector 47 (Gurgaon)
Enjoy Night ≽ 8448380779 ≼ Call Girls In Gurgaon Sector 47 (Gurgaon)
 
BDSM⚡Call Girls in Sector 143 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 143 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 143 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 143 Noida Escorts >༒8448380779 Escort Service
 
05052024_First India Newspaper Jaipur.pdf
05052024_First India Newspaper Jaipur.pdf05052024_First India Newspaper Jaipur.pdf
05052024_First India Newspaper Jaipur.pdf
 
Embed-4.pdf lkdiinlajeklhndklheduhuekjdh
Embed-4.pdf lkdiinlajeklhndklheduhuekjdhEmbed-4.pdf lkdiinlajeklhndklheduhuekjdh
Embed-4.pdf lkdiinlajeklhndklheduhuekjdh
 
Busty Desi⚡Call Girls in Sector 62 Noida Escorts >༒8448380779 Escort Service
Busty Desi⚡Call Girls in Sector 62 Noida Escorts >༒8448380779 Escort ServiceBusty Desi⚡Call Girls in Sector 62 Noida Escorts >༒8448380779 Escort Service
Busty Desi⚡Call Girls in Sector 62 Noida Escorts >༒8448380779 Escort Service
 
Group_5_US-China Trade War to understand the trade
Group_5_US-China Trade War to understand the tradeGroup_5_US-China Trade War to understand the trade
Group_5_US-China Trade War to understand the trade
 
Defensa de JOH insiste que testimonio de analista de la DEA es falso y solici...
Defensa de JOH insiste que testimonio de analista de la DEA es falso y solici...Defensa de JOH insiste que testimonio de analista de la DEA es falso y solici...
Defensa de JOH insiste que testimonio de analista de la DEA es falso y solici...
 
Nara Chandrababu Naidu's Visionary Policies For Andhra Pradesh's Development
Nara Chandrababu Naidu's Visionary Policies For Andhra Pradesh's DevelopmentNara Chandrababu Naidu's Visionary Policies For Andhra Pradesh's Development
Nara Chandrababu Naidu's Visionary Policies For Andhra Pradesh's Development
 
Powerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost Lover
Powerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost LoverPowerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost Lover
Powerful Love Spells in Phoenix, AZ (310) 882-6330 Bring Back Lost Lover
 
Kishan Reddy Report To People (2019-24).pdf
Kishan Reddy Report To People (2019-24).pdfKishan Reddy Report To People (2019-24).pdf
Kishan Reddy Report To People (2019-24).pdf
 
Enjoy Night ≽ 8448380779 ≼ Call Girls In Palam Vihar (Gurgaon)
Enjoy Night ≽ 8448380779 ≼ Call Girls In Palam Vihar (Gurgaon)Enjoy Night ≽ 8448380779 ≼ Call Girls In Palam Vihar (Gurgaon)
Enjoy Night ≽ 8448380779 ≼ Call Girls In Palam Vihar (Gurgaon)
 
{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...
{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...
{Qatar{^🚀^(+971558539980**}})Abortion Pills for Sale in Dubai. .abu dhabi, sh...
 
BDSM⚡Call Girls in Indirapuram Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Indirapuram Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Indirapuram Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Indirapuram Escorts >༒8448380779 Escort Service
 
BDSM⚡Call Girls in Sector 135 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 135 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 135 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 135 Noida Escorts >༒8448380779 Escort Service
 
China's soft power in 21st century .pptx
China's soft power in 21st century   .pptxChina's soft power in 21st century   .pptx
China's soft power in 21st century .pptx
 
Enjoy Night⚡Call Girls Rajokri Delhi >༒8448380779 Escort Service
Enjoy Night⚡Call Girls Rajokri Delhi >༒8448380779 Escort ServiceEnjoy Night⚡Call Girls Rajokri Delhi >༒8448380779 Escort Service
Enjoy Night⚡Call Girls Rajokri Delhi >༒8448380779 Escort Service
 
Gujarat-SEBCs.pdf pfpkoopapriorjfperjreie
Gujarat-SEBCs.pdf pfpkoopapriorjfperjreieGujarat-SEBCs.pdf pfpkoopapriorjfperjreie
Gujarat-SEBCs.pdf pfpkoopapriorjfperjreie
 

Leetaru, Kalev: lighntning talk, The GDELT Project

  • 1.
  • 2. Datasets • NEWS: Worldwide local news coverage in 100 languages (65 live translated) – online news preserved via Internet Archive • TELEVISION: Collaboration with the Internet Archive to process more than 100 television stations across the US, updating daily • ACADEMIC LITERATURE: 21 billion words covering 70 years (JSTOR/DTIC/CORE/CITESEER/IA) • BOOKS: Collaboration with Internet Archive and HathiTrust to process 3.5 million books 1800-2015 • HUMAN RIGHTS: Half century of worldwide human rights reports • IMAGERY: Large fraction of global news imagery processed via deep learning: objects/activities, OCR, logos, facial sentiment, geolocation
  • 3.
  • 4. Preserving Online News • World’s largest initiative to preserve online news – partnership with the Internet Archive • Only program to focus on worldwide local news in local languages • 1.5-2% of news articles disappear within 2 weeks • 5% disappear within a month • Up to 14% gone after 2 months – half with 404 and half ranging from sustained 500’s to domain removal (popular in some areas of the world) • Of GDELT-relevant coverage, 140,000 articles published today will be gone in 2 months • 14 million GDELT monitored articles disappeared over a 6 month period representing 2x the total output of the New York Times over the last half century • Nepal 2015 Earthquake: preserving coverage of sudden-onset natural disasters requires “always on” preservation – GDELT preserved 667,000 articles – 225,000 non-English, with top being Nepali
  • 5.
  • 6.
  • 7.

Editor's Notes

  1. Yet, timelines are just one of the ways we can visualize all of this data. Maps and network diagrams are especially powerful ways of understanding society. In clockwise order: 1) A map by BBVA of refugee inflows and outflows, 2) A map of global wildlife crime, 3) A network of the people most heavily associated with Russian economic sanctions, 4) A geographic network diagram of refugee flows, 5) A map of the countries discussing the Greek economic crisis, 6) A map of global anti-tank and aircraft missile activity
  2. Single map on the bottom combined 860 billion emotional scores, 1.5 billion location mentions, 89 million events and 1.4 million photographs from 200 million news articles in 65 languages from every country on earth covering 2015