SlideShare a Scribd company logo
1 of 54
Download to read offline
Revealing the Hidden Patterns of
News Photos:
Analysis of Millions of News Photos through
GDELT and Deep Learning-based Vision APIs
Haewoon Kwak Jisun An
Qatar Computing Research Institute
Hamad Bin Khalifa University
2
3
4
Roles of News Photos
● Influence people’s perception
● Enhance reader’s memory
● Deliver emotion otherwise hard to be
conveyed
5
6
7
Why was this photo not picked?
(source: https://www.donaldjtrump.com)
8
To Understand Messages of Photos
● We need to know
○ What are shown in the photos
○ How they are portrayed
9
Challenges in News Photo Analysis
● Text mining has been a useful tool for
analyzing news text
→ What is the appropriate tool for
examining news photos?
10
Conventional Tool for Photo Analysis
● Manual coding … hard to scale
11
Deep Learning for Image Recognition
12
Object Recognition
13
Emotion Detection
https://www.microsoft.com/cognitive-services/en-us/emotion-api 14
Deep learning enables us to study
news photos in large-scale
15
Goal of This Work
● To offer a general understanding of
news photos
○ What are shown in the photos?
○ How are people portrayed?
■ From the perspective of emotion
■ From the perspective of gender
● Case study: Portrayal of politicians
16
● We can crawl photos from news
websites and analyze them
● But, setting the deep learning
framework and training it take
time/money/...
Data Collection
17
18
GDELT Visual GKG (VGKG)
● Collects news articles around the world
● Extract photos from the articles
● Calls Google Cloud Vision API to analyze
photos
● VGKG is available since 1 Jan 2016 http:
//blog.gdeltproject.org/announcing-the-new-gdelt-visual-global-
knowledge-graph-vgkg/
19
Example of VGKG
20160101004500 http://www.bbc.co.uk/news/uk-35205943 http://ichef.bbci.co.
uk/news/1024/cpsprodpb/B89F/production/_87436274_87436273.jpg profession<FIELD>0.
95780987<FIELD>/m/063km<RECORD>person<FIELD>0.
85714287<FIELD>/m/01g317<RECORD>close up<FIELD>0.
82379222<FIELD>/m/02cqfm<RECORD>bishop<FIELD>0.
78259438<FIELD>/m/01b7b<RECORD>bishop<FIELD>0.
71334475<FIELD>/m/027k49j<RECORD>diocesan bishop<FIELD>0.
64282793<FIELD>/m/09sgrf<RECORD>auxiliary bishop<FIELD>0.
57118613<FIELD>/m/05mx3n<RECORD>clergy<FIELD>0.57113737<FIELD>/m/0db79 -2
<FIELD>-2<FIELD>-2<FIELD>-2 0.95443642<FIELD>3.199043<FIELD>12.419704<FIELD>-
7.179338<FIELD>0.621747<FIELD>433,215;575,215;575,357;433,357<FIELD>-2<FIELD>-
2<FIELD>-2<FIELD>2<FIELD>-2<FIELD>-2<FIELD>-2
20
Date, Document identifier (URL), Image URL, Labels (description,
confidence score, unique id), Geographic Landmarks, Logos, Safe
Search, Faces (Angle, Emotion, etc.), OCR
(Potential) Limitations of GDELT
● List of news sources is not explicitly
announced (also, growing) - coverage
bias might exist
● Our work of comparing GDELT with
another news dataset will be presented
in the poster session
Two Tales of the World: Comparison of Widely Used World News Datasets - GDELT and EventRegistry
Haewoon Kwak and Jisun An
ICWSM'16: The 10th International Conference on Web and Social Media (poster), 2016 21
Our Dataset - Full
● GKG and VGKG in January 2016
● Popularity measured by Alexa.com
22
Our Dataset - 7 Popular News Media
● Top 30 & > 1K records
23
Data Preprocessing
● Keep labels whose confidence score ≥ .8
http://i2.cdn.turner.com/cnnnext/dam/assets/160116174054-kerry-handshake-zarif-large-169.jpg
Person 0.84957772
Business 0.59667766
24
What Are Shown in the Photos?
Common Objects in News Photos
25
News Topics and Relevant Photos
● News photos should relate with topics
of news articles
→ Common objects might be different
across topics
● CNN has ‘section’ info. in its URL
http://edition.cnn.com/2016/04/07/travel/japan-
best-of-wakayama/index.html
http://edition.cnn.com/2016/05/05/politics/paul-
ryan-donald-trump-republican-resistance/index.html 26
Person is the Most Common Object
27
But, in Travel, Person is Uncommon
28
Region-related Sections
29
● Why does this matter?
Western Media and the Third World
● Golan reports that western mass media
strengthen the portrayal of the third
world by reporting war, poverty, famine,
conflicts, violence and conflicts and lead
to negative perception (Golan 2008).
30
How CNN deals with MENA region?
31
How Are People Portrayed?
From the Perspective of Emotion
32
Classification of Emotions
33https://articulation360.wordpress.com/2011/08/26/emotions-memory-game/
Google API Can Detect 4 Emotions
34https://articulation360.wordpress.com/2011/08/26/emotions-memory-game/
SURPRISE
SORROW ANGER
JOY
Neutral (75%) or Joy (24%)
● Among 11,127 faces (in 7 popular media),
2,740 faces (24.6%) have one of emotions
● Most of them (2,665 faces) express joy
35
Nonverbal & Verbal Communication
● Happy faces accelerate the cognitive
processing of positive words and slow
down that of negative words (Stenberg,
Wiking, and Dahl 1998)
36
We Use Microsoft Face API
● Measures smiling intensity (0.0~1.0)
37
0.998 0.0 (baby)
https://www.microsoft.com/cognitive-services/en-us/face-api
Smile Comes with Positive Text
● Positive correlation between smile
intensity and tone (sentiment) of the
text
⍴=0.225
38
How Are People Portrayed?
From the Perspective of Gender
39
Previous Studies on News Media
1. Men outnumber women
2. Men and women are associated with
particular roles
3. More women than men were depicted
as happy and calm.
→ We’ll verify this in large-scale
40
Again, We Use Microsoft Face API
41https://www.microsoft.com/cognitive-services/en-us/face-api
● Measures Gender and Age
Unequal Gender Representation
0.5
42
Stereotyping: Women in “Living”
0.5
43
Women Smile More Than Men
44
Younger Women, Older Men
45
Case Study
Portrayal of Politicians
46
Smiling Politicians
● Goodnow (2010) found that Obama
smiles more than Clinton in photos in
Time magazine
● Smile gives a positive, non-threatening
impression to viewers (Goffman 1979)
47
Bias of CNN Toward Sanders?
(Smiling faces / All faces)
* CNN even uses “Sorrow” faces for Sanders
48
Pro-Clinton Media Behave Similarly
49
Summary and Future Work
50
Key Findings
● What are shown in the news photos
○ People commonly appear (≥ 40.5% @top500)
● How they are portrayed
○ People are neutral (75%) or smiling (24%)
○ Gender representation is unequal
○ Gender role stereotyping is found
○ Women smile more and look younger than men
● Clinton smiles more than Sanders in some media
→We demonstrate the great potential of deep
learning
for computational journalism
51
Deeper Analysis on Text and Photos
● Headline and photos?
● Topic and photos?
● Keywords and photos?
52
Building PhotoBiasMeter.org
● Showing the preference of media
outlets toward candidates over time
● Challenges
○ Modeling complex dimension of
preference - “Smile” is only one
dimension
53
@haewoon
Full paper is available via
http://arxiv.org/abs/1603.04531
54

More Related Content

Similar to Revealing the Hidden Patterns of News Photos: Analysis of Millions of News Photos through GDELT and Deep Learning-based Vision APIs

Getting started in Data Science (April 2017, Los Angeles)
Getting started in Data Science (April 2017, Los Angeles)Getting started in Data Science (April 2017, Los Angeles)
Getting started in Data Science (April 2017, Los Angeles)Thinkful
 
Plouzennec - Innovative Analytics in the News Industry
Plouzennec - Innovative Analytics in the News IndustryPlouzennec - Innovative Analytics in the News Industry
Plouzennec - Innovative Analytics in the News IndustryLavaCon
 
Making Sense of Syria
Making Sense of Syria Making Sense of Syria
Making Sense of Syria Julia Plevin
 
Making Sense of Syria
Making Sense of SyriaMaking Sense of Syria
Making Sense of SyriaEliz Ayaydin
 
Using your Intranet to boost Employee Engagement
Using your Intranet to boost Employee EngagementUsing your Intranet to boost Employee Engagement
Using your Intranet to boost Employee EngagementStephan Schillerwein
 
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...
Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...Connie White
 
What Big Data Means for PR and Why It Matters to Us
What Big Data Means for PR and Why It Matters to UsWhat Big Data Means for PR and Why It Matters to Us
What Big Data Means for PR and Why It Matters to UsMSL
 
Getting Started in Data Science
Getting Started in Data ScienceGetting Started in Data Science
Getting Started in Data ScienceThinkful
 
People’s Insights Volume 1, Issue 52: Vicks Mobile Ad Campaign
People’s Insights Volume 1, Issue 52: Vicks Mobile Ad CampaignPeople’s Insights Volume 1, Issue 52: Vicks Mobile Ad Campaign
People’s Insights Volume 1, Issue 52: Vicks Mobile Ad CampaignMSL
 
Career in Data Science (July 2017, DTLA)
Career in Data Science (July 2017, DTLA)Career in Data Science (July 2017, DTLA)
Career in Data Science (July 2017, DTLA)Thinkful
 
Social Media in the Federal Community: Perceptions and Usage Among Government...
Social Media in the Federal Community: Perceptions and Usage Among Government...Social Media in the Federal Community: Perceptions and Usage Among Government...
Social Media in the Federal Community: Perceptions and Usage Among Government...Market Connections, Inc.
 
The H(app)athon Project Vision/Roadmap
The H(app)athon Project Vision/RoadmapThe H(app)athon Project Vision/Roadmap
The H(app)athon Project Vision/RoadmapJohn C. Havens
 
How the growth of R helps data-driven organizations succeed
How the growth of R helps data-driven organizations succeedHow the growth of R helps data-driven organizations succeed
How the growth of R helps data-driven organizations succeedRevolution Analytics
 
'Humans still needed' - research project reveals impact of artificial intelli...
'Humans still needed' - research project reveals impact of artificial intelli...'Humans still needed' - research project reveals impact of artificial intelli...
'Humans still needed' - research project reveals impact of artificial intelli...Chartered Institute of Public Relations
 
A Computational Analysis of Agenda Setting Theory
A Computational Analysis of Agenda Setting TheoryA Computational Analysis of Agenda Setting Theory
A Computational Analysis of Agenda Setting TheoryAlice Oh
 
Data Journalism 101: A Brief Survey
Data Journalism 101: A Brief SurveyData Journalism 101: A Brief Survey
Data Journalism 101: A Brief SurveyFlex.io
 
NewsWhip General Brochure
NewsWhip General BrochureNewsWhip General Brochure
NewsWhip General BrochureNewsWhip
 
Seth Grimes - Sentiment in Social Media
Seth Grimes - Sentiment in Social MediaSeth Grimes - Sentiment in Social Media
Seth Grimes - Sentiment in Social MediaInfluence People
 
Social Media & Web Mining for Public Services of Smart Cities - SSA Talk
Social Media & Web Mining for Public Services of Smart Cities - SSA TalkSocial Media & Web Mining for Public Services of Smart Cities - SSA Talk
Social Media & Web Mining for Public Services of Smart Cities - SSA TalkHemant Purohit
 
The future of measurement
The future of measurementThe future of measurement
The future of measurementColin Wheeler
 

Similar to Revealing the Hidden Patterns of News Photos: Analysis of Millions of News Photos through GDELT and Deep Learning-based Vision APIs (20)

Getting started in Data Science (April 2017, Los Angeles)
Getting started in Data Science (April 2017, Los Angeles)Getting started in Data Science (April 2017, Los Angeles)
Getting started in Data Science (April 2017, Los Angeles)
 
Plouzennec - Innovative Analytics in the News Industry
Plouzennec - Innovative Analytics in the News IndustryPlouzennec - Innovative Analytics in the News Industry
Plouzennec - Innovative Analytics in the News Industry
 
Making Sense of Syria
Making Sense of Syria Making Sense of Syria
Making Sense of Syria
 
Making Sense of Syria
Making Sense of SyriaMaking Sense of Syria
Making Sense of Syria
 
Using your Intranet to boost Employee Engagement
Using your Intranet to boost Employee EngagementUsing your Intranet to boost Employee Engagement
Using your Intranet to boost Employee Engagement
 
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...
Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...
 
What Big Data Means for PR and Why It Matters to Us
What Big Data Means for PR and Why It Matters to UsWhat Big Data Means for PR and Why It Matters to Us
What Big Data Means for PR and Why It Matters to Us
 
Getting Started in Data Science
Getting Started in Data ScienceGetting Started in Data Science
Getting Started in Data Science
 
People’s Insights Volume 1, Issue 52: Vicks Mobile Ad Campaign
People’s Insights Volume 1, Issue 52: Vicks Mobile Ad CampaignPeople’s Insights Volume 1, Issue 52: Vicks Mobile Ad Campaign
People’s Insights Volume 1, Issue 52: Vicks Mobile Ad Campaign
 
Career in Data Science (July 2017, DTLA)
Career in Data Science (July 2017, DTLA)Career in Data Science (July 2017, DTLA)
Career in Data Science (July 2017, DTLA)
 
Social Media in the Federal Community: Perceptions and Usage Among Government...
Social Media in the Federal Community: Perceptions and Usage Among Government...Social Media in the Federal Community: Perceptions and Usage Among Government...
Social Media in the Federal Community: Perceptions and Usage Among Government...
 
The H(app)athon Project Vision/Roadmap
The H(app)athon Project Vision/RoadmapThe H(app)athon Project Vision/Roadmap
The H(app)athon Project Vision/Roadmap
 
How the growth of R helps data-driven organizations succeed
How the growth of R helps data-driven organizations succeedHow the growth of R helps data-driven organizations succeed
How the growth of R helps data-driven organizations succeed
 
'Humans still needed' - research project reveals impact of artificial intelli...
'Humans still needed' - research project reveals impact of artificial intelli...'Humans still needed' - research project reveals impact of artificial intelli...
'Humans still needed' - research project reveals impact of artificial intelli...
 
A Computational Analysis of Agenda Setting Theory
A Computational Analysis of Agenda Setting TheoryA Computational Analysis of Agenda Setting Theory
A Computational Analysis of Agenda Setting Theory
 
Data Journalism 101: A Brief Survey
Data Journalism 101: A Brief SurveyData Journalism 101: A Brief Survey
Data Journalism 101: A Brief Survey
 
NewsWhip General Brochure
NewsWhip General BrochureNewsWhip General Brochure
NewsWhip General Brochure
 
Seth Grimes - Sentiment in Social Media
Seth Grimes - Sentiment in Social MediaSeth Grimes - Sentiment in Social Media
Seth Grimes - Sentiment in Social Media
 
Social Media & Web Mining for Public Services of Smart Cities - SSA Talk
Social Media & Web Mining for Public Services of Smart Cities - SSA TalkSocial Media & Web Mining for Public Services of Smart Cities - SSA Talk
Social Media & Web Mining for Public Services of Smart Cities - SSA Talk
 
The future of measurement
The future of measurementThe future of measurement
The future of measurement
 

More from Haewoon Kwak

Multiplex Media Attention and Disregard Network among 129 Countries
Multiplex Media Attention and Disregard Network among 129 CountriesMultiplex Media Attention and Disregard Network among 129 Countries
Multiplex Media Attention and Disregard Network among 129 CountriesHaewoon Kwak
 
Multi-level analysis on structures and dynamics of OSN
Multi-level analysis on structures and dynamics of OSNMulti-level analysis on structures and dynamics of OSN
Multi-level analysis on structures and dynamics of OSNHaewoon Kwak
 
Exploring cyberbullying and 
other toxic behavior in 
team competition online...
Exploring cyberbullying and 
other toxic behavior in 
team competition online...Exploring cyberbullying and 
other toxic behavior in 
team competition online...
Exploring cyberbullying and 
other toxic behavior in 
team competition online...Haewoon Kwak
 
Linguistic Analysis of Toxic Behavior in an Online Video Game
Linguistic Analysis of Toxic Behavior in an Online Video GameLinguistic Analysis of Toxic Behavior in an Online Video Game
Linguistic Analysis of Toxic Behavior in an Online Video GameHaewoon Kwak
 
Fragile Online Relationship: A First Look at Unfollow Dynamics in Twitter
Fragile Online Relationship: A First Look at Unfollow Dynamics in TwitterFragile Online Relationship: A First Look at Unfollow Dynamics in Twitter
Fragile Online Relationship: A First Look at Unfollow Dynamics in TwitterHaewoon Kwak
 
Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...
Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...
Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...Haewoon Kwak
 
Mining Communities in Networks: A Solution for Consistency and Its Evaluation
Mining Communities in Networks: A Solution for Consistency and Its EvaluationMining Communities in Networks: A Solution for Consistency and Its Evaluation
Mining Communities in Networks: A Solution for Consistency and Its EvaluationHaewoon Kwak
 

More from Haewoon Kwak (7)

Multiplex Media Attention and Disregard Network among 129 Countries
Multiplex Media Attention and Disregard Network among 129 CountriesMultiplex Media Attention and Disregard Network among 129 Countries
Multiplex Media Attention and Disregard Network among 129 Countries
 
Multi-level analysis on structures and dynamics of OSN
Multi-level analysis on structures and dynamics of OSNMulti-level analysis on structures and dynamics of OSN
Multi-level analysis on structures and dynamics of OSN
 
Exploring cyberbullying and 
other toxic behavior in 
team competition online...
Exploring cyberbullying and 
other toxic behavior in 
team competition online...Exploring cyberbullying and 
other toxic behavior in 
team competition online...
Exploring cyberbullying and 
other toxic behavior in 
team competition online...
 
Linguistic Analysis of Toxic Behavior in an Online Video Game
Linguistic Analysis of Toxic Behavior in an Online Video GameLinguistic Analysis of Toxic Behavior in an Online Video Game
Linguistic Analysis of Toxic Behavior in an Online Video Game
 
Fragile Online Relationship: A First Look at Unfollow Dynamics in Twitter
Fragile Online Relationship: A First Look at Unfollow Dynamics in TwitterFragile Online Relationship: A First Look at Unfollow Dynamics in Twitter
Fragile Online Relationship: A First Look at Unfollow Dynamics in Twitter
 
Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...
Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...
Comparison of Online Social Relations in terms of Volume vs. Interaction: A C...
 
Mining Communities in Networks: A Solution for Consistency and Its Evaluation
Mining Communities in Networks: A Solution for Consistency and Its EvaluationMining Communities in Networks: A Solution for Consistency and Its Evaluation
Mining Communities in Networks: A Solution for Consistency and Its Evaluation
 

Recently uploaded

Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesConf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesTimothy Spann
 
Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 217djon017
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxaleedritatuxx
 
Decoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectDecoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectBoston Institute of Analytics
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI  Streaming - AI Max PrincetonReal-Time AI  Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max PrincetonTimothy Spann
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our WorldEduminds Learning
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBoston Institute of Analytics
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsVICTOR MAESTRE RAMIREZ
 
What To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxWhat To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxSimranPal17
 
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...Amil Baba Dawood bangali
 
INTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingINTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingsocarem879
 
Cyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataCyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataTecnoIncentive
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Seán Kennedy
 
convolutional neural network and its applications.pdf
convolutional neural network and its applications.pdfconvolutional neural network and its applications.pdf
convolutional neural network and its applications.pdfSubhamKumar3239
 
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default  Presentation : Data Analysis Project PPTPredictive Analysis for Loan Default  Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPTBoston Institute of Analytics
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Thomas Poetter
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Seán Kennedy
 
Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Cathrine Wilhelmsen
 

Recently uploaded (20)

Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesConf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
 
Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
 
Decoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectDecoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis Project
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI  Streaming - AI Max PrincetonReal-Time AI  Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max Princeton
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our World
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business Professionals
 
What To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxWhat To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptx
 
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
NO1 Certified Black Magic Specialist Expert Amil baba in Lahore Islamabad Raw...
 
INTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingINTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processing
 
Cyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataCyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded data
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...
 
convolutional neural network and its applications.pdf
convolutional neural network and its applications.pdfconvolutional neural network and its applications.pdf
convolutional neural network and its applications.pdf
 
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default  Presentation : Data Analysis Project PPTPredictive Analysis for Loan Default  Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...Student Profile Sample report on improving academic performance by uniting gr...
Student Profile Sample report on improving academic performance by uniting gr...
 
Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)Data Factory in Microsoft Fabric (MsBIP #82)
Data Factory in Microsoft Fabric (MsBIP #82)
 

Revealing the Hidden Patterns of News Photos: Analysis of Millions of News Photos through GDELT and Deep Learning-based Vision APIs

  • 1. Revealing the Hidden Patterns of News Photos: Analysis of Millions of News Photos through GDELT and Deep Learning-based Vision APIs Haewoon Kwak Jisun An Qatar Computing Research Institute Hamad Bin Khalifa University
  • 2. 2
  • 3. 3
  • 4. 4
  • 5. Roles of News Photos ● Influence people’s perception ● Enhance reader’s memory ● Deliver emotion otherwise hard to be conveyed 5
  • 6. 6
  • 7. 7 Why was this photo not picked? (source: https://www.donaldjtrump.com)
  • 8. 8
  • 9. To Understand Messages of Photos ● We need to know ○ What are shown in the photos ○ How they are portrayed 9
  • 10. Challenges in News Photo Analysis ● Text mining has been a useful tool for analyzing news text → What is the appropriate tool for examining news photos? 10
  • 11. Conventional Tool for Photo Analysis ● Manual coding … hard to scale 11
  • 12. Deep Learning for Image Recognition 12
  • 15. Deep learning enables us to study news photos in large-scale 15
  • 16. Goal of This Work ● To offer a general understanding of news photos ○ What are shown in the photos? ○ How are people portrayed? ■ From the perspective of emotion ■ From the perspective of gender ● Case study: Portrayal of politicians 16
  • 17. ● We can crawl photos from news websites and analyze them ● But, setting the deep learning framework and training it take time/money/... Data Collection 17
  • 18. 18
  • 19. GDELT Visual GKG (VGKG) ● Collects news articles around the world ● Extract photos from the articles ● Calls Google Cloud Vision API to analyze photos ● VGKG is available since 1 Jan 2016 http: //blog.gdeltproject.org/announcing-the-new-gdelt-visual-global- knowledge-graph-vgkg/ 19
  • 20. Example of VGKG 20160101004500 http://www.bbc.co.uk/news/uk-35205943 http://ichef.bbci.co. uk/news/1024/cpsprodpb/B89F/production/_87436274_87436273.jpg profession<FIELD>0. 95780987<FIELD>/m/063km<RECORD>person<FIELD>0. 85714287<FIELD>/m/01g317<RECORD>close up<FIELD>0. 82379222<FIELD>/m/02cqfm<RECORD>bishop<FIELD>0. 78259438<FIELD>/m/01b7b<RECORD>bishop<FIELD>0. 71334475<FIELD>/m/027k49j<RECORD>diocesan bishop<FIELD>0. 64282793<FIELD>/m/09sgrf<RECORD>auxiliary bishop<FIELD>0. 57118613<FIELD>/m/05mx3n<RECORD>clergy<FIELD>0.57113737<FIELD>/m/0db79 -2 <FIELD>-2<FIELD>-2<FIELD>-2 0.95443642<FIELD>3.199043<FIELD>12.419704<FIELD>- 7.179338<FIELD>0.621747<FIELD>433,215;575,215;575,357;433,357<FIELD>-2<FIELD>- 2<FIELD>-2<FIELD>2<FIELD>-2<FIELD>-2<FIELD>-2 20 Date, Document identifier (URL), Image URL, Labels (description, confidence score, unique id), Geographic Landmarks, Logos, Safe Search, Faces (Angle, Emotion, etc.), OCR
  • 21. (Potential) Limitations of GDELT ● List of news sources is not explicitly announced (also, growing) - coverage bias might exist ● Our work of comparing GDELT with another news dataset will be presented in the poster session Two Tales of the World: Comparison of Widely Used World News Datasets - GDELT and EventRegistry Haewoon Kwak and Jisun An ICWSM'16: The 10th International Conference on Web and Social Media (poster), 2016 21
  • 22. Our Dataset - Full ● GKG and VGKG in January 2016 ● Popularity measured by Alexa.com 22
  • 23. Our Dataset - 7 Popular News Media ● Top 30 & > 1K records 23
  • 24. Data Preprocessing ● Keep labels whose confidence score ≥ .8 http://i2.cdn.turner.com/cnnnext/dam/assets/160116174054-kerry-handshake-zarif-large-169.jpg Person 0.84957772 Business 0.59667766 24
  • 25. What Are Shown in the Photos? Common Objects in News Photos 25
  • 26. News Topics and Relevant Photos ● News photos should relate with topics of news articles → Common objects might be different across topics ● CNN has ‘section’ info. in its URL http://edition.cnn.com/2016/04/07/travel/japan- best-of-wakayama/index.html http://edition.cnn.com/2016/05/05/politics/paul- ryan-donald-trump-republican-resistance/index.html 26
  • 27. Person is the Most Common Object 27
  • 28. But, in Travel, Person is Uncommon 28
  • 30. Western Media and the Third World ● Golan reports that western mass media strengthen the portrayal of the third world by reporting war, poverty, famine, conflicts, violence and conflicts and lead to negative perception (Golan 2008). 30
  • 31. How CNN deals with MENA region? 31
  • 32. How Are People Portrayed? From the Perspective of Emotion 32
  • 34. Google API Can Detect 4 Emotions 34https://articulation360.wordpress.com/2011/08/26/emotions-memory-game/ SURPRISE SORROW ANGER JOY
  • 35. Neutral (75%) or Joy (24%) ● Among 11,127 faces (in 7 popular media), 2,740 faces (24.6%) have one of emotions ● Most of them (2,665 faces) express joy 35
  • 36. Nonverbal & Verbal Communication ● Happy faces accelerate the cognitive processing of positive words and slow down that of negative words (Stenberg, Wiking, and Dahl 1998) 36
  • 37. We Use Microsoft Face API ● Measures smiling intensity (0.0~1.0) 37 0.998 0.0 (baby) https://www.microsoft.com/cognitive-services/en-us/face-api
  • 38. Smile Comes with Positive Text ● Positive correlation between smile intensity and tone (sentiment) of the text ⍴=0.225 38
  • 39. How Are People Portrayed? From the Perspective of Gender 39
  • 40. Previous Studies on News Media 1. Men outnumber women 2. Men and women are associated with particular roles 3. More women than men were depicted as happy and calm. → We’ll verify this in large-scale 40
  • 41. Again, We Use Microsoft Face API 41https://www.microsoft.com/cognitive-services/en-us/face-api ● Measures Gender and Age
  • 43. Stereotyping: Women in “Living” 0.5 43
  • 44. Women Smile More Than Men 44
  • 46. Case Study Portrayal of Politicians 46
  • 47. Smiling Politicians ● Goodnow (2010) found that Obama smiles more than Clinton in photos in Time magazine ● Smile gives a positive, non-threatening impression to viewers (Goffman 1979) 47
  • 48. Bias of CNN Toward Sanders? (Smiling faces / All faces) * CNN even uses “Sorrow” faces for Sanders 48
  • 49. Pro-Clinton Media Behave Similarly 49
  • 51. Key Findings ● What are shown in the news photos ○ People commonly appear (≥ 40.5% @top500) ● How they are portrayed ○ People are neutral (75%) or smiling (24%) ○ Gender representation is unequal ○ Gender role stereotyping is found ○ Women smile more and look younger than men ● Clinton smiles more than Sanders in some media →We demonstrate the great potential of deep learning for computational journalism 51
  • 52. Deeper Analysis on Text and Photos ● Headline and photos? ● Topic and photos? ● Keywords and photos? 52
  • 53. Building PhotoBiasMeter.org ● Showing the preference of media outlets toward candidates over time ● Challenges ○ Modeling complex dimension of preference - “Smile” is only one dimension 53
  • 54. @haewoon Full paper is available via http://arxiv.org/abs/1603.04531 54