SlideShare a Scribd company logo
1 of 107
Download to read offline
Real-Time	Processing	of	Social	Media	Content	
for	Social	Good	
Muhammad	Imran	
Research	Scien,st	
Qatar	Compu,ng	Research	Ins,tute	
Hamad	Bin	Khalifa	University	
Doha,	Qatar	
	
April	20th,	2017	
Data	Science	Workshop
Outline	
•  P1:	Background	of	Humanitarian	CompuBng	(10%)	
–  Sudden-onset	emergencies,	Time-cri,cal	situa,ons	
–  Social	Good	factors	
–  Aid	and	informa,on	needs	
•  P2:	The	Role	of	Social	Media	for	Social	Good	(20%)	
–  Par,cular	focus	on	micro-blogging	plaKorms	
–  Availability	of	various	types	of	informa,on	and	opportuni,es	
•  P3:	The	Role	of	ArBficial	Intelligence	for	Social	Good	(70%)	
–  How	AI	is	useful	in	crisis	response	
–  Various	AI	techniques,	approaches,	and	tools	
–  Work	of	crisis	compu,ng	group	at	QCRI	
–  Ongoing	research		
–  Future	direc,ons
Aid	Needs,	InformaBon	Needs,	and	Gaps	
Info.	 Info.	 Info.	
Disaster	event	(earthquake,	flood)	 Urgent	needs	of	affected	people	
InformaBon	gathering	
Humanitarian	organizaBons	and	local	administraBon	
InformaBon	gathering,	
especially	in	real-Bme,	is	
the	most	challenging	part	
Relief	operaBons	
-  Food,	water	
-  Shelter	
-  Medical	assistance	
-  DonaBons	
-  Service	and	uBliBes
Aid	Needs,	InformaBon	Needs,	and	Gaps	
Info.	 Info.	 Info.	
Disaster	event	(earthquake,	flood)	 Urgent	needs	of	affected	people	
InformaBon	gathering	
Humanitarian	organizaBons	and	local	administraBon	
InformaBon	gathering,	
especially	in	real-Bme,	is	
the	most	challenging	part	
Relief	operaBons	
-  Food,	water	
-  Shelter	
-  Medical	assistance	
-  DonaBons	
-  Service	and	uBliBes	
--Informa,on	Bestows	Power--	
Will	access	to	informaBon	solve	the	problem?
Decision-Making	and	Response	
Department	of	Community	Safety,	Queensland	Govt.	&	UNOCHA,	2011	
-  Delayed	decision-making	
-  Delayed	crisis	response	
-  High	community	harm	
-  Early	decision-making	
-  Rapid	crisis	response	
-  Low	community	harm	
Target
Decision-Making	and	Response	
Department	of	Community	Safety,	Queensland	Govt.	&	UNOCHA,	2011	
-  Delayed	decision-making	
-  Delayed	crisis	response	
-  High	community	harm	
-  Early	decision-making	
-  Rapid	crisis	response	
-  Low	community	harm	
Target	
--Need	Early	Informa,on--	
How	early	do	we	need	it?
The	Value	of	Timely	InformaBon	
During	Disasters	
Based	on	FEMA	large-scale	survey	among	emergency	management	professionals	across	the	US.	
InformaBon	value	
When	informaBon	is	too	late
The	Value	of	Timely	InformaBon	
During	Disasters	
Based	on	FEMA	large-scale	survey	among	emergency	management	professionals	across	the	US.	
InformaBon	value	
When	informaBon	is	too	late
InformaBon	Types	and	Needs	
•  Reports	of	Injured	or	dead	people	
•  Infrastructure	damage	(e.g.,	buildings,	bridges,	Roads)	
•  Urgent	needs	of	affected	people	(e.g.,	food,	water,	shelter)	
•  Dona,on	offers	and	requests	(e.g.,	money,	volunteers)	
•  Medical	Emergencies	
•  Disease	symptoms	reports	
•  Disease	treatment	reports	and	ques,ons	
•  …
Part	2:	
The	Role	of	Social	Media
CommunicaBons	Before	and	A_er		
ICT	and	Social	Media	
Gerald	Baron
InformaBon	Availability	in	the	Age	of	
ICT	and	Social	Media	
Based	on	FEMA	large-scale	survey	among	emergency	management	professionals	across	the	US.	
1990s	2000s	2010s	
InformaBon	value	
When	informaBon	is	too	late
Sandy	Hurricane	Twiaer	Data	Analysis	
@NYGovCuomo	orders	closing	of	NYC	bridges.	Only	Staten	Island	
bridges	unaffected	at	this	,me.	Bridges	must	close	by	7pm.	#Sandy	
#NYC.	
rt	@911buff:	public	help	needed:	2	boys	2	&	4	missing	nearly	24	hours	
aber	they	got	separated	from	their	mom	when	car	submerged	in	si.	
#sandy	#911buff		
freaking	out.	home	alone.	will	just	watch	tv	#Sandy	#NYC.	
400	Volunteers	are	needed	for	areas	that	#Sandy	destroyed.
@NYGovCuomo	orders	closing	of	NYC	bridges.	Only	Staten	Island	
bridges	unaffected	at	this	,me.	Bridges	must	close	by	7pm.	#Sandy	
#NYC.	
rt	@911buff:	public	help	needed:	2	boys	2	&	4	missing	nearly	24	hours	
aber	they	got	separated	from	their	mom	when	car	submerged	in	si.	
#sandy	#911buff		
freaking	out.	home	alone.	will	just	watch	tv	#Sandy	#NYC.	
400	Volunteers	are	needed	for	areas	that	#Sandy	destroyed.		
Personal	
Informa,ve	
Sandy	Hurricane	Twiaer	Data	Analysis
@NYGovCuomo	orders	closing	of	NYC	bridges.	Only	Staten	Island	
bridges	unaffected	at	this	,me.	Bridges	must	close	by	7pm.	#Sandy	
#NYC.	
rt	@911buff:	public	help	needed:	2	boys	2	&	4	missing	nearly	24	hours	
aber	they	got	separated	from	their	mom	when	car	submerged	in	si.	
#sandy	#911buff		
freaking	out.	home	alone.	will	just	watch	tv	#Sandy	#NYC.	
400	Volunteers	are	needed	for	areas	that	#Sandy	destroyed.		
Personal	
Informa,ve	
Cau,on	and	Advice	
Missing	people	report	
Dona,on	request	
Sandy	Hurricane	Twiaer	Data	Analysis
MERS	Outbreak:	Twiaer	Data	Analysis	
Middle	East	Respiratory	Syndrome	(MERS)	
Twicer	data	analysis	from:	2014-04-27	to	2014-07-14	
QualitaBve	analysis	categories:	
		
Reports	of	symptoms	 Affected	people	reports	 Death	reports	
Disease	transmission	reports	Preven,on	ques,ons	 Treatment	ques,ons		
Reports	of	signs	or	symptoms	
such	as	fever,	cough	or	
ques,ons	
Reports	of	affected	people	due	
to	the	MERS	disease	
Reports	of	deaths	due	to	the	
MERS	disease	
Ques,ons	or	sugges,ons	
related	to	the	preven,on	of	
disease	
Reports	or	ques,ons	related	to	
the	transmission	of	the	disease	
Ques,ons	or	sugges,ons	
regarding	the	treatment	of	the	
disease
Social	Media	During	MERS	Outbreak		
	
	
RT	@abecel:	Two	workers	at	FL	hospital	exposed	to	a	pa,ent	with	Middle	East	
Respiratory	Syndrome	are	showing	flu-like	symptoms	
	
	
Coronavirus	symptoms	include:	fever,	coughing,	shortness	of	breath,	congesBon	in	
the	nose	and	throat,	and	in	some	cases	diarrhea.	MERS	
	
	
#MERS	is	a	rela,vely	new	respiratory	illness,	spread	b/w	people	in	close	contact.	
Symptoms	are	fever,	cough,	&	shortness	of	breath.	
	
	
Saudi	Arabia	finds	another	32	MERS	cases	as	disease	spreads:	RIYADH	(Reuters)	-	
Saudi	Arabia	said	on	Thursday	...	hcp://t.co/cPhm0uTRCo	
	
Signs	and	symptoms	
Signs	and	symptoms	
Signs	and	symptoms	
Affected	individuals
Social	Media	During	MERS	Outbreak			
	
First	Case	of	Deadly	Middle	Eastern	Virus	Found	in	U.S.:	The	Centers	for	Disease	
Control	has	confirmed	that	a	case	of	the	deadly	Midd...	
	
	
Third	Case	of	MERS	Confirmed	in	the	U.S.:	The	U.S.	Centers	for	Disease	Control	and	
Preven,on	confirmed	on	Sat...	hcp://t.co/Sb8PMyxVUn	
	
	
No	clear	transmission	link	btwn	camels	and	humans	for	MERS.	94%	Egyp,an	camels	
seroposi,ve	but	no	human	cases	yet.	Hmm	#asm2014	
	
	
Saudi	health	authori,es	announced	on	Monday	that	the	death	toll	from	the	MERS	
coronavirus	has	reached	115	since	the	respiratory	disease	...	
	
	
Transmission	
Death	reports	
Affected	individuals	
Affected	individuals
Twiaer	Breaks	Events	Faster	
First	report	
Breaks	the	story	33	minutes	before	local	TV	
Hudson	Plane	Crash	
Westgate	Mall	Aaack
Twiaer	Breaks	Events	Faster	
First	report	on	Twiaer	 A_er	1	minute	
A_er	2	minutes	
Boston	Bombing
Types	of	InformaBon	on	Twiaer	
-  Twicer	data	from	13	
recent	crises	
-  Over	100,000	tweets	
-  InformaBon	types	
-  Types	of	sources	
Source:	Qatar	Compu,ng	Research	Ins,tute	-	Published	in	World	Humanitarian	Data	and	Trends	2014	(UN	OCHA)
2013	Pakistan	Earthquake	
September	28	at	07:34	UTC	
	
2010	HaiB	Earthquake	
January	12	at	21:53	UTC	
Data	and	OpportuniBes	
Social	Media	
Plaiorms	
	
Availability	of	Immense	Data:	
Around	16	thousands	tweets	
per	minute	were	posted	during	
the	hurricane	Sandy	in	the	US.	
OpportuniBes:	
-  Early	warning	and	event	detecBon	
-  SituaBonal	awareness	
-  AcBonable	informaBon	extracBon	
-  Rapid	response	
	
-  EffecBve	communicaBons	
	
Disease	outbreaks
Part	3	
The	Role	of	AI	and	Data	Science	for	
Social	Good
Big	Data	Challenges	–	4Vs	
(Under	Time-criBcal	SituaBons)	
•  Volume		
	Scale	of	data	(e.g.,	millions	of	tweets	aber	an	event)	
•  Velocity	
	High-velocity	streams	(e.g.,	thousands	of	tweets/min)	
•  Variety	
	Different	forms/types	of	data	(informa,on	types)	
•  Veracity	
	Uncertainty	of	data
Data	AcquisiBon
Twiaer	Data	CollecBon	
•  REST	APIs	
–  Provides	programma,c	access	to	post	a	new	tweet,	
read	profile,	and	followers.	
•  Streaming	APIs	
–  Receive	live	updates	on	the	latest	tweets	matching	a	
search	query.	
•  Ads	API,	MoPub,	and	Gnip	
–  Twicer	adver,sing	management,	MoPub	is	a	mobile	
ad	exchange	and	ad	server.	
–  Gnip	provides	commercial-grade	access	to	real-,me	
and	historical	Twicer	data.
REST	vs.	Streaming	API	
REST	API	
Streaming	API	
Public	streams	
User	streams	
Site	streams	
Streaming	endpoints	
Sample	code	
hcps://github.com/twicerdev
ProperBes	of	Social	Media	Data	
•  Mostly	SM	data	is	publicly	available	
•  Near	Real-Bme	access	
•  1%	to	3%	geo-tagged	
•  Highly	informal,	oben	brief,	and	non-
structured	
•  Wricen	by	different	people	in	many	languages	
•  Contains	rumors	and	misinforma,on
Slangs	and	Shortened	forms	
•  Single-word	slangs:	pls	(please),	srsly	(seriously)	
•  MulB-word	slangs:	imo	(in	my	opinion)	
•  Misspellings:	missin	(missing),	ovrcme	(overcome)	
•  PhoneBc	subsBtuBon:	2morrow	(tomorrow)	
•  Word	without	spaces:	prayfornepal	(pray	for	
nepal)	
Can	you	guess?	
“r	u	ok	m8”	??		
	
	
>>	“Are	you	OK,	mate?”
Data	Velocity	and	Volume	
High	velocity	
•  2012	Hurricane	Sandy:	18,000	tweets/min	
•  2013	Boston	bombings:	54,000	tweets/min	
•  2011	Japan	earthquake:	66,000	tweets/min	
High	volume	
•  2012	Hurricane	Sandy:	20	million	tweets	in	5	days	
Batch	 Periodic	 Near	real-,me	 Real-,me	 Stream	
Increase	in	Data	Velocity	
KB	 MB	 GB	 TB	 PB	
Increase	in	Data	Volume	
File	system	--	MySQL		--		Postgres	– MongoDB		--	Apache	Cassandra	--	Redis
Data	Processing
Social	Media	InformaBon	Processing	
•  Natural	Language	Processing	Methods	
– Informa,on	extrac,on	(e.g.	person,	loca,on,	
organiza,on)	
– ClassificaBon	and	clustering	
– Automa,c	summariza,on	
– Seman,c	search	
– Machine	transla,on	
•  Imagery	content	processing	
– Object	detec,on	&	recogni,on	
– Image	retrieval	and	filtering	
– Automa,c	annota,on
Supervised	ClassificaBon	
Data	collec,on	
1	 2	
Human	annota,ons	
on	sample	data	
Machine	training	
3	
Classifica,on	
4	
Event	Timeline:	
DATA	COLLECTION	
Humans	alone	cannot	
process	large	amounts	of	
data,	so	we	only	use	them	
to	help	process	a	subset	
We	train	machine	using	
human	input	to	
automa,cally	process	large	
Data	at	high	speed	
For	example	using		
Keywords,	hashtags	etc.
Data	Stream	Processing	
1.  Data	items	arrive	online	
2.  Streams	have	infinite	length	and	unbounded	in	size	
3.  No	control	over	the	order	in	which	data	items	arrive	
4.  Processed	items	are	either	discarded	or	archived	
5.  No	retrieval	unless	stored	in	memory	(oben	small	size)	
	
Credit	Card	fraud	detecBon	 Sensor	data	classificaBon	 Social	media	streams	mining	
Data	stream
TradiBonal	vs.	Stream	Processing	
Property	 TradiBonal	System	 Stream	Processing	System	
Number	of	passes	 Mul,ple	 Single	
Memory	availability	 Unlimited	 Restricted	
Processing	,me	 Unlimited	 Restricted	
Results	availability	 Delayed	 Real-,me	
Results	reliability	 Accurate	 Improvable
Pure	Stream	Processing	and	Issues	
•  Rely	en,rely	on	automated	algorithms	
•  SM	data	streams	can	be	imprecise,	highly	variable,	and	oben	
unseen	
–  Concept-dri_:	happens	due	to	slow	changes	in	the	concepts	
–  Concept-evoluBon:	happens	due	to	the	presence	of	unknown	classes	
	
	
Aurora	Stream	Processing	
	(Brown	University)	
Flu	pandemic	2009
Crowdsourced	Stream	Processing	
(CSP)	
In	cases	where	cri1cal—in	terms	of	cost,	2me	or	reliability—decision-making	needs	to	
take	place	in	real-1me,	based	on	data	streams	that	are	poten2ally	noisy	and	unseen,	
fully	automated	stream	processing	systems	do	not	meet	the	needs.	
Stream processing
systems (SPs)
Crowdsourcing
systems (Cs)
Crowdsourced stream
processing systems (CSPs)
Human
processing role
Automatic
processing role
Compostion
Binary classification
N-ary classification
Open-ended
Computation
Filtering
Task-generation
Task-assignment
Task-aggregation
Serial
Parallel
Complex
Hierarchal
taxonomy
Faceted
taxonomy
System
Ref.	Imran,	Muhammad,	Ioanna	Lykourentzou,	Yannick	Naudet,	and	Carlos	Cas2llo.	"Engineering	crowdsourced	stream	
processing	systems."	arXiv	preprint	arXiv:1310.5463.
hcp://aidr.qcri.org/	
AIDR	—Ar,ficial	Intelligence	for	Disaster	Response—	is	a	free,	open,	and	easy-to-use	
	plaKorm	to	automa,cally	filter	and	classify	relevant	tweets	posted	during	humanitarian	crises.	
1	 2	 3	
Collect	 Curate	 Classify	
Grand	Prize	Winner	from	the	Open	Source	So_ware	World	Challenge	2015
Data	collec,on	
1	 2	
Human	annota,ons	 Machine	training	
3	
Classifica,on	
4	
ONLINE	APPROACH	
DATA	COLLECTION	
H
A	
Learning-1	
CLASSIFICATION	OF	DATA	&	DECISION	MAKING	PROCESS	
Learning-2	 Learning-3	 …	 Learning-n	
Human	
annota,on	-	1		
Human	
annota,on	-	2	
Human	
annota,on	-	3	 …	
Human	
annota,on	-	n	
First	few	hours	
Near	Real-Bme	Processing
Data	ClassificaBon	
Apply	machine	learning	Apply	crowdsourcing	
Goal:	To	find	relevant	and	
ac,onable	informa,on	in	
near	real-,me.	
Growing	stack	of	data	
AIDR	
Machine	Learning	+	Crowdsourcing	
Filter-failure	 Need	human-labeled	examples
Real-Bme	ClassificaBon	of		
Social	Media	Data	
hcp://aidr.qcri.org/
AIDR	Architecture	
Tweets
collector
Twitter
streaming API
Features
extractor
ClassifierP/S
Task
generator
Q P/S
Annotator
model parameters
Learner
Output
adapters
Q
Q
load shedding load shedding
query tweets
〈tweet〉 〈tweet, features〉
〈task〉
〈task, label〉
〈tweet, label,
confidence〉
Redis	channel	
Redis	queue	
Human-in-the-loop	(crowdsourcing)	
-	Uni-grams	
-	Bi-grams	
-	InformaBon	gain	
Random	Forest	
(decision	trees)	
-	Task	selecBon	
-	Task	prioriBzaBon	
Database:	Postgres	
ApplicaBon	layer:	Java	EE,	RESTFul	services,	Weka	machine	learning	library	
Data	flow	and	control	flow:	Redis	
Front-end:	ExtJS	(JavaScript	library)
Data	CollecBon	in	AIDR	(Twiaer)	
CollecBon	details	dashboard	
hcp://aidr.qcri.org/	
Geographical	region	filter	Language	filter	
CollecBon	setup
Data	ClassificaBon	Approach	
3.	
Extrac,on	
2.	
Classifica,on	
1.	
Filtering
1.	Filtering	
Is	event-	
related?	
Contributes	to	
situaBonal	
awareness?	
Yes Yes
No No
2.	ClassificaBon	
Caution &
Advice
Information
Sources
Damage &
Casualties
Donations
Health
Shelter
Food
Water
Logistics
...
...
Filtered
tweets
hcp://aidr.qcri.org/	
Sesng	up	Classifiers
AIDR	–	Classifier	Sesng	(cont.)	
hcp://aidr.qcri.org/
Human	AnnotaBon	in	AIDR	
Internal	Tagging	Interface	
hcp://aidr.qcri.org/
Human	AnnotaBon	Using	MicroMappers	
MicroMapper	Interface	(web	clicker)	
hcp://aidr.qcri.org/	
Mobile	clicker
Tagged	Items	and	Machine	Output	
hcp://aidr.qcri.org/	
Training	examples	 Classifiers’	output
Quality,	Cost,	and	Performance	of	
AIDR
Quality	vs.	Cost	in	AIDR	
hcp://aidr.qcri.org/	
Goals:	Maximize	quality	– Minimize	cost	
•  Quality	
•  Classifica,on	accuracy	
•  Precision/AUC	
•  Cost	to	obtain	labeled	data		
•  Monetary	in	case	of	paid-workers	
•  Time	in	case	of	volunteers
Quality	vs.	Cost	in	AIDR	
hcp://aidr.qcri.org/	
Quality	vs.	cost	using	passive	learning	and	with/without	de-duplicaBon	
Quality	vs.	cost	using	acBve	learning	and	with/without	de-duplicaBon
Performance	
hcp://aidr.qcri.org/	
In	terms	of	throughput	and	latency	
Latency	of	feature	extractor,	classifier,	and	the	system	
Throughput	of	feature	extractor,	classifier,	and	the	system
Processing	Evolving	Data	Streams
Data	Stream	Processing	
1.  Data	items	in	the	stream	arrive	online	
2.  Streams	have	infinite	length	and	unbounded	in	size	
3.  No	control	over	the	order	in	which	data	items	arrive	
4.  Processed	items	are	either	discarded	or	archived	
5.  No	retrieval	unless	stored	in	memory	(oben	small	size)	
	
Credit	Card	fraud	detecBon	 Sensor	data	classificaBon	 Social	media	streams	mining	
Data	stream
Types	of	Changes	in	SM	Streams	
Types	of	Stream	Dribs	
Concept	Drib	 Feature	Evolu,on	 Concept	Evolu,on	
Class	
boundaries	
change	over	
,me	
Feature	
subspace	
may	change	
New	
features	
appear	
Feature	
distribu,on	
changes	
Novel	
classes	
emerge	
Recurrent	
novel	classes	
re-appear
Types	of	Changes	in	Streaming	Data	
Except	Noise	and	Blip,	all	the	presented	changes	are	treated	as	concept	drib	
and	require	model	adapta,on.	
Ref.	Brzeziński,	Dariusz.	"Mining	data	streams	with	concept	drib."	PhD	diss.,	Master’s	thesis,	Poznan	University	of	Technology,	2010.
InformaBon	Variability	on	Social	Media	
•  Different	events	present	different	informa,on	
categories	
•  Even	for	recurring	events,	categories	
propor,on	change
InformaBon	Variability	on	Social	Media	
•  Different	events	present	different	informa,on	
categories	
•  Even	for	recurring	events,	categories	
propor,on	change
InformaBon	Variability	on	Social	Media	
•  Different	events	present	different	informa,on	
categories	
•  Even	for	recurring	events,	categories	
propor,on	change
InformaBon	Variability	on	Social	Media	
•  Different	events	present	different	informa,on	
categories	
•  Even	for	recurring	events,	categories	
propor,on	change
InformaBon	Variability	on	Social	Media	
•  Different	events	present	different	informa,on	
categories	
•  Even	for	recurring	events,	categories	
propor,on	change
Social	Media	Data	Streams	ClassificaBon	
Two	major	issues	in	the	supervised	classifica,on	of	social	
media	streams:	
1.  How	to	keep	the	categories	used	for	classificaBon	up-to-date?	
	
2.  While	adding	new	categories,	how	to	maintain	high	
classificaBon	accuracy?	
by crowd
Automatic
processing
Automatic
processing
output output
Performing verification
Providing training data
a: Split automatic/manual processing b: Detect-verify paradigm
Automatic
processing
Automatic
processing
output
c: Improving quality through active learning
input input
input
IdenBficaBon	of	Novel	Categories	
Classes.	
-  Injured	people	
-  Infrastructure	damage	
-  Shelter	needs	
-  Dona,on	requests	
-  Missing	or	stranded	people	
-  Different	health	issues	
-  Novel	urgent	needs	like		
-  Blankets	
-  Medicine	
-  Schools	shut	
-  Airport	closed/open	
-  …	
Pre-defined	classes	 Unseen	classes	(Miscellaneous)	
Keep	in	mind	we	have	a	new	class	
“Miscellaneous”
Expert-Machine-Crowd	Sesng	
Constraints	Outlier	DetecBon	(COD-Means):	
1.  Constraints	forma,on	using	classified	items	
2.  Clustering	using	COD-Means	
3.  Labeling	errors	iden,fica,on	(using	outlier	detec,on)	
List of
categories
documents stream
Supervised
Learning System
Novel Categories Detector
Using COD-Means
Crowdsourcing
task generator
Emerging novel categories
Crowdsourcing tasks to
be labeled by crowd
An expert
Crowd workers
Crowd/machine classified items.
(Machine classified items with
confidence score >= 0.90)
Incoming uncategorized
documents stream
Machine categorized items
(item, category and machine
confidence score) triplet
Refined training set
Human
labels
Labels
1	
2	
3	
4
Input	and	Output	
Category	A	 Category	B	 Category	C	 Miscellaneous	Z	
Category	A’	 Category	B’	 Category	C’	
Z1	 Z2	
Z’	
INPUT	OUTPUT
Constraints	FormaBon	
1.	Items	in	same	category	have	Must-link	constraints	
2.	Items	belonging	to	different	categories	have	Cannot-link	
constraints	
	
Category	A	 Category	B	 Category	C	 Category	Z	
Must-link	
Cannot-link	Note:	Items	in	Z	do	not	have	any	constraints
ObjecBve	FuncBon	
Standard	distor2on	error	
If	an	ML	constraint	if	violated	
then	the	cost	of	the	viola2on	is	
equal	to	the	distance	between	
the	two	centroids	that	contain	
the	instances.	
If	a	CL	constraint	is	violated	then	
the	error	cost	is	the	distance	
between	the	centroid	C	assigned	
to	the	pair	and	its	nearest	
centroid	h(c).
Assignment	and	Update	Rules	
Rule	1:	For	items	without	any	constraints	(standard	distor,on	error)		
Rule	2:	For	items	with	Must-link	constraints;	cost	of	viola,on	is	distance	b/w	their	centroids		
Rule	3:	For	items	with	Cannot-link	constraints;	cost	is	the	distance	b/w	centroid	c	and		
Its	nearest	centroid		
is	the	Kronecker	delta	func2on	
	i.e.	it	is	1	if	x=y	and	0	if	x	!=	y	
Update	rule:	 The	update	rule	computes	a	modified	
average	of	all	points	that	belong	to	a	
cluster.
COD-Means	Algorithm	
Algorithm	
1	
2	
3	
Ini2aliza2on	(e.g.	random	pick	of	k	centroids)	
Assignment	of	items	based	on	3	assignment	
rules	considering	ML	and	CL	constraints	
Points	in	each	cluster	are	sorted	based	
on	their	distance	to	the	centroid	and	
top	l	are	removed	and	inserted	into	L
Dataset	and	Experiments	
1.  Are	the	new	clusters	iden,fied	by	the	COD-Means	algorithm	genuinely	different	and	
novel?	
2.  What	is	the	nature	of	outliers	(labeling	errors)	discovered	by	the	COD-Means	
algorithm?	Are	they	genuine	outliers?	
3.  What	is	the	impact	of	outlier	on	the	quality	of	clusters	generated	by	COD-Means?	
4.  Once	refined	clusters	(without	labeling	errors)	used	in	the	training	process,	does	the	
overall	accuracy	improves?	
8	disaster-related	datasets	were	used	from	Twiaer
Clusters	Novelty	and	Coherence	
K-Means	vs.	COD-Means	
•  The	proposed	approach	generates	more	cohesive	and	novel	clusters	by	removing	outliers		
•  As	the	value	of	L	increases,	more	,ght	and	coherent	clusters	emerge
Data	Improvements	EvaluaBon	
Affected individuals
Caution and advice
Donations and volunteering
Infrastructure and utilities
Sympathy and support
Misc. to other categories
Precision
0 0.25 0.5 0.75 1
Precision
0 0.25 0.5 0.75 1
2012 Colorado Wildfires 2013 Alberta Floods 2013 Boston Bombings
2013 Colorado Floods 2013 Train Crash
2013 Australia Bushfire
2013 Queensland Floods 2013 West Texas Explosion
Precision
0 0.25 0.5 0.75 1
Precision
0 0.25 0.5 0.75 1
Affected individuals
Caution and advice
Donations and volunteering
Infrastructure and utilities
Sympathy and support
Misc. to other categories
Precision
0 0.25 0.5 0.75 1
Precision
0 0.25 0.5 0.75 1
Precision
0 0.25 0.5 0.75 1
Precision
0 0.25 0.5 0.75 1
1.  Labeling	errors	in	non-miscellaneous	categories	
2.  Items	incorrectly	labeled	as	miscellaneous
Impact	on	ClassificaBon	Performance
Social	Media	Image	Processing	
An	ApplicaBon	of	Computer	Vision
“A	picture	is	worth	a	thousand	words.”
Research	Goals	
•  Social	media	image	filtering	
– Real-,me	image	retrieval,	processing,	and	storage	
– Duplicate	or	near-duplicate	detec,on	
– Irrelevant	image	detec,on	
•  AcBonable	informaBon	extracBon	
– Infrastructure	damage	assessment	
– Injured	people	detec,on
AutomaBc	Image	Processing	Pipeline	
Dat	Tien	Nguyen,	Firoj	Alam,	Ferda	Ofli,	Muhammad	Imran.	Automa2c	Image	Filtering	on	Social	Networks	Using	Deep	Learning	and	Perceptual	Hashing	During	Crises.	
Accepted	for	publica2on	at	the	14th	Interna2onal	Conference	on	Informa2on	Systems	for	Crisis	Response	And	Management	(ISCRAM).	2017	Albi,	France.
Disaster	Datasets	(Twiaer)	
Dataset	details	for	all	four	disaster	events	with	their	year	and	number	of	images	
Number	of	labeled	images	for	each	dataset	and	each	damage	category
Relevancy	Filtering	
Examples	of	irrelevant	images	showing	cartoons,	banners,	adver,sements,	celebri,es,	etc.	
Performance	of	the	relevancy	filtering	
Task:	Build	a	binary	classifier	
Approach:	Transfer	learning		
																			(fine-tune	a	pre-trained	convolu,onal	neural	network,	e.g.,	VGG16*)	
*	Simonyan,	K.	and	Zisserman,	A.	(2014).	“Very	deep	convolu,onal	networks	for		
large-scale	image	recogni,on”.	In:	arXiv	preprint	arXiv:1409.1556
Duplicate	Filtering	
Examples	of	near-duplicate	images	
Task:	Compute	similarity	between	a	pair	of	images	
Approach:	Perceptual	Hash*	+	Hamming	Distance	(w/	threshold)	
*	Lei,	Y.	et	al.	(2011).	“Robust	image	hash	in	Radon	transform	domain	for	authen,ca,on”.		
In:	Signal	Processing:	Image	Communica,on	26.6,	pp.	280–288.
Before/A_er	Image	Filtering	
Number	of	images	that	remain	in	our	dataset	aber	each	image	filtering	opera,on	
~	2	%	
~	2	%	
~	50	%	
~	58	%	
~	50	%	
~	30	%
Before/A_er	Image	Filtering	
Number	of	images	that	remain	in	our	dataset	aber	each	image	filtering	opera,on	
~	2	%	
~	2	%	
~	50	%	
~	58	%	
~	50	%	
~	30	%	
Assume	tagging	an	image	costs	$1,	we	could	have	gocen	the	same	job	done		
by	paying	$17k	less,	almost	saving	2/3s	of	the	budget!!!
Infrastructure	Damage	Assessment	
•  Three-class	classifica,on	
– Categories:	severe,	mild	&	licle-to-none	
•  Dis,nc,on	between	categories	is	ambiguous.	
•  Agreement	among	human	annotators	is	low.	
–  in	par,cular	for	mild	category	
•  Fine-tuning	a	pre-trained	CNN	(e.g.,	VGG16)
AIDR	SMS	Processing	
AIDR	Helps	Answer	Thousands	of	
Health	Queries
Public	Health:	AIDR	+	UNICEF	Zambia	
Manual	processing	
and	rou,ng	of	SMS	
Counselors	(experts	of	HIV,	STIs)	
SMS	service	
1	 2	
3	
4	
5	
6	
Vulnerable	people
Public	Health:	AIDR	+	UNICEF	Zambia	
Manual	processing	
and	rou,ng	of	SMS	
Counselors	(experts	of	HIV,	STIs)	
SMS	service	
1	 2	
3	
4	
5	
6	
Vulnerable	people
New	ScienBst	Featured	This	Work
Media	Coverage
Domain	AdaptaBon/Transfer	Learning	
Ability	of	a	system	to	apply	knowledge	and	skills	
learned	in	previous	domains	to	novel	domains	
Ongoing	Work	
Our	Goal:	
To	build	a	system	that	can	understand	natural	language
Domain	AdaptaBon	
Labeled	source,	but	unlabeled	target	
Feature	
extractor	
Machine	
learning	
algorithm	
Feature	
extractor	
Classifier	
model	
Input	documents	(blue	domain)	
Feature	vectors	
Labels	
Feature	vectors	
Machine	classified	items	
Input	documents	(orange	domain)	
Training	
PredicBon	
Source	event	data	 Target	event	data
Same	Domain	Learning	
Training	
data	
Machine	learning	model	
Tes,ng	
data	
infer	 predict	
Apples	 Apples	
Apples	
Oranges	
Different	shapes,	colors,	skins,	tastes,	etc.	
Source	domain	 Target	domain	
Oranges	
Oranges	
BUT
Crisis-related	Data	ClassificaBon	
Training	
data	
Machine	learning	model	
Tes,ng	
data	
infer	 predict	
Italy	earthquake	
	
Queensland	floods	
	
Sandy	hurricane	
	
	
Costa	Rica	earthquake	
	
Colorado	floods	
	
Typhoon	Haiyan	
Different	events,	languages,	and	needs	etc.	
Source	domain	 Target	domain
Domain	AdaptaBon
Model	AdaptaBon	EvaluaBon	
•  Model	adapta,on	using	single	source	
– Using	both:	in-domain	and	cross-domain	
•  Model	adapta,on	using	mulBple	sources	
– In-domain	
– Mul,ple	source	events	without	the	target	
– Mul,ple	source	events	with	the	target	
•  Model	adapta,on	in	special	cases	
– Same	languages	
– Similar	languages
Transfer	Learning	
Differences	in	classificaBon	tasks:	
•  Different	classifica,on	tasks	
•  Different	types	of	disasters,	stakeholders,	
informa,on	needs	
Task:	
•  Learn	from	source	to	classify	target	
•  Seman,c	similarity	between	tasks	
•  Zero-shot	learning	(no	training	examples)	
•  One-shot	learning	(few	training	examples)
SummarizaBon	and	PrioriBzaBon	of	
AcBonable	InformaBon	
InformaBon	needs	&	problem:	
•  Different	stakeholders	
•  Different	goals,	requirements,	and	info.	needs	
General	situaBonal	awareness	vs.	Target	situaBonal	
awareness	
•  High-level	general	updates	from	an	event	
•  Specific	updates	(infrastructure	damages)
InformaBon	SummarizaBon	
	In	Real-Time	
Class	A	 Class	B	 Class	C	 Class	D	
Summary	 Summary	 Summary	 Summary	
Classified	
documents	
	stream
Resources,	Datasets,	And	Tools
Towards	Standard	
Baselines	and	Datasets	
CrisisNLP.qcri.org	
-  Access	to	52	million	tweets	
-  Around	50k	labeled	tweets	into	humanitarian	categories	
-  Largest	word2vec	embeddings	trained	on	52m	crisis-related	tweets	
-  Out-of-vocabulary	dic,onaries	
-  Tweets	downloader
ACM	CompuBng	Survey	
Processing	Social	Media	Messages	in	Mass	
Emergency:	A	Survey		
[Imran	et	al.	2015]
27	Free	Data	Mining	Books	
hap://www.datasciencecentral.com/profiles/blogs/27-free-data-mining-books
Special	Issues	
Organizing	Editors	
	
Chris,an	Reuter	(University	of	Siegen)	
Muhammad	Imran	(Qatar	Compu,ng	Research	Ins,tute)	
Amanda	Hughes	(Utah	State	University)	
Starr	Roxanne	Hiltz	(New	Jersey	Ins,tute	of	Technology)	
Linda	Plotnick	(Jacksonville	State	University)	
Special	Issue	on	“ExploitaBon	of	Social	Media	for		
Emergency	Relief	and	Preparedness”		
Deadline:	July	1st	2017	
Marie-Francine	Moens,	KU	Leuven,	Belgium		
Gareth	Jones,	Dublin	City	University,	Ireland	
Muhammad	Imran,	Qatar	Compu,ng	Research	Ins,tute	
Saptarshi	Ghosh,	IIT	Kharagpur,	India	
Kripabandhu	Ghosh,	IIT	Kanpur,	India	
Debasis	Ganguly,	IBM	Research	Labs,	Dublin,	Ireland	
Tanmoy	Chakraborty,	University	of	Maryland,	College	Park,	
USA
Conclusions	
•  InformaBon	bestows	power	for	disaster	response	
–  People	need	informa,on	as	much	as	water,	shelter,	and	food	
–  Disasters	are	unavoidable,	but	planning	can	lessen	their	effects	
•  Social	media	as	Bme-criBcal	informaBon	source	
–  Early	warnings,	event	detec,on,	event	monitoring	
–  Availability	of	informa,on	opens	new	opportuni,es	
•  ArBficial	Intelligence	for	Social	Good	
–  Applied	research	at	its	best	
–  AI	+	humans-in-the-loop	can	enable	rapid	crisis	response	
–  AI	techniques	useful	for:	
•  Situa,onal	awareness	
•  Ac,onable	informa,on	extrac,on	
•  Summariza,on
THANK	YOU!	
CrisisNLP.qcri.org	AIDR.qcri.org	
Email:	mimran@hbku.edu.qa	
Homepage:	hap://mimran.me	
Twiaer:	@mimran15

More Related Content

What's hot

ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...
ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...
ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...Global Risk Forum GRFDavos
 
Social Media News Mining and Automatic Content Analysis of News
Social Media News Mining and Automatic Content Analysis of NewsSocial Media News Mining and Automatic Content Analysis of News
Social Media News Mining and Automatic Content Analysis of NewsCarlos Castillo (ChaTo)
 
Thesis Paper Robert Monné
Thesis Paper Robert MonnéThesis Paper Robert Monné
Thesis Paper Robert MonnéRobert Monné
 
Automatically Rank Social Media Requests for Emergency Services using Service...
Automatically Rank Social Media Requests for Emergency Services using Service...Automatically Rank Social Media Requests for Emergency Services using Service...
Automatically Rank Social Media Requests for Emergency Services using Service...Hemant Purohit
 
KNO.E.SIS Approach to Impactful Research, Creating Exceptional Careers & Eco...
KNO.E.SIS Approach to Impactful Research,  Creating Exceptional Careers & Eco...KNO.E.SIS Approach to Impactful Research,  Creating Exceptional Careers & Eco...
KNO.E.SIS Approach to Impactful Research, Creating Exceptional Careers & Eco...Amit Sheth
 
The case for integrating crisis response with social media
The case for integrating crisis response with social media The case for integrating crisis response with social media
The case for integrating crisis response with social media American Red Cross
 
Twitris in Action - a review of its many applications
Twitris in Action - a review of its many applications Twitris in Action - a review of its many applications
Twitris in Action - a review of its many applications Amit Sheth
 
Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...
Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...
Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...Artificial Intelligence Institute at UofSC
 
Leveraging A Wiki To Enhance Virtual Collaboration In The Emergency Domain
Leveraging A Wiki To Enhance Virtual Collaboration In The Emergency DomainLeveraging A Wiki To Enhance Virtual Collaboration In The Emergency Domain
Leveraging A Wiki To Enhance Virtual Collaboration In The Emergency DomainConnie White
 
Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...
Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...
Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...Keith Powell
 
World Disaster Report 2013
World Disaster Report 2013World Disaster Report 2013
World Disaster Report 2013Lavinia Baciu
 
Disaster Management: role of ICTs
Disaster Management: role of ICTsDisaster Management: role of ICTs
Disaster Management: role of ICTsglobal
 
CrisisCommons Statement for the Record
CrisisCommons Statement for the RecordCrisisCommons Statement for the Record
CrisisCommons Statement for the RecordHeather Blanchard
 
Emergency Risk Communication
Emergency Risk CommunicationEmergency Risk Communication
Emergency Risk CommunicationHeather Blanchard
 
Keynote talk: Big Crisis Data, an Open Invitation
Keynote talk: Big Crisis Data, an Open InvitationKeynote talk: Big Crisis Data, an Open Invitation
Keynote talk: Big Crisis Data, an Open InvitationCarlos Castillo (ChaTo)
 
The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...
The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...
The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...Ryan Burns
 
Applying citizen science model to disaster management
Applying citizen science model to disaster managementApplying citizen science model to disaster management
Applying citizen science model to disaster managementW. David Stephenson
 

What's hot (20)

How to Leverage Social Media Communities for Crisis Response Coordination
How to Leverage Social Media Communities for Crisis Response CoordinationHow to Leverage Social Media Communities for Crisis Response Coordination
How to Leverage Social Media Communities for Crisis Response Coordination
 
ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...
ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...
ICT for Disaster Risk Management-Managing Disaster Information-Global Risk Id...
 
Social Media News Mining and Automatic Content Analysis of News
Social Media News Mining and Automatic Content Analysis of NewsSocial Media News Mining and Automatic Content Analysis of News
Social Media News Mining and Automatic Content Analysis of News
 
Thesis Paper Robert Monné
Thesis Paper Robert MonnéThesis Paper Robert Monné
Thesis Paper Robert Monné
 
Automatically Rank Social Media Requests for Emergency Services using Service...
Automatically Rank Social Media Requests for Emergency Services using Service...Automatically Rank Social Media Requests for Emergency Services using Service...
Automatically Rank Social Media Requests for Emergency Services using Service...
 
KNO.E.SIS Approach to Impactful Research, Creating Exceptional Careers & Eco...
KNO.E.SIS Approach to Impactful Research,  Creating Exceptional Careers & Eco...KNO.E.SIS Approach to Impactful Research,  Creating Exceptional Careers & Eco...
KNO.E.SIS Approach to Impactful Research, Creating Exceptional Careers & Eco...
 
The case for integrating crisis response with social media
The case for integrating crisis response with social media The case for integrating crisis response with social media
The case for integrating crisis response with social media
 
Twitris in Action - a review of its many applications
Twitris in Action - a review of its many applications Twitris in Action - a review of its many applications
Twitris in Action - a review of its many applications
 
Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...
Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...
Crisis Mapping, Citizen Sensing and Social Media Analytics: Leveraging Citize...
 
Leveraging A Wiki To Enhance Virtual Collaboration In The Emergency Domain
Leveraging A Wiki To Enhance Virtual Collaboration In The Emergency DomainLeveraging A Wiki To Enhance Virtual Collaboration In The Emergency Domain
Leveraging A Wiki To Enhance Virtual Collaboration In The Emergency Domain
 
Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...
Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...
Humanitarian Diplomacy in the Digital Age: Analysis and use of digital inform...
 
Kenya red cross society
Kenya red cross societyKenya red cross society
Kenya red cross society
 
ICT IN DISASTER MANAGEMENT IN INDIA
ICT IN DISASTER MANAGEMENT IN INDIAICT IN DISASTER MANAGEMENT IN INDIA
ICT IN DISASTER MANAGEMENT IN INDIA
 
World Disaster Report 2013
World Disaster Report 2013World Disaster Report 2013
World Disaster Report 2013
 
Disaster Management: role of ICTs
Disaster Management: role of ICTsDisaster Management: role of ICTs
Disaster Management: role of ICTs
 
CrisisCommons Statement for the Record
CrisisCommons Statement for the RecordCrisisCommons Statement for the Record
CrisisCommons Statement for the Record
 
Emergency Risk Communication
Emergency Risk CommunicationEmergency Risk Communication
Emergency Risk Communication
 
Keynote talk: Big Crisis Data, an Open Invitation
Keynote talk: Big Crisis Data, an Open InvitationKeynote talk: Big Crisis Data, an Open Invitation
Keynote talk: Big Crisis Data, an Open Invitation
 
The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...
The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...
The Digital Humanitarian Moment: New Practices, Knowledge Politics, and Phila...
 
Applying citizen science model to disaster management
Applying citizen science model to disaster managementApplying citizen science model to disaster management
Applying citizen science model to disaster management
 

Similar to Real-Time Processing of Social Media Content for Social Good

Transforming Social Big Data into Timely Decisions and Actions for Crisis Mi...
Transforming Social Big Data into Timely Decisions  and Actions for Crisis Mi...Transforming Social Big Data into Timely Decisions  and Actions for Crisis Mi...
Transforming Social Big Data into Timely Decisions and Actions for Crisis Mi...Amit Sheth
 
Emergency Management in the age of social convergence
Emergency Management in the age of social convergenceEmergency Management in the age of social convergence
Emergency Management in the age of social convergencePatrice Cloutier
 
Humanitarian Informatics Approach for Cooperation between Citizens and Organi...
Humanitarian Informatics Approach for Cooperation between Citizens and Organi...Humanitarian Informatics Approach for Cooperation between Citizens and Organi...
Humanitarian Informatics Approach for Cooperation between Citizens and Organi...Hemant Purohit
 
Social Media in Government
Social Media in GovernmentSocial Media in Government
Social Media in GovernmentJudith A Newton
 
Tools and processes in digital voluntarism
Tools and processes in digital voluntarismTools and processes in digital voluntarism
Tools and processes in digital voluntarismperaarvik
 
U otago the frew group webinar
U otago the frew group webinar U otago the frew group webinar
U otago the frew group webinar COMP 113
 
Deploying digital tools for evaluation & citizen engagement
Deploying digital tools for evaluation & citizen engagementDeploying digital tools for evaluation & citizen engagement
Deploying digital tools for evaluation & citizen engagementGerard McCarthy
 
Multimedia rescue 161018
Multimedia rescue 161018Multimedia rescue 161018
Multimedia rescue 161018Ramesh Jain
 
Technologie & Humanitaire: Ressources & Liens
Technologie & Humanitaire: Ressources & LiensTechnologie & Humanitaire: Ressources & Liens
Technologie & Humanitaire: Ressources & LiensAgnes Lesage-Possolo
 
2nd Annual Geospatial Conference
2nd Annual Geospatial Conference2nd Annual Geospatial Conference
2nd Annual Geospatial ConferenceHeather Blanchard
 
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...
Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...Connie White
 
Gerard McCarthy Digital tools for evaluation
Gerard McCarthy Digital tools for evaluationGerard McCarthy Digital tools for evaluation
Gerard McCarthy Digital tools for evaluationDevelopment Futures
 
Copy of 1INTRO~4.Powerpoimt presentation
Copy of 1INTRO~4.Powerpoimt presentationCopy of 1INTRO~4.Powerpoimt presentation
Copy of 1INTRO~4.Powerpoimt presentationRouAnnAdobasNavarroz1
 
Dave Harte- The Timely Information Project
Dave Harte- The Timely Information ProjectDave Harte- The Timely Information Project
Dave Harte- The Timely Information ProjectPaul Hadley
 
Social Media as a Record for Public Services and Utilities in a Disaster
Social Media as a Record for Public Services and Utilities in a DisasterSocial Media as a Record for Public Services and Utilities in a Disaster
Social Media as a Record for Public Services and Utilities in a DisasterAndrew Steinitz
 
Digital intermediation: Towards Transparent Public Automated Media
Digital intermediation: Towards Transparent Public Automated MediaDigital intermediation: Towards Transparent Public Automated Media
Digital intermediation: Towards Transparent Public Automated MediaUniversity of Sydney
 
Big Data and Social Media Mining in Crisis and Emergency Management
Big Data and Social Media Mining in Crisis and Emergency ManagementBig Data and Social Media Mining in Crisis and Emergency Management
Big Data and Social Media Mining in Crisis and Emergency ManagementBYTE Project
 

Similar to Real-Time Processing of Social Media Content for Social Good (20)

Transforming Social Big Data into Timely Decisions and Actions for Crisis Mi...
Transforming Social Big Data into Timely Decisions  and Actions for Crisis Mi...Transforming Social Big Data into Timely Decisions  and Actions for Crisis Mi...
Transforming Social Big Data into Timely Decisions and Actions for Crisis Mi...
 
Emergency Management in the age of social convergence
Emergency Management in the age of social convergenceEmergency Management in the age of social convergence
Emergency Management in the age of social convergence
 
Humanitarian Informatics Approach for Cooperation between Citizens and Organi...
Humanitarian Informatics Approach for Cooperation between Citizens and Organi...Humanitarian Informatics Approach for Cooperation between Citizens and Organi...
Humanitarian Informatics Approach for Cooperation between Citizens and Organi...
 
Social Media in Government
Social Media in GovernmentSocial Media in Government
Social Media in Government
 
Tools and processes in digital voluntarism
Tools and processes in digital voluntarismTools and processes in digital voluntarism
Tools and processes in digital voluntarism
 
U otago the frew group webinar
U otago the frew group webinar U otago the frew group webinar
U otago the frew group webinar
 
Deploying digital tools for evaluation & citizen engagement
Deploying digital tools for evaluation & citizen engagementDeploying digital tools for evaluation & citizen engagement
Deploying digital tools for evaluation & citizen engagement
 
Multimedia rescue 161018
Multimedia rescue 161018Multimedia rescue 161018
Multimedia rescue 161018
 
Technologie & Humanitaire: Ressources & Liens
Technologie & Humanitaire: Ressources & LiensTechnologie & Humanitaire: Ressources & Liens
Technologie & Humanitaire: Ressources & Liens
 
2nd Annual Geospatial Conference
2nd Annual Geospatial Conference2nd Annual Geospatial Conference
2nd Annual Geospatial Conference
 
Smile presentation
Smile presentationSmile presentation
Smile presentation
 
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...
Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...Web 2.0 Technology  Building Situational Awareness:  Free and Open Source Too...
Web 2.0 Technology Building Situational Awareness: Free and Open Source Too...
 
Pier webinar
Pier webinarPier webinar
Pier webinar
 
Gerard McCarthy Digital tools for evaluation
Gerard McCarthy Digital tools for evaluationGerard McCarthy Digital tools for evaluation
Gerard McCarthy Digital tools for evaluation
 
Brightest Minds at CSR GCC Conference
Brightest Minds at CSR GCC ConferenceBrightest Minds at CSR GCC Conference
Brightest Minds at CSR GCC Conference
 
Copy of 1INTRO~4.Powerpoimt presentation
Copy of 1INTRO~4.Powerpoimt presentationCopy of 1INTRO~4.Powerpoimt presentation
Copy of 1INTRO~4.Powerpoimt presentation
 
Dave Harte- The Timely Information Project
Dave Harte- The Timely Information ProjectDave Harte- The Timely Information Project
Dave Harte- The Timely Information Project
 
Social Media as a Record for Public Services and Utilities in a Disaster
Social Media as a Record for Public Services and Utilities in a DisasterSocial Media as a Record for Public Services and Utilities in a Disaster
Social Media as a Record for Public Services and Utilities in a Disaster
 
Digital intermediation: Towards Transparent Public Automated Media
Digital intermediation: Towards Transparent Public Automated MediaDigital intermediation: Towards Transparent Public Automated Media
Digital intermediation: Towards Transparent Public Automated Media
 
Big Data and Social Media Mining in Crisis and Emergency Management
Big Data and Social Media Mining in Crisis and Emergency ManagementBig Data and Social Media Mining in Crisis and Emergency Management
Big Data and Social Media Mining in Crisis and Emergency Management
 

More from Muhammad Imran

Damage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During DisastersDamage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During DisastersMuhammad Imran
 
Image4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster ResponseImage4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster ResponseMuhammad Imran
 
AIDR Tutorial (Artificial Intelligence for Disaster Response)
AIDR Tutorial (Artificial Intelligence for Disaster Response)AIDR Tutorial (Artificial Intelligence for Disaster Response)
AIDR Tutorial (Artificial Intelligence for Disaster Response)Muhammad Imran
 
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...Muhammad Imran
 
Summarizing Situational Tweets in Crisis Scenario
Summarizing Situational Tweets in Crisis ScenarioSummarizing Situational Tweets in Crisis Scenario
Summarizing Situational Tweets in Crisis ScenarioMuhammad Imran
 
Artificial Intelligence for Disaster Response
Artificial Intelligence for Disaster ResponseArtificial Intelligence for Disaster Response
Artificial Intelligence for Disaster ResponseMuhammad Imran
 
A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...
A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...
A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...Muhammad Imran
 
Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...
Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...
Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...Muhammad Imran
 
Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...
Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...
Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...Muhammad Imran
 
Extracting Information Nuggets from Disaster-Related Messages in Social Media
Extracting Information Nuggets from Disaster-Related Messages in Social MediaExtracting Information Nuggets from Disaster-Related Messages in Social Media
Extracting Information Nuggets from Disaster-Related Messages in Social MediaMuhammad Imran
 
Domain Specific Mashups
Domain Specific MashupsDomain Specific Mashups
Domain Specific MashupsMuhammad Imran
 
Reseval Mashup Platform Talk at SECO
Reseval Mashup Platform Talk at SECOReseval Mashup Platform Talk at SECO
Reseval Mashup Platform Talk at SECOMuhammad Imran
 
ResEval: Resource-oriented Research Impact Evaluation platform
ResEval: Resource-oriented Research Impact Evaluation platformResEval: Resource-oriented Research Impact Evaluation platform
ResEval: Resource-oriented Research Impact Evaluation platformMuhammad Imran
 

More from Muhammad Imran (13)

Damage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During DisastersDamage Assessment from Social Media Imagery Data During Disasters
Damage Assessment from Social Media Imagery Data During Disasters
 
Image4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster ResponseImage4Act: Online Social Media Image Processing for Disaster Response
Image4Act: Online Social Media Image Processing for Disaster Response
 
AIDR Tutorial (Artificial Intelligence for Disaster Response)
AIDR Tutorial (Artificial Intelligence for Disaster Response)AIDR Tutorial (Artificial Intelligence for Disaster Response)
AIDR Tutorial (Artificial Intelligence for Disaster Response)
 
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...
A Robust Framework for Classifying Evolving Document Streams in an Expert-Mac...
 
Summarizing Situational Tweets in Crisis Scenario
Summarizing Situational Tweets in Crisis ScenarioSummarizing Situational Tweets in Crisis Scenario
Summarizing Situational Tweets in Crisis Scenario
 
Artificial Intelligence for Disaster Response
Artificial Intelligence for Disaster ResponseArtificial Intelligence for Disaster Response
Artificial Intelligence for Disaster Response
 
A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...
A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...
A Real-time Heuristic-based Unsupervised Method for Name Disambiguation in Di...
 
Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...
Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...
Coordinating Human and Machine Intelligence to Classify Microblog Communica0o...
 
Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...
Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...
Tweet4act: Using Incident-Specific Profiles for Classifying Crisis-Related Me...
 
Extracting Information Nuggets from Disaster-Related Messages in Social Media
Extracting Information Nuggets from Disaster-Related Messages in Social MediaExtracting Information Nuggets from Disaster-Related Messages in Social Media
Extracting Information Nuggets from Disaster-Related Messages in Social Media
 
Domain Specific Mashups
Domain Specific MashupsDomain Specific Mashups
Domain Specific Mashups
 
Reseval Mashup Platform Talk at SECO
Reseval Mashup Platform Talk at SECOReseval Mashup Platform Talk at SECO
Reseval Mashup Platform Talk at SECO
 
ResEval: Resource-oriented Research Impact Evaluation platform
ResEval: Resource-oriented Research Impact Evaluation platformResEval: Resource-oriented Research Impact Evaluation platform
ResEval: Resource-oriented Research Impact Evaluation platform
 

Recently uploaded

STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxSTOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxMurugaveni B
 
trihybrid cross , test cross chi squares
trihybrid cross , test cross chi squarestrihybrid cross , test cross chi squares
trihybrid cross , test cross chi squaresusmanzain586
 
User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)Columbia Weather Systems
 
GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024Jene van der Heide
 
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)Columbia Weather Systems
 
Organic farming with special reference to vermiculture
Organic farming with special reference to vermicultureOrganic farming with special reference to vermiculture
Organic farming with special reference to vermicultureTakeleZike1
 
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingBase editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingNetHelix
 
User Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather StationUser Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather StationColumbia Weather Systems
 
Introduction of Human Body & Structure of cell.pptx
Introduction of Human Body & Structure of cell.pptxIntroduction of Human Body & Structure of cell.pptx
Introduction of Human Body & Structure of cell.pptxMedical College
 
REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...
REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...
REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...Universidade Federal de Sergipe - UFS
 
User Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather StationUser Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather StationColumbia Weather Systems
 
PROJECTILE MOTION-Horizontal and Vertical
PROJECTILE MOTION-Horizontal and VerticalPROJECTILE MOTION-Horizontal and Vertical
PROJECTILE MOTION-Horizontal and VerticalMAESTRELLAMesa2
 
bonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girlsbonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girlshansessene
 
Topic 9- General Principles of International Law.pptx
Topic 9- General Principles of International Law.pptxTopic 9- General Principles of International Law.pptx
Topic 9- General Principles of International Law.pptxJorenAcuavera1
 
Pests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPirithiRaju
 
Ai in communication electronicss[1].pptx
Ai in communication electronicss[1].pptxAi in communication electronicss[1].pptx
Ai in communication electronicss[1].pptxsubscribeus100
 
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdfPests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdfPirithiRaju
 
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptxGENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptxRitchAndruAgustin
 
Environmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial BiosensorEnvironmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial Biosensorsonawaneprad
 
Four Spheres of the Earth Presentation.ppt
Four Spheres of the Earth Presentation.pptFour Spheres of the Earth Presentation.ppt
Four Spheres of the Earth Presentation.pptJoemSTuliba
 

Recently uploaded (20)

STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptxSTOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
STOPPED FLOW METHOD & APPLICATION MURUGAVENI B.pptx
 
trihybrid cross , test cross chi squares
trihybrid cross , test cross chi squarestrihybrid cross , test cross chi squares
trihybrid cross , test cross chi squares
 
User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)User Guide: Orion™ Weather Station (Columbia Weather Systems)
User Guide: Orion™ Weather Station (Columbia Weather Systems)
 
GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024GenAI talk for Young at Wageningen University & Research (WUR) March 2024
GenAI talk for Young at Wageningen University & Research (WUR) March 2024
 
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
User Guide: Pulsar™ Weather Station (Columbia Weather Systems)
 
Organic farming with special reference to vermiculture
Organic farming with special reference to vermicultureOrganic farming with special reference to vermiculture
Organic farming with special reference to vermiculture
 
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editingBase editing, prime editing, Cas13 & RNA editing and organelle base editing
Base editing, prime editing, Cas13 & RNA editing and organelle base editing
 
User Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather StationUser Guide: Capricorn FLX™ Weather Station
User Guide: Capricorn FLX™ Weather Station
 
Introduction of Human Body & Structure of cell.pptx
Introduction of Human Body & Structure of cell.pptxIntroduction of Human Body & Structure of cell.pptx
Introduction of Human Body & Structure of cell.pptx
 
REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...
REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...
REVISTA DE BIOLOGIA E CIÊNCIAS DA TERRA ISSN 1519-5228 - Artigo_Bioterra_V24_...
 
User Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather StationUser Guide: Magellan MX™ Weather Station
User Guide: Magellan MX™ Weather Station
 
PROJECTILE MOTION-Horizontal and Vertical
PROJECTILE MOTION-Horizontal and VerticalPROJECTILE MOTION-Horizontal and Vertical
PROJECTILE MOTION-Horizontal and Vertical
 
bonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girlsbonjourmadame.tumblr.com bhaskar's girls
bonjourmadame.tumblr.com bhaskar's girls
 
Topic 9- General Principles of International Law.pptx
Topic 9- General Principles of International Law.pptxTopic 9- General Principles of International Law.pptx
Topic 9- General Principles of International Law.pptx
 
Pests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdfPests of Bengal gram_Identification_Dr.UPR.pdf
Pests of Bengal gram_Identification_Dr.UPR.pdf
 
Ai in communication electronicss[1].pptx
Ai in communication electronicss[1].pptxAi in communication electronicss[1].pptx
Ai in communication electronicss[1].pptx
 
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdfPests of Blackgram, greengram, cowpea_Dr.UPR.pdf
Pests of Blackgram, greengram, cowpea_Dr.UPR.pdf
 
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptxGENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
GENERAL PHYSICS 2 REFRACTION OF LIGHT SENIOR HIGH SCHOOL GENPHYS2.pptx
 
Environmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial BiosensorEnvironmental Biotechnology Topic:- Microbial Biosensor
Environmental Biotechnology Topic:- Microbial Biosensor
 
Four Spheres of the Earth Presentation.ppt
Four Spheres of the Earth Presentation.pptFour Spheres of the Earth Presentation.ppt
Four Spheres of the Earth Presentation.ppt
 

Real-Time Processing of Social Media Content for Social Good