Examples of Real-World Big Data Application Specific examples of velocity challenge and how it is addressed in disaster coordination scenario (e.g., Jammu&Kashmir Floods). Prof Amit Sheth - Kno.e.sis

1. Semantic Approach to
Big Data and Event Processing
Examples of Real-World Big Data Application
Specific examples of velocity challenge and
how it is addressed in disaster coordination
scenario (e.g., Jammu&Kashmir Floods).
Prof. Amit Sheth
Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis)
Wright State University, USA
Tutorial @ Kno.e.sis Centre: Semantics Approach to Big Data and Event Processing, Oct 7-9, 2015

2. 2
Smart Data for Social Good
Mining human behavior to help
societal and humanitarian
development
• crisis response coordination,
harassment, gender-based
violence, human trafficking,
prescription drug abuse…

3. 20 million tweets with “sandy, hurricane”
keywords between Oct 27th and Nov 1st
2nd most popular topic on Facebook during 2012
Social (Big) Data during Crisis- Example of Hurricane Sandy
3
• http://www.guardian.co.uk/news/datablog/2
012/oct/31/twitter-sandy-flooding
• http://www.huffingtonpost.com/2012/11/02
/twitter-hurricane-sandy_n_2066281.html
• http://mashable.com/2012/10/31/hurricane-
sandy-facebook/

4. 0.5B Tweets per day
0.5B Users
60% on Mobile
5530 Tweets per second
related to the Japan earthquake and tsunami
17000 Tweets
per second
4
Twitter During Real-world Events of Interest
http://www.flickr.com/photos/twitteroffice/5897088517/sizes/o/in/photostream/
http://bayarea.sbnation.com/49ers/2013/2/3/3947738/super-bowl-prop-bets-2013-
twitterhttp://bayarea.sbnation.com/49ers/2013/2/3/3947738/super-bowl-prop-bets-2013-twitter
http://expandedramblings.com/index.php/march-2013-by-the-numbers-a-few-amazing-twitter-stats/

5. 5http://usatoday30.usatoday.com/news/politics/twitter-election-meter
http://twitris.knoesis.org/

6. Twitris’ Dimensions of Integrated Semantic Analysis
6Sheth et al. Twitris- a System for Collective Social Intelligence, ESNAM-2014

7. Semantic Approach to
Big Data and Event Processing
Metadata Extraction from
Informal Text
7
Meena Nagarajan, ‘Understanding User-Generated Content on Social Media,’ Ph.D. Dissertation, Wright State University, 2010

8. 8
Characteristics of Text on Social Media

9. 9
The Formality of Text

10. “I loved your music Yesterday!”
Yesterday is an album
“It was THE HANGOVER of the year..lasted
forever..
The Hangover is not a movie
So I went to the movies..badchoice picking “GI
Jane”worse now”
GI Jane is a movie
10
Named Entity Recognition: Identifying and classifying tokens

11. 11
Cultural NER - Two Flavors

12. 12
(a) Multiple Senses in the Same Domain

13. Scoped Relationship graphs
– Using context cues from the
content, webpage title, url…
e.g. new Merry Christmas tune
– Reduce potential entity spot size
e.g. new albums/songs
• Generate candidate entities
• Spot and Disambiguate
Which ‘Merry Christmas’?; ‘So Good’is also a song!
13
Approach Overview

14. Career Restrictions
- “release your third album already..”
Recent Album restrictions
- “I loved your new album..”
Artist age restrictions
-”happy 25th rihanna, loved alfie btw..”
etc.
Which ‘Merry Christmas’?; ‘So Good’is also a
song!
14
Sample Real-world Constraints

15. Observations:
• The opinion clues may not be toward the given target
(1,2,3,6)
• The opinion clues are domain and context dependent
(5,7)
• Single words are not enough (4,7,8)
Simple lexicon-based method doesn't work well.
15
Target of “sexy” is “Helena”
Target of “terrific” is “reviews”
“free” is not opinionated in
movie domain.
Target of “loving” is “telling”
“well” in “as well” is not
opinionated
Sentiment Analysis:
Target-specific Opinion Identification & Classification
of Tweets-Unsupervised Approach

16. 16
[The screenshots of Twitris+ were taken on Nov. 6th 6 PM EST]
/t

17. 17
Twitris: Sentiment Analysis- Smart Answers with reasoning!
How was Obama doing in the first debate?

18. 18
Red Color: Negative Topics
Green Color: Positive Topics
Twitris: Sentiment Analysis- Smart Answers with reasoning!
How was Obama doing in the second debate?
SMART DATA IS ABOUT ANALYSIS FOR REASONING
(what caused the positive sentiment for Democrats)
BEHIND THE REAL-WORLD ACTIONS (Democrats’ win)
http://knoesis.wright.edu/library/resource.php?id=1787

19. Top 100 influential users that
talks about Barack Obama
Positive or Negative
Influence
Twitris: Network Analysis
SMART DATA TELLS YOU HOW CAN A SYSTEM BE
TWEAKED FOR THE DESIRED ACTIONS!
Could we engage with users (targeted) with extreme
polarity leaning for Obama to spark an agenda in the whole
network of voters (ACTION)?
19

20. Twitris: Community Evolution
SMART DATA FOCUSES ON THE CAUSALITY
OF CHANGES IN REAL-WORLD ACTIONS!
Romney
Obama
Evolution of influencer interaction networks for Romney vs. Obama
topical communities, during U.S. Presidential Election 2012 debates
Before 1st
debate
After 1st
debate
After
Hurricane Sandy
After 3rd
debate
20

21. How People from Different
parts of the world talked
about US Election
Images and Videos
Related to US Election
Twitris: Analysis by Location
21

22. What is Smart Data in the context of
Disaster Management
ACTIONABLE: Timely delivery of
right resources and information to
the right people at right location!
22
Because everyone wants to Help, but DON’T KNOW HOW!

23. 23
Emergency Management Coordination

24. • Illustrious scenario: #Oklahoma-tornado 2013
24
Disaster Response Coordination:
Anecdote for the value of Smart Data
FEMA asked us to quickly filter
out gas-leak related data
Mining the data for smart nuggets
to inform FEMA (Timely needs)
Engaged with the author of this
information to confirm (Veracity)
e.g., All gas leaks in #moore were capped and stopped by
11:30 last night (at 5/22/2013 1:41:37)
Lot of tweets for ‘how to/where to’ assist (‘pseudo’ responders)
e.g., I want to go to Oklahoma this weekend & do what i can to help those people with
food,cloths & supplies,im in the feel of wanting to help ! :)

25. Really sparse Signal to Noise:
• 2M tweets during the first 48 hrs. of #Oklahoma-tornado-2013
- 1.3% as the precise resource donation requests to help
- 0.02% as the precise resource donation offers to help
25
• Anyone know how to get involved to
help the tornado victims in
Oklahoma??#tornado #oklahomacity
(OFFER)
• I want to donate to the Oklahoma cause
shoes clothes even food if I can (OFFER)
Disaster Response Coordination:
Finding Actionable Nuggets for Responders to act
• Text REDCROSS to 909-99 to donate to
those impacted by the Moore tornado!
http://t.co/oQMljkicPs (REQUEST)
• Please donate to Oklahoma disaster
relief efforts.: http://t.co/crRvLAaHtk
(REQUEST)
For responders, most important information is the scarcity and
availability of resources
Blog by our colleague Patrick Meier on this analysis: http://irevolution.net/2013/05/29/analyzing-tweets-tornado/

26. RT @OpOKRelief:
Southgate Baptist Church
on 4th Street in Moore
has food, water, clothes,
diapers, toys, and more.
If you can't go,call 794
Text "FOOD" to
32333, REDCROSS to
90999, or STORM to
80888 to donate $10
in storm relief.
#moore #oklahoma
#disasterrelief
#donate
Want to help animals in
#Oklahoma? @ASPCA tells
how you can help:
http://t.co/mt8l9PwzmO
CITIZEN SENSORS
RESPONSE TEAMS
(including humanitarian
org. and ‘pseudo’ responders)
VICTIM SITE
Coordination of
needs and offers
Using Social Media
Does anyone
know where to
send a check to
donate to the
tornado
victims?
Where do I go
to help out for
volunteer work
around Moore?
Anyone know?
Anyone know
where to donate
to help the
animals from the
Oklahoma
disaster? #oklah
oma #dogs
Matched
Matched
Matched
Serving the need!
If you would like to volunteer
today, help is desperately
needed in Shawnee. Call
273-5331 for more info
http://www.slideshare.net/knoesis/iccm-2013ignitetalkhemantpurohitunnairobi 26Purohit et al. Emergency-relief coordination on social media: Automatically matching resource requests and offers, 2014. With Int’l collaborator QCRI

27. 27
Disaster Response Coordination:
Twitris Real-time information for needs
Incoming Tweets with need
types to give quick idea of
what is needed and where
currently #OKC
Legends for Different
needs #OKC
(It is real-time widget for monitoring of needs, so will not be active after the event has passed)
http://twitris.knoesis.org/oklahomatornado

28. 28
Disaster Response Coordination:
Influencers to engage with for specific needs
Influential users are respective
needs and their interaction
network on the right.

29. 29
Disaster Response Coordination:
Engagement Interface for responders
What-Where-How-Who-Why
Coordination
Influential users to engage
with and resources for
seekers/supplies at a location,
at a timestamp
Contextual
Information for a
chosen topical tags

30. 30
Google Crisis Map for Cyclone Phalin, which
was used by planners and responders on the
ground. The system used data from
international participants spearheaded by
Twitris team at the Kno.e.sis center.
Details: http://www.thehindu.com/sci-tech/technology/gadgets/using-
crisis-mapping-to-aid-uttarakhand/article4854027.ece.

31. 31
Snapshot of Twitris platform based simulation tool for filtering the social
stream during functional exercise of emergency response teams in
Dayton on the May 28, 2014. Data used for this simulation was based on
repurposing of 2013 Boston Bombing given the focus on man-made
disasters with Urban focus. The tool provides filtering via search (top
left), evolving topics (left pane), and by location (middle pane) for the
intractable real-time stream (right pane). The local disaster management
and response officials found this to be a highly valuable tool for
training/exercise and planning.

32. 32
Use of a Twitris component by Digital Volunteers during JK Floods.
Digital volunteers use social media, come together during Jammu floods:
http://www.oneindia.com/india/digital-volunteers-use-social-media-for-rescue-efforts-during-jammu-floods-1524180.html

33. Dynamic Model Creation
Continuous Semantics
33

34. Dynamic Model Creation:
34
Example of how background knowledge help
understand situation described in the tweets, while
also updating knowledge model also

35. Semantic Approach to
Big Data and Event Processing
Thank you!
Any Question?