Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Semantic Approach to
Big Data and Event Processing
Examples of Real-World Big Data Application
Specific examples of veloci...
2
Smart Data for Social Good
Mining human behavior to help
societal and humanitarian
development
• crisis response coordin...
20 million tweets with “sandy, hurricane”
keywords between Oct 27th and Nov 1st
2nd most popular topic on Facebook during ...
0.5B Tweets per day
0.5B Users
60% on Mobile
5530 Tweets per second
related to the Japan earthquake and tsunami
17000 Twee...
5http://usatoday30.usatoday.com/news/politics/twitter-election-meter
http://twitris.knoesis.org/
Twitris’ Dimensions of Integrated Semantic Analysis
6Sheth et al. Twitris- a System for Collective Social Intelligence, ES...
Semantic Approach to
Big Data and Event Processing
Metadata Extraction from
Informal Text
7
Meena Nagarajan, ‘Understandin...
8
Characteristics of Text on Social Media
9
The Formality of Text
“I loved your music Yesterday!”
Yesterday is an album
“It was THE HANGOVER of the year..lasted
forever..
The Hangover is n...
11
Cultural NER - Two Flavors
12
(a) Multiple Senses in the Same Domain
Scoped Relationship graphs
– Using context cues from the
content, webpage title, url…
e.g. new Merry Christmas tune
– Redu...
Career Restrictions
- “release your third album already..”
Recent Album restrictions
- “I loved your new album..”
Artist a...
Observations:
• The opinion clues may not be toward the given target
(1,2,3,6)
• The opinion clues are domain and context ...
16
[The screenshots of Twitris+ were taken on Nov. 6th 6 PM EST]
/t
17
Twitris: Sentiment Analysis- Smart Answers with reasoning!
How was Obama doing in the first debate?
18
Red Color: Negative Topics
Green Color: Positive Topics
Twitris: Sentiment Analysis- Smart Answers with reasoning!
How ...
Top 100 influential users that
talks about Barack Obama
Positive or Negative
Influence
Twitris: Network Analysis
SMART DAT...
Twitris: Community Evolution
SMART DATA FOCUSES ON THE CAUSALITY
OF CHANGES IN REAL-WORLD ACTIONS!
Romney
Obama
Evolution ...
How People from Different
parts of the world talked
about US Election
Images and Videos
Related to US Election
Twitris: An...
What is Smart Data in the context of
Disaster Management
ACTIONABLE: Timely delivery of
right resources and information to...
23
Emergency Management Coordination
• Illustrious scenario: #Oklahoma-tornado 2013
24
Disaster Response Coordination:
Anecdote for the value of Smart Data
FEM...
Really sparse Signal to Noise:
• 2M tweets during the first 48 hrs. of #Oklahoma-tornado-2013
- 1.3% as the precise resour...
RT @OpOKRelief:
Southgate Baptist Church
on 4th Street in Moore
has food, water, clothes,
diapers, toys, and more.
If you ...
27
Disaster Response Coordination:
Twitris Real-time information for needs
Incoming Tweets with need
types to give quick i...
28
Disaster Response Coordination:
Influencers to engage with for specific needs
Influential users are respective
needs an...
29
Disaster Response Coordination:
Engagement Interface for responders
What-Where-How-Who-Why
Coordination
Influential use...
30
Google Crisis Map for Cyclone Phalin, which
was used by planners and responders on the
ground. The system used data fro...
31
Snapshot of Twitris platform based simulation tool for filtering the social
stream during functional exercise of emerge...
32
Use of a Twitris component by Digital Volunteers during JK Floods.
Digital volunteers use social media, come together d...
Dynamic Model Creation
Continuous Semantics
33
Dynamic Model Creation:
34
Example of how background knowledge help
understand situation described in the tweets, while
al...
Semantic Approach to
Big Data and Event Processing
Thank you!
Any Question?
Upcoming SlideShare
Loading in …5
×

Examples of Real-World Big Data Application

542 views

Published on

Examples of Real-World Big Data Application Specific examples of velocity challenge and how it is addressed in disaster coordination scenario (e.g., Jammu&Kashmir Floods).
Prof Amit Sheth - Kno.e.sis

Published in: Data & Analytics
  • Be the first to comment

Examples of Real-World Big Data Application

  1. 1. Semantic Approach to Big Data and Event Processing Examples of Real-World Big Data Application Specific examples of velocity challenge and how it is addressed in disaster coordination scenario (e.g., Jammu&Kashmir Floods). Prof. Amit Sheth Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis) Wright State University, USA Tutorial @ Kno.e.sis Centre: Semantics Approach to Big Data and Event Processing, Oct 7-9, 2015
  2. 2. 2 Smart Data for Social Good Mining human behavior to help societal and humanitarian development • crisis response coordination, harassment, gender-based violence, human trafficking, prescription drug abuse…
  3. 3. 20 million tweets with “sandy, hurricane” keywords between Oct 27th and Nov 1st 2nd most popular topic on Facebook during 2012 Social (Big) Data during Crisis- Example of Hurricane Sandy 3 • http://www.guardian.co.uk/news/datablog/2 012/oct/31/twitter-sandy-flooding • http://www.huffingtonpost.com/2012/11/02 /twitter-hurricane-sandy_n_2066281.html • http://mashable.com/2012/10/31/hurricane- sandy-facebook/
  4. 4. 0.5B Tweets per day 0.5B Users 60% on Mobile 5530 Tweets per second related to the Japan earthquake and tsunami 17000 Tweets per second 4 Twitter During Real-world Events of Interest http://www.flickr.com/photos/twitteroffice/5897088517/sizes/o/in/photostream/ http://bayarea.sbnation.com/49ers/2013/2/3/3947738/super-bowl-prop-bets-2013- twitterhttp://bayarea.sbnation.com/49ers/2013/2/3/3947738/super-bowl-prop-bets-2013-twitter http://expandedramblings.com/index.php/march-2013-by-the-numbers-a-few-amazing-twitter-stats/
  5. 5. 5http://usatoday30.usatoday.com/news/politics/twitter-election-meter http://twitris.knoesis.org/
  6. 6. Twitris’ Dimensions of Integrated Semantic Analysis 6Sheth et al. Twitris- a System for Collective Social Intelligence, ESNAM-2014
  7. 7. Semantic Approach to Big Data and Event Processing Metadata Extraction from Informal Text 7 Meena Nagarajan, ‘Understanding User-Generated Content on Social Media,’ Ph.D. Dissertation, Wright State University, 2010
  8. 8. 8 Characteristics of Text on Social Media
  9. 9. 9 The Formality of Text
  10. 10. “I loved your music Yesterday!” Yesterday is an album “It was THE HANGOVER of the year..lasted forever.. The Hangover is not a movie So I went to the movies..badchoice picking “GI Jane”worse now” GI Jane is a movie 10 Named Entity Recognition: Identifying and classifying tokens
  11. 11. 11 Cultural NER - Two Flavors
  12. 12. 12 (a) Multiple Senses in the Same Domain
  13. 13. Scoped Relationship graphs – Using context cues from the content, webpage title, url… e.g. new Merry Christmas tune – Reduce potential entity spot size e.g. new albums/songs • Generate candidate entities • Spot and Disambiguate Which ‘Merry Christmas’?; ‘So Good’is also a song! 13 Approach Overview
  14. 14. Career Restrictions - “release your third album already..” Recent Album restrictions - “I loved your new album..” Artist age restrictions -”happy 25th rihanna, loved alfie btw..” etc. Which ‘Merry Christmas’?; ‘So Good’is also a song! 14 Sample Real-world Constraints
  15. 15. Observations: • The opinion clues may not be toward the given target (1,2,3,6) • The opinion clues are domain and context dependent (5,7) • Single words are not enough (4,7,8) Simple lexicon-based method doesn't work well. 15 Target of “sexy” is “Helena” Target of “terrific” is “reviews” “free” is not opinionated in movie domain. Target of “loving” is “telling” “well” in “as well” is not opinionated Sentiment Analysis: Target-specific Opinion Identification & Classification of Tweets-Unsupervised Approach
  16. 16. 16 [The screenshots of Twitris+ were taken on Nov. 6th 6 PM EST] /t
  17. 17. 17 Twitris: Sentiment Analysis- Smart Answers with reasoning! How was Obama doing in the first debate?
  18. 18. 18 Red Color: Negative Topics Green Color: Positive Topics Twitris: Sentiment Analysis- Smart Answers with reasoning! How was Obama doing in the second debate? SMART DATA IS ABOUT ANALYSIS FOR REASONING (what caused the positive sentiment for Democrats) BEHIND THE REAL-WORLD ACTIONS (Democrats’ win) http://knoesis.wright.edu/library/resource.php?id=1787
  19. 19. Top 100 influential users that talks about Barack Obama Positive or Negative Influence Twitris: Network Analysis SMART DATA TELLS YOU HOW CAN A SYSTEM BE TWEAKED FOR THE DESIRED ACTIONS! Could we engage with users (targeted) with extreme polarity leaning for Obama to spark an agenda in the whole network of voters (ACTION)? 19
  20. 20. Twitris: Community Evolution SMART DATA FOCUSES ON THE CAUSALITY OF CHANGES IN REAL-WORLD ACTIONS! Romney Obama Evolution of influencer interaction networks for Romney vs. Obama topical communities, during U.S. Presidential Election 2012 debates Before 1st debate After 1st debate After Hurricane Sandy After 3rd debate 20
  21. 21. How People from Different parts of the world talked about US Election Images and Videos Related to US Election Twitris: Analysis by Location 21
  22. 22. What is Smart Data in the context of Disaster Management ACTIONABLE: Timely delivery of right resources and information to the right people at right location! 22 Because everyone wants to Help, but DON’T KNOW HOW!
  23. 23. 23 Emergency Management Coordination
  24. 24. • Illustrious scenario: #Oklahoma-tornado 2013 24 Disaster Response Coordination: Anecdote for the value of Smart Data FEMA asked us to quickly filter out gas-leak related data Mining the data for smart nuggets to inform FEMA (Timely needs) Engaged with the author of this information to confirm (Veracity) e.g., All gas leaks in #moore were capped and stopped by 11:30 last night (at 5/22/2013 1:41:37) Lot of tweets for ‘how to/where to’ assist (‘pseudo’ responders) e.g., I want to go to Oklahoma this weekend & do what i can to help those people with food,cloths & supplies,im in the feel of wanting to help ! :)
  25. 25. Really sparse Signal to Noise: • 2M tweets during the first 48 hrs. of #Oklahoma-tornado-2013 - 1.3% as the precise resource donation requests to help - 0.02% as the precise resource donation offers to help 25 • Anyone know how to get involved to help the tornado victims in Oklahoma??#tornado #oklahomacity (OFFER) • I want to donate to the Oklahoma cause shoes clothes even food if I can (OFFER) Disaster Response Coordination: Finding Actionable Nuggets for Responders to act • Text REDCROSS to 909-99 to donate to those impacted by the Moore tornado! http://t.co/oQMljkicPs (REQUEST) • Please donate to Oklahoma disaster relief efforts.: http://t.co/crRvLAaHtk (REQUEST) For responders, most important information is the scarcity and availability of resources Blog by our colleague Patrick Meier on this analysis: http://irevolution.net/2013/05/29/analyzing-tweets-tornado/
  26. 26. RT @OpOKRelief: Southgate Baptist Church on 4th Street in Moore has food, water, clothes, diapers, toys, and more. If you can't go,call 794 Text "FOOD" to 32333, REDCROSS to 90999, or STORM to 80888 to donate $10 in storm relief. #moore #oklahoma #disasterrelief #donate Want to help animals in #Oklahoma? @ASPCA tells how you can help: http://t.co/mt8l9PwzmO CITIZEN SENSORS RESPONSE TEAMS (including humanitarian org. and ‘pseudo’ responders) VICTIM SITE Coordination of needs and offers Using Social Media Does anyone know where to send a check to donate to the tornado victims? Where do I go to help out for volunteer work around Moore? Anyone know? Anyone know where to donate to help the animals from the Oklahoma disaster? #oklah oma #dogs Matched Matched Matched Serving the need! If you would like to volunteer today, help is desperately needed in Shawnee. Call 273-5331 for more info http://www.slideshare.net/knoesis/iccm-2013ignitetalkhemantpurohitunnairobi 26Purohit et al. Emergency-relief coordination on social media: Automatically matching resource requests and offers, 2014. With Int’l collaborator QCRI
  27. 27. 27 Disaster Response Coordination: Twitris Real-time information for needs Incoming Tweets with need types to give quick idea of what is needed and where currently #OKC Legends for Different needs #OKC (It is real-time widget for monitoring of needs, so will not be active after the event has passed) http://twitris.knoesis.org/oklahomatornado
  28. 28. 28 Disaster Response Coordination: Influencers to engage with for specific needs Influential users are respective needs and their interaction network on the right.
  29. 29. 29 Disaster Response Coordination: Engagement Interface for responders What-Where-How-Who-Why Coordination Influential users to engage with and resources for seekers/supplies at a location, at a timestamp Contextual Information for a chosen topical tags
  30. 30. 30 Google Crisis Map for Cyclone Phalin, which was used by planners and responders on the ground. The system used data from international participants spearheaded by Twitris team at the Kno.e.sis center. Details: http://www.thehindu.com/sci-tech/technology/gadgets/using- crisis-mapping-to-aid-uttarakhand/article4854027.ece.
  31. 31. 31 Snapshot of Twitris platform based simulation tool for filtering the social stream during functional exercise of emergency response teams in Dayton on the May 28, 2014. Data used for this simulation was based on repurposing of 2013 Boston Bombing given the focus on man-made disasters with Urban focus. The tool provides filtering via search (top left), evolving topics (left pane), and by location (middle pane) for the intractable real-time stream (right pane). The local disaster management and response officials found this to be a highly valuable tool for training/exercise and planning.
  32. 32. 32 Use of a Twitris component by Digital Volunteers during JK Floods. Digital volunteers use social media, come together during Jammu floods: http://www.oneindia.com/india/digital-volunteers-use-social-media-for-rescue-efforts-during-jammu-floods-1524180.html
  33. 33. Dynamic Model Creation Continuous Semantics 33
  34. 34. Dynamic Model Creation: 34 Example of how background knowledge help understand situation described in the tweets, while also updating knowledge model also
  35. 35. Semantic Approach to Big Data and Event Processing Thank you! Any Question?

×