1

2011
How much data?
48
(2013)
500
(2013)
2http://www.knowledgeinfusion.com/blog/2011/11/get-your-head-out-of-the-clouds-and-into-big-data/

1% of the data is
used for analysis.
3
http://www.csc.com/insights/flxwd/78931-big_data_growth_just_beginning_to_explode
http://www.guardian.co.uk/news/datablog/2012/dec/19/big-data-study-digital-universe-global-volume

Variety
Semi structured
4

Velocity
Fast Data
Rapid Changes
Real-Time/Stream Analysis
Current application examples: financial services, stock brokerage, weather tracking, movies/entertainment and online retail 5

• Focus on verticals: advertising‚ social media‚ retail‚
financial services‚ telecom‚ and healthcare
– Aggregate data, focused on transactions, limited
integration (limited complexity), analytics to find
(simple) patterns
– Emphasis on technologies to handle
volume/scale, and to lesser extent velocity:
Hadoop, NoSQL,MPP warehouse ….
– Full faith in the power of data (no
hypothesis), bottom up analysis
6
Current Focus on Big Data

• What if your data volume gets so large and
varied you don't know how to deal with it?
• Do you store all your data?
• Do you analyze it all?
• How can you find out which data points are
really important?
• How can you use it to your best advantage?
7
Questions typically asked on Big Data
http://www.sas.com/big-data/

http://techcrunch.com/2012/10/27/big-data-right-now-five-trendy-open-source-technologies/
Variety of Data Analytics Enablers
8

• Prediction of the spread of flu in real time during H1N1 2009
– Google tested a mammoth of 450 million different mathematical
models to test the search terms, comparing their predictions against
the actual flu cases; 45 important parameters were founds
– Model was tested when H1N1 crisis struck in 2009 and gave more
meaningful and valuable real time information than any public health
official system [Big Data, Viktor Mayer-Schonberger and Kenneth Cukier, 2013]
• FareCast: predict the direction of air fares over different
routes [Big Data, Viktor Mayer-Schonberger and Kenneth Cukier, 2013]
• NY city manholes problem [ICML Discussion, 2012]
9
Illustrative Big Data Applications

• Current focus mainly to serve business intelligence and targeted
analytics needs, not to serve complex individual and collective
human needs (e.g., empower human in health, fitness and well-
being; better disaster coordination) that is highly
personalized/individualized/contextualized
– Incorporate real-world complexity: multi-modal and multi-sensory nature
of real-world and human perception
– Need deeper understanding of data and its role to information (e.g., skew,
coverage)
• Human involvement and guidance: Leading to actionable
information, understanding and insight right in the context of
human activities
– Bottom-up & Top-down processing: Infusion of models and background
knowledge (data + knowledge + reasoning)
10
What is missing?

Makes Sense
Actionable or help decision support/making
11

Smart Data
Smart data makes sense out of Big data
It provides value from harnessing the
challenges posed by
volume, velocity, variety and veracity of big
data, in-turn providing actionable
information and improve decision
making.
12

“OF human, BY human and FOR human”
Smart data is focused on the actionable
value achieved by human involvement in
data creation, processing and consumption
phases for improving
the human experience.
Another perspective on Smart Data
13

Descriptive
Exploratory
Inferential
Predictive
Causal
Improved
Analytics CREATION
PROCESSING
EXPERIENCE
& DECISION
MAKING
14
Human Centric Computing

“OF human, BY human and FOR human”
Another perspective on Smart Data
15

Petabytes of Physical(sensory)-Cyber-Social Data everyday!
More on PCS Computing: http://wiki.knoesis.org/index.php/PCS 16
‘OF human’ : Relevant Real-time Data
Streams for Human Experience

“OF human, BY human and FOR human”
17
Another perspective on Smart Data

Use of Prior Human-created Knowledge Models
18
‘BY human’: Involving
Crowd Intelligence in data processing workflows
Crowdsourcing and Domain-expert guided
Machine Learning Modeling

“OF human, BY human and FOR human”
Another perspective on Smart Data
19

Detection of events, such as wheezing
sound, indoor
temperature, humidity, dust, and CO2
level
Weather Application
Asthma Healthcare
Application
Close the window at home
during day to avoid CO2 in
gush, to avoid asthma attacks
at night
20
‘FOR human’ :
Improving Human Experience
Population Level
Personal
Public Health
Action in the Physical World

21
Why do we care about Smart Data
rather than Big Data?

Transforming Big Data into Smart Data:
Deriving Value via harnessing Volume, Variety and Velocity
using semantics and Semantic Web
Put Knoesis Banner
Keynote at SEBD 2013, July 1, 2013 and invited talk in universities in Spain, June 2013.
The Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis), Wright State, USA
Pavan
Kapanipathi
Pramod
Anantharam
Amit Sheth
Cory
Henson
Dr. T.K.
Prasad
Maryam
Panahiazar
Contributions by many, but Special Thanks to:
Hemant
Purohit

Second-costliest hurricane in United States
history estimated damage $75 billion
90-115 mph winds
State of Emergency in New York
285 people killed on the track of Sandy
750,000 without power (NY)
Immense devastation and Human suffering
23
Big Data to Smart Data: Disaster Management example
http://www.huffingtonpost.com/2012/10/30/hurricane-sandy-power-outage-map-infographic_n_2044411.html

20 million tweets with “sandy, hurricane”
keywords between Oct 27th and Nov 1st
2nd most popular topic on Facebook during 2012
Social (Big) Data during Hurricane Sandy
24
• http://www.guardian.co.uk/news/datablog/2
012/oct/31/twitter-sandy-flooding
• http://www.huffingtonpost.com/2012/11/02
/twitter-hurricane-sandy_n_2066281.html
• http://mashable.com/2012/10/31/hurricane-
sandy-facebook/

For information seeking
For timely information
For unique information
For unfiltered information
To determine disaster magnitude
To check in with family and friends
To self-mobilize
To maintain a sense of community
To seek emotional support and healing
Governments
Emergency management
organizations
Journalists
Disaster responders
Public
BIG DATA TO SMART DATA: WHY? and FOR WHOM?
25
Fraustino et al. Social Media Use
during Disasters: A Review of the
Knowledge Base and Gaps. US Dept.
of Homeland Security, START 2012.

Improving situational awareness
- Timely delivery of necessary
information to the right people
Improving coordination between
resource seekers and suppliers
Detecting the magnitude of
disaster by people sentiments.
Many more challenges…
Can SNS’s make Disaster Management easier –
Giving Actionable Information (Smart Data)
26
http://www.buzzfeed.com/annanorth/how-social-media-is-aiding-the-hurricane-sandy-rec
http://blog.twitter.com/2012/10/hurricane-sandy-resources-on-twitter.html
http://www.treehugger.com/culture/12-ways-help-hurricane-sandy-relief-efforts.html

Volume
Twitter hits half a billion tweets a day!
Challenges
Delivering the necessary
actionable/information to the right people
27
http://news.cnet.com/8301-1023_3-57541566-93/report-twitter-hits-half-a-billion-tweets-a-day/
http://semiocast.com/en/publications/2012_07_30_Twitter_reaches_half_a_billion_accounts_140m_in_the_US

Velocity
Volume
@ConEdison Twitter handle that the company had only
set up in June gained an extra 16,000 followers over the
storm. – Did the information reach everyone?
Challenges
Delivering the necessary/actionable
information to the right people
Rate of Data Arrival
Approximately 7000 TPS
10 images per second on instagram
28
http://news.cnet.com/8301-1023_3-57541566-93/report-twitter-hits-half-a-billion-tweets-a-day/
http://semiocast.com/en/publications/2012_07_30_Twitter_reaches_half_a_billion_accounts_140m_in_the_US
http://www.internews.org/sites/default/files/resources/InternewsEurope_Report_Japan_Connecting%20the%20last%20mile%20Japan_2013.pdf

http://news.cnet.com/8301-1023_3-57541566-93/report-twitter-hits-half-a-billion-
tweets-a-day/
http://semiocast.com/en/publications/2012_07_30_Twitter_reaches_half_a_billion
_accounts_140m_in_the_US
Velocity
Variety
Volume
Semi Structured
Structured
Unstructured
Sensors
Linked Open Data
Wikipedia
Challenges
Delivering the necessary/actionable
information to the right people
29

Velocity
Variety
Veracity
Volume
Challenges
Delivering the necessary/actionable
information to the right people
30http://www.buzzfeed.com/jackstuef/the-man-behind-comfortablysmug-hurricane-sandys

Velocity
Variety
Veracity
Volume
31

Value
-Makes Sense
-Actionable Information
-Decision support/making
Data http://www.wired.com/insights/2013/04/big-data-fast-data-smart-data/ 32
Smart Data
focuses on the
value

Value
-Makes Sense
-Actionable Information
-Decision support/making
Disaster Management
Victims
Timely and Contextual Information about
• Electricity, Food, Water, Shelter and
donation offers related to the disaster.
Data http://www.wired.com/insights/2013/04/big-data-fast-data-smart-data/ 33

Descriptive
Exploratory
Inferential
Predictive
Causal
Human Centric Computing
Improved
Analytics Creation
Processing
Experience
34
Revisiting..

• Healthcare
– kHealth
– SemHeath
• Social event coordination
– Twitris
• Traffic monitoring
– kTraffic
35
Applications of Smart Data Analytics

The Patient of the Future
MIT Technology Review, 2012
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/ 36

To gain new insight in
patient care &
early indications of
disease
37
Smart Data in Healthcare

Sensing is a key enabler of the Internet of Things
BUT, how do we make sense of the resulting avalanche
of sensor data?
50 Billion Things by 2020 (Cisco)
38

Parkinson’s disease (PD) data from The Michael J. Fox Foundation
for Parkinson’s Research.
39
1https://www.kaggle.com/c/predicting-parkinson-s-disease-progression-with-smartphone-data
8 weeks of data from 5 sensors on a smart phone, collected for 16 patients
resulting in ~12 GB (with lot of missing data).
Variety Volume
VeracityVelocity
Value
Can we detect the onset of Parkinson’s disease?
Can we characterize the disease progression?
Can we provide actionable information to the patient?
semantics
Representing prior knowledge of PD
led to a focused exploration of this
massive dataset
WHY Big Data to Smart Data: Healthcare example

40
Big Data to Smart Data Using a Knowledge Based Approach
ParkinsonMild(person) = Tremor(person) ∧ PoorBalance(person)
ParkinsonModerate(person) = MoveSlow(person) ∧ PoorSleep(person) ∧ MonotoneSpeech(person)
ParkinsonAdvanced(person) = Fall(person)
Control Group PD Patients
Movements of an active
person has a good
distribution over X, Y, and
Z axis
Restricted movements by
a PD patient can be seen
in the acceleration
readings
Audio is well modulated
with good variations in
the energy of the voice
Audio is not well
modulated represented a
monotone speech
Declarative Knowledge of
Parkinson’s Disease used to focus
our attention on symptom
manifestations in sensor
observations

• 25 million people in the U.S. are diagnosed with
asthma (7 million are children)1.
• 300 million people suffering from asthma
worldwide2.
• Asthma related healthcare costs alone are around
$50 billion a year2.
• 155,000 hospital admissions and 593,000 emergency
department visits in 20063.
41
1http://www.nhlbi.nih.gov/health/health-topics/topics/asthma/
2http://www.lung.org/lung-disease/asthma/resources/facts-and-figures/asthma-in-adults.html
3Akinbami et al. (2009). Status of childhood asthma in the United States, 1980–2007. Pediatrics,123(Supplement 3), S131-S145.
Asthma: Severity of the problem

Asthma is a multifactorial disease with health signals spanning
personal, public health, and population levels.
42
Real-time health signals from personal level (e.g., Wheezometer, NO in
breath, accelerometer, microphone), public health (e.g., CDC, Hospital EMR), and
population level (e.g., pollen level, CO2) arriving continuously in fine grained
samples potentially with missing information and uneven sampling frequencies.
Variety Volume
VeracityVelocity
Value
Can we detect the asthma severity level?
Can we characterize asthma control level?
What risk factors influence asthma control?
What is the contribution of each risk factor?semantics
Understanding relationships between
health signals and asthma attacks
for providing actionable information
WHY Big Data to Smart Data: Healthcare example

43
Population Level
Personal
Public Health
Variety: Health signals span heterogeneous sources
Volume: Health signals are fine grained
Velocity: Real-time change in situations
Veracity: Reliability of health signals may be compromised
Value: Can I reduce my asthma attacks at night?
Decision support to doctors
by providing them with
deeper insights into patient
asthma care
Asthma: Demonstration of Value

44
Sensordrone – for monitoring
environmental air quality
Wheezometer – for monitoring
wheezing sounds
Can I reduce my asthma attacks at night?
What are the triggers?
What is the wheezing level?
What is the propensity toward asthma?
What is the exposure level over a day?
What is the air quality indoors?
Commute to Work
Personal
Public Health
Population Level
Closing the window at home
in the morning and taking an
alternate route to office may
lead to reduced asthma attacks
Actionable
Information
Asthma: Actionable Information for Asthma Patients

Personal, Public Health, and Population Level Signals for Monitoring Asthma
Asthma Control => Daily Medication
Choices for starting
therapy
Not Well Controlled Poor Controlled
Severity Level of
Asthma
(Recommended Action) (Recommended Action) (Recommended Action)
Intermittent Asthma SABA prn - -
Mild Persistent Asthma Low dose ICS Medium ICS Medium ICS
Moderate Persistent
Asthma
Medium dose ICS alone
Or with
LABA/montelukast
Medium ICS +
LABA/Montelukast
Or High dose ICS
Medium ICS +
LABA/Montelukast
Or High dose ICS*
Severe Persistent Asthma High dose ICS with
LABA/montelukast
Needs specialist care Needs specialist care
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ;
*consider referral to specialist
Asthma Control
and Actionable Information
Sensors and their observations
for understanding asthma
45

46
Personal
Level Signals
Societal Level
Signals
(Personal Level Signals)
(Personalized
Societal Level Signal)
(Societal Level Signals)
Societal Level Signals
Relevant to the
Personal Level
Personal Level Sensors
(kHealth**) (EventShop*)
Qualify Quantify
Action
Recommendation
What are the features influencing my asthma?
What is the contribution of each of these features?
How controlled is my asthma? (risk score)
What will be my action plan to manage asthma?
Storage
Societal Level Sensors
Asthma Early Warning Model (AEWM)
Query AEWM
Verify & augment
domain knowledge
Recommended
Action
Action
Justification
Asthma Early Warning Model
*http://www.slideshare.net/jain49/eventshop-120721, ** http://www.youtube.com/watch?v=btnRi64hJp4

47
Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,
Activity, Wheezing, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert
Knowledge
Background
Knowledge
tweet reporting pollution level
and asthma attacks
Acceleration readings from
on-phone sensors
Sensor and personal
observations
Signals from personal, personal
spaces, and community spaces
Risk Category assigned by
doctors
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Health Signal Extraction to Understanding
Well Controlled - continue
Not Well Controlled – contact nurse
Poor Controlled – contact doctor

… and do it efficiently and at scale
What if we could automate this
sense making ability?
48

People are good at making sense of sensory input
What can we learn from cognitive models of perception?
• The key ingredient is prior knowledge
49

* based on Neisser’s cognitive model of perception
Observe
Property
Perceive
Feature
Explanation
Discrimination
1
2
Perception Cycle*
Translating low-level signals
into high-level knowledge
Focusing attention on those
aspects of the environment that
provide useful information
Prior Knowledge
50

To enable machine perception,
Semantic Web technology is used to integrate
sensor data with prior knowledge on the Web
51

Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
52

Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
53

Observe
Property
Perceive
Feature
Explanation
1
Translating low-level signals
into high-level knowledge
Explanation
Explanation is the act of choosing the objects or events that best account for a
set of observations; often referred to as hypothesis building
54

Explanation
Inference to the best explanation
• In general, explanation is an abductive problem; and
hard to compute
Finding the sweet spot between abduction and OWL
• Single-feature assumption* enables use of OWL-DL
deductive reasoner
* An explanation must be a single feature which accounts for
all observed properties
Explanation is the act of choosing the objects or events that best account for a set of
observations; often referred to as hypothesis building
55

Explanation
Explanatory Feature: a feature that explains the set of observed properties
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{p1} ⊓ … ⊓ ∃ssn:isPropertyOf—.{pn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Observed Property Explanatory Feature
56

Discrimination is the act of finding those properties that, if observed, would help distinguish
between multiple explanatory features
Observe
Property
Perceive
Feature
Explanation
Discrimination
2
Focusing attention on those
aspects of the environment that
provide useful information
Discrimination
57

Discrimination
Expected Property: would be explained by every explanatory feature
ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Expected Property Explanatory Feature
58

Discrimination
Not Applicable Property: would not be explained by any explanatory feature
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ¬∃ssn:isPropertyOf.{fn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Not Applicable Property Explanatory Feature
59

Discrimination
Discriminating Property: is neither expected nor not-applicable
DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Discriminating Property Explanatory Feature
60

Through physical monitoring and
analysis, our cellphones could act as
an early warning system to detect
serious health conditions, and
provide actionable information
canary in a coal mine
Our Motivation
kHealth: knowledge-enabled healthcare
61

Qualities
-High BP
-Increased Weight
Entities
-Hypertension
-Hypothyroidism
kHealth
Machine Sensors
Personal Input
EMR/PHR
Comorbidity risk score
e.g., Charlson Index
Longitudinal studies of
cardiovascular risks
- Find correlations
- Validation
- domain knowledge
- domain expert
Parameterize the
model
Risk Assessment Model
Current Observations
-Physical
-Physiological
-History
Risk Score
(Actionable Information)
Model CreationValidate correlations
Historical observations
of each patient
Risk Score: from Data to Abstraction and Actionable Information
62

How do we implement machine perception efficiently on a
resource-constrained device?
Use of OWL reasoner is resource intensive
(especially on resource-constrained devices),
in terms of both memory and time
• Runs out of resources with prior knowledge >> 15 nodes
• Asymptotic complexity: O(n3)
63

intelligence at the edge
Approach 1: Send all sensor observations
to the cloud for processing
Approach 2: downscale semantic
processing so that each device is capable
of machine perception
64
Henson et al. 'An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained
Devices, ISWC 2012.

Efficient execution of machine perception
Use bit vector encodings and their operations to encode prior knowledge and
execute semantic reasoning
010110001101
0011110010101
1000110110110
101100011010
0111100101011
000110101100
0110100111
65

O(n3) < x < O(n4) O(n)
Efficiency Improvement
• Problem size increased from 10’s to 1000’s of nodes
• Time reduced from minutes to milliseconds
• Complexity growth reduced from polynomial to linear
Evaluation on a mobile device
66

2 Prior knowledge is the key to perception
Using SW technologies, machine perception can be formalized and
integrated with prior knowledge on the Web
3 Intelligence at the edge
By downscaling semantic inference, machine perception can
execute efficiently on resource-constrained devices
Semantic Perception for smarter analytics: 3 ideas to takeaway
1 Translate low-level data to high-level knowledge
Machine perception can be used to convert low-level sensory
signals into high-level knowledge useful for decision making
67

• Real Time Feature Streams:
http://www.youtube.com/watch?v=_ews4w_eCpg
• kHealth: http://www.youtube.com/watch?v=btnRi64hJp4
68
Demos

73
Smart Data in Social Media Analytics
To Understand the
human social
dynamics in real
world events

0.5B Tweets per day
0.5B Users
60% on Mobile
5530 Tweets per second
related to the Japan earthquake and tsunami
17000 Tweets
per second
74
Twitter During Real-world Events of Interest
http://www.flickr.com/photos/twitteroffice/5897088517/sizes/o/in/photostream/
http://bayarea.sbnation.com/49ers/2013/2/3/3947738/super-bowl-prop-bets-2013-
twitterhttp://bayarea.sbnation.com/49ers/2013/2/3/3947738/super-bowl-prop-bets-2013-twitter
http://expandedramblings.com/index.php/march-2013-by-the-numbers-a-few-amazing-twitter-stats/

75http://usatoday30.usatoday.com/news/politics/twitter-election-meter
http://twitris.knoesis.org/

State of the Art – Uni/Bi Dimensional Analysis During Elections
Topics
Sentiments
76

Twitris’ Dimensions of Integrated Semantic Analysis
77Sheth et al. Twitris- a System for Collective Social Intelligence, ESNAM-2013

78
http://semanticweb.com/picking-the-president-twindex-twitris-track-social-media-electorate_b31249
http://semanticweb.com/election-2012-the-semantic-recap_b33278

79
[The screenshots of Twitris+ were taken on Nov. 6th 6 PM EST]
/t

80
Twitris: Sentiment Analysis- Smart Answers with reasoning!
How was Obama doing in the first debate?

81
Red Color: Negative Topics
Green Color: Positive Topics
Twitris: Sentiment Analysis- Smart Answers with reasoning!
How was Obama doing in the second debate?
SMART DATA IS ABOUT ANALYSIS FOR REASONING
(what caused the positive sentiment for Democrats)
BEHIND THE REAL-WORLD ACTIONS (Democrats’ win)
http://knoesis.wright.edu/library/resource.php?id=1787

Top 100 influential users that
talks about Barack Obama
Positive or Negative
Influence
Twitris: Network Analysis
SMART DATA TELLS YOU HOW CAN A SYSTEM BE
TWEAKED FOR THE DESIRED ACTIONS!
Could we engage with users (targeted) with extreme
polarity leaning for Obama to spark an agenda in the whole
network of voters (ACTION)? 82

Twitris: Community Evolution
SMART DATA FOCUSES ON THE CAUSALITY
OF CHANGES IN REAL-WORLD ACTIONS!
Romney
Obama
Evolution of influencer interaction networks for Romney vs. Obama
topical communities, during U.S. Presidential Election 2012 debates
Before 1st
debate
After 1st
debate
After
Hurricane Sandy
After 3rd
debate
83

The Dead People mentioned
in the event OWC
Twitris: Impact of Background Knowledge
84

How People from Different
parts of the world talked
about US Election
Images and Videos
Related to US Election
Twitris: Analysis by Location
85

What is Smart Data in the context of
Disaster Management
ACTIONABLE: Timely delivery of
right resources and information to
the right people at right location!
86
Because everyone wants to Help, but DON’T KNOW HOW!

Join us for the Social
Good!
http://twitris.knoesis.org
RT @OpOKRelief:
Southgate Baptist Church
on 4th Street in Moore
has
food, water, clothes, diap
ers, toys, and more. If
you can't go,call 794
Text "FOOD" to
32333, REDCROSS to
90999, or STORM to
80888 to donate $10
in storm relief.
#moore #oklahoma
#disasterrelief
#donate
Want to help animals in
#Oklahoma? @ASPCA tells
how you can help:
http://t.co/mt8l9PwzmO
CITIZEN SENSORS
RESPONSE TEAMS
(including humanitarian
org. and ‘pseudo’ responders)
VICTIM SITE
Coordination of
needs and offers
Using Social Media
Does anyone
know where to
send a check to
donate to the
tornado
victims?
Where do I go
to help out for
volunteer work
around Moore?
Anyone know?
Anyone know
where to donate
to help the
animals from the
Oklahoma
disaster? #oklah
oma #dogs
Matched
Matched
Matched
Serving the need!
If you would like to volunteer
today, help is desperately
needed in Shawnee. Call
273-5331 for more info
http://www.slideshare.net/hemant_knoesis/cscw-2012-hemantpurohit-11531612
87
Purohit et al. Framework to Analyze Coordination in Crisis Response, 2012. Int’l Collaboration in-progress:

Smart Data from Twitris system for
Disaster Response Coordination
Which are the primary locations with
most negative sentiments/emotions?
Who are all the people to engage
with for better information
diffusion?Which are the most important
organizations acting at my
location?
Smart data provides actionable information and improve decision making through
semantic analysis of Big Data.
Who are the resource seekers and
suppliers? How can one donate?
88

Source: Purohit et. al 2013, Information Filtering and Management Model for Disaster Response Coordination 89
Disaster Response Coordination Framework

Disaster Response Coordination:
Twitris Summary for Actionable Nuggets
90
Important tags to
summarize Big Data flow
Related to Oklahoma
tornado
Images and Videos Related
to Oklahoma tornado

91
Disaster Response Coordination:
Twitris Real-time information for needs
Incoming Tweets with need
types to give quick idea of
what is needed and where
currently #OKC
Legends for Different
needs #OKC
(It is real-time widget for monitoring of needs, so will not be active after the event has passed)
http://twitris.knoesis.org/oklahomatornado

92
Disaster Response Coordination:
Influencers to engage with for specific needs
Influential users are respective
needs and their interaction
network on the right.

Really sparse Signal to Noise:
• 2M tweets during the first week after #Oklahoma-tornado-2013
- 1.3% as the highly precise donation requests to help
- 0.02% as the highly precise donation offers to help
93
• Anyone know how to get involved to
help the tornado victims in
Oklahoma??#tornado #oklahomacity
(OFFER)
• I want to donate to the Oklahoma cause
shoes clothes even food if I can (OFFER)
Disaster Response Coordination:
Finding Actionable Nuggets for Responders to act
• Text REDCROSS to 909-99 to donate to
those impacted by the Moore tornado!
http://t.co/oQMljkicPs (REQUEST)
• Please donate to Oklahoma disaster
relief efforts.: http://t.co/crRvLAaHtk
(REQUEST)
For responders, most important information is the scarcity and
availability of resources, can we mine it via Social Media?

• Features driven by the experience of domain experts at the
responder organizations
• Examples,
– ‘I want to <donate/ help/ bring>’ for extraction of offering
intention
– ‘tent house’ OR ‘cots’ for shelter need types
94
Disaster Response Coordination:
Human Knowledge to drive information extraction

• A knowledge-driven approach
– A rich inventory of metadata for tweets
– Semantic matching for
needs (query) vs. offers (documents)
• Example,
– @bladesofmilford please help get the word out,we are accepting kid clothes to send
to the lil angels in Oklahoma.Drop off @MilfordGreenPiz (REQUEST)
– I want to donate to the Oklahoma cause shoes clothes even food if I can (OFFER)
95
Disaster Response Coordination:
Automatic Matching of needs and offers
Matching the
competitive intentions
(Needs and Offers) can
offload humans for the
task of resource
matchmaking for
coordination.

96
Disaster Response Coordination:
Engagement Interface for responders
What-Where-How-Who-Why
Coordination
Influential users to engage
with and resources for
seekers/supplies at a
location, at a timestamp
Contextual
Information for a
chosen topical tags

• Illustrious scenario: #Oklahoma-tornado 2013
97
Disaster Response Coordination:
Anecdote for the value of Smart Data
FEMA asked us to quickly filter
out gas-leak related data
Mining the data for smart nuggets
to inform FEMA (Timely needs)
Engaged with the author of this
information to confirm (Veracity)
e.g., All gas leaks in #moore were capped and stopped by
11:30 last night (at 5/22/2013 1:41:37)
Lot of tweets for ‘how to/where to’ assist (‘pseudo’ responders)
e.g., I want to go to Oklahoma this weekend & do what i can to help those people with
food,cloths & supplies,im in the feel of wanting to help ! :)

An event is a dynamic topic that evolves and
might later fork into several distinct events.
Smart Data analytics to capture rapidly evolving social data events
98
Social Media is the pulse of the
populace, a true reflection of
events all over the globe!

Continuous Semantics
99

Dynamic Model Creation
Continuous Semantics 100

Dynamic Model Creation:
101
Example of how background knowledge help
understand situation described in the tweets, while
also updating knowledge model also

How is Continuous Semantics a form of
Smart Data Analytics?
Keeping the Background Knowledge
abreast with the changes of the event
Smartly learning and adapting data acquisition
(Temporally apt Big Data, i.e. Fast Data)
In-turn providing temporally relevant
Smart Data through analysis
102

103
Smart Data Analytics in Traffic Management
To improve the
everyday life
entangled due
to our most
common
problem of
sticking in
traffic

By 2001 over 285 million Indians lived in cities, more than in all
North American cities combined (Office of the Registrar General of India 2001)1
1The Crisis of Public Transport in India
2IBM Smarter Traffic
Modes of transportation in Indian Cities
Texas Transportation Institute (TTI)
Congestion report in U.S.
104
Severity of the Traffic Problem

Vehicular traffic data from San Francisco Bay Area aggregated from on-road
sensors (numerical) and incident reports (textual)
105
http://511.org/
Every minute update of speed, volume, travel time, and occupancy resulting in
178 million link status observations, 738 active events, and 146 scheduled
events with many unevenly sampled observations collected over 3 months.
Variety Volume
VeracityVelocity
Value
Can we detect the onset of traffic congestion?
Can we characterize traffic congestion based on events?
Can we provide actionable information to decision makers?
semantics
Representing prior knowledge of
traffic lead to a focused exploration
of this massive dataset
Big Data to Smart Data: Traffic Management example

Slow moving
traffic
Link
Description
Scheduled
Event
Scheduled
Event
511.org
511.org
Schedule Information
511.org
Traffic Monitoring
106
Heterogeneity in a Physical-Cyber-Social System

107
Heterogeneity in a Physical-Cyber-Social System

• Observation: Slow Moving Traffic
• Multiple Causes (Uncertain about the cause):
– Scheduled Events: music events, fair, theatre events, concerts, road
work, repairs, etc.
– Active Events: accidents, disabled vehicles, break down of
roads/bridges, fire, bad weather, etc.
– Peak hour: e.g. 7 am – 9 am OR 4 pm – 6 pm
• Each of these events may have a varying impact on traffic.
• A delay prediction algorithm should process multimodal and
multi-sensory observations.
Uncertainty in a Physical-Cyber-Social System
108

• Internal observations
– Speed, volume, and travel time observations
– Correlations may exist between these variables
across different parts of the network
• External events
– Accident, music event, sporting event, and
planned events
– External events and internal observations may
exhibit correlations
Modeling Traffic Events
109

Accident
Music event
Sporting event
Road Work
Theatre event
External events
<ActiveEvents, ScheduledEvents>
Internal observations
<speed, volume, traveTime>
Weather
Time of Day
Modeling Traffic Events
110

Domain Experts
cold
PoorVisibility
SlowTraffic
IcyRoad
Declarative domain knowledge
Causal
knowledge
Linked Open Data
Cold (YES/NO) IcyRoad (ON/OFF) PoorVisibility (YES/NO) SlowTraffic (YES/NO)
1 0 1 1
1 1 1 0
1 1 1 1
1 0 1 0
Domain Observations
Domain Knowledge
Structure and parameters
Complementing Probabilistic Models with Declarative Knowledge
112
Correlations to causations using
Declarative knowledge on the
Semantic Web

• Declarative knowledge about various domains
are increasingly being published on the web1,2.
• Declarative knowledge describes concepts and
relationships in a domain (structure).
• Linked Open Data may be used to derive
priors probability of events (parameters).
• Explored the use declarative knowledge for
structure using ConceptNet 5.
1http://conceptnet5.media.mit.edu/
2http://linkeddata.org/
Domain Knowledge
113

http://conceptnet5.media.mit.edu/web/c/en/traffic_jam
Delay
go to baseball game
traffic jam
traffic accident
traffic jam
ActiveEvent
ScheduledEvent
Causes
traffic jam
Causes
traffic jam
CapableOf
slow traffic
CapableOf
occur twice each day
Causes
is_a
bad weather
CapableOf
slow traffic
road ice
Causes
accident
TimeOfDay
go to concert
HasSubevent
car crash
accident
RelatedTo
car crash
BadWeather
Causes
Causes
is_a
is_a
is_a is_a is_a
is_a
is_a
ConceptNet 5
114

Traffic jam
Link
Description
Scheduled
Event
traffic jambaseball game
Add missing random variables
Time of day
bad weather CapableOf slow traffic
bad weather
Traffic data from sensors deployed on road
network in San Francisco Bay Area
time of day
traffic jambaseball game
time of day
slow traffic
Three Operations: Complementing graphical model structure extraction
Add missing links bad weather
traffic jambaseball game
time of day
slow traffic
Add link direction
bad weather
traffic jambaseball game
time of day
slow traffic
go to baseball game Causes traffic jam
Knowledge from ConceptNet5
traffic jam CapableOfoccur twice each day
traffic jam CapableOf slow traffic
115

116
Scheduled Event
Active Event
Day of week Time of day
delay
Travel time
speed
volume
Structure extracted form
traffic observations
(sensors + textual) using
statistical techniques
Scheduled Event
Active Event
Day of week
Time of day
delayTravel time
speed
volume
Bad Weather
Enriched structure which has
link directions and new nodes
such as “Bad Weather”
potentially leading to better
delay predictions
Enriched Probabilistic Models using ConceptNet 5

Take Away
• It is all about the human – not computing, not
device
– Computing for human experience
• Whatever we do in Smart Data, focus on human-
in-the-loop (empowering machine computing!):
– Of Human, By Human, For Human
– But in serving human needs, there is a lot more than
what current big data analytics handle –
variety, contextual, personalized, subjective, spanning
data and knowledge across P-C-S dimensions
118

Acknowledgements
• Kno.e.sis team
• Funds: NSF, NIH, AFRL, Industry…
• Note:
• For images and sources, if not on slides, please see slide notes
• Some images were taken from the Web Search results and all such images belong
to their respective owners, we are grateful to the owners for usefulness of these
images in our context.
119

• OpenSource: http://knoesis.org/opensource
• Showcase: http://knoesis.org/showcase
• Vision: http://knoesis.org/node/266
• Publications: http://knoesis.org/library
120
References and Further Readings

Thanks …
121

122
Physical Cyber Social Computing
Amit Sheth, Kno.e.sis, Wright State

Amit Sheth’s
PHD students
Ashutosh Jadhav
Hemant
Purohit
Vinh
Nguyen
Lu Chen
Pavan
Kapanipathi
Pramod
Anantharam
Sujan
Perera
Alan Smith
Pramod Koneru
Maryam Panahiazar
Sarasi Lalithsena
Cory Henson
Kalpa
Gunaratna
Delroy
Cameron
Sanjaya
Wijeratne
Wenbo
Wang
Kno.e.sis in 2012 = ~100 researchers (15 faculty, ~50 PhD students)

124
thank you, and please visit us at
http://knoesis.org
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, Ohio, USA
Smart Data