Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Amit Sheth
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing:
Wright State University, Dayton, Ohio
Sem...
2
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7433358
Credit: Looi Consulting (http://www.looiconsulting.com/home/enterprise-big-data/)
● In 2008, data generated > storage avai...
How would an enterprise get actionable information?
http://www.slideshare.net/NamrataChatterjee/nokias-supply-chain-manage...
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
The Patient of the Future
MIT Technology R...
Smart Data
I introduced the term in 2004; redefined in 2013:
http://wiki.knoesis.org/index.php/Smart_Data
● The astounding bandwidth of
your senses is 11 million bits of
information every second.
● In conscious activities like
r...
● How can we take inspiration from human brain and derive an intelligent
processing of big data
● Take inspiration from co...
http://www.research.ibm.com/cognitive-computing/brainpower/, https://newtonsapplevce.wordpress.com/2016/01/06/neuromorphic...
A Second Approach to Intelligence
Two Use Cases: Healthcare and Traffic
Prior Medical Knowledge
D1
D3
D1
Medical History and
Past observations
S1
S2
S3
Sn
..
..
D1
D2
D3
S1
S2
S3
..
..
..
..
..
...
Personalized Health and Objectives: one size does not fit all
Millions of people - > one treatment
Wearable and Sensor data
1Marcus, Philip, Kevin R. Murphy, Abid Rahman, and Christopher D. O’Brien. "Intrapatient symptom variability in
adults and...
ACTIONS
situation awareness useful
for decision making
ABSTRACTIONS
make sense to humans
KNOWLEDGE
for interpretation of o...
Semantics, perception, and cognition interact seamlessly.
● Semantic Computing can deal with basic big data challenges - m...
http://www.mezzacotta.net/garfield/?comic=1470, https://in.pinterest.com/pin/222435669066482336/
● Semantics is the “meani...
Semantics attaches meaning to observation by providing a definition
within a system context or the knowledge that people p...
Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Healt...
Cognition is the process by which an autonomous system perceives its environment, learns
from experiences, anticipates the...
DARPA launched a Cognitive Computing project in 2002. Cognitive Computing was defined as the
ability to “reason, use repre...
http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI
Need to be Cautious with Claims (Roger Schank)
“where we see the world we do not decide to see it” Daniel Kahneman
Perception is an active cyclical process of exploratio...
http://www.ted.com/talks/blaise_aguera_y_arcas_how_computers_are_learning_to_be_creative#t-345661
Image of a bird and a co...
Perceptual computing is the ability for a computer to recognize what is going on around it.
Computer can perceive the envi...
http://www.jfsowa.com/pubs/cogcycle.pdf, http://www.jfsowa.com/talks/interop.pdf
Peirce’s Cycle of Pragmatism
John Sowa’s ...
Perceptual Computing supports the ability to:
● Ask contextually relevant and personalized questions to explore and interp...
Hyperthyroidism
Elevated
Blood
Pressure
Systolic blood pressure of 150 mmHg
“150”
... ...
Levels of Abstraction
Making Sense of Physical-Cyber-Social Data With
Henson, et al. An Ontological Approach to Focusing Attention and Enhancing...
1
2
Perception Cycle
http://knoesis.org/node/2835
Prior Knowledge on the Web
Explanation is the act of choosing the objects or events that best account for a set of
observations; often referred to as...
Explanatory Feature: a feature that explains the set of observed properties
elevated blood pressure Hypertension
Hyperthyr...
Discriminating Property: is neither expected nor not-applicable
clammy skin
Hypertension
Hyperthyroidism
Pulmonary EdemaPa...
Augmented Personalized Healthcare (APH) is expected to enhance healthcare by
personalizing the use of all relevant Physica...
Asthma
kHealth Kit: Android Application
kHealth Personalized Digital Health
Providing actionable information in a timely manner is
crucial to avoid information overload or fatigue
Sleep data Communi...
For collecting observations from both machine sensors
and from patients in the form of a questionnaire.
kHealth Kit: Andro...
kHealth Dashboard: a visual environment to identify correlations
Domain Knowledge
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-a...
Personal level
Signals
Public level
Signals
Population level
Signals
Domain
Knowledge
Risk Model
Events from
social stream...
Risk assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population level
Si...
Risk assessment
model
Semantic
Perception
Personal level
Signals
Public level
Signals
Domain
Knowledge
Population level
Si...
Foobot – for monitoring
environmental air quality
Wheezometer – for monitoring
wheezing sounds
Can I reduce my asthma atta...
Another Example
with a Focus on a Richer Model
We are still working on the simpler representations of the real world!
http://artint.info/html/ArtInt_8.html, http://en.wi...
We need computational paradigms to tap into the rich pulse of the human
populace, and utilize diverse continuous stream of...
By 2001 over 285 million Indians lived in cities, more than in all North American
cities combined (Office of the Registrar...
• What time to start?
• What route to take?
• What is the reason for traffic?
– Wait for some time or re-route?
Questions ...
Complementary Data Sources
Image credit: http://traffic.511.org/index
Multiple Events
Varying influence
Interact with Each Other
Challenge: Non-linea...
7 × 24
LDS(1,1), LDS(1,2) ,…., LDS(1,24)
LDS(7,1), LDS(7,2) ,…., LDS(7,24)
.
.
.
di
hj
Mon.
Tue.
Wed.
Thu.
Fri.
Sat.
Sun.
...
Compute Log Likelihood for
each hour of observed data
(di,hj) LDS(hj,di)
7 × 24
Lik(1,1), Lik(1,2) ,…., Lik(1,24)
Lik(7,1)...
Hourly Traffic Dynamics Over a Day
Most of the drivers tend to go
5 km/h over the posted speed limit.
There are relatively few drivers who go more
than 10 km...
Public Safety
Urban Planning
Gov. & Agency
Admin.
Energy & water
Environmental
TransportationSocial Programs
Healthcare
Ed...
Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting City Traffic Events from S...
Complementary Events
Traffic Incident; road-construction
Textual Events from Tweets vs. 511.org: Complementary
Corroborative Events
Fog visibility-air-quality; fog
Textual Events from Tweets vs. 511.org: Corroborative
Timeliness
Concert Concert
Textual Events from Tweets vs. 511.org: Timeliness
Image Credit: http://traffic.511.org/index
Overturned Truck
Domain knowledge in the
form of traffic vocabulary
Domain know...
This example demonstrates use of:
– Multimodal data streams (types of events from text - signature from sensor data).
– Mu...
Thank you, and please visit us at http://knoesis.org
For more information on kHealth, please visit us at
http://knoesis.or...
Part of my group @ Kno.e.sis
with some faculty colleagues
Ohio Center of Excellence in
Knowledge-enabled Computing
Wright State University
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing
Upcoming SlideShare
Loading in …5
×

Semantic, Cognitive, and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing

1,867 views

Published on

Keynote at Web Intelligence 2017: http://webintelligence2017.com/program/keynotes/
Video: https://youtu.be/EIbhcqakgvA Paper: http://knoesis.org/node/2698

Abstract: While Bill Gates, Stephen Hawking, Elon Musk, Peter Thiel, and others engage in OpenAI discussions of whether or not AI, robots, and machines will replace humans, proponents of human-centric computing continue to extend work in which humans and machine partner in contextualized and personalized processing of multimodal data to derive actionable information.
In this talk, we discuss how maturing towards the emerging paradigms of semantic computing (SC), cognitive computing (CC), and perceptual computing (PC) provides a continuum through which to exploit the ever-increasing and growing diversity of data that could enhance people’s daily lives. SC and CC sift through raw data to personalize it according to context and individual users, creating abstractions that move the data closer to what humans can readily understand and apply in decision-making. PC, which interacts with the surrounding environment to collect data that is relevant and useful in understanding the outside world, is characterized by interpretative and exploratory activities that are supported by the use of prior/background knowledge. Using the examples of personalized digital health and a smart city, we will demonstrate how the trio of these computing paradigms form complementary capabilities that will enable the development of the next generation of intelligent systems. For background: http://bit.ly/PCSComputing

  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Semantic, Cognitive, and Perceptual Computing – three intertwined strands of a golden braid of intelligent computing

  1. 1. Amit Sheth Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing: Wright State University, Dayton, Ohio Semantic, Cognitive, and Perceptual Computing: three intertwined strands of a golden braid of intelligent computing Keynote @ Web Intelligence 2017, Leipzig, Germany, 24 Aug 2017 http://webintelligence2017.com/program/keynotes/
  2. 2. 2
  3. 3. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7433358
  4. 4. Credit: Looi Consulting (http://www.looiconsulting.com/home/enterprise-big-data/) ● In 2008, data generated > storage available. Less than 0.5% of data gets analyzed. ● Vast variety of data: text << images << A/V < genome sequencing < IoT ● Of all the data generated, which data is relevant, and why? Which data to analyze? Which data can offer insight? Who cares for what data? How to get attention to a human decision maker? What we need is intelligent processing to get actionable, smart data. A Big Challenge and Opportunity in Recent Times
  5. 5. How would an enterprise get actionable information? http://www.slideshare.net/NamrataChatterjee/nokias-supply-chain-management-case-study, http://www.economist.com/node/7032258 ● Weak crisis judgement. ● Failure to take prompt action. ● Single supplier reliability. Fire at Royal Philips electronic semiconductor plant, New Mexico in March 2000. 8 trays of wafers containing the miniature circuitry to make several thousand chips for mobile phones was destroyed. The expected time to recover was estimated to be a week. ● Fire breakout in clean room. ● Inability to determine the exact damage to the clean room. ● Lack of emergency preparation. ● Early speculation of possible crisis. ● Preparedness against supply crisis. ● Finding alternative source of chip supply.
  6. 6. http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/ The Patient of the Future MIT Technology Review, 2012 How would an individual get actionable information?
  7. 7. Smart Data I introduced the term in 2004; redefined in 2013: http://wiki.knoesis.org/index.php/Smart_Data
  8. 8. ● The astounding bandwidth of your senses is 11 million bits of information every second. ● In conscious activities like reading, the human brain distills approximately 40 bits of information per second. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/mobile-white-paper-c11-520862.html The Brain: Inspiration for Intelligent Processing: What if we could automate such interpretation of data?
  9. 9. ● How can we take inspiration from human brain and derive an intelligent processing of big data ● Take inspiration from cognition and perception, and for pedagogy, outline three computing paradigms for building future intelligent systems focused on converting data into abstractions that humans act upon for making decision and taking action ● CHE: Humans and machines partner to enhance human experience * 2010 article: http://wiki.knoesis.org/index.php/Computing_For_Human_Experience; distinct from “Human Experience of Computing by Booch: Dec. 9, 2011 at http://computingthehumanexperience.com/category/computing/. Sheth - 2008+ * What is this talk about?
  10. 10. http://www.research.ibm.com/cognitive-computing/brainpower/, https://newtonsapplevce.wordpress.com/2016/01/06/neuromorphic-technology/ Recreating the brain in silicon One Approach to Intelligence
  11. 11. A Second Approach to Intelligence
  12. 12. Two Use Cases: Healthcare and Traffic
  13. 13. Prior Medical Knowledge D1 D3 D1 Medical History and Past observations S1 S2 S3 Sn .. .. D1 D2 D3 S1 S2 S3 .. .. .. .. .. .. .. .. D1 D1 Doctor Patient Q1 Q2 Qn A1 A2 An Blood Pressure Heart Rate Breathing Rate Body Temperature Multi-model Observations Current: Observing a Snapshot of the Patient
  14. 14. Personalized Health and Objectives: one size does not fit all Millions of people - > one treatment Wearable and Sensor data
  15. 15. 1Marcus, Philip, Kevin R. Murphy, Abid Rahman, and Christopher D. O’Brien. "Intrapatient symptom variability in adults and children with asthma: Results of a survey." Advances in therapy 22, no. 5 (2005): 488-497. “…pediatric patients report variability in asthma symptoms over time, even when asthma medications are taken.”1 Near Future: Analyzing a Multifaceted Continuous Stream of Diverse Data Personal level Signals Public level Signals Population level Signals
  16. 16. ACTIONS situation awareness useful for decision making ABSTRACTIONS make sense to humans KNOWLEDGE for interpretation of observations Contextualization Personalization DATA Observations from machine and social sensors Converting Data to Actions
  17. 17. Semantics, perception, and cognition interact seamlessly. ● Semantic Computing can deal with basic big data challenges - make data more meaningful, improve interoperability and integration. ● Cognitive Computing can use relevant knowledge to improve information understanding for decision-making. ● Perceptual Computing can provide personalized and contextual abstractions over massive amounts of multimodal data from the physical, cyber, and social realms, enabling action. https://www.linkedin.com/pulse/perceptual-computing-third-strand-golden-braid-amit-sheth Semantic Computing, Perceptual Computing, Cognitive Computing
  18. 18. http://www.mezzacotta.net/garfield/?comic=1470, https://in.pinterest.com/pin/222435669066482336/ ● Semantics is the “meaning or relationship of meanings, or relating to meaning ” (Webster). ● Meaning and use of data (Information System). ● It is concerned with the relationship between the linguistic symbols and their meaning or real-world objects. Semantics
  19. 19. Semantics attaches meaning to observation by providing a definition within a system context or the knowledge that people possess. Semantic Computing encompasses the technology required to represent concepts and their relationships in an integrated semantic network that loosely mimics the brain’s conceptual interrelationships. Web of data Semantics attached to objects in the world Semantic Computing
  20. 20. Population Level Personal Wheeze – Yes Do you have tightness of chest? –Yes ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding <Wheezing=Yes, time, location> <ChectTightness=Yes, time, location> <PollenLevel=Medium, time, location> <Pollution=Yes, time, location> <Activity=High, time, location> Wheezing ChestTightness PollenLevel Pollution Activity Wheezing ChestTightness PollenLevel Pollution Activity RiskCategory <PollenLevel, ChectTightness, Pollution, Activity, Wheezing, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> <2, 1, 1,3, 1, RiskCategory> . . . Expert Knowledge Background Knowledge tweet reporting pollution level and asthma attacks Acceleration readings from on-phone sensors Sensor and personal observations Signals from personal, personal spaces, and community spaces Risk Category assigned by doctors Qualify Quantify Enrich Outdoor pollen and pollution Public Health Well Controlled - continue Not Well Controlled – contact nurse Poor Controlled – contact doctor SSN Semantic Computing for Our Personalized Digital Health (Asthma) Application
  21. 21. Cognition is the process by which an autonomous system perceives its environment, learns from experiences, anticipates the outcome of the events, acts to pursue goals and adapts to the changing environment. It is the result of a developmental process through which the system becomes progressively more skilled and acquires the ability to understand events, contexts, and action… https://mitpress.mit.edu/books/artificial-cognitive-systems PerceptionAction Anticipate Adapt Assimilate Cognition is a cycle of anticipation, assimilation, adaptation: embedded in, contributing to, and benefitting from a continuous process of action and perception. What is Cognition?
  22. 22. DARPA launched a Cognitive Computing project in 2002. Cognitive Computing was defined as the ability to “reason, use represented knowledge, learn from experience, accumulate knowledge, explain itself, accept direction, be aware of its own behavior and capabilities, [and] respond in a robust manner to surprises.” IBM describes the components used to develop, and behaviors resulting from, “systems that learn at scale, reason with purpose and interact with humans naturally.” According to them, while sharing many attributes with the field of artificial intelligence, it differentiates itself via the complex interplay of disparate components, each of which comprise their own individual mature disciplines [1]. Our Take Cognitive computing uses annotated observations obtained from Semantic computing , or raw observations from diverse sources and makes it meaningful in context –it provides the ability to understand events, contexts, and action. Cognitive computing systems learns from their experiences and improve when performing repeated tasks. [1] https://en.wikipedia.org/wiki/Cognitive_computing Cognitive Computing
  23. 23. http://www.rogerschank.com/fraudulent-claims-made-by-IBM-about-Watson-and-AI Need to be Cautious with Claims (Roger Schank)
  24. 24. “where we see the world we do not decide to see it” Daniel Kahneman Perception is an active cyclical process of exploration and interpretation. Perception enables individual to focus on most promising course of action by incorporating background knowledge that provided a comprehensive contextual understanding. Perception
  25. 25. http://www.ted.com/talks/blaise_aguera_y_arcas_how_computers_are_learning_to_be_creative#t-345661 Image of a bird and a computer says it’s a bird. Perception is an active imagination. Perception: Google
  26. 26. Perceptual computing is the ability for a computer to recognize what is going on around it. Computer can perceive the environment and the users in that environment. The computer determines what needs a user might have and react to those needs without giving or receiving any additional information. Perception: Intel
  27. 27. http://www.jfsowa.com/pubs/cogcycle.pdf, http://www.jfsowa.com/talks/interop.pdf Peirce’s Cycle of Pragmatism John Sowa’s Perception Cycle
  28. 28. Perceptual Computing supports the ability to: ● Ask contextually relevant and personalized questions to explore and interpret observations. ● Builds upon Semantic Computing and Cognitive Computing by providing the machinery to ask the next question or derive a hypothesis. ● It helps generate abstractions that lead to actions. Perceptual Computing: Our View
  29. 29. Hyperthyroidism Elevated Blood Pressure Systolic blood pressure of 150 mmHg “150” ... ... Levels of Abstraction
  30. 30. Making Sense of Physical-Cyber-Social Data With Henson, et al. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web, Applied Ontology, 2011. Our First Approach to Implement Perceptual Computing
  31. 31. 1 2 Perception Cycle
  32. 32. http://knoesis.org/node/2835
  33. 33. Prior Knowledge on the Web
  34. 34. Explanation is the act of choosing the objects or events that best account for a set of observations; often referred to as hypothesis building Discrimination is the act of finding those properties that, if observed, would help distinguish between multiple explanatory features 1 2 Explanation and Discrimination
  35. 35. Explanatory Feature: a feature that explains the set of observed properties elevated blood pressure Hypertension Hyperthyroidism Pulmonary Edema Clammy Skin Palpitations Explanation
  36. 36. Discriminating Property: is neither expected nor not-applicable clammy skin Hypertension Hyperthyroidism Pulmonary EdemaPalpitations elevated blood pressure Discrimination
  37. 37. Augmented Personalized Healthcare (APH) is expected to enhance healthcare by personalizing the use of all relevant Physical, Cyber, and Social data obtained from wearables, sensors and Internet of Things, mobile applications, electronic medical records, web-based information, and social media for better health for an individual. Converting big data into smart data through contextual and personalized processing such that patient and clinician can make better decisions and take timely actions for Data include traditional clinical data, PGHD and public health data, as well as environmental and social data that could impact an individual’s health. Augmented Personalized Healthcare
  38. 38. Asthma
  39. 39. kHealth Kit: Android Application kHealth Personalized Digital Health
  40. 40. Providing actionable information in a timely manner is crucial to avoid information overload or fatigue Sleep data Community dataPersonal Schedule Activity data Personal health records Data Overload for Patients/Health Aficionados
  41. 41. For collecting observations from both machine sensors and from patients in the form of a questionnaire. kHealth Kit: Android Application
  42. 42. kHealth Dashboard: a visual environment to identify correlations
  43. 43. Domain Knowledge ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist; *consider referral to specialist Asthma Control and Actionable Information Asthma Domain Knowledge
  44. 44. Personal level Signals Public level Signals Population level Signals Domain Knowledge Risk Model Events from social streams Take medication before going to work. Avoid going out in the evening due to high pollen levels. Contact a doctor. Analysis Personalized Actionable Information Data Acquisition & aggregation kHealth: Health Signal Processing Architecture
  45. 45. Risk assessment model Semantic Perception Personal level Signals Public level Signals Domain Knowledge Population level Signals GREEN: Well controlled YELLOW: Not well controlled Red: Poor controlled How controlled is my asthma? Patient Health Score (Diagnostic)
  46. 46. Risk assessment model Semantic Perception Personal level Signals Public level Signals Domain Knowledge Population level Signals Patient health Score How vulnerable* is my control level today? *considering changing environmental conditions and current control level Patient Health Score (Prognostic)
  47. 47. Foobot – for monitoring environmental air quality Wheezometer – for monitoring wheezing sounds Can I reduce my asthma attacks at night? What are the triggers? What is the wheezing level? What is the propensity toward asthma? What is the exposure level over a day? Commute to work Luminosity CO Level CO in gush during the daytime. Actionable Information Personal level Signals Public level Signals Population level Signals What is the air quality indoors? Close the window at home during day to avoid CO2 inflow, to avoid asthma attacks at night Decision Support for Doctors and Patients: A Scenario
  48. 48. Another Example with a Focus on a Richer Model
  49. 49. We are still working on the simpler representations of the real world! http://artint.info/html/ArtInt_8.html, http://en.wikipedia.org/wiki/Traffic_congestion Solve Represent Interpret Real world Simplified representation Compute What did not change in data processing for quite some time?
  50. 50. We need computational paradigms to tap into the rich pulse of the human populace, and utilize diverse continuous stream of data Represent, capture, and compute with richer and fine-grained representations of real-world problems Solve Represent Interpret Real world Richer representation Compute + Richer representation of traffic observations. Effective solutions People interpreting a real-world event. What should change?
  51. 51. By 2001 over 285 million Indians lived in cities, more than in all North American cities combined (Office of the Registrar General of India 2001) 1. 1 The Crisis of Public Transport in India 2 IBM Smarter Traffic Modes of Transportation in Indian Cities Texas Transportation Institute (TTI) Congestion report in U.S. Severity of the Traffic Problem
  52. 52. • What time to start? • What route to take? • What is the reason for traffic? – Wait for some time or re-route? Questions Asked Daily
  53. 53. Complementary Data Sources
  54. 54. Image credit: http://traffic.511.org/index Multiple Events Varying influence Interact with Each Other Challenge: Non-linearity in Traffic Dynamics
  55. 55. 7 × 24 LDS(1,1), LDS(1,2) ,…., LDS(1,24) LDS(7,1), LDS(7,2) ,…., LDS(7,24) . . . di hj Mon. Tue. Wed. Thu. Fri. Sat. Sun. Mon. Tue. Wed. Thu. Fri. Sat. Sun. Speed/travel-time time series data from a link. Time series data for each hour of day (1-24) for each day of week (Monday – Sunday). Mean time series computed for each day of week and hour of day along with the medoid. 168 LDS models for each link; Total models learned = 425,712 i.e., (2,534 links × 168 models per link). Step 1: Index data for each link for day of week and hour of day utilizing the traffic domain knowledge for piece-wise linear approximation Step 2: Find the “typical” dynamics by computing the mean and choosing the medoid for each hour of day and day of week Step 3: Learn LDS parameters for the medoid for each hour of day (24 hours) and each day of week (7 days) resulting in 24 × 7 = 168 models for each link Learning Context-specific LDS Models
  56. 56. Compute Log Likelihood for each hour of observed data (di,hj) LDS(hj,di) 7 × 24 Lik(1,1), Lik(1,2) ,…., Lik(1,24) Lik(7,1), Lik(7,2) ,…., Lik(7,24) . . . Train? Yes (Training phase) Tag Anomalous hours using the Log Likelihood Range No (di,hj) (min. likelihood) Anomalies L = Partition based on (di,hj) Speed and travel-time time Observations from a link Log likelihood min. and max. values obtained from five number summary Partition based on (di,hj) 7 × 24 LDS(1,1), LDS(1,2) ,…., LDS(1,24) LDS(7,1), LDS(7,2) ,…., LDS(7,24) . . . di hj (Input) (Output) Tagging Anomalies with LDS Models
  57. 57. Hourly Traffic Dynamics Over a Day
  58. 58. Most of the drivers tend to go 5 km/h over the posted speed limit. There are relatively few drivers who go more than 10 km/h over the posted speed limit. There are situations in a day where the drivers are going (forced) below the speed limit e.g., rush hour traffic. Do these histograms resemble any probability distribution? Traffic Data: Possible Explanation
  59. 59. Public Safety Urban Planning Gov. & Agency Admin. Energy & water Environmental TransportationSocial Programs Healthcare Education Twitter as a Source of City Events
  60. 60. Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting City Traffic Events from Social Streams. ACM Trans. Intell. Syst. Technol. 6, 4, Article 43 (July 2015), 27 pages. DOI=10.1145/2717317 http://doi.acm.org/10.1145/2717317 Last O night O in O CA... O (@ O Half B-LOCATION Moon I-LOCATION Bay B-LOCATION Brewing I-LOCATION Company O w/ O 8 O others) O http://t.co/w0eGEJjApY O Extracting City Events from Textual Data
  61. 61. Complementary Events Traffic Incident; road-construction Textual Events from Tweets vs. 511.org: Complementary
  62. 62. Corroborative Events Fog visibility-air-quality; fog Textual Events from Tweets vs. 511.org: Corroborative
  63. 63. Timeliness Concert Concert Textual Events from Tweets vs. 511.org: Timeliness
  64. 64. Image Credit: http://traffic.511.org/index Overturned Truck Domain knowledge in the form of traffic vocabulary Domain knowledge of traffic flow synthesized from sensor data Explained-by Horizontal operator: relating/mapping data from different modality to a concept (theme) within a spatio-temporal context; Spatial context even include what it means to have a slow traffic for the type of road Understanding: Semantic Annotation of Sensor + Textual Data Utilizing Background Knowledge
  65. 65. This example demonstrates use of: – Multimodal data streams (types of events from text - signature from sensor data). – Multiple sources of knowledge/ontologies. – Semantic annotations and enrichments. – Use of rich representation (PGM). – Statistical approach to create normalcy models and understand anomalies using historical data. – Explain anomalies using extracted events. – Provide actionable information. How traffic analysis captures complexity of the real-world?
  66. 66. Thank you, and please visit us at http://knoesis.org For more information on kHealth, please visit us at http://knoesis.org/projects/khealth Cognitive Computing Semantic Computing Perceptual Computing Contributors and collaborators for this talk: Pramod Anantharam Cory Henson Dr. T.K. Prasad Sujan Perera Utkarshani Jaimini Thank You Sanjaya Wijeratne
  67. 67. Part of my group @ Kno.e.sis with some faculty colleagues
  68. 68. Ohio Center of Excellence in Knowledge-enabled Computing Wright State University

×