Cory Andrew Henson, profile picture
Uploaded byCory Andrew Henson
PPTX, PDF1,530 views

A Semantics-based Approach to Machine Perception

1) The document discusses a semantics-based approach to machine perception that uses semantic web technologies to derive abstractions from sensor data using background knowledge on the web. 2) It addresses three primary issues: annotation of sensor data, developing a semantic sensor web, and enabling semantic perception intelligence at the edge on resource-constrained devices. 3) The approach represents background knowledge and sensor observations using ontologies, and uses deductive and abductive reasoning over these representations to interpret sensor data at multiple levels of abstraction.

Embed presentation

Downloaded 34 times
A SEMANTICS-BASED APPROACH TO MACHINE PERCEPTION Cory Andrew Henson August 27, 2013 1 Committee: Amit Sheth (advisor) Krishnaprasad Thirunarayan John Gallagher Payam Barnaghi Satya Sahoo Ph.D. Dissertation Defense Wright State University Dept. of Computer Science and Engineering
2
3 Thesis Machine perception can be formalized using semantic web technologies in order to derive abstractions from sensor data using background knowledge on the Web, and efficiently executed on resource-constrained devices.
4 3 primary issues to be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices 1 2 3
5 lives in has pet is ahas pet Person Animal Concrete Facts Resource Description Framework Semantic Web (according to Farside) General Knowledge Web Ontology Language “Now! – That should clear up a few things around here!” is a
Government Media Publications Life Sciences Geographic Semantic Web 6
7http://www.opengeospatial.org/projects/groups/sensorwebdwg Semantic Sensor Web
8 Sensor systems are too often stovepiped
9 With freedom comes responsibility 1. discovery, access, and search 2. integration and interpretation We want to set this data free
10 OGC Sensor Web Enablement (SWE)
11 With freedom comes responsibility 1. discovery, access, and search 2. integration and interpretation We want to set this data free
12 RDF OWL How are machines supposed to integrate and interpret sensor data? Semantic Sensor Networks (SSN)
13 W3C Semantic Sensor Network Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
14 W3C Semantic Sensor Network Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
15 W3C Semantic Sensor Network Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
16 Semantic Annotation of SWE Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
Semantic Sensor Observation Service (SemSOS) Cory Henson, Josh Pschorr,Amit Sheth, Krishnaprasad Thirunarayan, SemSOS: Semantic Sensor Observation Service, In Proceedings of the 2009 International Symposium on Collaborative Technologies and Systems (CTS 2009), Baltimore, MD, May 18-22, 2009. 17
Semantic Sensor Observation Service (SemSOS) Joshua Pschorr, Cory Henson, Harshal Patni, and Amit P. Sheth. Sensor Discovery on Linked Data. Kno.e.sis Center Technical Report 2010. 18
19 3 primary issues to be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices1 2 3
20 Semantic Perception • The role of perception is to transform raw sensory data into a meaningful and correct representation of the external world. • The systematic automation of this ability is the focus of machine perception. • For correct interpretation of this representation, we need a formal account of how this is done.
21 What can we learn from cognitive models of perception? People are good at making sense of sensory input
22 Perception is an active, cyclical process of exploration and interpretation. The perception cycle is driven by prior knowledge, in order to generate and test hypotheses. Some observations are more informative than others (in order to effectively test hypotheses*). Ulric Neisser Richard Gregory Kenneth Norwich 1970’s 1980’s 1990’s Cognitive Models of Perception * Applies to machine perception within Intellego, NOT human perception.
Observed Properties Perceived Features 23 Background knowledge Explanation Ontology of Perception Focus An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
Example: Medical diagnosis as perception 24
Proactive, Preventative Healthcare 25
The Patient of the Future MIT Technology Review, 2012 26http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
Digital Doctor Let’s provide people with the tools needed to monitor and manage their own health http://worldofdtcmarketing.com/mobile-health-apps-a-new-opportunity-for-healthcare-marketers/mobile-healthcare-marketing-trends/ 27
Medical/healthcare expert systems have been around for a long time 28
1. Ubiquitous Sensing 2. Always-on Computing 3. Knowledge on the Web 3 recent developments have changed the technological landscape … 29
30 Making sense of sensor data with
DATA sensor observations KNOWLEDGE situation awareness useful for decision making Primary challenge is to bridge the gap between data and knowledge 31
SSN Ontology 2 Interpreted data (deductive) [in OWL] e.g., threshold 1 Annotated Data [in RDF] e.g., label 0 Raw Data [in TEXT] e.g., number Levels of Abstraction 3 Interpreted data (abductive) [in OWL] e.g., diagnosis Intellego ―150‖ Systolic blood pressure of 150 mmHg Elevated Blood Pressure Hyperthyroidism …… 32
Observed Properties Perceived Features 33 Background knowledge on the Web Low-level observed properties suggest explanatory hypotheses through abduction Explanation Focus Ontology of Perception An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
34 Semantics of Explanation Abduction – or, inference to the best EXPLANATION Task • Given background knowledge of the environment (SIGMA), and • given a set of sensor observation data (RHO), • find a consistent explanation of the situation (DELTA) Background knowledge Features (objects/events) in the world Sensor observation data
35 Semantics of Explanation Background knowledge is represented as a causal network between features (objects or events) in the world and the sensor observations they give rise to.
36 Semantics of Explanation Finding the sweet spot between abduction and OWL • Simulation of Parsimonious Covering Theory in OWL-DL (using the single-feature assumption*) * An explanation must be a single feature which accounts for all observed properties Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012) Theorem: Given a PCT problem P and its translation o(P) into OWL, Δ = {e} is a PCT explanation if and only if ExplanatoryFeature(e) is deduced by an OWL-DL reasoner — that is, if and only if o(P) ⊧ ExplanatoryFeature(e).
37 Finding the Sweet Spot minimize explanations degrade gracefully w/ incomplete info decidable web reasoning Abductive Logic (e.g., PCT) high complexity Deductive Logic (e.g., OWL) (relatively) low complexity Explanation (in Intellego)
Explanatory Feature: A feature is explanatory w.r.t. a set of observed properties if it causes each property in the set. ExplanatoryFeature ≡ ∃isPropertyOf—.{p1} ⊓ … ⊓ ∃isPropertyOf—.{pn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Observed Property Explanatory Feature 38 Semantics of Explanation
Observed Properties Perceived Features 39 Background knowledge on the Web Hypotheses imply the informational value of future observations through deduction Explanation Focus Ontology of Perception An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
Universe of observable properties 40 Semantics of Focus To predict which future observations have informational value, find those observable properties that can discriminate between the set of hypotheses. Expected Properties Not-applicable Properties Discriminating Properties
Expected Property: A property is expected w.r.t. a set of features if it is caused by each feature in the set. ExpectedProperty ≡ ∃isPropertyOf.{f1} ⊓ … ⊓ ∃isPropertyOf.{fn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Expected Property Explanatory Feature 41 Semantics of Focus
Not Applicable Property: A property is not-applicable w.r.t. a set of features if it is not caused by any feature in the set. NotApplicableProperty ≡ ¬∃isPropertyOf.{f1} ⊓ … ⊓ ¬∃isPropertyOf.{fn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Not Applicable Property Explanatory Feature 42 Semantics of Focus
Discriminating Property: A property is discriminating w.r.t. a set of features if it is neither expected nor not-applicable. DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Discriminating Property Explanatory Feature 43 Semantics of Focus
Off-the-shelf OWL-DL reasoners are too resource intensive in terms of both memory and time • Runs out of resources with background knowledge >> 20 nodes • Asymptotic complexity: O(n3) 44 O(n3) < x < O(n4) Semantic perception on resource-constrained devices
45 3 primary issues to be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices1 2 3
Internet of Things 46http://www.idgconnect.com/blog-abstract/900/the-internet-things-breaking-down-barriers-connected-world
47http://share.cisco.com/internet-of-things.html
48 Basis Watch • Heart Rate Monitor • Accelerometer • Skin Temperature • Galvanic Skin Response
Homo Digitus 49 How do we make sense of this data … and do it efficiently and at scale?
Approach 1: Send all sensor observations to the cloud for processing 50 Approach 2: downscale semantic processing so that each device is capable of machine perception Intelligence at the Edge
Use bit vector encodings and their operations to encode background knowledge and execute perceptual inference 51 Efficient execution of semantic perception 01100011 10001110 11001110 01010110 01110101 OWL-DL An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (ISWC, 2012)
lift lower Translate background knowledge, observations, and explanations between Semantic Web and bit vector representation 52 Lifting and lowering of knowledge
53 Efficient execution of semantic perception Bit vector algorithms are provably equivalent to the OWL inference (i.e., semantics preserving) Intuition: discover and dismiss those features that cannot explain the set of observed properties. Intuition: discover and assemble those properties that discriminate between the explanatory features
bp 1 cs 0 pa 1 HN HM PE bp 1 1 1 cs 0 1 0 pa 1 1 0 HN HM PE 1 1 1 HN HM PE 1 1 1 1 1 0 AND => AND 1 1 1 Observed Property Prior Knowledge Previous Explanatory Feature Current Explanatory Feature => INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties. 54 Explanation: efficient algorithm
bp 1 cs 0 pa 1 HN HM PE bp 0 1 1 cs 0 1 0 pa 1 1 0 HN HM PE 1 1 0 HN HM PE 0 1 0AND => 0 1 0 Observed Property Prior Knowledge Previous Explanatory Feature Current Explanatory Feature = INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble those properties that discriminate between the explanatory features bp 0 cs 0 pa 0 Discriminatin g Property 1 1 0 … expected? 0 0 0 … not-applicable? ZERO Bit Vector 0 1 0 = => FALSE => FALSE 1 Is the property discriminating? 55 Focus: efficient algorithm
O(n3) < x < O(n4) O(n) 56 Evaluation on a mobile device Efficiency Improvement • Problem size increased from 10’s to 1000’s of nodes • Time reduced from minutes to milliseconds • Complexity growth reduced from polynomial to linear
57 Technical contributions in a nutshell 1. Semantic Sensor Web: Developed technologies for the semantic annotation of sensor data on the Web - Semantic Sensor Web (IEEE Internet Computing, 2008) – 276 citations (as of Aug. 2013) - SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009) - Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011) 2. Semantic Perception: Designed a declarative specification of perception, capable of utilizing an off-the-shelf OWL-DL reasoner - An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology Journal, 2011) - Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing: Special Issue on Context- Aware Computing, 2012) – most downloaded paid paper in IEEE-IC 2012 3. Intelligence at the Edge: Implemented efficient algorithms for executing perceptual inference on resource-constrained devices - An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012)
58 W3C Reports 1. Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011) Journal Publications 1. Physical-Cyber-Social Computing: An Early 21st Century Approach (IEEE Intelligent Systems, 2013) 2. Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012) 3. Semantics for the Internet of Things: Early Progress and Back to the Future (International Journal on Semantic Web and Information Systems, 2012) 4. The SSN ontology of the W3C semantic sensor network incubator group (Web Semantics: Science, Services and Agents on the World Wide Web, 2012) 5. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011) 6. Semantic Sensor Web (IEEE Internet Computing, 2008) Conference Publications 1. An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012) 2. Computing Perception from Sensor Data (IEEE Sensors Conference, 2012) 3. SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009) 4. Situation Awareness via Abductive Reasoning for Semantic Sensor Data: A Preliminary Report (International Symposium on Collaborative Technologies and Systems, 2009). Workshop Publications 1. SECURE: Semantics Empowered Rescue Environment (International Workshop on Semantic Sensor Networks, 2011) 2. Representation of Parsimonious Covering Theory in OWL-DL (International Workshop on OWL: Experiences and Directions, 2011) 3. Provenance Aware Linked Sensor Data (Workshop on Trust and Privacy on the Social and Semantic Web, 2010) 4. Linked Sensor Data (International Symposium on Collaborative Technologies and Systems, 2010) 5. A Survey of the Semantic Specification of Sensors (International Workshop on Semantic Sensor Networks, 2009) 6. An Ontological Representation of Time Series Observations on the Semantic Sensor Web (International Workshop on the Semantic Sensor Web) 7. Video on the Semantic Sensor Web (W3C Video on the Web Workshop, 2007) Relevant Publications
59 Application Proactive, preventative healthcare
Heart disease is a critical issue ~815,000 (2011) http://millionhearts.hhs.gov/abouthds/cost-consequences.html 60
Acute Decompensated Heart Failure (ADHF) • Affects nearly 6 million people (in U.S.) • 555,000 new cases are diagnosed each year 61U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
• 4.8 million hospitalizations per year • 50% are readmitted within 6 months • 25% are readmitted within 30 days • 70% due to worsening conditions • Costing $17 billion per year ADHF hospital readmission rates are too high 62U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
Congress has incentivized hospitals to lower readmission rates 63U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
Current state-of-the-art 64
Score (0: Not at all, 1: A little, 2: A great deal, 3: Extremely) Heart Failure Somatic Awareness Scale (HFSAS) Current state-of-the-art 65
kHealth – knowledge-enabled healthcare Approach: • Use semantic perception inference • with data from cardio-related sensors • and curated medical background knowledge on the Web 1. to monitor and abstract health conditions 2. to ask the patient contextually relevant questions 66
Cardiology Background Knowledge • Symptoms: 284 • Disorders: 173 • Causal Relations: 1944 Unified Medical Language System Causal Network 67
kHealth Kit Weight Scale Heart Rate Monitor Blood Pressure Monitor 68 Sensors Android Device (w/ kHealth App) Total cost: < $500
69 Explanation in kHealth • Abnormal heart rate • High blood pressure • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Observed Property Explanatory Feature via Bluetooth
70 Focus in kHealth Are you feeling lightheaded? Are you have trouble taking deep breaths? yes yes • Abnormal heart rate • High blood pressure • Lightheaded • Trouble breathing • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Contextually dependent questioning based on prior observations (from 284 possible questions) Observed Property Explanatory Feature
71 Evaluation of kHealth Evaluate the ability to discriminate between sets of potential disorders using: 1. HFSAS/WANDA’s restricted set of observable symptoms (12) 2. kHealth’s more comprehensive set of observable symptoms (284)
72 Evaluation of kHealth Evaluation Metrics: 1. Efficiency: How many observations (or questions) required to minimize the set of explanations? 2. Specificity: How specific is the resulting minimum set of explanations (i.e., how many explanatory disorders in the set)? Explanatory Disorders (computed by Intellego) Actual Disorder (extracted from EMR) Possible Disorders (derived from cardiology KB)
73 Evaluation of kHealth – Early Results HFSAS/WANDA Efficiency: ~7.45 (# questions asked) Specificity: ~11.95* (# minimum explanations) * Converged to 1 explanation 20% of the time • 496 EMRs • ~3.2 diagnosed disorders per EMR • 173 possible disorders The approach utilized by kHealth is more efficient and more specific than HFSAS/WANDA. kHealth Efficiency: ~7.28 (# questions asked) Specificity: 1 (# minimum explanations) Explanatory Disorders (computed by Intellego) Actual Disorder (extracted from EMR) Possible Disorders (derived from cardiology KB)
Pre-clinical usability trial Dr. William Abraham, M.D. Director of Cardiovascular Medicine 74
Sensing and Perception Health Care Academic Standards Org. Industry Government Research Collaborators by Research Topic and Organization Type 75
Special Thanks to AFRL and DAGSI 76 AFRL/DAGSI Research Topic SN08-8: Architectures for Secure Semantic Sensor Networks for Multi-Layered Sensing
77
Semantic Sensor Web Team 78
A SEMANTICS-BASED APPROACH TO MACHINE PERCEPTION Cory Andrew Henson August 27, 2013 79 Thank you. For additional information visit: http://knoesis.org/researchers/cory

More Related Content

PPT
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
byArtificial Intelligence Institute at UofSC
 
PPTX
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
byAmit Sheth
 
PPTX
Smart Data - How you and I will exploit Big Data for personalized digital hea...
byAmit Sheth
 
PPTX
Knowledge Will Propel Machine Understanding of Big Data
byAmit Sheth
 
PPTX
Smart IoT for Connected Manufacturing
byAmit Sheth
 
PPTX
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
byAmit Sheth
 
PDF
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
byArtificial Intelligence Institute at UofSC
 
PPTX
Physical Cyber Social Computing
byAmit Sheth
 
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
byArtificial Intelligence Institute at UofSC
 
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...
byAmit Sheth
 
Smart Data - How you and I will exploit Big Data for personalized digital hea...
byAmit Sheth
 
Knowledge Will Propel Machine Understanding of Big Data
byAmit Sheth
 
Smart IoT for Connected Manufacturing
byAmit Sheth
 
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...
byAmit Sheth
 
Knowledge-empowered Probabilistic Graphical Models for Physical-Cyber-Social ...
byArtificial Intelligence Institute at UofSC
 
Physical Cyber Social Computing
byAmit Sheth
 

What's hot

PPTX
Web and Complex Systems Lab @ Kno.e.sis
byArtificial Intelligence Institute at UofSC
 
PPTX
What's up at Kno.e.sis?
byAmit Sheth
 
PPTX
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
byAmit Sheth
 
PPTX
Citizen Sensor Data Mining, Social Media Analytics and Applications
byAmit Sheth
 
PPTX
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
byArtificial Intelligence Institute at UofSC
 
PPTX
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
byArtificial Intelligence Institute at UofSC
 
PPTX
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
byAmit Sheth
 
PPTX
Understanding City Traffic Dynamics Utilizing Sensor and Textual Observations
byArtificial Intelligence Institute at UofSC
 
PPTX
Physical Cyber Social Computing: An early 21st century approach to Computing ...
byAmit Sheth
 
PPTX
Knoesis Student Achievement
byArtificial Intelligence Institute at UofSC
 
PPT
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
byAmit Sheth
 
PDF
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
byArtificial Intelligence Institute at UofSC
 
PPTX
Extracting City Traffic Events from Social Streams
byPramod Anantharam
 
PPTX
Semantics-empowered Smart City applications: today and tomorrow
byAmit Sheth
 
PDF
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
byAmit Sheth
 
PDF
Smart Data and real-world semantic web applications (2004)
byAmit Sheth
 
PDF
The human face of AI: how collective and augmented intelligence can help sol...
byElena Simperl
 
PDF
The NEEDS vs. the WANTS in IoT
byPrasant Misra
 
PDF
Challenges in Analytics for BIG Data
byPrasant Misra
 
PDF
Sensors, Signals and Sense-making in Human-Energy Relationships
byMartha Russell
 
Web and Complex Systems Lab @ Kno.e.sis
byArtificial Intelligence Institute at UofSC
 
What's up at Kno.e.sis?
byAmit Sheth
 
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...
byAmit Sheth
 
Citizen Sensor Data Mining, Social Media Analytics and Applications
byAmit Sheth
 
Computing for Human Experience [v4]: Keynote @ OnTheMove Federated Conferences
byArtificial Intelligence Institute at UofSC
 
Semantics-empowered Approaches to Big Data Processing for Physical-Cyber-Soci...
byArtificial Intelligence Institute at UofSC
 
Transforming Big Data into Smart Data for Smart Energy: Deriving Value via ha...
byAmit Sheth
 
Understanding City Traffic Dynamics Utilizing Sensor and Textual Observations
byArtificial Intelligence Institute at UofSC
 
Physical Cyber Social Computing: An early 21st century approach to Computing ...
byAmit Sheth
 
Knoesis Student Achievement
byArtificial Intelligence Institute at UofSC
 
Semantics for Bioinformatics: What, Why and How of Search, Integration and An...
byAmit Sheth
 
Hemant Purohit PhD Defense: Mining Citizen Sensor Communities for Cooperation...
byArtificial Intelligence Institute at UofSC
 
Extracting City Traffic Events from Social Streams
byPramod Anantharam
 
Semantics-empowered Smart City applications: today and tomorrow
byAmit Sheth
 
Semantic Web for 360-degree Health: State-of-the-Art & Vision for Better Inte...
byAmit Sheth
 
Smart Data and real-world semantic web applications (2004)
byAmit Sheth
 
The human face of AI: how collective and augmented intelligence can help sol...
byElena Simperl
 
The NEEDS vs. the WANTS in IoT
byPrasant Misra
 
Challenges in Analytics for BIG Data
byPrasant Misra
 
Sensors, Signals and Sense-making in Human-Energy Relationships
byMartha Russell
 

Viewers also liked

PPTX
Text Mining using LDA with Context
bySteffen Staab
 
PDF
How to Leverage Social Media Communities for Crisis Response Coordination
byArtificial Intelligence Institute at UofSC
 
PDF
SIAM SDM2014 tutorial - Social Media and Web of Data to Assist Crisis Respons...
byArtificial Intelligence Institute at UofSC
 
ODP
Presentación de la firma
byAPelagio
 
PDF
NCSU invited talk: Leveraging Social Media for Tourism Marketplace Coordination
byArtificial Intelligence Institute at UofSC
 
PPTX
20 cv mil_models_for_words
byzukun
 
PPTX
Analysis of Reviews on Sony Z3
byKrishna Bollojula
 
PDF
Mathematical approach for Text Mining 1
byKyunghoon Kim
 
PPTX
Latent Semantic Indexing and Search Engines Optimimization (SEO)
bymuzzy4friends
 
PDF
Geometric Aspects of LSA
byHeinrich Hartmann
 
PDF
AutoCardSorter - Designing the Information Architecture of a web site using L...
byChristos Katsanos
 
PPTX
Recommending Tags with a Model of Human Categorization
byChristoph Trattner
 
PPTX
SpeakerLDA: Discovering Topics in Transcribed Multi-Speaker Audio Contents @ ...
byDamiano Spina
 
PPTX
LSI latent (par HATOUM Saria et DONGO ESCALANTE Irvin Franco)
byrchbeir
 
PDF
Practical Machine Learning
byJaganadh Gopinadhan
 
PPT
Mining Features from the Object-Oriented Source Code of a Collection of Softw...
byRa'Fat Al-Msie'deen
 
PPT
SNAPP - Learning Analytics and Knowledge Conference 2011
byaneeshabakharia
 
PDF
Latent Semantic Transliteration using Dirichlet Mixture
byRakuten Group, Inc.
 
PDF
BigML Summer 2016 Release
byBigML, Inc
 
PPTX
An approach to source code plagiarism
byvarsha_bhat
 
Text Mining using LDA with Context
bySteffen Staab
 
How to Leverage Social Media Communities for Crisis Response Coordination
byArtificial Intelligence Institute at UofSC
 
SIAM SDM2014 tutorial - Social Media and Web of Data to Assist Crisis Respons...
byArtificial Intelligence Institute at UofSC
 
Presentación de la firma
byAPelagio
 
NCSU invited talk: Leveraging Social Media for Tourism Marketplace Coordination
byArtificial Intelligence Institute at UofSC
 
20 cv mil_models_for_words
byzukun
 
Analysis of Reviews on Sony Z3
byKrishna Bollojula
 
Mathematical approach for Text Mining 1
byKyunghoon Kim
 
Latent Semantic Indexing and Search Engines Optimimization (SEO)
bymuzzy4friends
 
Geometric Aspects of LSA
byHeinrich Hartmann
 
AutoCardSorter - Designing the Information Architecture of a web site using L...
byChristos Katsanos
 
Recommending Tags with a Model of Human Categorization
byChristoph Trattner
 
SpeakerLDA: Discovering Topics in Transcribed Multi-Speaker Audio Contents @ ...
byDamiano Spina
 
LSI latent (par HATOUM Saria et DONGO ESCALANTE Irvin Franco)
byrchbeir
 
Practical Machine Learning
byJaganadh Gopinadhan
 
Mining Features from the Object-Oriented Source Code of a Collection of Softw...
byRa'Fat Al-Msie'deen
 
SNAPP - Learning Analytics and Knowledge Conference 2011
byaneeshabakharia
 
Latent Semantic Transliteration using Dirichlet Mixture
byRakuten Group, Inc.
 
BigML Summer 2016 Release
byBigML, Inc
 
An approach to source code plagiarism
byvarsha_bhat
 

Similar to A Semantics-based Approach to Machine Perception

PPTX
Semantic perception tkp(1)
byArtificial Intelligence Institute at UofSC
 
PPTX
Seminar
byravi raushan kumar chaudhary
 
PPTX
Ingredients for Semantic Sensor Networks
byOscar Corcho
 
PDF
Big dataprocessing cts2015
byArtificial Intelligence Institute at UofSC
 
PPTX
Active Perception over Machine and Citizen Sensing
byCory Andrew Henson
 
PPTX
An Efficient Bit Vector Approach to Semantics-based Machine Perception in Res...
byCory Andrew Henson
 
PPT
Dynamic Semantics for Semantics for Dynamic IoT Environments
byPayamBarnaghi
 
PPTX
Spatial Semantics for Better Interoperability and Analysis: Challenges and Ex...
byAmit Sheth
 
PPTX
Semantic Sensor Networks and Linked Stream Data
byOscar Corcho
 
PPTX
Computing for Human Experience: Sensors, Perception, Semantics, Social Comput...
byAmit Sheth
 
PPTX
Computing for Human Experience [v3, Aug-Oct 2010]
byArtificial Intelligence Institute at UofSC
 
PPTX
Spatial Semantics for Better Interoperability and Analysis: Challenges and Ex...
byArtificial Intelligence Institute at UofSC
 
PPT
Semantic Technologies for the Internet of Things: Challenges and Opportunities
byPayamBarnaghi
 
PPTX
Semantic Web: introduction & overview
byAmit Sheth
 
PPTX
Making the Web Searchable - Keynote ICWE 2015
byPeter Mika
 
PPTX
(Keynote) Peter Mika - “Making the Web Searchable”
byicwe2015
 
PPTX
Ontology Tutorial: Semantic Technology for Intelligence, Defense and Security
byBarry Smith
 
PDF
Semantic Web concepts used in Web 3.0 applications
byIRJET Journal
 
PPT
Spivack Blogtalk 2008
byBlogtalk 2008
 
PPT
4 semantic web and ontology
bySanthosh Kannan
 
Semantic perception tkp(1)
byArtificial Intelligence Institute at UofSC
 
Seminar
byravi raushan kumar chaudhary
 
Ingredients for Semantic Sensor Networks
byOscar Corcho
 
Big dataprocessing cts2015
byArtificial Intelligence Institute at UofSC
 
Active Perception over Machine and Citizen Sensing
byCory Andrew Henson
 
An Efficient Bit Vector Approach to Semantics-based Machine Perception in Res...
byCory Andrew Henson
 
Dynamic Semantics for Semantics for Dynamic IoT Environments
byPayamBarnaghi
 
Spatial Semantics for Better Interoperability and Analysis: Challenges and Ex...
byAmit Sheth
 
Semantic Sensor Networks and Linked Stream Data
byOscar Corcho
 
Computing for Human Experience: Sensors, Perception, Semantics, Social Comput...
byAmit Sheth
 
Computing for Human Experience [v3, Aug-Oct 2010]
byArtificial Intelligence Institute at UofSC
 
Spatial Semantics for Better Interoperability and Analysis: Challenges and Ex...
byArtificial Intelligence Institute at UofSC
 
Semantic Technologies for the Internet of Things: Challenges and Opportunities
byPayamBarnaghi
 
Semantic Web: introduction & overview
byAmit Sheth
 
Making the Web Searchable - Keynote ICWE 2015
byPeter Mika
 
(Keynote) Peter Mika - “Making the Web Searchable”
byicwe2015
 
Ontology Tutorial: Semantic Technology for Intelligence, Defense and Security
byBarry Smith
 
Semantic Web concepts used in Web 3.0 applications
byIRJET Journal
 
Spivack Blogtalk 2008
byBlogtalk 2008
 
4 semantic web and ontology
bySanthosh Kannan
 

Recently uploaded

PPTX
Greengnorance Toolkit Module 4 Active Mobility and Transport
byKarl Donert
 
PPTX
How to Track & Manage My Time Section in Odoo 18 Time Off
byCeline George
 
PDF
Bones by Sadu Kassam (play) story ppt pdf
byRonnieMaeBorres
 
PPTX
Renal Physiology- Functional anatomy of Kidney.pptx
byDisha Soriya
 
PPTX
Plato's Insight model teaching and learning.pptx
byNikhitaDS
 
PPTX
Overview of How to set priority in Odoo 19 Todo
byCeline George
 
PPTX
Fluorimetric Analysis- Theory, Instrumentation and Application
bySayali Powar
 
PDF
Orlando by Virginia Woolf: The Granite and The Rainbow
byAdityarajsinh Gohil
 
PDF
Structure-of-the-Atom PPT.pdf/CBSE CLASS 11 CHEMISTRY/ PREPARED BY K SANDEEP ...
bySandeep Swamy
 
PPTX
Q4_PPT MUSIC & ARTS 7 week 1-2, matatag curriculum
byZEN
 
PDF
Beyond the Absurd: A Comparative Reading of Waiting for Godot and the Bhagava...
byKruti Vyas
 
PDF
Artificial Intelligence in Research and Academic Writing, Workshop on Researc...
byProf. Vinod Kumar Kanvaria
 
PDF
The Sheep and the Goat: Beckett’s Subversion of Divine Justice in Waiting for...
bysejadchokiya
 
PDF
Workshop 29 Crystal Review All Students by YogiGoddess
by©LDMMIA, ©Reiki Yoga
 
PDF
Conservation of Earthen Structures in India Preserving Traditional and Sustai...
byAman Kumar Singh
 
PDF
Judgement Regarding Land Acquisition Act not Applicable to Encroachers.pdf
byssuser9d8e4e
 
PDF
PHARMACOLOGY 1 ( BP 404 T) Unit 1 (Cology 1)
byChetan Mali
 
PPTX
Renal Physiology- Juxtaglomerular Apparatus.pptx
byDisha Soriya
 
PDF
java full stack programming and interview questions
byrajuashokit
 
PPTX
WEEK 2 (2).pptx TLE COOKERY 10 QUARTER 4
byResynFayeCortez2
 
Greengnorance Toolkit Module 4 Active Mobility and Transport
byKarl Donert
 
How to Track & Manage My Time Section in Odoo 18 Time Off
byCeline George
 
Bones by Sadu Kassam (play) story ppt pdf
byRonnieMaeBorres
 
Renal Physiology- Functional anatomy of Kidney.pptx
byDisha Soriya
 
Plato's Insight model teaching and learning.pptx
byNikhitaDS
 
Overview of How to set priority in Odoo 19 Todo
byCeline George
 
Fluorimetric Analysis- Theory, Instrumentation and Application
bySayali Powar
 
Orlando by Virginia Woolf: The Granite and The Rainbow
byAdityarajsinh Gohil
 
Structure-of-the-Atom PPT.pdf/CBSE CLASS 11 CHEMISTRY/ PREPARED BY K SANDEEP ...
bySandeep Swamy
 
Q4_PPT MUSIC & ARTS 7 week 1-2, matatag curriculum
byZEN
 
Beyond the Absurd: A Comparative Reading of Waiting for Godot and the Bhagava...
byKruti Vyas
 
Artificial Intelligence in Research and Academic Writing, Workshop on Researc...
byProf. Vinod Kumar Kanvaria
 
The Sheep and the Goat: Beckett’s Subversion of Divine Justice in Waiting for...
bysejadchokiya
 
Workshop 29 Crystal Review All Students by YogiGoddess
by©LDMMIA, ©Reiki Yoga
 
Conservation of Earthen Structures in India Preserving Traditional and Sustai...
byAman Kumar Singh
 
Judgement Regarding Land Acquisition Act not Applicable to Encroachers.pdf
byssuser9d8e4e
 
PHARMACOLOGY 1 ( BP 404 T) Unit 1 (Cology 1)
byChetan Mali
 
Renal Physiology- Juxtaglomerular Apparatus.pptx
byDisha Soriya
 
java full stack programming and interview questions
byrajuashokit
 
WEEK 2 (2).pptx TLE COOKERY 10 QUARTER 4
byResynFayeCortez2
 

A Semantics-based Approach to Machine Perception

  • 1.
    A SEMANTICS-BASED APPROACH TOMACHINE PERCEPTION Cory Andrew Henson August 27, 2013 1 Committee: Amit Sheth (advisor) Krishnaprasad Thirunarayan John Gallagher Payam Barnaghi Satya Sahoo Ph.D. Dissertation Defense Wright State University Dept. of Computer Science and Engineering
  • 2.
  • 3.
    3 Thesis Machine perception canbe formalized using semantic web technologies in order to derive abstractions from sensor data using background knowledge on the Web, and efficiently executed on resource-constrained devices.
  • 4.
    4 3 primary issuesto be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices 1 2 3
  • 5.
    5 lives in has pet isahas pet Person Animal Concrete Facts Resource Description Framework Semantic Web (according to Farside) General Knowledge Web Ontology Language “Now! – That should clear up a few things around here!” is a
  • 6.
  • 7.
  • 8.
    8 Sensor systems aretoo often stovepiped
  • 9.
    9 With freedom comesresponsibility 1. discovery, access, and search 2. integration and interpretation We want to set this data free
  • 10.
    10 OGC Sensor WebEnablement (SWE)
  • 11.
    11 With freedom comesresponsibility 1. discovery, access, and search 2. integration and interpretation We want to set this data free
  • 12.
    12 RDF OWL How aremachines supposed to integrate and interpret sensor data? Semantic Sensor Networks (SSN)
  • 13.
    13 W3C Semantic SensorNetwork Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
  • 14.
    14 W3C Semantic SensorNetwork Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
  • 15.
    15 W3C Semantic SensorNetwork Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
  • 16.
    16 Semantic Annotation ofSWE Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
  • 17.
    Semantic Sensor ObservationService (SemSOS) Cory Henson, Josh Pschorr,Amit Sheth, Krishnaprasad Thirunarayan, SemSOS: Semantic Sensor Observation Service, In Proceedings of the 2009 International Symposium on Collaborative Technologies and Systems (CTS 2009), Baltimore, MD, May 18-22, 2009. 17
  • 18.
    Semantic Sensor ObservationService (SemSOS) Joshua Pschorr, Cory Henson, Harshal Patni, and Amit P. Sheth. Sensor Discovery on Linked Data. Kno.e.sis Center Technical Report 2010. 18
  • 19.
    19 3 primary issuesto be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices1 2 3
  • 20.
    20 Semantic Perception • Therole of perception is to transform raw sensory data into a meaningful and correct representation of the external world. • The systematic automation of this ability is the focus of machine perception. • For correct interpretation of this representation, we need a formal account of how this is done.
  • 21.
    21 What can welearn from cognitive models of perception? People are good at making sense of sensory input
  • 22.
    22 Perception is anactive, cyclical process of exploration and interpretation. The perception cycle is driven by prior knowledge, in order to generate and test hypotheses. Some observations are more informative than others (in order to effectively test hypotheses*). Ulric Neisser Richard Gregory Kenneth Norwich 1970’s 1980’s 1990’s Cognitive Models of Perception * Applies to machine perception within Intellego, NOT human perception.
  • 23.
    Observed Properties Perceived Features 23 Background knowledge Explanation Ontology ofPerception Focus An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
  • 24.
  • 25.
  • 26.
    The Patient ofthe Future MIT Technology Review, 2012 26http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
  • 27.
    Digital Doctor Let’s providepeople with the tools needed to monitor and manage their own health http://worldofdtcmarketing.com/mobile-health-apps-a-new-opportunity-for-healthcare-marketers/mobile-healthcare-marketing-trends/ 27
  • 28.
    Medical/healthcare expert systemshave been around for a long time 28
  • 29.
    1. Ubiquitous Sensing2. Always-on Computing 3. Knowledge on the Web 3 recent developments have changed the technological landscape … 29
  • 30.
    30 Making sense ofsensor data with
  • 31.
    DATA sensor observations KNOWLEDGE situation awarenessuseful for decision making Primary challenge is to bridge the gap between data and knowledge 31
  • 32.
    SSN Ontology 2 Interpreted data (deductive) [inOWL] e.g., threshold 1 Annotated Data [in RDF] e.g., label 0 Raw Data [in TEXT] e.g., number Levels of Abstraction 3 Interpreted data (abductive) [in OWL] e.g., diagnosis Intellego ―150‖ Systolic blood pressure of 150 mmHg Elevated Blood Pressure Hyperthyroidism …… 32
  • 33.
    Observed Properties Perceived Features 33 Background knowledge on theWeb Low-level observed properties suggest explanatory hypotheses through abduction Explanation Focus Ontology of Perception An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
  • 34.
    34 Semantics of Explanation Abduction– or, inference to the best EXPLANATION Task • Given background knowledge of the environment (SIGMA), and • given a set of sensor observation data (RHO), • find a consistent explanation of the situation (DELTA) Background knowledge Features (objects/events) in the world Sensor observation data
  • 35.
    35 Semantics of Explanation Backgroundknowledge is represented as a causal network between features (objects or events) in the world and the sensor observations they give rise to.
  • 36.
    36 Semantics of Explanation Findingthe sweet spot between abduction and OWL • Simulation of Parsimonious Covering Theory in OWL-DL (using the single-feature assumption*) * An explanation must be a single feature which accounts for all observed properties Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012) Theorem: Given a PCT problem P and its translation o(P) into OWL, Δ = {e} is a PCT explanation if and only if ExplanatoryFeature(e) is deduced by an OWL-DL reasoner — that is, if and only if o(P) ⊧ ExplanatoryFeature(e).
  • 37.
    37 Finding the SweetSpot minimize explanations degrade gracefully w/ incomplete info decidable web reasoning Abductive Logic (e.g., PCT) high complexity Deductive Logic (e.g., OWL) (relatively) low complexity Explanation (in Intellego)
  • 38.
    Explanatory Feature: Afeature is explanatory w.r.t. a set of observed properties if it causes each property in the set. ExplanatoryFeature ≡ ∃isPropertyOf—.{p1} ⊓ … ⊓ ∃isPropertyOf—.{pn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Observed Property Explanatory Feature 38 Semantics of Explanation
  • 39.
    Observed Properties Perceived Features 39 Background knowledge on theWeb Hypotheses imply the informational value of future observations through deduction Explanation Focus Ontology of Perception An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
  • 40.
    Universe of observableproperties 40 Semantics of Focus To predict which future observations have informational value, find those observable properties that can discriminate between the set of hypotheses. Expected Properties Not-applicable Properties Discriminating Properties
  • 41.
    Expected Property: Aproperty is expected w.r.t. a set of features if it is caused by each feature in the set. ExpectedProperty ≡ ∃isPropertyOf.{f1} ⊓ … ⊓ ∃isPropertyOf.{fn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Expected Property Explanatory Feature 41 Semantics of Focus
  • 42.
    Not Applicable Property:A property is not-applicable w.r.t. a set of features if it is not caused by any feature in the set. NotApplicableProperty ≡ ¬∃isPropertyOf.{f1} ⊓ … ⊓ ¬∃isPropertyOf.{fn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Not Applicable Property Explanatory Feature 42 Semantics of Focus
  • 43.
    Discriminating Property: Aproperty is discriminating w.r.t. a set of features if it is neither expected nor not-applicable. DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Discriminating Property Explanatory Feature 43 Semantics of Focus
  • 44.
    Off-the-shelf OWL-DL reasonersare too resource intensive in terms of both memory and time • Runs out of resources with background knowledge >> 20 nodes • Asymptotic complexity: O(n3) 44 O(n3) < x < O(n4) Semantic perception on resource-constrained devices
  • 45.
    45 3 primary issuesto be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices1 2 3
  • 46.
  • 47.
  • 48.
    48 Basis Watch • HeartRate Monitor • Accelerometer • Skin Temperature • Galvanic Skin Response
  • 49.
    Homo Digitus 49 How dowe make sense of this data … and do it efficiently and at scale?
  • 50.
    Approach 1: Sendall sensor observations to the cloud for processing 50 Approach 2: downscale semantic processing so that each device is capable of machine perception Intelligence at the Edge
  • 51.
    Use bit vectorencodings and their operations to encode background knowledge and execute perceptual inference 51 Efficient execution of semantic perception 01100011 10001110 11001110 01010110 01110101 OWL-DL An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (ISWC, 2012)
  • 52.
    lift lower Translate background knowledge,observations, and explanations between Semantic Web and bit vector representation 52 Lifting and lowering of knowledge
  • 53.
    53 Efficient execution ofsemantic perception Bit vector algorithms are provably equivalent to the OWL inference (i.e., semantics preserving) Intuition: discover and dismiss those features that cannot explain the set of observed properties. Intuition: discover and assemble those properties that discriminate between the explanatory features
  • 54.
    bp 1 cs 0 pa1 HN HM PE bp 1 1 1 cs 0 1 0 pa 1 1 0 HN HM PE 1 1 1 HN HM PE 1 1 1 1 1 0 AND => AND 1 1 1 Observed Property Prior Knowledge Previous Explanatory Feature Current Explanatory Feature => INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties. 54 Explanation: efficient algorithm
  • 55.
    bp 1 cs 0 pa1 HN HM PE bp 0 1 1 cs 0 1 0 pa 1 1 0 HN HM PE 1 1 0 HN HM PE 0 1 0AND => 0 1 0 Observed Property Prior Knowledge Previous Explanatory Feature Current Explanatory Feature = INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble those properties that discriminate between the explanatory features bp 0 cs 0 pa 0 Discriminatin g Property 1 1 0 … expected? 0 0 0 … not-applicable? ZERO Bit Vector 0 1 0 = => FALSE => FALSE 1 Is the property discriminating? 55 Focus: efficient algorithm
  • 56.
    O(n3) < x< O(n4) O(n) 56 Evaluation on a mobile device Efficiency Improvement • Problem size increased from 10’s to 1000’s of nodes • Time reduced from minutes to milliseconds • Complexity growth reduced from polynomial to linear
  • 57.
    57 Technical contributions ina nutshell 1. Semantic Sensor Web: Developed technologies for the semantic annotation of sensor data on the Web - Semantic Sensor Web (IEEE Internet Computing, 2008) – 276 citations (as of Aug. 2013) - SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009) - Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011) 2. Semantic Perception: Designed a declarative specification of perception, capable of utilizing an off-the-shelf OWL-DL reasoner - An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology Journal, 2011) - Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing: Special Issue on Context- Aware Computing, 2012) – most downloaded paid paper in IEEE-IC 2012 3. Intelligence at the Edge: Implemented efficient algorithms for executing perceptual inference on resource-constrained devices - An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012)
  • 58.
    58 W3C Reports 1. SemanticSensor Network XG Final Report (W3C Incubator Group Report, 2011) Journal Publications 1. Physical-Cyber-Social Computing: An Early 21st Century Approach (IEEE Intelligent Systems, 2013) 2. Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012) 3. Semantics for the Internet of Things: Early Progress and Back to the Future (International Journal on Semantic Web and Information Systems, 2012) 4. The SSN ontology of the W3C semantic sensor network incubator group (Web Semantics: Science, Services and Agents on the World Wide Web, 2012) 5. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011) 6. Semantic Sensor Web (IEEE Internet Computing, 2008) Conference Publications 1. An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012) 2. Computing Perception from Sensor Data (IEEE Sensors Conference, 2012) 3. SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009) 4. Situation Awareness via Abductive Reasoning for Semantic Sensor Data: A Preliminary Report (International Symposium on Collaborative Technologies and Systems, 2009). Workshop Publications 1. SECURE: Semantics Empowered Rescue Environment (International Workshop on Semantic Sensor Networks, 2011) 2. Representation of Parsimonious Covering Theory in OWL-DL (International Workshop on OWL: Experiences and Directions, 2011) 3. Provenance Aware Linked Sensor Data (Workshop on Trust and Privacy on the Social and Semantic Web, 2010) 4. Linked Sensor Data (International Symposium on Collaborative Technologies and Systems, 2010) 5. A Survey of the Semantic Specification of Sensors (International Workshop on Semantic Sensor Networks, 2009) 6. An Ontological Representation of Time Series Observations on the Semantic Sensor Web (International Workshop on the Semantic Sensor Web) 7. Video on the Semantic Sensor Web (W3C Video on the Web Workshop, 2007) Relevant Publications
  • 59.
  • 60.
    Heart disease isa critical issue ~815,000 (2011) http://millionhearts.hhs.gov/abouthds/cost-consequences.html 60
  • 61.
    Acute Decompensated HeartFailure (ADHF) • Affects nearly 6 million people (in U.S.) • 555,000 new cases are diagnosed each year 61U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
  • 62.
    • 4.8 millionhospitalizations per year • 50% are readmitted within 6 months • 25% are readmitted within 30 days • 70% due to worsening conditions • Costing $17 billion per year ADHF hospital readmission rates are too high 62U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
  • 63.
    Congress has incentivizedhospitals to lower readmission rates 63U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
  • 64.
  • 65.
    Score (0: Notat all, 1: A little, 2: A great deal, 3: Extremely) Heart Failure Somatic Awareness Scale (HFSAS) Current state-of-the-art 65
  • 66.
    kHealth – knowledge-enabledhealthcare Approach: • Use semantic perception inference • with data from cardio-related sensors • and curated medical background knowledge on the Web 1. to monitor and abstract health conditions 2. to ask the patient contextually relevant questions 66
  • 67.
    Cardiology Background Knowledge •Symptoms: 284 • Disorders: 173 • Causal Relations: 1944 Unified Medical Language System Causal Network 67
  • 68.
    kHealth Kit Weight Scale HeartRate Monitor Blood Pressure Monitor 68 Sensors Android Device (w/ kHealth App) Total cost: < $500
  • 69.
    69 Explanation in kHealth •Abnormal heart rate • High blood pressure • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Observed Property Explanatory Feature via Bluetooth
  • 70.
    70 Focus in kHealth Areyou feeling lightheaded? Are you have trouble taking deep breaths? yes yes • Abnormal heart rate • High blood pressure • Lightheaded • Trouble breathing • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Contextually dependent questioning based on prior observations (from 284 possible questions) Observed Property Explanatory Feature
  • 71.
    71 Evaluation of kHealth Evaluatethe ability to discriminate between sets of potential disorders using: 1. HFSAS/WANDA’s restricted set of observable symptoms (12) 2. kHealth’s more comprehensive set of observable symptoms (284)
  • 72.
    72 Evaluation of kHealth EvaluationMetrics: 1. Efficiency: How many observations (or questions) required to minimize the set of explanations? 2. Specificity: How specific is the resulting minimum set of explanations (i.e., how many explanatory disorders in the set)? Explanatory Disorders (computed by Intellego) Actual Disorder (extracted from EMR) Possible Disorders (derived from cardiology KB)
  • 73.
    73 Evaluation of kHealth– Early Results HFSAS/WANDA Efficiency: ~7.45 (# questions asked) Specificity: ~11.95* (# minimum explanations) * Converged to 1 explanation 20% of the time • 496 EMRs • ~3.2 diagnosed disorders per EMR • 173 possible disorders The approach utilized by kHealth is more efficient and more specific than HFSAS/WANDA. kHealth Efficiency: ~7.28 (# questions asked) Specificity: 1 (# minimum explanations) Explanatory Disorders (computed by Intellego) Actual Disorder (extracted from EMR) Possible Disorders (derived from cardiology KB)
  • 74.
    Pre-clinical usability trial Dr.William Abraham, M.D. Director of Cardiovascular Medicine 74
  • 75.
  • 76.
    Special Thanks toAFRL and DAGSI 76 AFRL/DAGSI Research Topic SN08-8: Architectures for Secure Semantic Sensor Networks for Multi-Layered Sensing
  • 77.
  • 78.
  • 79.
    A SEMANTICS-BASED APPROACH TOMACHINE PERCEPTION Cory Andrew Henson August 27, 2013 79 Thank you. For additional information visit: http://knoesis.org/researchers/cory

Editor's Notes

  • #27 - Larry Smarr is a professor at the University of California, San DiegoAnd he was diagnosed with Chrones DiseaseWhat’s interesting about this case is that Larry diagnosed himselfHe is a pioneer in the area of Quantified-Self, which uses sensors to monitor physiological symptomsThrough this process he discovered inflamation, which led him to discovery of Chrones DiseaseThis type of self-tracking is becoming more and more common
  • #49 My qs story