A SEMANTICS-BASED APPROACH
TO MACHINE PERCEPTION
Cory Andrew Henson
August 27, 2013
1
Committee:
Amit Sheth (advisor)
Kris...
2
3
Thesis
Machine perception can be formalized using semantic web technologies
in order to derive abstractions from sensor ...
4
3 primary issues to be addressed
Annotation of
sensor data
Semantic
Sensor
Web
Semantic
Perception
Intelligence
at the E...
5
lives in
has pet
is ahas pet
Person Animal
Concrete Facts
Resource Description Framework
Semantic Web
(according to Fars...
Government
Media
Publications
Life Sciences
Geographic
Semantic Web
6
7http://www.opengeospatial.org/projects/groups/sensorwebdwg
Semantic Sensor Web
8
Sensor systems are too often stovepiped
9
With freedom comes responsibility
1. discovery, access, and search
2. integration and interpretation
We want to set this...
10
OGC Sensor Web Enablement (SWE)
11
With freedom comes responsibility
1. discovery, access, and search
2. integration and interpretation
We want to set thi...
12
RDF OWL
How are machines supposed to integrate and interpret sensor data?
Semantic Sensor Networks (SSN)
13
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-...
14
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-...
15
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-...
16
Semantic Annotation of SWE
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R....
Semantic Sensor Observation Service (SemSOS)
Cory Henson, Josh Pschorr,Amit Sheth, Krishnaprasad Thirunarayan, SemSOS: Sem...
Semantic Sensor Observation Service (SemSOS)
Joshua Pschorr, Cory Henson, Harshal Patni, and Amit P. Sheth. Sensor Discove...
19
3 primary issues to be addressed
Annotation of
sensor data
Semantic
Sensor
Web
Semantic
Perception
Intelligence
at the ...
20
Semantic Perception
• The role of perception is to transform raw sensory
data into a meaningful and correct representat...
21
What can we learn from
cognitive models of perception?
People are good at
making sense
of sensory input
22
Perception is an active,
cyclical process of exploration
and interpretation.
The perception cycle is driven by
prior kn...
Observed
Properties
Perceived
Features
23
Background knowledge
Explanation
Ontology of Perception
Focus
An Ontological App...
Example: Medical diagnosis as perception
24
Proactive,
Preventative
Healthcare
25
The Patient of the Future
MIT Technology Review, 2012
26http://www.technologyreview.com/featuredstory/426968/the-patient-o...
Digital Doctor
Let’s provide people with the
tools needed to monitor and
manage their own health
http://worldofdtcmarketin...
Medical/healthcare expert systems have been around for a long time
28
1. Ubiquitous Sensing 2. Always-on Computing 3. Knowledge on the Web
3 recent developments have changed the technological ...
30
Making sense of sensor data with
DATA
sensor observations
KNOWLEDGE
situation awareness useful
for decision making
Primary challenge is to bridge the gap b...
SSN
Ontology
2 Interpreted data
(deductive)
[in OWL]
e.g., threshold
1 Annotated Data
[in RDF]
e.g., label
0 Raw Data
[in ...
Observed
Properties
Perceived
Features
33
Background knowledge
on the Web
Low-level observed properties suggest
explanator...
34
Semantics of Explanation
Abduction – or, inference to the best EXPLANATION
Task
• Given background knowledge of the env...
35
Semantics of Explanation
Background knowledge is represented as a causal network between features (objects
or events) i...
36
Semantics of Explanation
Finding the sweet spot between abduction and OWL
• Simulation of Parsimonious Covering Theory ...
37
Finding the Sweet Spot
minimize
explanations
degrade gracefully
w/ incomplete info
decidable
web reasoning
Abductive Lo...
Explanatory Feature: A feature is explanatory w.r.t. a set of observed properties if it
causes each property in the set.
E...
Observed
Properties
Perceived
Features
39
Background knowledge
on the Web
Hypotheses imply the informational value of
futu...
Universe of observable properties
40
Semantics of Focus
To predict which future observations have informational value, fin...
Expected Property: A property is expected w.r.t. a set of features if it is caused by each
feature in the set.
ExpectedPro...
Not Applicable Property: A property is not-applicable w.r.t. a set of features if it is not
caused by any feature in the s...
Discriminating Property: A property is discriminating w.r.t. a set of features if it is
neither expected nor not-applicabl...
Off-the-shelf OWL-DL reasoners are too
resource intensive in terms of both
memory and time
• Runs out of resources with
ba...
45
3 primary issues to be addressed
Annotation of
sensor data
Semantic
Sensor
Web
Semantic
Perception
Intelligence
at the ...
Internet of Things
46http://www.idgconnect.com/blog-abstract/900/the-internet-things-breaking-down-barriers-connected-world
47http://share.cisco.com/internet-of-things.html
48
Basis Watch
• Heart Rate Monitor
• Accelerometer
• Skin Temperature
• Galvanic Skin Response
Homo Digitus
49
How do we make sense of this data
… and do it efficiently and at scale?
Approach 1: Send all sensor
observations to the cloud for
processing
50
Approach 2: downscale semantic
processing so that ...
Use bit vector encodings and their operations to encode background knowledge and
execute perceptual inference
51
Efficient...
lift
lower
Translate background knowledge, observations, and explanations between Semantic
Web and bit vector representati...
53
Efficient execution of semantic perception
Bit vector algorithms are provably equivalent to the OWL inference (i.e., se...
bp 1
cs 0
pa 1
HN HM PE
bp 1 1 1
cs 0 1 0
pa 1 1 0
HN HM PE
1 1 1
HN HM PE
1 1 1
1 1 0
AND =>
AND
1 1 1
Observed
Property ...
bp 1
cs 0
pa 1
HN HM PE
bp 0 1 1
cs 0 1 0
pa 1 1 0
HN HM PE
1 1 0
HN HM PE
0 1 0AND => 0 1 0
Observed
Property Prior Knowl...
O(n3) < x < O(n4) O(n)
56
Evaluation on a mobile device
Efficiency Improvement
• Problem size increased from 10’s to 1000’...
57
Technical contributions in a nutshell
1. Semantic Sensor Web: Developed technologies for the semantic annotation
of sen...
58
W3C Reports
1. Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011)
Journal Publications
1. Physi...
59
Application
Proactive,
preventative
healthcare
Heart disease is a critical issue
~815,000 (2011)
http://millionhearts.hhs.gov/abouthds/cost-consequences.html 60
Acute Decompensated Heart Failure (ADHF)
• Affects nearly 6 million people (in U.S.)
• 555,000 new cases are diagnosed eac...
• 4.8 million hospitalizations per year
• 50% are readmitted within 6 months
• 25% are readmitted within 30 days
• 70% due...
Congress has incentivized hospitals to lower readmission rates
63U.S. Department of Health & Human Services. (2011). Hospi...
Current state-of-the-art
64
Score (0: Not at all, 1: A little, 2: A great deal, 3: Extremely)
Heart Failure Somatic Awareness Scale (HFSAS)
Current st...
kHealth – knowledge-enabled healthcare
Approach:
• Use semantic perception inference
• with data from cardio-related senso...
Cardiology Background Knowledge
• Symptoms: 284
• Disorders: 173
• Causal Relations: 1944
Unified Medical Language System
...
kHealth Kit
Weight Scale
Heart Rate Monitor
Blood Pressure
Monitor
68
Sensors
Android Device
(w/ kHealth App)
Total cost: ...
69
Explanation in kHealth
• Abnormal heart rate
• High blood pressure
• Panic Disorder
• Hypoglycemia
• Hyperthyroidism
• ...
70
Focus in kHealth
Are you feeling lightheaded?
Are you have trouble taking deep breaths?
yes
yes
• Abnormal heart rate
•...
71
Evaluation of kHealth
Evaluate the ability to discriminate between sets of potential disorders
using:
1. HFSAS/WANDA’s ...
72
Evaluation of kHealth
Evaluation Metrics:
1. Efficiency: How many observations (or questions) required to minimize the
...
73
Evaluation of kHealth – Early Results
HFSAS/WANDA
Efficiency: ~7.45 (# questions asked)
Specificity: ~11.95* (# minimum...
Pre-clinical usability trial
Dr. William Abraham, M.D.
Director of Cardiovascular Medicine
74
Sensing
and
Perception
Health
Care
Academic
Standards
Org.
Industry Government
Research
Collaborators
by Research Topic
an...
Special Thanks to AFRL and DAGSI
76
AFRL/DAGSI Research Topic SN08-8:
Architectures for Secure Semantic Sensor Networks fo...
77
Semantic Sensor Web Team
78
A SEMANTICS-BASED APPROACH
TO MACHINE PERCEPTION
Cory Andrew Henson
August 27, 2013
79
Thank you.
For additional informati...
Upcoming SlideShare
Loading in …5
×

A Semantics-based Approach to Machine Perception

2,748 views

Published on

Cory Henson defended his thesis on "A Semantics-based Approach to Machine Perception".
Video can be found at: http://www.youtube.com/watch?v=L8M7eoGKtSE

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,748
On SlideShare
0
From Embeds
0
Number of Embeds
2,084
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • - Larry Smarr is a professor at the University of California, San DiegoAnd he was diagnosed with Chrones DiseaseWhat’s interesting about this case is that Larry diagnosed himselfHe is a pioneer in the area of Quantified-Self, which uses sensors to monitor physiological symptomsThrough this process he discovered inflamation, which led him to discovery of Chrones DiseaseThis type of self-tracking is becoming more and more common
  • My qs story
  • A Semantics-based Approach to Machine Perception

    1. 1. A SEMANTICS-BASED APPROACH TO MACHINE PERCEPTION Cory Andrew Henson August 27, 2013 1 Committee: Amit Sheth (advisor) Krishnaprasad Thirunarayan John Gallagher Payam Barnaghi Satya Sahoo Ph.D. Dissertation Defense Wright State University Dept. of Computer Science and Engineering
    2. 2. 2
    3. 3. 3 Thesis Machine perception can be formalized using semantic web technologies in order to derive abstractions from sensor data using background knowledge on the Web, and efficiently executed on resource-constrained devices.
    4. 4. 4 3 primary issues to be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices 1 2 3
    5. 5. 5 lives in has pet is ahas pet Person Animal Concrete Facts Resource Description Framework Semantic Web (according to Farside) General Knowledge Web Ontology Language “Now! – That should clear up a few things around here!” is a
    6. 6. Government Media Publications Life Sciences Geographic Semantic Web 6
    7. 7. 7http://www.opengeospatial.org/projects/groups/sensorwebdwg Semantic Sensor Web
    8. 8. 8 Sensor systems are too often stovepiped
    9. 9. 9 With freedom comes responsibility 1. discovery, access, and search 2. integration and interpretation We want to set this data free
    10. 10. 10 OGC Sensor Web Enablement (SWE)
    11. 11. 11 With freedom comes responsibility 1. discovery, access, and search 2. integration and interpretation We want to set this data free
    12. 12. 12 RDF OWL How are machines supposed to integrate and interpret sensor data? Semantic Sensor Networks (SSN)
    13. 13. 13 W3C Semantic Sensor Network Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
    14. 14. 14 W3C Semantic Sensor Network Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
    15. 15. 15 W3C Semantic Sensor Network Ontology Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
    16. 16. 16 Semantic Annotation of SWE Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
    17. 17. Semantic Sensor Observation Service (SemSOS) Cory Henson, Josh Pschorr,Amit Sheth, Krishnaprasad Thirunarayan, SemSOS: Semantic Sensor Observation Service, In Proceedings of the 2009 International Symposium on Collaborative Technologies and Systems (CTS 2009), Baltimore, MD, May 18-22, 2009. 17
    18. 18. Semantic Sensor Observation Service (SemSOS) Joshua Pschorr, Cory Henson, Harshal Patni, and Amit P. Sheth. Sensor Discovery on Linked Data. Kno.e.sis Center Technical Report 2010. 18
    19. 19. 19 3 primary issues to be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices1 2 3
    20. 20. 20 Semantic Perception • The role of perception is to transform raw sensory data into a meaningful and correct representation of the external world. • The systematic automation of this ability is the focus of machine perception. • For correct interpretation of this representation, we need a formal account of how this is done.
    21. 21. 21 What can we learn from cognitive models of perception? People are good at making sense of sensory input
    22. 22. 22 Perception is an active, cyclical process of exploration and interpretation. The perception cycle is driven by prior knowledge, in order to generate and test hypotheses. Some observations are more informative than others (in order to effectively test hypotheses*). Ulric Neisser Richard Gregory Kenneth Norwich 1970’s 1980’s 1990’s Cognitive Models of Perception * Applies to machine perception within Intellego, NOT human perception.
    23. 23. Observed Properties Perceived Features 23 Background knowledge Explanation Ontology of Perception Focus An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
    24. 24. Example: Medical diagnosis as perception 24
    25. 25. Proactive, Preventative Healthcare 25
    26. 26. The Patient of the Future MIT Technology Review, 2012 26http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
    27. 27. Digital Doctor Let’s provide people with the tools needed to monitor and manage their own health http://worldofdtcmarketing.com/mobile-health-apps-a-new-opportunity-for-healthcare-marketers/mobile-healthcare-marketing-trends/ 27
    28. 28. Medical/healthcare expert systems have been around for a long time 28
    29. 29. 1. Ubiquitous Sensing 2. Always-on Computing 3. Knowledge on the Web 3 recent developments have changed the technological landscape … 29
    30. 30. 30 Making sense of sensor data with
    31. 31. DATA sensor observations KNOWLEDGE situation awareness useful for decision making Primary challenge is to bridge the gap between data and knowledge 31
    32. 32. SSN Ontology 2 Interpreted data (deductive) [in OWL] e.g., threshold 1 Annotated Data [in RDF] e.g., label 0 Raw Data [in TEXT] e.g., number Levels of Abstraction 3 Interpreted data (abductive) [in OWL] e.g., diagnosis Intellego ―150‖ Systolic blood pressure of 150 mmHg Elevated Blood Pressure Hyperthyroidism …… 32
    33. 33. Observed Properties Perceived Features 33 Background knowledge on the Web Low-level observed properties suggest explanatory hypotheses through abduction Explanation Focus Ontology of Perception An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
    34. 34. 34 Semantics of Explanation Abduction – or, inference to the best EXPLANATION Task • Given background knowledge of the environment (SIGMA), and • given a set of sensor observation data (RHO), • find a consistent explanation of the situation (DELTA) Background knowledge Features (objects/events) in the world Sensor observation data
    35. 35. 35 Semantics of Explanation Background knowledge is represented as a causal network between features (objects or events) in the world and the sensor observations they give rise to.
    36. 36. 36 Semantics of Explanation Finding the sweet spot between abduction and OWL • Simulation of Parsimonious Covering Theory in OWL-DL (using the single-feature assumption*) * An explanation must be a single feature which accounts for all observed properties Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012) Theorem: Given a PCT problem P and its translation o(P) into OWL, Δ = {e} is a PCT explanation if and only if ExplanatoryFeature(e) is deduced by an OWL-DL reasoner — that is, if and only if o(P) ⊧ ExplanatoryFeature(e).
    37. 37. 37 Finding the Sweet Spot minimize explanations degrade gracefully w/ incomplete info decidable web reasoning Abductive Logic (e.g., PCT) high complexity Deductive Logic (e.g., OWL) (relatively) low complexity Explanation (in Intellego)
    38. 38. Explanatory Feature: A feature is explanatory w.r.t. a set of observed properties if it causes each property in the set. ExplanatoryFeature ≡ ∃isPropertyOf—.{p1} ⊓ … ⊓ ∃isPropertyOf—.{pn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Observed Property Explanatory Feature 38 Semantics of Explanation
    39. 39. Observed Properties Perceived Features 39 Background knowledge on the Web Hypotheses imply the informational value of future observations through deduction Explanation Focus Ontology of Perception An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
    40. 40. Universe of observable properties 40 Semantics of Focus To predict which future observations have informational value, find those observable properties that can discriminate between the set of hypotheses. Expected Properties Not-applicable Properties Discriminating Properties
    41. 41. Expected Property: A property is expected w.r.t. a set of features if it is caused by each feature in the set. ExpectedProperty ≡ ∃isPropertyOf.{f1} ⊓ … ⊓ ∃isPropertyOf.{fn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Expected Property Explanatory Feature 41 Semantics of Focus
    42. 42. Not Applicable Property: A property is not-applicable w.r.t. a set of features if it is not caused by any feature in the set. NotApplicableProperty ≡ ¬∃isPropertyOf.{f1} ⊓ … ⊓ ¬∃isPropertyOf.{fn} elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Not Applicable Property Explanatory Feature 42 Semantics of Focus
    43. 43. Discriminating Property: A property is discriminating w.r.t. a set of features if it is neither expected nor not-applicable. DiscriminatingProperty ≡ ¬ExpectedProperty ⊓ ¬NotApplicableProperty elevated blood pressure clammy skin palpitations Hypertension Hyperthyroidism Pulmonary Edema Discriminating Property Explanatory Feature 43 Semantics of Focus
    44. 44. Off-the-shelf OWL-DL reasoners are too resource intensive in terms of both memory and time • Runs out of resources with background knowledge >> 20 nodes • Asymptotic complexity: O(n3) 44 O(n3) < x < O(n4) Semantic perception on resource-constrained devices
    45. 45. 45 3 primary issues to be addressed Annotation of sensor data Semantic Sensor Web Semantic Perception Intelligence at the Edge Interpretation of sensor data Efficient execution on resource-constrained devices1 2 3
    46. 46. Internet of Things 46http://www.idgconnect.com/blog-abstract/900/the-internet-things-breaking-down-barriers-connected-world
    47. 47. 47http://share.cisco.com/internet-of-things.html
    48. 48. 48 Basis Watch • Heart Rate Monitor • Accelerometer • Skin Temperature • Galvanic Skin Response
    49. 49. Homo Digitus 49 How do we make sense of this data … and do it efficiently and at scale?
    50. 50. Approach 1: Send all sensor observations to the cloud for processing 50 Approach 2: downscale semantic processing so that each device is capable of machine perception Intelligence at the Edge
    51. 51. Use bit vector encodings and their operations to encode background knowledge and execute perceptual inference 51 Efficient execution of semantic perception 01100011 10001110 11001110 01010110 01110101 OWL-DL An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (ISWC, 2012)
    52. 52. lift lower Translate background knowledge, observations, and explanations between Semantic Web and bit vector representation 52 Lifting and lowering of knowledge
    53. 53. 53 Efficient execution of semantic perception Bit vector algorithms are provably equivalent to the OWL inference (i.e., semantics preserving) Intuition: discover and dismiss those features that cannot explain the set of observed properties. Intuition: discover and assemble those properties that discriminate between the explanatory features
    54. 54. bp 1 cs 0 pa 1 HN HM PE bp 1 1 1 cs 0 1 0 pa 1 1 0 HN HM PE 1 1 1 HN HM PE 1 1 1 1 1 0 AND => AND 1 1 1 Observed Property Prior Knowledge Previous Explanatory Feature Current Explanatory Feature => INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties. 54 Explanation: efficient algorithm
    55. 55. bp 1 cs 0 pa 1 HN HM PE bp 0 1 1 cs 0 1 0 pa 1 1 0 HN HM PE 1 1 0 HN HM PE 0 1 0AND => 0 1 0 Observed Property Prior Knowledge Previous Explanatory Feature Current Explanatory Feature = INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble those properties that discriminate between the explanatory features bp 0 cs 0 pa 0 Discriminatin g Property 1 1 0 … expected? 0 0 0 … not-applicable? ZERO Bit Vector 0 1 0 = => FALSE => FALSE 1 Is the property discriminating? 55 Focus: efficient algorithm
    56. 56. O(n3) < x < O(n4) O(n) 56 Evaluation on a mobile device Efficiency Improvement • Problem size increased from 10’s to 1000’s of nodes • Time reduced from minutes to milliseconds • Complexity growth reduced from polynomial to linear
    57. 57. 57 Technical contributions in a nutshell 1. Semantic Sensor Web: Developed technologies for the semantic annotation of sensor data on the Web - Semantic Sensor Web (IEEE Internet Computing, 2008) – 276 citations (as of Aug. 2013) - SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009) - Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011) 2. Semantic Perception: Designed a declarative specification of perception, capable of utilizing an off-the-shelf OWL-DL reasoner - An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology Journal, 2011) - Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing: Special Issue on Context- Aware Computing, 2012) – most downloaded paid paper in IEEE-IC 2012 3. Intelligence at the Edge: Implemented efficient algorithms for executing perceptual inference on resource-constrained devices - An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012)
    58. 58. 58 W3C Reports 1. Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011) Journal Publications 1. Physical-Cyber-Social Computing: An Early 21st Century Approach (IEEE Intelligent Systems, 2013) 2. Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012) 3. Semantics for the Internet of Things: Early Progress and Back to the Future (International Journal on Semantic Web and Information Systems, 2012) 4. The SSN ontology of the W3C semantic sensor network incubator group (Web Semantics: Science, Services and Agents on the World Wide Web, 2012) 5. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011) 6. Semantic Sensor Web (IEEE Internet Computing, 2008) Conference Publications 1. An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012) 2. Computing Perception from Sensor Data (IEEE Sensors Conference, 2012) 3. SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009) 4. Situation Awareness via Abductive Reasoning for Semantic Sensor Data: A Preliminary Report (International Symposium on Collaborative Technologies and Systems, 2009). Workshop Publications 1. SECURE: Semantics Empowered Rescue Environment (International Workshop on Semantic Sensor Networks, 2011) 2. Representation of Parsimonious Covering Theory in OWL-DL (International Workshop on OWL: Experiences and Directions, 2011) 3. Provenance Aware Linked Sensor Data (Workshop on Trust and Privacy on the Social and Semantic Web, 2010) 4. Linked Sensor Data (International Symposium on Collaborative Technologies and Systems, 2010) 5. A Survey of the Semantic Specification of Sensors (International Workshop on Semantic Sensor Networks, 2009) 6. An Ontological Representation of Time Series Observations on the Semantic Sensor Web (International Workshop on the Semantic Sensor Web) 7. Video on the Semantic Sensor Web (W3C Video on the Web Workshop, 2007) Relevant Publications
    59. 59. 59 Application Proactive, preventative healthcare
    60. 60. Heart disease is a critical issue ~815,000 (2011) http://millionhearts.hhs.gov/abouthds/cost-consequences.html 60
    61. 61. Acute Decompensated Heart Failure (ADHF) • Affects nearly 6 million people (in U.S.) • 555,000 new cases are diagnosed each year 61U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
    62. 62. • 4.8 million hospitalizations per year • 50% are readmitted within 6 months • 25% are readmitted within 30 days • 70% due to worsening conditions • Costing $17 billion per year ADHF hospital readmission rates are too high 62U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
    63. 63. Congress has incentivized hospitals to lower readmission rates 63U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
    64. 64. Current state-of-the-art 64
    65. 65. Score (0: Not at all, 1: A little, 2: A great deal, 3: Extremely) Heart Failure Somatic Awareness Scale (HFSAS) Current state-of-the-art 65
    66. 66. kHealth – knowledge-enabled healthcare Approach: • Use semantic perception inference • with data from cardio-related sensors • and curated medical background knowledge on the Web 1. to monitor and abstract health conditions 2. to ask the patient contextually relevant questions 66
    67. 67. Cardiology Background Knowledge • Symptoms: 284 • Disorders: 173 • Causal Relations: 1944 Unified Medical Language System Causal Network 67
    68. 68. kHealth Kit Weight Scale Heart Rate Monitor Blood Pressure Monitor 68 Sensors Android Device (w/ kHealth App) Total cost: < $500
    69. 69. 69 Explanation in kHealth • Abnormal heart rate • High blood pressure • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Observed Property Explanatory Feature via Bluetooth
    70. 70. 70 Focus in kHealth Are you feeling lightheaded? Are you have trouble taking deep breaths? yes yes • Abnormal heart rate • High blood pressure • Lightheaded • Trouble breathing • Panic Disorder • Hypoglycemia • Hyperthyroidism • Heart Attack • Septic Shock Contextually dependent questioning based on prior observations (from 284 possible questions) Observed Property Explanatory Feature
    71. 71. 71 Evaluation of kHealth Evaluate the ability to discriminate between sets of potential disorders using: 1. HFSAS/WANDA’s restricted set of observable symptoms (12) 2. kHealth’s more comprehensive set of observable symptoms (284)
    72. 72. 72 Evaluation of kHealth Evaluation Metrics: 1. Efficiency: How many observations (or questions) required to minimize the set of explanations? 2. Specificity: How specific is the resulting minimum set of explanations (i.e., how many explanatory disorders in the set)? Explanatory Disorders (computed by Intellego) Actual Disorder (extracted from EMR) Possible Disorders (derived from cardiology KB)
    73. 73. 73 Evaluation of kHealth – Early Results HFSAS/WANDA Efficiency: ~7.45 (# questions asked) Specificity: ~11.95* (# minimum explanations) * Converged to 1 explanation 20% of the time • 496 EMRs • ~3.2 diagnosed disorders per EMR • 173 possible disorders The approach utilized by kHealth is more efficient and more specific than HFSAS/WANDA. kHealth Efficiency: ~7.28 (# questions asked) Specificity: 1 (# minimum explanations) Explanatory Disorders (computed by Intellego) Actual Disorder (extracted from EMR) Possible Disorders (derived from cardiology KB)
    74. 74. Pre-clinical usability trial Dr. William Abraham, M.D. Director of Cardiovascular Medicine 74
    75. 75. Sensing and Perception Health Care Academic Standards Org. Industry Government Research Collaborators by Research Topic and Organization Type 75
    76. 76. Special Thanks to AFRL and DAGSI 76 AFRL/DAGSI Research Topic SN08-8: Architectures for Secure Semantic Sensor Networks for Multi-Layered Sensing
    77. 77. 77
    78. 78. Semantic Sensor Web Team 78
    79. 79. A SEMANTICS-BASED APPROACH TO MACHINE PERCEPTION Cory Andrew Henson August 27, 2013 79 Thank you. For additional information visit: http://knoesis.org/researchers/cory

    ×