High-Level Context Inference for Human Behavior Identi cation

Oresti Banos
Oresti BanosAssociate Professor
Claudia Villalonga, Oresti Banos,
Wahajat Ali, Taqdir Ali, Asif Rassaq,
Sungyong Lee, Hector Pomares, Ignacio Rojas
International Work-conference on
Ambient Assisted Living an Active
Ageing (IWAAL 2015), Patagonia,
Chile, December 1-4, (2015)
High-Level Context Inference for
Human Behavior Identification
/“The Slow-Moving Public Health Disaster”
Diseases linked to lifestyle choices are
currently the biggest cause of death
worldwide:
• Cardiovascular conditions, cancers, chronic respiratory
disorders, obesity and diabetes, represent more than 60% of
global deceases, half of which are of premature nature
• Most of these diseases are fairly associated to common risk
factors, namely, tobacco and alcohol use, unwholesome diet
and physical inactivity
• This "lifestyle disease" epidemic causes a much greater
public health threat than any other epidemic known to man
• Millions of lives could be saved if the world over the
next decade invests $1-3 per person on promoting
healthier habits
Global targets for prevention and control of
“lifestyle diseases” to be attained by 2025
Source: WHO, “Global status report on noncommunicable diseases 2014,” World Health Organization, Tech. Rep., 2014.
2
/Mining Minds in a nutshell
“Collection of innovative
services, tools, and
techniques, working
collaboratively to
investigate on human's
daily-life routines data
generated from
heterogeneous resources,
for personalized wellbeing
and healthcare support”
3
/Context Information Curation Layer
High Level Context-Awareness
Low Level Context-Awareness
Sensory Data Router
Inertial
Activity
Recognizer
Activity Unifier
Audio
Activity
Recognizer
Video
Activity
Recognizer
Emotion Unifier Location Unifier
Context Ontology Manager
High-Level Context Reasoner
High-Level Context Builder
Physiological
Emotion
Recognizer
Video
Emotion
Recognizer
Audio
Emotion
Recognizer
Inertial
Location
Detector
Video
Location
Detector
Geopositioning
Location
Detector
Context
Ontology
Storage
High-Level Context Notifier
Classification
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Classification
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Classification
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Classification
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Classification
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Classification
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
InertialNavigation
Tracking
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Video
Tracking
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
GPS Tracking
Feature
Extraction
Segmentation
Preprocessing
Input Adapter
Output Adapter
Activity Notifier Emotion Notifier Location Notifier
Context Synchronizer
Context Instantiator
Context Mapper
Context Verifier
Context Classifier
Context Query Generator
Context Handler
Ontology Model Manager
High
Level
Context
Low
Level
Context
Multimodal
Data
4
/Mining Minds Context Ontology 5
ContextOntologyMetrics:
9High-LevelContexts
16Activities(Low-LevelContext)
8Locations(Low-LevelContext)
8Emotions(Low-LevelContext)
/Context Ontology: High-Level Context Classes Definition 6
Activity and Location
(Emotion is not required)
Activity, Location and
Emotion (if available)
Activity, Location and
Emotion (mandatory)
None of the other Contexts
and sedentary Activity
/Context Ontology: Examples of High-Level Context Instances 7
Activity, Location and
Emotion
Activity, Location and
Emotion
Activity and Location,
without Emotion
Activity and Location,
without Emotion
/Context Ontology: Examples of High-Level Context Instances 8
Activity, Location and
Emotion
Activity and Location,
without Emotion
/
HighLevelContext-Awareness
HLCA Operation 9
LLCA Activity Recognizer Emotion RecognizerLocation Detector
High-Level Context Builder
High-Level Context Reasoner
High-Level Context Notifier
Context Ontology Manager
Context
Ontology
Storage
Ontology Model
Manager
Context Query
Generator
Context Handler
Data
Curation
Layer
act_sitting rdf:type Sitting .
act_sitting hasStartTime “2015-08-10T11:05:30”^^dateTime .
act_sitting isContextOf user9876 .
loc_office rdf:type Office .
loc_office hasStartTime “2015-08-10T11:04:55”^^dateTime .
loc_office isContextOf user9876 .
ctx rdf:type Context .
ctx hasActivity act_sitting .
ctx hasLocation loc_office .
ctx hasEmotion emo_boredom .
ctx isContextOf user9876 .
ctx hasStartTime “2015-08-10T11:05:30”^^dateTime .
ctx rdf:type hasActivity only ({act_sitting}) .
ctx rdf:type hasLocation only ({loc_office }) .
ctx rdf:type hasEmotion only ({emo_boredom}) .
ctx rdf:type Context .
ctx rdf:type OfficeWork .
ctx hasActivity act_sitting .
ctx hasLocation loc_office .
ctx hasEmotion emo_boredom .
ctx isContextOf user9876 .
ctx hasStartTime “2015-08-10T11:05:30”^^dateTime .
ctx rdf:type hasActivity only ({act_sitting}) .
ctx rdf:type hasLocation only ({loc_office }) .
ctx rdf:type hasEmotion only ({emo_boredom}) .
emo_boredom type Boredom .
emo_boredom hasStartTime “2015-08-10T11:05:12”^^dateTime .
emo_boredom isContextOf user9876 .
/HLCA Operation 10
11:05:30 11:08:0011:06:45
11:05:50 11:07:00
User9876
llc_1777
Sitting
llc_1780
Walking
llc_1778
Office
llc_1779
Boredom
llc_2501
Sitting
llc_2500
Mall
User5555
llc_2502
Happiness
Context Ontology Manager
Context
Ontology
Storage
Ontology Model
Manager
Context Query
Generator
Context Handler
Context Mapper
/HLCA Operation 11
11:05:30 11:08:0011:06:45
11:05:50 11:07:00
User9876
llc_1777
Sitting
llc_1780
Walking
llc_1778
Office
llc_1779
Boredom
llc_2501
Sitting
llc_2500
Mall
User5555
llc_2502
Happiness
Context Synchronizer
• LLC instances starting within the
window: llc_1777
• LLC instances ending within the
window: -
• Order chronologically
• Concurrent LLC for llc_1777:
llc_1778 and llc_1779
Notify Context Instantiator
1
2
3
4
5
11:05:15 11:05:30
/HLCA Operation 12
11:05:30
User9876
llc_1777
Sitting
llc_1778
Office
llc_1779
Boredom
hlc_0001
Trigger LLC
Concurrent LLC
Unclassified HLC
11:05:30
User9876
O
I INPUT:
OUTPUT:
Context Instantiator
/HLCA Operation 13
HLC Reasoner:
11:05:30 11:08:0011:06:45
11:05:50 11:07:00
User9876
llc_1777
Sitting
llc_1780
Walking
llc_1778
Office
llc_1779
Boredom
llc_2501
Sitting
llc_2500
Mall
User5555
llc_2502
Happiness
hlc_0002
OfficeWork
hlc_0001
OfficeWork
hlc_0101
Inactivity
hlc_0003
Unknown
hlc_0102
Amusement
/HLCA Operation 14
HLC Notifier:
11:05:30 11:08:0011:06:45
11:05:50 11:07:00
User9876
llc_1777
Sitting
llc_1780
Walking
llc_1778
Office
llc_1779
Boredom
llc_2501
Sitting
llc_2500
Mall
User5555
llc_2502
Happiness
hlc_0002
OfficeWork
hlc_0001
OfficeWork
hlc_0001
OfficeWork
hlc_0101
Inactivity
hlc_0101
Inactivity
hlc_0003
Unknown
hlc_0102
Amusement
hlc_0002
OfficeWork
Context Ontology Manager
Context
Ontology
Storage
Ontology Model
Manager
Context Query
Generator
Context Handler
/Demo 15
/Conclusions
• Design and implementation of a framework for the online identification of
high-level context based on low-level information (activities, locations,
and emotions)
• Definition of an ontology for the comprehensive and holistic identification
of human behavior:
• activity and location information might not be enough to detect some of the high-
level contexts
• emotion enables a more accurate high-level context identification
• Flexible methodology and ontology to operate in real life scenarios in
which recognition systems may not always be available
16
Thank you for
your
attention.
Questions?
Claudia Villalonga
Ubiquitous Computing Lab (UCLab)
Kyung Hee University (KHU), South Korea
Email: cvillalonga@oslab.khu.ac.kr
/
1 of 17

More Related Content

Similar to High-Level Context Inference for Human Behavior Identi cation(20)

Aaai 2011 event processing tutorialAaai 2011 event processing tutorial
Aaai 2011 event processing tutorial
Opher Etzion2.6K views
Xomia_20220602.pptxXomia_20220602.pptx
Xomia_20220602.pptx
Longhow Lam134 views
Persuading your organisation to do awesome thingsPersuading your organisation to do awesome things
Persuading your organisation to do awesome things
Jacqueline (Jax) Wechsler2.4K views
Biocatalogue Talk SlidesBiocatalogue Talk Slides
Biocatalogue Talk Slides
BioCatalogue518 views

More from Oresti Banos(20)

Biodata analysisBiodata analysis
Biodata analysis
Oresti Banos1.9K views
Biosignal ProcessingBiosignal Processing
Biosignal Processing
Oresti Banos15.4K views

High-Level Context Inference for Human Behavior Identi cation

  • 1. Claudia Villalonga, Oresti Banos, Wahajat Ali, Taqdir Ali, Asif Rassaq, Sungyong Lee, Hector Pomares, Ignacio Rojas International Work-conference on Ambient Assisted Living an Active Ageing (IWAAL 2015), Patagonia, Chile, December 1-4, (2015) High-Level Context Inference for Human Behavior Identification
  • 2. /“The Slow-Moving Public Health Disaster” Diseases linked to lifestyle choices are currently the biggest cause of death worldwide: • Cardiovascular conditions, cancers, chronic respiratory disorders, obesity and diabetes, represent more than 60% of global deceases, half of which are of premature nature • Most of these diseases are fairly associated to common risk factors, namely, tobacco and alcohol use, unwholesome diet and physical inactivity • This "lifestyle disease" epidemic causes a much greater public health threat than any other epidemic known to man • Millions of lives could be saved if the world over the next decade invests $1-3 per person on promoting healthier habits Global targets for prevention and control of “lifestyle diseases” to be attained by 2025 Source: WHO, “Global status report on noncommunicable diseases 2014,” World Health Organization, Tech. Rep., 2014. 2
  • 3. /Mining Minds in a nutshell “Collection of innovative services, tools, and techniques, working collaboratively to investigate on human's daily-life routines data generated from heterogeneous resources, for personalized wellbeing and healthcare support” 3
  • 4. /Context Information Curation Layer High Level Context-Awareness Low Level Context-Awareness Sensory Data Router Inertial Activity Recognizer Activity Unifier Audio Activity Recognizer Video Activity Recognizer Emotion Unifier Location Unifier Context Ontology Manager High-Level Context Reasoner High-Level Context Builder Physiological Emotion Recognizer Video Emotion Recognizer Audio Emotion Recognizer Inertial Location Detector Video Location Detector Geopositioning Location Detector Context Ontology Storage High-Level Context Notifier Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter InertialNavigation Tracking Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Video Tracking Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter GPS Tracking Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Activity Notifier Emotion Notifier Location Notifier Context Synchronizer Context Instantiator Context Mapper Context Verifier Context Classifier Context Query Generator Context Handler Ontology Model Manager High Level Context Low Level Context Multimodal Data 4
  • 5. /Mining Minds Context Ontology 5 ContextOntologyMetrics: 9High-LevelContexts 16Activities(Low-LevelContext) 8Locations(Low-LevelContext) 8Emotions(Low-LevelContext)
  • 6. /Context Ontology: High-Level Context Classes Definition 6 Activity and Location (Emotion is not required) Activity, Location and Emotion (if available) Activity, Location and Emotion (mandatory) None of the other Contexts and sedentary Activity
  • 7. /Context Ontology: Examples of High-Level Context Instances 7 Activity, Location and Emotion Activity, Location and Emotion Activity and Location, without Emotion Activity and Location, without Emotion
  • 8. /Context Ontology: Examples of High-Level Context Instances 8 Activity, Location and Emotion Activity and Location, without Emotion
  • 9. / HighLevelContext-Awareness HLCA Operation 9 LLCA Activity Recognizer Emotion RecognizerLocation Detector High-Level Context Builder High-Level Context Reasoner High-Level Context Notifier Context Ontology Manager Context Ontology Storage Ontology Model Manager Context Query Generator Context Handler Data Curation Layer act_sitting rdf:type Sitting . act_sitting hasStartTime “2015-08-10T11:05:30”^^dateTime . act_sitting isContextOf user9876 . loc_office rdf:type Office . loc_office hasStartTime “2015-08-10T11:04:55”^^dateTime . loc_office isContextOf user9876 . ctx rdf:type Context . ctx hasActivity act_sitting . ctx hasLocation loc_office . ctx hasEmotion emo_boredom . ctx isContextOf user9876 . ctx hasStartTime “2015-08-10T11:05:30”^^dateTime . ctx rdf:type hasActivity only ({act_sitting}) . ctx rdf:type hasLocation only ({loc_office }) . ctx rdf:type hasEmotion only ({emo_boredom}) . ctx rdf:type Context . ctx rdf:type OfficeWork . ctx hasActivity act_sitting . ctx hasLocation loc_office . ctx hasEmotion emo_boredom . ctx isContextOf user9876 . ctx hasStartTime “2015-08-10T11:05:30”^^dateTime . ctx rdf:type hasActivity only ({act_sitting}) . ctx rdf:type hasLocation only ({loc_office }) . ctx rdf:type hasEmotion only ({emo_boredom}) . emo_boredom type Boredom . emo_boredom hasStartTime “2015-08-10T11:05:12”^^dateTime . emo_boredom isContextOf user9876 .
  • 10. /HLCA Operation 10 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness Context Ontology Manager Context Ontology Storage Ontology Model Manager Context Query Generator Context Handler Context Mapper
  • 11. /HLCA Operation 11 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness Context Synchronizer • LLC instances starting within the window: llc_1777 • LLC instances ending within the window: - • Order chronologically • Concurrent LLC for llc_1777: llc_1778 and llc_1779 Notify Context Instantiator 1 2 3 4 5 11:05:15 11:05:30
  • 12. /HLCA Operation 12 11:05:30 User9876 llc_1777 Sitting llc_1778 Office llc_1779 Boredom hlc_0001 Trigger LLC Concurrent LLC Unclassified HLC 11:05:30 User9876 O I INPUT: OUTPUT: Context Instantiator
  • 13. /HLCA Operation 13 HLC Reasoner: 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness hlc_0002 OfficeWork hlc_0001 OfficeWork hlc_0101 Inactivity hlc_0003 Unknown hlc_0102 Amusement
  • 14. /HLCA Operation 14 HLC Notifier: 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness hlc_0002 OfficeWork hlc_0001 OfficeWork hlc_0001 OfficeWork hlc_0101 Inactivity hlc_0101 Inactivity hlc_0003 Unknown hlc_0102 Amusement hlc_0002 OfficeWork Context Ontology Manager Context Ontology Storage Ontology Model Manager Context Query Generator Context Handler
  • 16. /Conclusions • Design and implementation of a framework for the online identification of high-level context based on low-level information (activities, locations, and emotions) • Definition of an ontology for the comprehensive and holistic identification of human behavior: • activity and location information might not be enough to detect some of the high- level contexts • emotion enables a more accurate high-level context identification • Flexible methodology and ontology to operate in real life scenarios in which recognition systems may not always be available 16
  • 17. Thank you for your attention. Questions? Claudia Villalonga Ubiquitous Computing Lab (UCLab) Kyung Hee University (KHU), South Korea Email: cvillalonga@oslab.khu.ac.kr /

Editor's Notes

  1. 1
  2. 'Lifestyle' diseases linked to unhealthy habits kill millions of people prematurely
  3. Context ::: class Sitting ::: subclass (these are disjoint) hasActivity ::: object property hasStartTime ::: data property Literal (string), and concretely the time has the W3C standard format XML schema
  4. These are our defined classes (only defined classes can be used for the classification). HLC classes are “defined” (i.e., both necessary, “e.g., hasUser some User”, and sufficient conditions, “e.g., hasActivity some Sitting”) while LLC classes are simply “described” (i.e., only necessary conditions, “e.g., hasUser some User”). Due to the “open world assumption”: you cannot assume that something does not exist if you do not explicitly state that it does not exist Some – existencial restriction ::: it must exist Only – universal restriction ::: if it exists, it can only be of the given type (e.g., the property (“hasActivity”) can only relate to an instance which is a member of class “Sitting” and not at the same link to another instance of type “Walking”)
  5. Once we have defined our ontology, how can we use it? The idea is to create an instance of this context whenever a new context is experienced by the user. What we do is to set in this instance the low-level contexts that are taking place. In this case for example the emotions… By applying reasoning techniques we can identify the high-level context, in this case Amusement. Instance of the “Context” class, i.e., parent class. (Left) Type assertions | (Right) Property assertions Assertion of the values of the properties. E.g., hasActivity act_sitting (where act_sitting is an instance of the LLC class Sitting) Due to the Open World Assumption, type assertions are used as closure axioms to indicate something does not exist. E.g., the value of the “hasActivity” property is ONLY Sitting. Or the emotion does not exist (not (hasEmotion some Emotion))