High-Level Context Inference for Human Behavior Identication

1. Claudia Villalonga, Oresti Banos, Wahajat Ali, Taqdir Ali, Asif Rassaq, Sungyong Lee, Hector Pomares, Ignacio Rojas International Work-conference on Ambient Assisted Living an Active Ageing (IWAAL 2015), Patagonia, Chile, December 1-4, (2015) High-Level Context Inference for Human Behavior Identification

2. /“The Slow-Moving Public Health Disaster” Diseases linked to lifestyle choices are currently the biggest cause of death worldwide: • Cardiovascular conditions, cancers, chronic respiratory disorders, obesity and diabetes, represent more than 60% of global deceases, half of which are of premature nature • Most of these diseases are fairly associated to common risk factors, namely, tobacco and alcohol use, unwholesome diet and physical inactivity • This "lifestyle disease" epidemic causes a much greater public health threat than any other epidemic known to man • Millions of lives could be saved if the world over the next decade invests $1-3 per person on promoting healthier habits Global targets for prevention and control of “lifestyle diseases” to be attained by 2025 Source: WHO, “Global status report on noncommunicable diseases 2014,” World Health Organization, Tech. Rep., 2014. 2

3. /Mining Minds in a nutshell “Collection of innovative services, tools, and techniques, working collaboratively to investigate on human's daily-life routines data generated from heterogeneous resources, for personalized wellbeing and healthcare support” 3

4. /Context Information Curation Layer High Level Context-Awareness Low Level Context-Awareness Sensory Data Router Inertial Activity Recognizer Activity Unifier Audio Activity Recognizer Video Activity Recognizer Emotion Unifier Location Unifier Context Ontology Manager High-Level Context Reasoner High-Level Context Builder Physiological Emotion Recognizer Video Emotion Recognizer Audio Emotion Recognizer Inertial Location Detector Video Location Detector Geopositioning Location Detector Context Ontology Storage High-Level Context Notifier Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Classification Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter InertialNavigation Tracking Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Video Tracking Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter GPS Tracking Feature Extraction Segmentation Preprocessing Input Adapter Output Adapter Activity Notifier Emotion Notifier Location Notifier Context Synchronizer Context Instantiator Context Mapper Context Verifier Context Classifier Context Query Generator Context Handler Ontology Model Manager High Level Context Low Level Context Multimodal Data 4

5. /Mining Minds Context Ontology 5 ContextOntologyMetrics: 9High-LevelContexts 16Activities(Low-LevelContext) 8Locations(Low-LevelContext) 8Emotions(Low-LevelContext)

6. /Context Ontology: High-Level Context Classes Definition 6 Activity and Location (Emotion is not required) Activity, Location and Emotion (if available) Activity, Location and Emotion (mandatory) None of the other Contexts and sedentary Activity

7. /Context Ontology: Examples of High-Level Context Instances 7 Activity, Location and Emotion Activity, Location and Emotion Activity and Location, without Emotion Activity and Location, without Emotion

8. /Context Ontology: Examples of High-Level Context Instances 8 Activity, Location and Emotion Activity and Location, without Emotion

9. / HighLevelContext-Awareness HLCA Operation 9 LLCA Activity Recognizer Emotion RecognizerLocation Detector High-Level Context Builder High-Level Context Reasoner High-Level Context Notifier Context Ontology Manager Context Ontology Storage Ontology Model Manager Context Query Generator Context Handler Data Curation Layer act_sitting rdf:type Sitting . act_sitting hasStartTime “2015-08-10T11:05:30”^^dateTime . act_sitting isContextOf user9876 . loc_office rdf:type Office . loc_office hasStartTime “2015-08-10T11:04:55”^^dateTime . loc_office isContextOf user9876 . ctx rdf:type Context . ctx hasActivity act_sitting . ctx hasLocation loc_office . ctx hasEmotion emo_boredom . ctx isContextOf user9876 . ctx hasStartTime “2015-08-10T11:05:30”^^dateTime . ctx rdf:type hasActivity only ({act_sitting}) . ctx rdf:type hasLocation only ({loc_office }) . ctx rdf:type hasEmotion only ({emo_boredom}) . ctx rdf:type Context . ctx rdf:type OfficeWork . ctx hasActivity act_sitting . ctx hasLocation loc_office . ctx hasEmotion emo_boredom . ctx isContextOf user9876 . ctx hasStartTime “2015-08-10T11:05:30”^^dateTime . ctx rdf:type hasActivity only ({act_sitting}) . ctx rdf:type hasLocation only ({loc_office }) . ctx rdf:type hasEmotion only ({emo_boredom}) . emo_boredom type Boredom . emo_boredom hasStartTime “2015-08-10T11:05:12”^^dateTime . emo_boredom isContextOf user9876 .

10. /HLCA Operation 10 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness Context Ontology Manager Context Ontology Storage Ontology Model Manager Context Query Generator Context Handler Context Mapper

11. /HLCA Operation 11 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness Context Synchronizer • LLC instances starting within the window: llc_1777 • LLC instances ending within the window: - • Order chronologically • Concurrent LLC for llc_1777: llc_1778 and llc_1779 Notify Context Instantiator 1 2 3 4 5 11:05:15 11:05:30

12. /HLCA Operation 12 11:05:30 User9876 llc_1777 Sitting llc_1778 Office llc_1779 Boredom hlc_0001 Trigger LLC Concurrent LLC Unclassified HLC 11:05:30 User9876 O I INPUT: OUTPUT: Context Instantiator

13. /HLCA Operation 13 HLC Reasoner: 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness hlc_0002 OfficeWork hlc_0001 OfficeWork hlc_0101 Inactivity hlc_0003 Unknown hlc_0102 Amusement

14. /HLCA Operation 14 HLC Notifier: 11:05:30 11:08:0011:06:45 11:05:50 11:07:00 User9876 llc_1777 Sitting llc_1780 Walking llc_1778 Office llc_1779 Boredom llc_2501 Sitting llc_2500 Mall User5555 llc_2502 Happiness hlc_0002 OfficeWork hlc_0001 OfficeWork hlc_0001 OfficeWork hlc_0101 Inactivity hlc_0101 Inactivity hlc_0003 Unknown hlc_0102 Amusement hlc_0002 OfficeWork Context Ontology Manager Context Ontology Storage Ontology Model Manager Context Query Generator Context Handler

15. /Demo 15

16. /Conclusions • Design and implementation of a framework for the online identification of high-level context based on low-level information (activities, locations, and emotions) • Definition of an ontology for the comprehensive and holistic identification of human behavior: • activity and location information might not be enough to detect some of the high- level contexts • emotion enables a more accurate high-level context identification • Flexible methodology and ontology to operate in real life scenarios in which recognition systems may not always be available 16

17. Thank you for your attention. Questions? Claudia Villalonga Ubiquitous Computing Lab (UCLab) Kyung Hee University (KHU), South Korea Email: cvillalonga@oslab.khu.ac.kr /

High-Level Context Inference for Human Behavior Identi cation

Oresti Banos