Summit2013 ho-jin choi - summit2013

2,175 views

Published on

Published in: Health & Medicine, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,175
On SlideShare
0
From Embeds
0
Number of Embeds
408
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Summit2013 ho-jin choi - summit2013

  1. 1. 18 July 2013 Ho-Jin Choi Dept. of Computer Science, KAIST Systems Biomedical Informatics Research Center (SBI-NCRC), SNU College of Medicine Personalized Context-Aware Health Avatar in Smart Phone Environment The 2013 STI Semantic Summit Suzdal, Russia, July 17-19, 2013 1
  2. 2. Outline  SBI-NCRC – A brief introduction  Research Topics  Activity Recognition for Personalized Life-Care  Healthcare Service Framework for Continuous Context Monitoring  Text Mining for Extracting Knowledge from Web Contents  Personalized Bio and Medical Data Analysis 2
  3. 3. Systems Biomedical Informatics National Core Research Center (SBI-NCRC) - A brief introduction - 3
  4. 4. SBI-NCRC  NCRC (National Core Research Center)  Government initiative to support interdisciplinary research & education  Since 2004, one or two centers newly selected each year  Funding scale, 2 million USD/year * 6.5 years  Systems Biomedical Informatics (SBI) Research Center  An NCRC established jointly by SNU Hospital and KAIST Computer Science  Born in September 2010  24 professors/researchers participating from 4 organizations  SNUH, KAIST, Ajou University, ETRI  Goals for SBI-NCRC  To define and realize “Digital Self” or “Health Avatar” prescriptive medicine  To integrate clinical information and bio-information using IT  To launch an interdisciplinary program in Biomedical Informatics  To collaborate towards Joint KAIST-SNUH BIT Campus in Inchon area 4
  5. 5. 4P Medicine  Preventive medicine  Predictive medicine  Personalized medicine  Participatory medicine Tests for early detection Risk evaluation Prevention Targeted monitoring Diagnosis Treatment Results monitoring Caring Diseases Caring Health 5
  6. 6. Middleware Integrative analyses OCS PACS EMR LIS Seq. Exp. Prot. Tissue CGH HL-7 DICOM CDA LOINC BSML MAGE MIAPE TMA ? SNP HapMap 2DPage ProtChip Tissue MA BAC Chip Phenomic Self Extractor Genomic Self Extractor Clinical Genomics BioData Acquisition Pattern Recognizer DNA Chip Transformer XML Binder DBConnector HL-7 / CDA Protocols Hospital caBIO EVS caDSR CRF CDE/CTEP caCORE IDE Foundation Self Warehouse 유방암 Application 폐암 Application 혈액암 Application Legacy System Authentication/Authorization Interface 자료 수용기 CGI- gateway WebServer XML-Validation Ontology-Enhancement Data Indexing Clinical Trials Knowledge Base XML CGI-gateway Retrieval engine Query Constructor Clinical Research & Clinical Trial KB Application Processor Search engine Statistical analysis Visualization Simulation Communications Workflow Middleware Ontology Server Vocabulary Server Taxonomy Server Public Bio-DBs Digital Self Simulated Self Individuated Second Self Foundation Self Molecular & Cellular Foundations of Self Ubiquitous Self Life Logs and Distributed Collaborations Genomic Self: Translational Bioinformatics for Genomic Health and Molecular Medicine Phenomic Self: Data and Measurement driven Discovery and Understanding of Human Disorders Physiomic Self: Multi-scale Modeling of Physical and Physiological Systems of Human Body Semantic Self: Ontological Representation and Engineering of Health Avatar Augmented Self: Multi-modal Assessment and Treatment to Retain and Enhance Human Performance Connected Self: Life Logs and Stream- Type Data Mining for Health Protection Distributed Self: Customized and Context-aware Healthcare Service Agents in Smart Phone Environment
  7. 7. Teaming 7 Group/Project Title PI’s Major Group 1 Foundation Self: Molecular and Cellular Foundations of Self SNUH, Ajou U. Project 1-1 Genomic Self: Translational Bioinformatics for Genomic Health and Molecular Medicine Psychiatry(1), Surgery(1), Bioinformatics(1), Statistics(1) Project 1-2 Phenomic Self: Data and Measurement driven Discovery and Understanding of Human Disorders Pathology(2), Bioinformatics(1) Project 1-3 Physiomic Self: Multi-scale Modeling of Physical and Physiological Systems of Human Body Biomedical Engineering(2), Neurosurgery(1), General Practice(1) Group 2 Simulated Self: Individuated Second Self SNUH, KAIST Project 2-1 Semantic Self: Ontological Representation and Engineering of Health Avatar Nursing Informatics(2), Internal Medicine(1), Pathology(1) Project 2-2 Augmented Self: Multi-modal Assessment and Treatment to Retain and Enhance Human Performance NLP(1), Graphics(1), Image Processing(1), Psychiatry(1) Group 3 Ubiquitous Self: Life Logs and Distributed Collaborations KAIST, ETRI Project 3-1 Distributed Self: Customized and Context-aware Healthcare Service Agents in Smart Phone Environment AI(1), Software Engineering(1), Bioinformatics(1) Project 3-2 Connected Self: Life Logs and Stream-Type Data Mining for Health Protection Information Systems(1), Data Mining(1)
  8. 8. Life-style Logging Medical Logging Genetic Logging Virtual Self Supercomputer (Collective Intelligence, Data Mining) Healthainment Smart EMR SNS Health Avatars * *Autonomous Health Avatar Virtual Self and Health Avatar 8
  9. 9. 9  Target healthcare domains  Obesity, diabetes, dementia  On-going research topics  Activity Recognition for Personalized Life-Care (Prof. Ho-Jin Choi)  Healthcare Service Framework for Continuous Context Monitoring (Prof. Jun-Hwa Song)  Text Mining for Extracting Knowledge from Web Contents (Prof. Key-Sun Choi)  Personalized Bio and Medical Data Analysis (Prof. Gwan-Su Yi) Research Topics
  10. 10. Activity Recognition for Personalized Life-Care Prof. Ho-Jin Choi Dept. of Computer Science KAIST 10
  11. 11. Multi-Sensor Surveillance for Elderly Care 11 “Patient #1234 is in a risky situation” Data observed from microphones helps the system detect the potentially risky situations . The agent estimates patient #1234’s behaviors. When preliminary conditions of dangerous situations are occurred to the patient, the agent alarms to the caregiver.
  12. 12. 12 Activity Recognition from Video Image with Depth Sensor  Action Recognition with Automatically Detected Essential Body Joints
  13. 13. Technologies Involved Understand Image Data - RGB images (camera) and depth images (depth sensor) are sent to the system - System then do -Find a patient in a scene -Track the patient -Understand behaviors of the patient ★ Issues to challenge - The level of complexity of scenes and behaviors - Scenes may contain various objects and backgrounds - Human-behaviors should be understood as much as possible. 13 Understand Audio Data -Audio data (microphones) is sent to the system -System then do -Detect abnormal sounds ★ Issues to challenge - How accurate the system detects abnormal sounds Detect Risky Situation -After analyzing data from various sensors, the system determines whether the situation is potentially risky -System constructs a database for predefined risky situations -For every situation, the system calculates the likelihood of being risky -If the likelihood scores more than a threshold, it alarms to the caregiver ★ Issues to challenge -How well the system constructs the database -The accuracy of likelihoods Find Patient’s Location -Smartphone gives and receives various signals to update patient’s geographic information ★ Issues to challenge - How accurately the system locates the patient
  14. 14. Wrist-Type Device Based Human Behavior Recognition 14  Mediated Interface for human-robot interaction …. Health Care Care Services Raw data (Behavior pattern, Vital Signal, etc) Old People How to get “Raw Data” From Old People? Robots Care-giver Activity, Gesture, Vital signal, Location, Identification(MI: Mediated Interface) Ex: Watch, Ring Robots Care-giver Elderly Care Services Using Robots Suggestion Fall Detection Wandering Monitoring Location Monitoring Care Services
  15. 15. Wrist-Type Device Based Human Behavior Recognition 15  Wrist-type and waist-type monitors MCU Cortex-M3 (STM32F100) RF(Zigbee) CC2520 Sensors 3-axis accelerometer(LIS331DLH) 3-axis gyro (L3G4200D) Temperature/humidity (SHT21) Brightness (TCS3414CS) IR Photodiode (TSOP85238) Emergency button 1개 (front side) Memory card MicroSD Battery [Li-Ion 600mAh] Recharger External rechager Strap Wrist: nato band Waist: elastic belt
  16. 16. Lifestyle Manager Using SNS and Activity Recognition 16 Life-style patterns Clinical history Genetic information Server Smartphone users Lifestyle ranking Life log Lifestyle disease risk Default behavior registration Location - time elapse threshold Localization by Wifi signal
  17. 17. 17  Analysis of life log and SNS  Lifestyle = Eating habit(timing and food types) + CAR(Circadian activity rhythm) Server Many smart phone users Location dimension Sleep : My room, Park, Motel Rest : TV room, Living room, Lounge Work or study : Work place, Study place Enjoy : Shopping place, Cultural place, Attractions Usual food : Restaurant Exercise : Exercise place Religious activities : Church, Buddhist temple Fast food or snack : Fast food place, Mc. Donald, Convenience store Sugar-sweetened beverage : Cafeteria, Convenience store Smoking : Smoking place, Convenience store, Alcohol : Alcohol place, Bar Drug : Drug store UNKNOWN 1. Lifestyle recommendation 2. Measurement of Lifestyle Metric Data matrix Tries for users’ visiting patterns on location & time dimension People Healthcare organizationsProactive/Reactive Services Lifestyle Manager Using SNS and Activity Recognition
  18. 18. Lifestyle Manager Using SNS and Activity Recognition 18 - Home, Hotel - Smoking place - Drinking place -Working place, Studying place - Restaurant - Cafeteria, Coffee shop - Exercising place - Religious place -Attractions, Shopping place, Cultural place, Enjoy place - Hospital, Pharmacy - Unknown Location manager - Facebook : contains more daily lives than others ( e.g., “I ate a hamburger, so cool.“ 11:45AM ) Social Network Services -Walking - Exercise / Sport - Running - Riding an automobile - Riding a bicycle - No activity Activity Recognition -Accelerometer - Illuminance sensor -WiFi - GPS Sensor Handler - Name - SNS information -Address ( GPS ) - BMI - … User profile - Lifelog - Carlorie counting (day, week, …) - Lifestyle disease risk Service Provider
  19. 19. 19 Reuse Experience Base Developer Community Social Network Service Extracted Experiences (XML-based) Experience Structure Problem Solution Environments Flag (success, fail) Problem Topics Solution Topics Document Parser Social Network Analyzer Topic Miner Relationship Creator Co-related Problem-Solution Topic Pairs Constructing case-base Reusing the cases as experience Experience Mining for Healthcare Social Network
  20. 20. 20 Problem Document1 Solution Document1 Original Document1 Original Document2 Original Document3 Original Document4 Original Document10 ... Problem Document10. . . Solution Document10. . . Split to problem and solution documents Topic Mining for Problems and Solutions
  21. 21. 21 Keywords window button taskbar bitmap tiff hchuldwnd msdn hwnd wparam ... Rank Doc #Words Ratio 1 D07 99 80.5% 2 D10 71 67.0% 3 D08 24 24.2% 4 D02 23 29.5% 5 D06 18 13.0% 6 D05 12 13.6% 7 D01 12 14.6% 8 D04 5 27.8% 9 D03 3 8.1% 10 D09 0 0.0% ProblemTopic1 Keywords http bit alpha wrp aspx en work system trusted ... Rank Doc #Words Ratio 1 D02 54 69.2% 2 D08 42 42.4% 3 D01 39 47.6% 4 D05 34 38.6% 5 D07 15 12.2% 6 D06 12 8.7% 7 D10 10 9.4% 8 D09 5 71.4% 9 D04 4 22.2% 10 D03 4 10.8% ProblemTopic2 Keywords windows vista microsoft logo partner program application support certification ... Rank Doc #Words Ratio 1 D06 108 78.3% 2 D05 42 47.7% 3 D08 33 33.3% 4 D01 31 37.8% 5 D03 30 81.1% 6 D10 25 23.6% 7 D07 9 7.3% 8 D04 9 50.0% 9 D09 2 28.6% 10 D02 1 0.0% ProblemTopic3 Keywords office run microsoft community aurigma test en prevent channel ... Rank Doc #Words Ratio 1 D04 20 50.0% 2 D01 17 27.4% 3 D05 15 0.0% 4 D03 9 0.0% 5 D02 7 25.9% 6 D10 0 14.0% 7 D09 0 44.4% 8 D08 0 30.0% 9 D07 0 0.0% 10 D06 0 0.0% SolutionTopic1 Keywords window bitmap net http feedback error read link application ... Rank Doc #Words Ratio 1 D05 22 50.0% 2 D01 15 24.2% 3 D03 12 50.0% 4 D02 11 40.7% 5 D04 7 0.0% 6 D10 0 20.6% 7 D09 0 15.6% 8 D08 0 40.0% 9 D07 0 0.0% 10 D06 0 0.0% SolutionTopic2 Keywords windows taskbar system ws msft www dp question bob ... Rank Doc #Words Ratio 1 D05 36 50.0% 2 D04 13 50.0% 3 D01 11 17.7% 4 D03 5 50.0% 5 D02 5 18.5% 6 D10 0 33.6% 7 D09 0 28.9% 8 D08 0 16.7% 9 D07 0 50.0% 10 D06 0 50.0% SolutionTopic3 Keywords button app win msdn cp support creating usi bao ... Rank Doc #Words Ratio 1 D05 34 0.0% 2 D01 19 40.7% 3 D04 5 0.0% 4 D03 4 0.0% 5 D02 4 14.8% 6 D10 0 31.8% 7 D09 0 11.1% 8 D08 0 13.3% 9 D07 0 50.0% 10 D06 0 50.0% SolutionTopic4
  22. 22. Healthcare Service Framework for Continuous Context Monitoring Prof. Jun-Hwa Song Dept. of Computer Science KAIST 22
  23. 23. Context-Aware Healthcare Service Scenarios 23  Example Scenarios  Obesity monitoring  Continuously monitors people’s activity level and consumed calories, and suggests proper exercises to the people.  Elderly people monitoring  Continuously monitors an elderly people’s emergency situation such as slipping down on a wet floor, and expedites an emergency call.  Cardiac patient monitoring  Continuously monitors a cardiac patient’s ECG, and expedites an emergency call.
  24. 24. DietSense: Smartphone-Based Diet Monitoring for Enhancing Obesity Self-Care 24 Comparing diet and physical activity Monitoring Physical ActivityCapturing Diet Calories consumed from food Calories burned during physical activity camera microphone accelerometer
  25. 25. Activity Log Smartphone 1. Collecting activity data from the patient 2. Training ML algorithms for analyzing activity patterns 3. Figuring out the right does time without interrupting the current work activity 4. Notify the does time and subsequent reminder MedicineTaker Motion Sensor Activity Log Place A Place A Task 1 Task 2 Task 3 Action 1-1 Action 1-2 Action 1-3 Action 2-1 Action 2-2 Action 3-1 Type 1 Type 2 Smart Alarming for Long-Term Medicine Adherence 25
  26. 26. Continuous Context Monitoring 26  Continuous monitoring of user’s context  A key building block for personal context-aware applications  Often requires complex, multi-step, continuous processing for multiple devices  E.g., Running situation -> sensing in three 3-axis accelerometers, FFT processing, recognition Location-based Services HealthMonitoring U-Trainer U-Secretary U-Reminder Dietdiary U-Learning Behaviorcorrection S F C S S F F C SS F C S S F C S F C Sensing Feature extraction Context recognition PAN-scale dynamic distributed computing platform Context monitoring (e.g., sensing, feature extraction, recognition) Application logic Location-based Services HealthMonitoring U-Trainer U-Secretary U-Reminder Dietdiary U-Learning Behaviorcorrection A A A App logic
  27. 27. Mobile Healthcare Service Framework 27  Develop a healthcare service framework  To support multiple and long running healthcare services with highly scarce and dynamic resources  Efficient resource utilization  Longer lasting operation (and service) under highly scarce resource situation  Quick and efficient abnormal situation detection  Seamless (stable) operation even under high resource dynamics  Challenges  Limited battery power due to mobility  Scarce computing resources of mobile devices  Dynamic join/leave of heterogeneous sensors  Multiple healthcare services share highly limited resources Resource Manager Policy Manager Energy Manager Sensor Broker Sensor Detector Communication Manager … …Heartbeat monitoringFall monitoring Healthcare services API Message Interpreter Sensor Data Processor Resource Monitor Network protocols (e.g., ZigBee, BT) Health Context Monitor Sensor data Requests Sensor availability/status Results Sensor detection/control, Data/status report Sensors in BAN/PAN Application Broker Application Interface Result Manager Message Parser Diet diary GPSBVP/GSR Accelerometers …… Anomaly Detector Feature Extractor Resource Coordinator
  28. 28. Healthtopia – Healthcare Platform  Providing API for utilizing health sensors  Saving power consumption for concurrent multiple apps 28
  29. 29. Text Mining for Extracting Knowledge from Web Contents Prof. Key-Sun Choi Dept. of Computer Science KAIST 29
  30. 30. Food Ingredients and Recipe Advice for Controlling Obesity Web Environment Ontology Mobile Environment Recipe Extraction Web /Wikipedia Automatic User Experience Extraction Target Food/Dish Recognition Manual User Experience Input Web Log / SNS Equipment How-to Food/Dish Restaurant DB Scenario 2. New Recipe (Low calories) Suggestion w/ same Ingredients Scenario 1. Food-NutritionAssociationVisualization Nutrition Ingredients Food-Nutrition Extraction 30
  31. 31. Mining Connections between Multiple Sources 31 Literature Web Clinical Data HeterogeneousTextual Sources -Textbook - PubMed - Blogs -Wikipedia - Personal health record  Medical information sources  Literature contents affect the Web contents  As background factual knowledge  Web contents have other benefits  Wide coverage  Huge collaborators (confidence)  Aggregating information from multiple sources  Analysis of trend evolving on literature/Web to identify factors that will improve the quality of patient care  Reliability: Literature > News > Web (Wikipedia, blog, SNS)  Accessibility: Web ≥ News > Literature
  32. 32. Detecting MeSH Keywords from Web Pages 32  Medical Subject Headings (MeSH)  NLM controlled vocabulary thesaurus used for indexing articles for PubMed  Tree structure (http://www.nlm.nih.gov/mesh/trees.html)  Provide an efficient way of accessing and organizing biomedical information  Examples of MeSH Headings  Body Weight, Kidney, Dental Cavity Preparation, Self Medication, Brain Edema Extracting Candidates Matching Obesity is a medical condition in which excess body fat has a ccumulated to the extent that it may have an adverse effect on health, leading to reduced life expectancy and/or increased he alth problems.[1][2] Body mass index (BMI), a measurement which compares weight and height, defines people as overweig ht (pre-obese) if their BMI is between 25 and 30 kg/m2, and o bese when it is greater than 30 kg/m2. • Hyperinked terms are extracted as term candidatesLanguage Handling
  33. 33. Detecting MeSH Keywords from Web Pages 33 Extracting Candidates Matching Obesity Medical condition Body fat Body Mass Index weight Dieting Obesity Medical condition Body fat Body Mass Index Body Weight Diet Link Structure MeSH term Language Handling  Language Handling  Polysemy and homonymy problem
  34. 34. 34 Semi-Automatic Infobox Construction for Korean Wikipedia
  35. 35. 35 맛있는 감자탕 1그릇을 먹을경우 177Kcal를 섭취하게 된다고 합니다. Parsing the sentence Extracting knowledge <감자탕 1그릇, 열량, 177Kcal> Infobox DB Extracting Knowledge from Unstructured Texts using Infobox DB
  36. 36. Personalized Bio and Medical Data Analysis Prof. Gwan-SuYi Dept. of Bio and Brain Engineering KAIST 36
  37. 37. Personalized Diseases Risk Analysis User Agent Disease risk Prediction model Personal genome Data processing model Drug response Prediction model Disease risk info. (SNP-Disease) Drug response info. (SNP-Drug) Personal genome info. Personal sequence data New info. on disease risk New info. on drug response Update Update request result Personalized disease risk Genome profile Personalized drug response Storage Build database Personalized Personalized Drug response Diseases risk Obesity, Diabetes Obesity, Diabetes 37
  38. 38. Constructing Databases for Diseases Risk and Drug Response 38 184(Type I Diabetes), 203(Type II Diabetes), 82(Obesity) entries for diseases related SNP markers collected 228 drugs, 830 SNP markers, 1341entries for drug-SNP related information collected Diseases Drugs Diseases risk info. SNP ID Gene Gene Region (Locus) Risk Allele Odds Ratio P-value Reference … PharmGKB Drug Bank Drug response info. SNP ID Gene Gene Region (Locus) Drug Condition Reference … Integrated database for diseases risks and drug responses Public database Drug related info. 23andMe Navigenics Pathway Genomics Gene sequencing service drug related info. 23andMe Navigenics deCODEme Gene sequencing service GWAS info. HugeNavigator GAD NCBI (HapMap & NHGRI catalog) Public database GWAS info.
  39. 39. Developing Methods for Analyzing Diseases Risk and Drug Response 39 Diseases OMIM PharmGKB DrugBank Drugs PharmGKB DrugBank SNPs dbSNP HapMap Genome type WTCCC Genome body UCSC Ensembl AceView Genome Entrez Bio. pathway KEGG Reactome NCI pathway Panther SNPs dbSNP HapMap Genome type WTCCC Biological information SNP SNP Analysis tech. for diseases risk Analysis tech. for drug response Obesity, Diabetes • Extraction of drug response related SNP’s • Drug targeting and biological pathway based function analysis • Drug response prediction • Obesity (Diabetes) related SNP or SNP combinations info. • Genomic and biological pathway based function analysis • Diseases risk prediction
  40. 40. Service Platform for Personalized Information about Diseases Risk and Drug Response Agent User Plug-ins from life-logging team Diseases risk prediction model Personal genome info. data processing model Drug response prediction model Diseases risk info. (SNP-Disease) Drug response info. (SNP-Drug) Diseases DrugsGenome body SNP PathwayGenome type Life-logging database Personal genome info. 40
  41. 41. Thanks for Attention Contact: hojinc@kaist.ac.kr 41

×