Personalized robotic intervention strategy by using semantics for people with dementia in nursing homes
•Download as PPTX, PDF•
1 like•153 views
Presentation giving during the1st International Workshop on Application of Semantic Web Technologies in Robotics (AnSWeR) at ESWC 2017
Recommended
Recommended
More Related Content
Similar to Personalized robotic intervention strategy by using semantics for people with dementia in nursing homes
Similar to Personalized robotic intervention strategy by using semantics for people with dementia in nursing homes (20)
Recently uploaded
Recently uploaded (20)
Personalized robotic intervention strategy by using semantics for people with dementia in nursing homes
- 1. PUBLIC PERSONALIZED ROBOTIC INTERVENTION STRATEGY BY USING SEMANTICS FOR PEOPLE WITH DEMENTIA IN NURSING HOMES FEMKE ONGENAE & FEMKE DE BACKERE GHENT UNIVERSITY - IDLAB
- 2. 2 100,000 PWD in Flanders, exhibiting Behavioral Disturbances (BDs) Up to 80% Agression/Agitation Sleep/wake disturbance 42% Apathy 72% 9 - 63% Delusions
- 3. 3 BDs can be prevented and reduced by preserving dignity and promoting quality of life Financial constraints and lack of technology hinder the adoption of person-centric care principles
- 4. PERSONALIZED ROBOTIC INTERVENTIONS WONDER project 2 nursing homes, ZoraBots, XETAL & Flemish Universities Design of an Internet of Robotic Things platform Enabling contextualized steering of robot actions & intervention strategy Enabling personalized interactions Design of personalized intervention algorithm to intercept BDs
- 5. INTERNET OF ROBOTICS THINGS PLATFORM
- 6. CAPTURING SENSOR DATA WITH DYAMAND technology devices Integration Detection & Interaction Plugin-based No manual configuration needed Mapping to SSN Ontology Produces JSON-LD
- 7. DYAMAND Robotics Staff devices MESSAGE BROKER observations actuations updates alerts updates commands Static Data Provider Static data Processing Services Sensor Gateway Process Management Action Executor Task Assignmen t Services Action Assignmen t Services
- 8. USER-DRIVEN APPROACH TO DETERMINE PERSONALIZED INTERVENTION STRATEGY
- 9. Sensor data Profile info BD detection algorithm Assignment algorithm Task assignment algorithm 3 ALGORITHMS UNDER DESIGN... Detected BD Send Zora Send message to caregiver
- 10. USER-DRIVEN ONTOLOGY & ALGORITHM DESIGN APPROACH Observation s & interviews Mind mapping Decision tree workshops
- 11. DECISION TREE WORKSHOP Goal: Capturing algorithms/decision processes that stakeholders/domain experts perform intuitively (“experience thaught me how to do it, there is no process”, “we don’t think about it”) Formalize those processes in a decision tree Capturing the context/parameters on which people base those decisions Prioritize the different decisions that make up the process Wonder: How do you decide when & how to intervene when a PwD is exhibiting a BD? 4 workshops with each time 4 caregives of the involved residential care centers
- 13. DECISION TREE WORKSHOP Knowledge engineer Asked questions insight into which concepts needed Example answers possible data/sub concepts Order of questions insight into importance 14
- 14. RESULTING DECISION TREES PRIORITY TREE
- 15. RESULTING DECISION TREES TASK ASSIGNMENT
- 16. DECISION TREES Taking into account: Type of incident & its priority (wandering, yelling,...) Resident’s historical behavior (BDs exhibited, agression, ...) Resident’s profile (liked/disliked activities, family (history), ...) Staff information (who is working, locations, relationship to residents, ...) Robot information (where are they, what are there capabilities, ...) Previous interactions of robot and PwD When robot is sent constant updates to staff & allowance to intervene Staff preferred to send the robot as much as possible when there is low probability of agression Personalization of actions of the robot Based on context & profile information create list of possible actions personalized to situation & resident Learning through reinforcement, what works and what not
- 17. RESULTING DECISION TREES DISTANCE ASSESSMENT
- 18. USER-DRIVEN ONTOLOGY & ALGORITHM DESIGN APPROACH Observation s & interviews Mind mapping Ontology re-use Scenario & persona definitions Decision tree workshops IoRT architecture Ontology & algorithm developmen t Wizard-of-oz testing
- 19. PERSONALIZED SERVICES THROUGH SEMANTIC REASONING
- 20. AMBIENT-AWARE CONTINUOUS CARE ONTOLOGY (ACCIO) HTTPS://GITHUB.COM/IBCNSERVICES/ACCIO-ONTOLOGY/ SSNIoTSAREFIoT PhysicalAss et TaskAccio RoleCompetenc eACCIO PersonProfil eACCIO Localization ContextAcci o DUL NurseCall FallRisk CareCoBots General Medical SS N SARE F Time OWL-S GEO- SPARQ L V-card Organ ization Unit Galen Know Rob
- 21. OBSERVATION PATTERN Observation hasSymptom Symptom hasFault isSymptomOf isFaultOf hasSolution LongLoudNoiseSymptom isSolutionOf Task requiresTask isTaskOf Fault Solution is a yellingPersonFault is a RobotSoothPersonSolution is a sendNurseNotificationAction RobotPlaysSoothingMusicAction is a is a NoiseObservation is a
- 22. TASK ASSIGNMENT SERVICES Yelling BD Detection Service Wandering BD Detection Service Distance Measure to Robot Service Priority BD Assessme nt Service Wandering BD Solution Assignme nt Service Yelling BD Solution Assignme nt Service Robot Task Assignme nt Service Staff Comm Service Semantic Communication Bus Robot Comm Service Adapters LocationFilter ≣ hasContext some (BLETagObservation or XetalObservation)
- 23. RobotPlaysSoothing MusicAction RobotSolution TASK ASSIGNMENT SERVICES Yelling BD Detection Service Wanderin g BD Detection Service Distance Measure to Robot Service Priority BD Assessme nt Service Wanderin g BD Solution Assignme nt Service Yelling BD Solution Assignme nt Service Robot Task Assignme nt Service Staff Comm Service Semantic Communication Bus Robot Comm Service Adapters NoiseObservation SoundObservation YellingPersonFault YellingPersonFault WanderingPersonFault YellingPersonFault High Priority YellingPersonFault Priority RobotSooth PersonSolution RobotSolution SendNurseNotification RobotAction RobotPlaysSoothing MusicAction Distance RobotAction Distance
- 24. WORK-IN-PROGRESS: BD DETECTION & PLATFORM EVALUATION
- 26. ANNOTATION OF BDS BY STAFF Wandering 4 types (purposefully, pacing, circles, random) Escaping Standing by the door Escaped outside Yelling Short Long A B A B B A D C
- 28. CONCLUSION
- 29. CONCLUSION User-driven approach towards design of personalized care cobot algorithms IoRT platform Contexualized task generation & assignment for robots Plug-in of new situational & personalized services based on semantic reasoning Realizing P4 Medicine through Semantic Reasoning enabling: - Heterogeneous data integration - Personalized event detection & intervention
- 30. PUBLIC THANK YOU, QUESTIONS? FEMKE ONGENAE & FEMKE DE BACKERE
Editor's Notes
- Along with the ageing population, the number of people with dementia (PWD) is steadily increasing [1]. The Alzheimer Liga Vlaanderen estimates a prevalence of 100,000 PWD in Flanders. Nursing homes around the globe are pulling out all the stops to provide the best possible care for residents with dementia. Almost all PWD exhibit behavioral disturbances (BDs) [2] like agitation (e.g., wandering or aggression), mood disorders (e.g., depression or apathy), sleep disorders and psychotic symptoms (e.g., delusions or hallucinations).
- These BDs can be prevented by non-pharmaceutical interventions [3], like personal interaction, revisiting positive personal memories and promoting comfort and quality of life of the PWD. However, because of increased strain on the available resources within healthcare, a dwindling number of caregivers needs to care for an increasing number of elderly [4]. This inhibits the staff from allocating a lot of their time to these interactions.
- A robot solution could help to provide such person-centric care by interacting with the PwD in order to alleviate BDs [5]. By audio-visual stimuli or by engaging communication, humanoid robots can elicit memories with associated positive feelings that have a calming effect on PwD. This improves the well-being of the PwD, but it can also be used to temporarily distract the PwD until the staff arrives in acute situations. As manifestations of dementia and the stimuli they react to, vary widely amongst PwD, a personalized approach is required. The key idea is that personalized interaction by robots improves 1) the well-being of residents, thus reducing the prevalence of behavioral disturbances, and 2) can temporarily distract PWD until the alerted staff arrives in acute situations. Personalization is especially important for companion robots, as it has been shown that this leads to more enjoyable experiences for the person and prevents losing interest in in the long-term [6], [7].
- In the WONDER project1, we are developing an Internet of Robotic Things (IoRT) platform to enable personalized robot interactions with PwD to reduce and intercept BDs. In an IoRT platform the robot is integrated in a smart en- vironment outfitted with a variability of sensors and wearables that capture the current context. A semantic cloud back-end then analyzes this captured information and combines it with other context information sources (e.g. profile of the PwD or day schedule of the institution) to extract valuable knowledge about the context and activities of the persons active in it. This derived knowledge is then used to steer the actions of the robot. The IoRT platform autonomously detects whether a BD occurred and determines which actions should be per- formed by the robot to respond to it in a personalized manner. Although several researchers have looked towards semantics to support the care for people with dementia, none of them focused on the combination of IoT and semantics in or- der to detect behavioral disturbances and derive the accompanying personalized intervention strategy [8], [9], [10].
- 7
- Task Assignment Services: uses semantic reasoning to decide which tasks should be executed by the nurses and robots Action Assignment Services: Path planning, charging, translating tasks into lower level steps Before we dive into the different task assignment services, Femke will explain the methodology we used to design these services
- Start: stakeholders each describe a complex situation pertaining to the goal of the workshop Layout of the institution = central discussion point Stakeholders are asked to play the “system” This system has a task e.g. a BD is occurring assign it to a person or robot and give them something to do (action) As the system, they have a bird-eye view, they can ask additional questions about the context (queries) Not immediately answered Discuss importance of question Discuss example answers Answer + visualize
- 1 2 Markov logic chains 3, 4, 5 semantic reasoning 7 Reinforcement learning
- 1 2 Markov logic chains 3, 4, 5 semantic reasoning 7 Reinforcement learning
- 1 2 Markov logic chains 3, 4, 5 semantic reasoning 7 Reinforcement learning
- Heterogeneous data integration & personalization of services through Semantic Reasoning leads us a step closer towards truly realizing P4 Medicine