The document outlines the Soprano Ambient Middleware (SAM) designed to facilitate interactions between sensors and actuators in ambient systems, highlighting its requirements for flexibility, adaptability, and extensibility. It discusses a modular architecture that incorporates semantic technologies for context management and reasoning, enabling various applications in telecare and customized services. Innovations in the system include procedural management for specific workflows, context awareness, and support for multiple stakeholders through clear interfaces and contracts.

1. SOPRANO Ambient Middleware (SAM) Andreas Schmidt (FZI) Michael Klein (CAS) 2nd Review Meeting March 19-20, 2009, Cologne

2. SOPRANO Ambient Middleware SOPRANO Ambient Middleware (SAM) is the glue between sensors and actuators is responsible for the ambient system behaviour

3. Overview SAM

4. Requirements Flexible Support a wide range of use cases Adaptability of pre-defined use cases to specific homes Extensible Allow for new uses cases without redesign New services New hardware sensors and actuators Open Clear and easy to understand interfaces, tailored to different market stakeholders Enable a market ecology

5. Market stakeholders contributing to SAM Hardware sensors/actuators manufacturers Software service developers Processing and enriching sensor information Telecare solution providers Definition of use cases and respective system behaviour independent of individual homes Case Manager / Care Provider Customization for individual customers

6. Ontology-based approach The ontology-centred design approach taken by SOPRANO is an enabler for a flexible and extensible middleware Ontology is an explicit representation of a shared understanding of the different stakeholders and forms the basis for semantic coherence of the system Semantic technologies (reasoning, service matchmaking) can deal with different levels of abstraction and make use of background knowledge in an automated way … but we have also taken care of keeping everything modular and manageable

7. SAM in Action Composer OSGi Service Middleware Context Manager AP at door Kettle is on Window is open No bell Door Sensor Procedure : AP about to leave home  start warning workflow Loudsp. Display Loudspeaker/touch screen “ Kettle is still on.” “ Window is still open.” Actuators Sensors Procedural Manager State: Door open Event : AP about to leave home Workflow instantiation: (1) Inform AP about problems Services: (1a) audio warning (1b) warning on touch screen

8. SOPRANO Architecture

9. Context Manager Sensor Service Context Storage Conflict Resolution Sensor Service Sensor Service Context Uplifter provide state information according to low-level context ontology Translate low-level context into high-level context Store all context facts with timestamp and uncertainty Provide a consistent view on context at a specific point in time can be extended by software developers

10. Context Reasoning Mechanisms Mechanism Characteristics Examples of Usage Rule Based System - easy to realize Door sensor  Door open Decision Trees - decision chains “ Inside” vs. “Outside” “ Sleeping” Neural Networks - handle uncertainty - adaptive - for complex problems - need training data “ Leaving“ “ Preparing Meal” Bayesian Networks - handle uncertainty - chains of actions - need domain experts “ Leaving“ “ Preparing Meal” Hidden Markov Models - sequences of events - for lots of sensors - need training data “ Exercising“ “ Being Bored” “ Socially interacting"

11. Collaboration of Uplifters - Example Sensor s Fires Door d Open s attached to Door d RFID chip c Detected by reader r AP a wears chip c Reader r is lo- cated near door d AP a near door d AP a leaving triggers workflow Doorbell db pressed AP outside Rule-based Rule-based Bayesian Written to context storage Written to context storage Written to context storage

12. Context Manager: Key Innovations Naive approach: RDF Triple-Store (+reasoning) but what about temporal data (histories!), uncertainty, ... ? SOPRANO Context Manager arbitrary methods for context uplifting not only rule or logic-based, but also statistical (mining approach) can provide snapshot views and histories (e.g., for mining) inconsistencies in the context storage possible conflict resolution according to configurable heuristics can deal with uncertainty and provides controlled aging can deal with negative facts asynchronous notifications

13. SOPRANO Architecture

14. Concretisation of Procedures Procedure Template Procedure Procedure Run Medication ?m is due  - Inform ?ap - wait (x) minutes for acceptance - on problem: inform (person) Medication ?m is due  - Inform ?ap - wait 10 minutes for acceptance - on problem: inform carer Richard Medication Sopranol is due  - Inform Peter - wait 10 minutes for acceptance - on problem: inform carer Richard Telecare Solution Provider Case Manager SAM – Procedural Mgr SAM – Composer Service invocation - inform Richard via SMS on +4912345

15. Procedure Manager Procedures are used to detect situations for which specific ambient system behaviour is modelled: event, context condition: state patterns workflow: BPEL workflow with abstract goals Situation Event Workflow Procedure conditions

16. SOPRANO Architecture

17. Inner Architecture of Composer Context-Aware Workflow Engine Service Matchmaker Service Invocator Composer Procedural Manager Abstract workflow Abstract service goal Concrete service OSGi Repository Query Service Service Parameterised invocation Context Manager Context query Context query

18. Context-Aware Workflow Engine Based on Standard: Business Process Execution Language (BPEL) Standard Constructs Context-Aware Extensions Sequence If Loop Assign … Query context manager Store statement Wait for statement Call semantic service

19. Example Procedure: Medication AP outside? Medication is-due [yes] [no] Inform AP about medication (visually & aurally, AP’s location) Wait for AP acknowledgement (service request) Inform carer of med problem [timeout] [ok] Wait for AP not outside Inform AP about medication (visually & aurally, near door) workflow event

20. Semantic Service Description Language Requirements Provide abstract and technical description of services Compatible with OSGi Fully automatically processable Service offers and requests are created independently by different stakeholders  Different levels of abstraction. Powerful matchmaker Take context information into account Cope with missing information Real-time Derive invocation parameters

21. Different Layers of Abstraction Inform AP about medication Play-speech On loudspeaker-1 Show message Via GUI on tv-1 Send SMS To Number {healthcare consultants} {case manager} {device service providers} Show message Via GUI on touchscreen-1 Speak via Avatar On tv-1 ?

22. Solution: Virtual Services Inform AP about medication Play sound message on loudspeaker-1 Show text message via GUI on tv-1 Send SMS to Number Show text message via GUI on touchscreen-1 Play avatar message on tv-1 Inform AP If not hearing-impaired Play sound message on loudspeaker in this or connected room of AP Inform AP If not vision-impaired Show message on GUI on switched-on device in room of AP

23. Proposal for SSD in OSGi …

24. Key innovation - Procedure execution Rule-Based Workflow-Based Based on semantical Service requests Event-driven  ambient system Procedural  deterministic system Declarative  context-aware, intelligent system

25. Key innovation: Semantic contracts Hardware sensors/actuators manufacturers contract: low-level context ontology, context storage Software developers can provide context uplifters contract: context ontology and context fact model can be packaged with sensors Telecare solution providers can develop procedure templates for certain use cases contract: procedure formalism, high-level context ontology can be packaged with sensors, actuators, and uplifters Care providers can customize procedures contract: procedure template, high-level context ontology

26. Summary Will be exploited as part of an open source initiative

27. WP2.2/2.3/2.4 SOPRANO Ambient Middleware (SAM) Andreas Schmidt (FZI) Michael Klein (CAS) 2nd Review Meeting March 19-20, 2009, Cologne Thank you!