A Distributed Staged Architecture for Multimodal Applications

604 views
443 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
604
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • No process
    Reverse approach: re-interpretation and refinement of architecture after new findings
  • Anders erzählen (heute, dann, Vorteile handsfree, kein Büro…)
  • role nicht component
  • Überleitung zum nächsten: DialogManager & AdaptationManager not merged
    map to different roles in the MMI-F
  • Überleitung zum nächsten: DialogManager & AdaptationManager not merged
    Adaptation Manager is not bound to this application
    map to different roles in the MMI-F (indication)
  • A Distributed Staged Architecture for Multimodal Applications

    1. 1. A Distributed Staged Architecture for Multimodal Applications Alessandro Costa Pereira, Falk Hartmann, Kay Kadner SAP Research/TU Dresden
    2. 2. Agenda  The Project SNOW  Prerequisites  Enabling Multimodal Access (MMI-F)  The SNOW Architecture  The Implementation  Lessons Learned/Research Questions © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications 2/13
    3. 3. Project SNOW 3/13  Services for Nomadic Workers  EU FP6 Project, 2004-2006, 6 Partners  www.snow-project.org A large-scale industrial diffusion of multimodal mobile documentation for maintenance operations.  How to author multimodal mobile maintenance documentation?  How to exploit this documentation through robust interaction modalities? © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    4. 4. 4/13 Project SNOW From paper … a ob C J rd Maintenance procedures and related information …to wearable electronic documentation  Airplane maintenance worker uses paper-based documentation  Unanticipated problems cause high expenses  Multimodality essential  Increased productivity for future maintenance © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    5. 5. Prerequisites 5/13  Multimodality is the use of multiple types of interaction between a user and a computer.  W3C Activity "Multimodal Interaction"  "Multimodal Interaction Framework" (MMI-F)  "Extensible MultiModal Annotation markup language" (EMMA)  A role model is "the description of a (possibly) infinite set of object collaborations using role types".  Riehle/Gross "Role model based framework design and integration"  A staged architecture is "a sequence of n subarchitectures, in which stage produces the (data for the) next stage"  Aßmann "Invasive Software Composition" © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    6. 6. Enabling Multimodal Access 6/13 W3C Multimodal Interaction Framework (MMI-F) The multimodal interaction framework is not an architecture . The multimodal interaction framework is a level of abstraction above an architecture. […] High-Level View OutputComponents Input Components © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    7. 7. Enabling Multimodal Access MMI-F is a role model  Short illustration of the core of the MMI-F  Applicable to the software architecture of your choice © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications 7/13
    8. 8. The SNOW Architecture Architectural Requirements  Client-/Server architecture  Domain independence  Device independence (⇐ multimodality)  (Smallest) End-user device: PDA  No speech-/gesture-recognition, no speech synthesis  Available modalities depend on network connection  Reuse standard software on PDA (Browser) © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications 8/13
    9. 9. 9/13 The SNOW Architecture Staged Architecture Browser Client Server Device-specific Adaptation Manager Dialog Manager Device-independent, Domain-independent Domain-independent Documentation Application Domain-specific Procedures © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    10. 10. 10/13 The SNOW Architecture Keyboard Control Integration Manager Speaker Control Camera Control Microphone Control Text to Speech Gesture Recognition Speech Recognition Browser Client Server Adaptation Manager Dialog Manager Procedures © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications Rendering Interpretation Styling Integration Documentation Application Recognition Generation Interaction Manager
    11. 11. The Implementation 11/13 Staged Architecture → XML transformation pipeline  Problem 1: Implementation effort  More components  Use of a common base technology in the components  Special template engine with bypassing (XTL Engine)  Problem 2: Performance penalty  Several parse/serialize processes  Mitigated using StAX for XML processing  Compensated by deployment options (esp. within development) © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    12. 12. Lessons Learned/Research Questions 12/13 Lessons Learned  Mapping the MMI-F to a concrete architecture can be difficult  Not very suitable for client/server architectures  Improvement of the MMI-F (fission role)  Staged Architecture can enable domain independence  Evaluated the architecture in a medical scenario (WearIT@Work)  Only the Documentation Application needs to be substituted  But: there is no best-suited architecture for multimodality Research Questions  How to generically apply the MMI-F role model? © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications
    13. 13. 13/13 Questions WLAN Access Point “Server” (Text-to-speech, Speech Recognition, Gesture recognition) Mobile Client (PDA w/ camera and headset) …Tablet PC based demonstrator now also available © SAP AG 2007, A Distributed Staged Architecture for Multimodal Applications

    ×