Use of hierarchical model-view-controller                architecture for user interaction in AAL                         ...
Usually elements are classified in a number of subclasseswhich further specifies their properties. The most generalclassifica...
C. Maintainability     M                     C                                           Maintainability is the measure of...
MH                                                                                L&F Extension                           ...
the given extension.B. UniversAAL Technological Solution   UniversAAL is an European research funded project [11],which co...
displays. Other interesting L&F packages are AAL impairment                         Communication and Interaction, Human-C...
Upcoming SlideShare
Loading in …5

Paper Alejandro Medrano - Use of hierarchical model-view-controller architecture for user interaction in AAL environments


Published on

Published in: Technology, News & Politics
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Paper Alejandro Medrano - Use of hierarchical model-view-controller architecture for user interaction in AAL environments

  1. 1. Use of hierarchical model-view-controller architecture for user interaction in AAL environments Alejandro M. Medrano Gil ∗ , Dario Salvi ∗ , María Teresa Arredondo Waldmeyer ∗ and Patricia Abril Jimenez ∗ ∗ Life Supporting Technologies, Departamento de Tecnología Fotónica y Bioingeniería, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid Avenida Complutense n. 30, "Ciudad Universitaria". 28040 - Madrid (SPAIN) {amedrano, dsalvi, mta, pabril} Abstract—The Ambient Assisted Living (AAL) concept applies On the methodological side also User Centered Designa model of provision of various services for elderly and people methodologies are based on the assumption that the productwith special needs. AAL services should include invisible, but must be created around and for its final users, independentlyalways present technology; natural, simple and effortless interac-tion with users; device-independent and reactive user interfaces; of their abilities. These methodologies has been successfullyand continuous, autonomous and ubiquitous service adaptation. applied in many projects related to AAL [6], [3], [1]. In this context, the presentation of the information is a User Centered Design methodologies have been the reasonchallenge. The system installed in a AAL environment should be for the development of multiple user interaction (UI) solutionsable to support dynamic presentation of data according user’s like gesture recognition, voice activation, different kinds ofpreferences and needs and environment conditions. Separatingthe view from the data and logic present an advantage when visual representations that mean that application developersestablishing coding patterns and software architecture for the should be extremely flexible when designing the interaction.domain. However, as each of these modalities has its own particular- This paper reveals the details and advantages of using Hierar- ities, the information to be provided to the users have to bechical Model View (HMVC), proving its feasibility with a generic adapted and this causes some problems and difficulties in thedesign for User Interaction (UI) Renderers and the concretesolutions implemented in an EU funded research project. development process. II. D EFINITIONS I. I NTRODUCTION In order to represent the User Interaction (UI) independently Ambient Assisted Living (AAL) is a technological frame- of the interaction modality, a meta model of UI must bework whose main focus is to make life easier to elderly defined. This meta model is then interpreted and translatedand disabled people by introducing smart technology in their into a concrete UI mean by what can be called a UI renderer.daily lives. This kind of users are particularly demanding in The following paragraphs offer a deeper overview of these twoterms of user interaction requirements. Elderly users do not concepts.usually have the required knowledge or skills to fully interactwith modern electronic systems, while other users can present A. UI Meta Modelsdisabilities that can pose limits to their daily activities, and The objective of the meta model is to model the user inter-thus also when interacting with software. Although addressing action neutral to the form of modality or the used technology.these impaiments is particulary challenging for software devel- This model represents the type of information that can be sentopers, there is also a long-established experience in application to users and can received from them and the actual content ofdevelopments for this particular users. the message. Universal Design1 , or Design for All2 (D4All) are fields that An example of a standard meta models for UI is XFormsattracted big efforts in the research [2] and from European [10], other specific examples can be found within AAL plat-policies3 . ICT D4All aims to build systems usable by the vast forms, where user interaction models have proliferated [8], [7],majority of people, regardless of factors such as their abilities [11].or age, the interaction platform and the context in which they Meta models are defined by examining carefully the UIoperate [4]. requirements, trying to factorize and categorize the informa- 1 tion types exchanged by user and applications. The basic 2 concept behind these models is the element, a piece of UI that 3 contains a message. Elements are then grouped in containersaccessibility/dfa that represent the user interaction in a given point in time.
  2. 2. Usually elements are classified in a number of subclasseswhich further specifies their properties. The most generalclassification of elements include input elements, output, labels GUI Renderer VUI Rendererand groups. • Input elements are elements which value can be changed by the user. Input elements may also be seen as output- input as they show the state before and after the user interaction. Meta Model • Output elements, on the other hand, do not enable the user to change their status, and they are used to display static information to the user. • Labels help modelling the tag that the rest of elements App1 App2 App3 should have, displaying the name of the user or an icon. The use for these labels also includes modelling of the options provided for elements that involve the user Figure 1: Layered UI architecture making a selection out of a discrete set of options. • Groups are used to bond a set of elements together to help the UI renderer, and eventually the user to understand that so. The task for the UI renderer is to translate from the meta those elements relate conceptually. model into the underlying technology’s components, and the For each of these classes there are specializations, for in- user input back into the meta model.stance, to display and retrieve boolean values, others specialize As most of the information is modelled as modality neutral,in simple text, or numerical values within a specific range. there can be many UI renderers, each one managing different The choice of these subclasses is domain of the application. set of channels with different modalities.Applications use the meta model by making instances of the C. Interaction model among Applications, Meta Models anddesired container, and filling it with instances of the elements UI Renderersthey need to show to the user, or need to retrieve fromthe user. This model is then presented to the user using a The global picture of how the whole system works isUI renderer, when the user interacts with the renderer the presented in figure 1. Application developers focus on thelatter will inject the information provided by the user into the whole stack using a canonical MVC approach, they do notmodel. The application then receive this information through need a meta model nor an independent UI renderer. Bythe mechanism the model provides, either a callback or an introducing the meta model application developers can focusevent. on the requirements of their application and ignoring details This process forces the application developers to design UI about the user interface. The meta model also allow severalin neutral modality terms, preventing them from manipulating applications to share a common interface, and look.the end result, and keeping them agnostic of the hardware or Applications create instances of the meta model, includingtechnology used in the actual presentation. the required containers with their properties, and pass them to The use of the meta model forces also the use of MVC UI renderers. In some cases this process of transporting thepattern, as the meta model itself represents the view part of meta model instance from the application to the UI renderer isany application, and provides mechanisms to help the design done through another component. When this occurs normallyof the controller section. it is because the meta model is part of a platform, and the mechanism of transportation is also a sub system ofB. UI Renderers the UI management. The case of platforms with this kind The function of a UI renderer is to interpret the meta of UI approach usually enable different applications and UImodel into a useful form that the user understands and can renderers to run at the same time, just like shown in figure 1.interact with. Usually one UI renderer is associated to onetechnology (and the compatible hardware) and one or more D. Nested and Hierarchical MVCcommunication channels. This approach implies a MVC stacking. The applications A UI renderer may be designed for a wide spectrum of use MVC where the view is a meta model instance and theusers, but can also be specialized to fit a set of users with UI renderer also uses MVC, but this very same instance isspecial needs. For example a GUI renderer may be configured the model. This concept is called nested MVC, as the MVCto show bigger fonts if the user has mild visual impairment or for the UI renderer plays within the view of the MVC of themay be purposely built for cognitive deprived users that are application (see figure 2 a).unable to remember short term events or can not read. This concept should not be confused with hierarchical The containers defined in the meta model are rendered ac- MVC (HMVC) (see figure 2). HMVC refers to the MVCcording to their definition as close as the hardware, technology being applied several times at different levels. In the scopeand/or communication channel enables the UI renderer to do of this paper, HMVC is the application of MVC for meta
  3. 3. C. Maintainability M C Maintainability is the measure of how the project can be re- M C paired, evolve, or be adapted in the future. The most significant contribution to maintainability is a clear and understandable V V design, where the key features are immediately spotted and developers have easy access to them. M C D. Testability V M C M C Testability in user interaction always implies tricky tech- niques, as the UI by definition needs a user to interact with. V V Nevertheless testability means also the easiness to test indi- vidual parts of the design independently; with the decoupling (a) NMVC (b) HMVC of the model and view, both can be tested independently. The proposed design also provides an easy way to test individ-Figure 2: Diference between Nested and Hierarchical MVC ual meta model container interpretations and even individual elements.model containers in top level, then grouping elements, and E. Scalabilitysimple elements on the bottom level. HMVC means that each Scalability means that the design is equally applicable forcomponent of the meta model is treated autonomously, and increased functionality. In this context increased functionalityindependent from all other components of the meta model, is the extension of the meta model, with new containersthe only relationship being a hierarchical one. or elements. We propose the use of reflective properties, applicable in the objective programming language like Java, III. M ETHODOLOGY in order to guarantee dynamic mapping between model and view. This way any change in the meta model is immediately UI renderers should follow the MVC pattern, not just applicable by either modifying an existing class or adding abecause the MVC is imposed by the usage of a meta model, new class per new element.but because it helps the design and clarity of the implementa-tion. In this scope the model is represented by the incoming IV. R ESULTSinstances of the meta model, the view role is played by the This section presents the results of the design and theunderlying rendering technology, and the controller is the UI implementation of a UI renderer within the AAL projectrenderer’s logic. universAAL. Section IV-A proposes an architectural design The main idea behind this design is to make highly config- which is independent from Meta model and the underlyingurable, extensible and maintainable UI renderer. technology used for generating the user interaction. Section The advantages of the adoption of such methodology can be IV-B1 shows how we implemented this design in the univer-classified in terms of extensibility, re-usability, maintainabil- sAAL platform using Java Swing as the graphical renderer.ity, testability and scalability of the produced UI renderers.Following, these five aspects are analysed separately. A. UI Renderer Design The proposed architecture is composed of 4 main com-A. Extensibility ponents, a Meta Model Component (MMC), a Container Expected extensions of the renderer comprise meta model Management Unit (CMU), a View Mapper (VM) and a Mainextensions, with new containers and/or new elements, and the Hub (MH), along with view modules per container and perconfiguration of the actual look and feel of the components of element, as shown in figure 3.the underlying technology (for example the usage of different The MMC is the module in charge of interfacing with thecolours, fonts, or layouts for GUI renderers). By separating meta model, listening to the meta model events and delegatingthe model from the controller and the view, these extension the received instances to CMU.are dealt separately in different modules of the solution. The CMU decides which container to show at each time, according to the context or meta-data included in the meta model. Meta-data included in the meta model can describeB. Reusability specific needs, for example they can describe the priority a Reusability is the capability of applying the same solution to container has, or the language used in the content, to theother UI renderers, in fact not only the design can be applied emphasis of using one modality over another. The CMUbut also great part of the implementation has great potential to may implement the logic to queue a container that has lessbe reused. The MVC model decouples model form view this priority than the container that is currently being displayed atway the controller part keeps unaltered when the view part is that moment, or filter those containers that do not match thechanged to develop a new UI renderer. user’s language or the renderer’s modality. The CMU is then
  4. 4. MH L&F Extension Meta Model CMU VM MMC Extension UI Renderer Container View Element View Figure 3: Proposed UI renderer architecture Meta Model Figure 4: Extensions contextresponsible for obtaining the implementation of the containerby querying the the VM to get the container view component. The VM is a piece of software that locates the viewcomponent for any container or element instance of the meta developers a simple, yet powerful, way to specialize the UImodel. For example when the VM is required to search for an renderer for certain impairments. For example the rendererelement or container of type T, then the viewing component will normally use fancy colours and fonts, while in presencefor that type is loaded, for instance on the basis of a naming of visual impaired users, the renderer would adopt anotherconvention that associates T with TView. Reflection can be template with bigger sizes, simplified fonts and high-contrastused to easily implement this mechanism. colours. The MH will act as a place holder for all common needed This design inherently allows meta model extensions andvariables, including the MMC and the CMU. This allows to L&F extensions. These extensions can be provided in separatereplace these components, when for example another meta packages and dynamically incorporated to the UI renderer (seemodel is used, or when another display management is re- figure 4). Being reflection used in VM, the new meta modelquired. components are directly mapped to the provided viewing The container view component will make use of the VM find the element view component for each of it’s children For instance it is possible to enrich the choice of L&F(matching the expected behaviour of top level components of templates by adding them to the VM (e.g. simplified L&F,the HMVC hierarchy). Grouping element views would do the customised colours etc.), or it is possible to enrich the metasame technique but their hierarchical level is lower as they can model by extending it and by providing proper viewingbe “owned” by containers or other grouping elements. components for it (e.g. a viewer that plots arrays on a two View components in general (element and container view axis graph).components) are atomic translators for the meta model com- L&F packages are loaded by instructing the VM to loadponents. For each meta model component’s MVC (as part of the L&F components instead of the proper view packages.the HMVC) there would be a view component, which will As L&F components are extensions of the proper component,allocate all the necessary instructions to do the translation into they incorporate all the needed functionality required from athe underlying technology’s model. viewing component. Furthermore, each view component can be decoupled into Meta model extensions can be very useful, for exampletwo major parts, the information translation and the look and an application may want to display graphs, which are notfeel (L&F) layers. The idea is to decouple what a component usually defined in the original meta model; as it will beshows from how it shows it. defined in neutral modality and graphs only make sense in The L&F is independent from proper viewing component graphical modalities. The way to provide this functionality(information translation layer), and several L&F templates can is by providing meta model extensions, in some sense thesebe developed for the same UI renderer. L&F templates will extensions are stackable, so they may be thought of as plugins.take care of colours, fonts, layouts, shapes, sizes and every The proposed design automatically incorporates these plu-displaying detail that the proper view component does not gins. When a reflection VM is used, the new meta modeltake care of. The capability of providing different L&Fs is components are directly mapped to the provided viewingsomething appreciated by any user, but in the case of AAL components. So the packaged meta model extension pluginusers is specially important. This mechanism offers AAL for a UI renderer just contains the viewing components for
  5. 5. the given extension.B. UniversAAL Technological Solution UniversAAL is an European research funded project [11],which copes with the challenge to create a standard openplatform that makes technically feasible and economicallyviable the development of personalized AAL solutions. As part of its platform universAAL provides UI supportas a basic functionality. The main software artefacts of theplatform around which the UI management is built are theinput and output buses. In universAAl a bus is a virtual channelof communication through which semantic rich informationtravels, these buses connect pieces of software within the samenode and between nodes providing the power of distributed Figure 5: Simple test showing some of the key features of theprocessing. The output bus is the mechanism through which default look and feel package.output events are presented to the user, likewise the input busdoes the same job but inversely, taking user input events andpresenting them to the applications. to commit their mission, that is: to send all user input when The meta model in universAAL, wrapped in these input and the submit is activated by the user.output events, is a definition of resources called IO RDF, even The handler had no need for a queued CMU as univer-though it’s implementation is not in actual RDF [9], but a Java sAAL’s DM already manages the priority.equivalent representation. This meta model features 4 types of Look and feel is supported by means of the standard Javaforms, grouping elements, several input and output elements, Swing look and feel templates. A standard template for alland a special kind of element called submit, which triggers the universAAL applications has been produced, although it leavesuser’s input events and terminates dialogs. This meta model the possibility of extending it or implementing a new very basic, as in any AAL environments UI is not expected The visual results of the implementation produce an elegantto be very complex due to users’ limitations. The IO RDF is and simple graphical output which can run seamlessly on dif-based on XForms [10], and like it, it is susceptible of being ferent environments and operating systems and with differentextended. screen resolutions. UniversAAL adds another component to the IO buses and V. C ONCLUSIONS AND FUTURE WORKthe IO RDF: the Dialog Manager (DM). The DM is a dialog This paper shows how to apply hierarchical MVC patternsorchestrator which coordinates all output and input events to the design of AAL user interaction renderers. The proposedcoming from and to applications, playing the role of the architecture and the example of implementation provided forCMU at system wise level. This coordination involves multi the universAAL platform prove that the use of this patternuser coordination, multi modality synchronization, priority simplifies the effort in developing UI, impairment mapping, and many other important The Swing Handler has been compared with an alreadyadministration of the whole UI. existing handler of the platform. Thanks to our design, locating Lastly but not least important, the UI renderers, here called functionalities of each IO RDF resource representation wasUI handlers, cope with human-machine interaction and map- much easier on our implementation than the existing to and from the IO RDF (meta model). Moreover, the Swing Handler allows the redefinition of the 1) The Swing Handler: Within this framework, we have IO RDF, extending it without hardly the need to change thedeveloped a UI renderer based on the well-established Java central logic of the handler. The adoption of Swing templatesSwing library [5] as graphical rendering engine, namely the allows also the capability of changing the look and feel, evenSwing Handler, to prove the applicability of the HMVC in at runtime.UI renderers for an AAL environment. This Swing Handler Nevertheless the implementation of a single class for eachfollows the design proposed in section IV-A, but specially element in the meta model was tedious work, and for largeadapted to univesAAL unique features. meta models it would require a bigger development team. UniversAAL provides two elements to receive and publish The duality of view component and L&F template, makes thatoutput and input events respectively, these are the output the development of new templates is equally human resourcesubscriber and the input publisher, both form the meta model demanding. But usually already created L&F packages willcomponent as their role is to interact directly with the IO serve as reference and source templates to develop new l&Fbuses. Elements, or Form Controls as IO RDF defines them, packages, which eases the procedure to create new L&Fs.have their view following a HMVC paradigm. In the case of As future improvements we propose the development ofsubmit form controls have a main role in the control part of plugins for the Swing handler. Including L&F packages spe-the handler’s MVC. They are connected to the input publisher cialized for specific hardware like hand held devices or TV
  6. 6. displays. Other interesting L&F packages are AAL impairment Communication and Interaction, Human-Computer Interaction Series,specific like short sighted or cognitive deprived users. As far pages 114–131. Springer London, 2008. 10.1007/978-1-84628-927-9_7. [2] Laura Burzagli, Pier Luigi Emiliani, and Francesco Gabbanini. Designas L&F template enables there should be a specific template for all in action: An example of analysis and implementation. Expertper hardware and impairment, so we propose the research on Systems with Applications, 36(2, Part 1):985 – 994, to combine L&F packages. This will enable to easily [3] Annette De Vito Dabbs, Brad A. Myers, Kenneth R. Mc Curry, Jacque- line Dunbar-Jacob, Robert P. Hawkins, Alex Begey, and Mary Amandadevelop a specific handler, for example a new L&F template Dew. User-centered design and interactive health technologies forcould be created from combining existing templates for hand patients. CIN: Computers, Informatics, Nursing, 27, May/June 2009.held device and mild sight impairment, in stead of creating [4] P. L. Emiliani and C. Stephanidis. Universal access to ambient intelligence environments: opportunities and challenges for people withthe same handler from scratch. disabilities. IBM Syst. J., 44:605–619, August 2005. Finally, another future improvements involves the other type [5] Amy Fowler. A swing architecture overview. plugin, the meta model extension. We propose the creation products/jfc/tsc/articles/architecture/. [6] Sonja Müller, Ingo Meyer, Ilse Bierhoff, Sarah Delaney, Andrew Six-of GUI modal specific features, like for example a graph and smith, and Sandra Sproll. Iterative User Involvement in Ambient Assistedchart element which will draw a graphical representation of Living Research and Development Processes: Does It Really Make asignal data or any type of numerical data series. Difference? 2011. [7] AALuis project. Facilitate the connection of different services to ACKNOWLEDGEMENTS different types of user interfaces and thus enable the future users of ambient assisted living systems to use more services interacting in their We would like to thank the whole universAAL Consortium preferred way. their valuable contribution for the realization of this work. [8] GoldUI Project. European aal project for adaptive embedded human interfaces designed for older people. work has been partially funded by European Union under [9] W3C RDF Working Group. Resource description framework a standardthe 7th Framework Program through the universAAL (FP7- model for data interchange on the web. research project. [10] W3C The Forms Working Group. Next generation of forms technology for the world wide web. R EFERENCES [11] universAAL project. Universal open platform and reference specification for ambient assisted living. [1] Jan O. Borchers. A pattern approach to interaction design. In Satinder Gill, John Karat, and Jean Vanderdonckt, editors, Cognition,