This document discusses using a hierarchical model-view-controller (HMVC) architecture for user interfaces in ambient assisted living (AAL) environments. It proposes separating the user interface meta model from the presentation logic and user interaction modalities. This allows applications to define modality-neutral interfaces that can be rendered across different modalities like graphical user interfaces, voice interfaces, etc. It describes defining a common meta model for user interface elements and using UI renderers to translate this model to specific modalities. The HMVC approach keeps applications independent of interaction details and allows multiple applications to share interfaces.

1. Use of hierarchical model-view-controller
architecture for user interaction in AAL
environments
Alejandro M. Medrano Gil ∗ , Dario Salvi ∗ , María Teresa Arredondo Waldmeyer ∗ and Patricia Abril Jimenez ∗
∗
Life Supporting Technologies, Departamento de Tecnología Fotónica y Bioingeniería,
Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madrid
Avenida Complutense n. 30, "Ciudad Universitaria". 28040 - Madrid (SPAIN)
{amedrano, dsalvi, mta, pabril}@lst.tfo.upm.es
Abstract—The Ambient Assisted Living (AAL) concept applies On the methodological side also User Centered Design
a model of provision of various services for elderly and people methodologies are based on the assumption that the product
with special needs. AAL services should include invisible, but must be created around and for its final users, independently
always present technology; natural, simple and effortless interac-
tion with users; device-independent and reactive user interfaces; of their abilities. These methodologies has been successfully
and continuous, autonomous and ubiquitous service adaptation. applied in many projects related to AAL [6], [3], [1].
In this context, the presentation of the information is a User Centered Design methodologies have been the reason
challenge. The system installed in a AAL environment should be for the development of multiple user interaction (UI) solutions
able to support dynamic presentation of data according user’s like gesture recognition, voice activation, different kinds of
preferences and needs and environment conditions. Separating
the view from the data and logic present an advantage when visual representations that mean that application developers
establishing coding patterns and software architecture for the should be extremely flexible when designing the interaction.
domain. However, as each of these modalities has its own particular-
This paper reveals the details and advantages of using Hierar- ities, the information to be provided to the users have to be
chical Model View (HMVC), proving its feasibility with a generic adapted and this causes some problems and difficulties in the
design for User Interaction (UI) Renderers and the concrete
solutions implemented in an EU funded research project.
development process.
II. D EFINITIONS
I. I NTRODUCTION
In order to represent the User Interaction (UI) independently
Ambient Assisted Living (AAL) is a technological frame- of the interaction modality, a meta model of UI must be
work whose main focus is to make life easier to elderly defined. This meta model is then interpreted and translated
and disabled people by introducing smart technology in their into a concrete UI mean by what can be called a UI renderer.
daily lives. This kind of users are particularly demanding in The following paragraphs offer a deeper overview of these two
terms of user interaction requirements. Elderly users do not concepts.
usually have the required knowledge or skills to fully interact
with modern electronic systems, while other users can present A. UI Meta Models
disabilities that can pose limits to their daily activities, and The objective of the meta model is to model the user inter-
thus also when interacting with software. Although addressing action neutral to the form of modality or the used technology.
these impaiments is particulary challenging for software devel- This model represents the type of information that can be sent
opers, there is also a long-established experience in application to users and can received from them and the actual content of
developments for this particular users. the message.
Universal Design1 , or Design for All2 (D4All) are fields that An example of a standard meta models for UI is XForms
attracted big efforts in the research [2] and from European [10], other specific examples can be found within AAL plat-
policies3 . ICT D4All aims to build systems usable by the vast forms, where user interaction models have proliferated [8], [7],
majority of people, regardless of factors such as their abilities [11].
or age, the interaction platform and the context in which they Meta models are defined by examining carefully the UI
operate [4]. requirements, trying to factorize and categorize the informa-
1 http://www.ncsu.edu/project/design-projects/udi/
tion types exchanged by user and applications. The basic
2 http://www.designforall.org/ concept behind these models is the element, a piece of UI that
3 http://ec.europa.eu/information_society/activities/einclusion/policy/ contains a message. Elements are then grouped in containers
accessibility/dfa that represent the user interaction in a given point in time.

2. Usually elements are classified in a number of subclasses
which further specifies their properties. The most general
classification of elements include input elements, output, labels
GUI Renderer VUI Renderer
and groups.
• Input elements are elements which value can be changed
by the user. Input elements may also be seen as output-
input as they show the state before and after the user
interaction.
Meta Model
• Output elements, on the other hand, do not enable the
user to change their status, and they are used to display
static information to the user.
• Labels help modelling the tag that the rest of elements App1
App2 App3
should have, displaying the name of the user or an
icon. The use for these labels also includes modelling
of the options provided for elements that involve the user Figure 1: Layered UI architecture
making a selection out of a discrete set of options.
• Groups are used to bond a set of elements together to help
the UI renderer, and eventually the user to understand that so. The task for the UI renderer is to translate from the meta
those elements relate conceptually. model into the underlying technology’s components, and the
For each of these classes there are specializations, for in- user input back into the meta model.
stance, to display and retrieve boolean values, others specialize As most of the information is modelled as modality neutral,
in simple text, or numerical values within a specific range. there can be many UI renderers, each one managing different
The choice of these subclasses is domain of the application. set of channels with different modalities.
Applications use the meta model by making instances of the
C. Interaction model among Applications, Meta Models and
desired container, and filling it with instances of the elements
UI Renderers
they need to show to the user, or need to retrieve from
the user. This model is then presented to the user using a The global picture of how the whole system works is
UI renderer, when the user interacts with the renderer the presented in figure 1. Application developers focus on the
latter will inject the information provided by the user into the whole stack using a canonical MVC approach, they do not
model. The application then receive this information through need a meta model nor an independent UI renderer. By
the mechanism the model provides, either a callback or an introducing the meta model application developers can focus
event. on the requirements of their application and ignoring details
This process forces the application developers to design UI about the user interface. The meta model also allow several
in neutral modality terms, preventing them from manipulating applications to share a common interface, and look.
the end result, and keeping them agnostic of the hardware or Applications create instances of the meta model, including
technology used in the actual presentation. the required containers with their properties, and pass them to
The use of the meta model forces also the use of MVC UI renderers. In some cases this process of transporting the
pattern, as the meta model itself represents the view part of meta model instance from the application to the UI renderer is
any application, and provides mechanisms to help the design done through another component. When this occurs normally
of the controller section. it is because the meta model is part of a platform, and
the mechanism of transportation is also a sub system of
B. UI Renderers the UI management. The case of platforms with this kind
The function of a UI renderer is to interpret the meta of UI approach usually enable different applications and UI
model into a useful form that the user understands and can renderers to run at the same time, just like shown in figure 1.
interact with. Usually one UI renderer is associated to one
technology (and the compatible hardware) and one or more D. Nested and Hierarchical MVC
communication channels. This approach implies a MVC stacking. The applications
A UI renderer may be designed for a wide spectrum of use MVC where the view is a meta model instance and the
users, but can also be specialized to fit a set of users with UI renderer also uses MVC, but this very same instance is
special needs. For example a GUI renderer may be configured the model. This concept is called nested MVC, as the MVC
to show bigger fonts if the user has mild visual impairment or for the UI renderer plays within the view of the MVC of the
may be purposely built for cognitive deprived users that are application (see figure 2 a).
unable to remember short term events or can not read. This concept should not be confused with hierarchical
The containers defined in the meta model are rendered ac- MVC (HMVC) (see figure 2). HMVC refers to the MVC
cording to their definition as close as the hardware, technology being applied several times at different levels. In the scope
and/or communication channel enables the UI renderer to do of this paper, HMVC is the application of MVC for meta

3. C. Maintainability
M C Maintainability is the measure of how the project can be re-
M C paired, evolve, or be adapted in the future. The most significant
contribution to maintainability is a clear and understandable
V V design, where the key features are immediately spotted and
developers have easy access to them.
M C D. Testability
V M C M C Testability in user interaction always implies tricky tech-
niques, as the UI by definition needs a user to interact with.
V V Nevertheless testability means also the easiness to test indi-
vidual parts of the design independently; with the decoupling
(a) NMVC (b) HMVC of the model and view, both can be tested independently. The
proposed design also provides an easy way to test individ-
Figure 2: Diference between Nested and Hierarchical MVC
ual meta model container interpretations and even individual
elements.
model containers in top level, then grouping elements, and E. Scalability
simple elements on the bottom level. HMVC means that each Scalability means that the design is equally applicable for
component of the meta model is treated autonomously, and increased functionality. In this context increased functionality
independent from all other components of the meta model, is the extension of the meta model, with new containers
the only relationship being a hierarchical one. or elements. We propose the use of reflective properties,
applicable in the objective programming language like Java,
III. M ETHODOLOGY in order to guarantee dynamic mapping between model and
view. This way any change in the meta model is immediately
UI renderers should follow the MVC pattern, not just
applicable by either modifying an existing class or adding a
because the MVC is imposed by the usage of a meta model,
new class per new element.
but because it helps the design and clarity of the implementa-
tion. In this scope the model is represented by the incoming IV. R ESULTS
instances of the meta model, the view role is played by the
This section presents the results of the design and the
underlying rendering technology, and the controller is the UI
implementation of a UI renderer within the AAL project
renderer’s logic.
universAAL. Section IV-A proposes an architectural design
The main idea behind this design is to make highly config- which is independent from Meta model and the underlying
urable, extensible and maintainable UI renderer. technology used for generating the user interaction. Section
The advantages of the adoption of such methodology can be IV-B1 shows how we implemented this design in the univer-
classified in terms of extensibility, re-usability, maintainabil- sAAL platform using Java Swing as the graphical renderer.
ity, testability and scalability of the produced UI renderers.
Following, these five aspects are analysed separately. A. UI Renderer Design
The proposed architecture is composed of 4 main com-
A. Extensibility ponents, a Meta Model Component (MMC), a Container
Expected extensions of the renderer comprise meta model Management Unit (CMU), a View Mapper (VM) and a Main
extensions, with new containers and/or new elements, and the Hub (MH), along with view modules per container and per
configuration of the actual look and feel of the components of element, as shown in figure 3.
the underlying technology (for example the usage of different The MMC is the module in charge of interfacing with the
colours, fonts, or layouts for GUI renderers). By separating meta model, listening to the meta model events and delegating
the model from the controller and the view, these extension the received instances to CMU.
are dealt separately in different modules of the solution. The CMU decides which container to show at each time,
according to the context or meta-data included in the meta
model. Meta-data included in the meta model can describe
B. Reusability
specific needs, for example they can describe the priority a
Reusability is the capability of applying the same solution to container has, or the language used in the content, to the
other UI renderers, in fact not only the design can be applied emphasis of using one modality over another. The CMU
but also great part of the implementation has great potential to may implement the logic to queue a container that has less
be reused. The MVC model decouples model form view this priority than the container that is currently being displayed at
way the controller part keeps unaltered when the view part is that moment, or filter those containers that do not match the
changed to develop a new UI renderer. user’s language or the renderer’s modality. The CMU is then

4. MH
L&F Extension
Meta Model
CMU VM MMC
Extension
UI Renderer
Container View Element View
Figure 3: Proposed UI renderer architecture
Meta Model
Figure 4: Extensions context
responsible for obtaining the implementation of the container
by querying the the VM to get the container view component.
The VM is a piece of software that locates the view
component for any container or element instance of the meta developers a simple, yet powerful, way to specialize the UI
model. For example when the VM is required to search for an renderer for certain impairments. For example the renderer
element or container of type T, then the viewing component will normally use fancy colours and fonts, while in presence
for that type is loaded, for instance on the basis of a naming of visual impaired users, the renderer would adopt another
convention that associates T with TView. Reflection can be template with bigger sizes, simplified fonts and high-contrast
used to easily implement this mechanism. colours.
The MH will act as a place holder for all common needed This design inherently allows meta model extensions and
variables, including the MMC and the CMU. This allows to L&F extensions. These extensions can be provided in separate
replace these components, when for example another meta packages and dynamically incorporated to the UI renderer (see
model is used, or when another display management is re- figure 4). Being reflection used in VM, the new meta model
quired. components are directly mapped to the provided viewing
The container view component will make use of the VM components.
to find the element view component for each of it’s children For instance it is possible to enrich the choice of L&F
(matching the expected behaviour of top level components of templates by adding them to the VM (e.g. simplified L&F,
the HMVC hierarchy). Grouping element views would do the customised colours etc.), or it is possible to enrich the meta
same technique but their hierarchical level is lower as they can model by extending it and by providing proper viewing
be “owned” by containers or other grouping elements. components for it (e.g. a viewer that plots arrays on a two
View components in general (element and container view axis graph).
components) are atomic translators for the meta model com- L&F packages are loaded by instructing the VM to load
ponents. For each meta model component’s MVC (as part of the L&F components instead of the proper view packages.
the HMVC) there would be a view component, which will As L&F components are extensions of the proper component,
allocate all the necessary instructions to do the translation into they incorporate all the needed functionality required from a
the underlying technology’s model. viewing component.
Furthermore, each view component can be decoupled into Meta model extensions can be very useful, for example
two major parts, the information translation and the look and an application may want to display graphs, which are not
feel (L&F) layers. The idea is to decouple what a component usually defined in the original meta model; as it will be
shows from how it shows it. defined in neutral modality and graphs only make sense in
The L&F is independent from proper viewing component graphical modalities. The way to provide this functionality
(information translation layer), and several L&F templates can is by providing meta model extensions, in some sense these
be developed for the same UI renderer. L&F templates will extensions are stackable, so they may be thought of as plugins.
take care of colours, fonts, layouts, shapes, sizes and every The proposed design automatically incorporates these plu-
displaying detail that the proper view component does not gins. When a reflection VM is used, the new meta model
take care of. The capability of providing different L&Fs is components are directly mapped to the provided viewing
something appreciated by any user, but in the case of AAL components. So the packaged meta model extension plugin
users is specially important. This mechanism offers AAL for a UI renderer just contains the viewing components for

5. the given extension.
B. UniversAAL Technological Solution
UniversAAL is an European research funded project [11],
which copes with the challenge to create a standard open
platform that makes technically feasible and economically
viable the development of personalized AAL solutions.
As part of its platform universAAL provides UI support
as a basic functionality. The main software artefacts of the
platform around which the UI management is built are the
input and output buses. In universAAl a bus is a virtual channel
of communication through which semantic rich information
travels, these buses connect pieces of software within the same
node and between nodes providing the power of distributed Figure 5: Simple test showing some of the key features of the
processing. The output bus is the mechanism through which default look and feel package.
output events are presented to the user, likewise the input bus
does the same job but inversely, taking user input events and
presenting them to the applications. to commit their mission, that is: to send all user input when
The meta model in universAAL, wrapped in these input and the submit is activated by the user.
output events, is a definition of resources called IO RDF, even The handler had no need for a queued CMU as univer-
though it’s implementation is not in actual RDF [9], but a Java sAAL’s DM already manages the priority.
equivalent representation. This meta model features 4 types of Look and feel is supported by means of the standard Java
forms, grouping elements, several input and output elements, Swing look and feel templates. A standard template for all
and a special kind of element called submit, which triggers the universAAL applications has been produced, although it leaves
user’s input events and terminates dialogs. This meta model the possibility of extending it or implementing a new one.
is very basic, as in any AAL environments UI is not expected The visual results of the implementation produce an elegant
to be very complex due to users’ limitations. The IO RDF is and simple graphical output which can run seamlessly on dif-
based on XForms [10], and like it, it is susceptible of being ferent environments and operating systems and with different
extended. screen resolutions.
UniversAAL adds another component to the IO buses and V. C ONCLUSIONS AND FUTURE WORK
the IO RDF: the Dialog Manager (DM). The DM is a dialog This paper shows how to apply hierarchical MVC patterns
orchestrator which coordinates all output and input events to the design of AAL user interaction renderers. The proposed
coming from and to applications, playing the role of the architecture and the example of implementation provided for
CMU at system wise level. This coordination involves multi the universAAL platform prove that the use of this pattern
user coordination, multi modality synchronization, priority simplifies the effort in developing UI renderers.
management, impairment mapping, and many other important The Swing Handler has been compared with an already
administration of the whole UI. existing handler of the platform. Thanks to our design, locating
Lastly but not least important, the UI renderers, here called functionalities of each IO RDF resource representation was
UI handlers, cope with human-machine interaction and map- much easier on our implementation than the existing one.
ping to and from the IO RDF (meta model). Moreover, the Swing Handler allows the redefinition of the
1) The Swing Handler: Within this framework, we have IO RDF, extending it without hardly the need to change the
developed a UI renderer based on the well-established Java central logic of the handler. The adoption of Swing templates
Swing library [5] as graphical rendering engine, namely the allows also the capability of changing the look and feel, even
Swing Handler, to prove the applicability of the HMVC in at runtime.
UI renderers for an AAL environment. This Swing Handler Nevertheless the implementation of a single class for each
follows the design proposed in section IV-A, but specially element in the meta model was tedious work, and for large
adapted to univesAAL unique features. meta models it would require a bigger development team.
UniversAAL provides two elements to receive and publish The duality of view component and L&F template, makes that
output and input events respectively, these are the output the development of new templates is equally human resource
subscriber and the input publisher, both form the meta model demanding. But usually already created L&F packages will
component as their role is to interact directly with the IO serve as reference and source templates to develop new l&F
buses. Elements, or Form Controls as IO RDF defines them, packages, which eases the procedure to create new L&Fs.
have their view following a HMVC paradigm. In the case of As future improvements we propose the development of
submit form controls have a main role in the control part of plugins for the Swing handler. Including L&F packages spe-
the handler’s MVC. They are connected to the input publisher cialized for specific hardware like hand held devices or TV

6. displays. Other interesting L&F packages are AAL impairment Communication and Interaction, Human-Computer Interaction Series,
specific like short sighted or cognitive deprived users. As far pages 114–131. Springer London, 2008. 10.1007/978-1-84628-927-9_7.
[2] Laura Burzagli, Pier Luigi Emiliani, and Francesco Gabbanini. Design
as L&F template enables there should be a specific template for all in action: An example of analysis and implementation. Expert
per hardware and impairment, so we propose the research on Systems with Applications, 36(2, Part 1):985 – 994, 2009.
how to combine L&F packages. This will enable to easily [3] Annette De Vito Dabbs, Brad A. Myers, Kenneth R. Mc Curry, Jacque-
line Dunbar-Jacob, Robert P. Hawkins, Alex Begey, and Mary Amanda
develop a specific handler, for example a new L&F template Dew. User-centered design and interactive health technologies for
could be created from combining existing templates for hand patients. CIN: Computers, Informatics, Nursing, 27, May/June 2009.
held device and mild sight impairment, in stead of creating [4] P. L. Emiliani and C. Stephanidis. Universal access to ambient
intelligence environments: opportunities and challenges for people with
the same handler from scratch. disabilities. IBM Syst. J., 44:605–619, August 2005.
Finally, another future improvements involves the other type [5] Amy Fowler. A swing architecture overview. http://java.sun.com/
of plugin, the meta model extension. We propose the creation products/jfc/tsc/articles/architecture/.
[6] Sonja Müller, Ingo Meyer, Ilse Bierhoff, Sarah Delaney, Andrew Six-
of GUI modal specific features, like for example a graph and smith, and Sandra Sproll. Iterative User Involvement in Ambient Assisted
chart element which will draw a graphical representation of Living Research and Development Processes: Does It Really Make a
signal data or any type of numerical data series. Difference? 2011.
[7] AALuis project. Facilitate the connection of different services to
ACKNOWLEDGEMENTS different types of user interfaces and thus enable the future users of
ambient assisted living systems to use more services interacting in their
We would like to thank the whole universAAL Consortium preferred way. http://www.aaluis.eu/.
for their valuable contribution for the realization of this work. [8] GoldUI Project. European aal project for adaptive embedded human
interfaces designed for older people. http://www.goldui.eu/.
This work has been partially funded by European Union under [9] W3C RDF Working Group. Resource description framework a standard
the 7th Framework Program through the universAAL (FP7- model for data interchange on the web. http://www.w3.org/RDF/.
247950) research project. [10] W3C The Forms Working Group. Next generation of forms technology
for the world wide web. http://www.w3.org/MarkUp/Forms/.
R EFERENCES [11] universAAL project. Universal open platform and reference specification
for ambient assisted living. http://www.universaal.org/.
[1] Jan O. Borchers. A pattern approach to interaction design. In
Satinder Gill, John Karat, and Jean Vanderdonckt, editors, Cognition,