WallTalk_K.Beard_N.Post

1
WallTalk: Connecting Elderly with Their
(Grand)Children in a Way That Is Suitable for All
Generations Involved
Kayleigh Beard (2521477) and Nathalie Post (2526555)
Faculty of Sciences
VU University Amsterdam
k.l.beard@student.vu.nl, n3.post@student.vu.nl
July 1, 2015
!
Abstract—A high quality relationship between elderly and their
(grand)children can be beneficial for all generations involved. However, a
communication barrier can exist between the younger and older genera-
tions, because generally each generation prefers different communication
methods. This can negatively influence the grandparent-(grand)child
relationship. To address this problem, WallTalk is developed: a system
that aims to connect elderly with their (grand)children, in a way that is
suitable for all generations involved. WallTalk projects (grand)children’s
Facebook content on the wall of elderly. The Facebook content of each
(grand)child is projected around a physical photo frame containing a
picture of that (grand)child. Elderly can ‘like’ and ‘comment’ on this
Facebook content in an intuitive manner: by smiling and speaking. It
was evaluated whether WallTalk is a suitable system for both grand-
children and grandparents. A questionnaire was conducted among 254
grandchildren, and a field experiment with semi-structured interviews
was carried out among 5 grandparents. The sampling methods used
were non probabilistic (convenience sampling), therefore the results of
this study are not generalizable to a larger population. The findings
of this research indicate that the developed prototype of WallTalk is
not completely suitable for grandchildren and grandparents. For both
groups, suggestions are made to enhance the suitability of WallTalk in
future research.
1 Introduction
Modern communication technologies could enhance the con-
nectedness between elderly and their (grand)children [1]. Yet,
communication preferences of younger generations mostly
differ from the preferences of elderly people [2]. According to
Lindley et al. (2012) [1], younger generations regularly use
modern communication methods, such as text-messaging and
social media platforms. However, the design of these modern
technologies is mostly focused on younger generations, and
often involves asynchronous (‘lightweight’) communication,
such as posting Facebook messages. In general, elderly have
more affinity towards traditional synchronous (‘heavyweight’)
communication, for instance calling and face-to-face contact.
Older generations often experience difficulties when adapting
to new technologies, or are simply not interested in using
them. This can create a communication barrier between
grandparents and their children and grandchildren. Therefore,
the research goal of this paper is to develop and evaluate a sys-
tem that connects grandparents with their (grand)children, in
a way that is suitable for all generations involved.
Previous research [3][4] has emphasized the importance
of the grandparent-(grand)child relationship. High quality
relationships between these generations can be beneficial
for the health and well-being of both parties. Several stud-
ies [5][6][7] have focused on developing communication sys-
tems to enhance the relationship between grandparents and
their (grand)children. However, these systems establish one
communication method for all users, and are therefore either
suitable for elderly or for (grand)children, but rarely for all
generations involved. Text-messaging, posting a status update
on Facebook, and tweeting on Twitter, are all communi-
cation methods that are suitable for the busy lifestyles of
younger generations. On the other hand, elderly often have
less busy lifestyles; therefore, communication methods such
as calling and face-to-face contact are more suitable for their
generation. This is the main challenge of developing a sys-
tem for grandparent-grandchild communication: addressing
the different needs of all generations involved. Therefore,
‘WallTalk’ is proposed to improve upon current grandparent-
(grand)child communication technologies, and meet the needs
of both elderly and their (grand)children.
WallTalk is a system that projects (grand)children’s Face-
book content on the wall of elderly (Figure 1). The Facebook
content of each (grand)child is projected around a physical
photo frame that contains a picture of that (grand)child;
adding value to the familiarity of physical photo frames.
Elderly can ‘like’ and ‘comment’ on this Facebook content in
an intuitive manner: by smiling and speaking. Choices for the
development of WallTalk have been derived from literature
considering similar areas of study. In this paper, WallTalk is
described, and evaluation studies are conducted to determine
whether WallTalk is a suitable communication system for
both elderly and their grandchildren.

Figure 1. An example of the WallTalk projection.
2 Related work
In the field of Human-Computer Interaction, various systems
have been developed that aim at establishing a connection
between elderly and their children and grandchildren. When
people are separated by distance, communication often takes
place using traditional communication methods, such as call-
ing. These communication methods are heavyweight, mean-
ing that communication is likely to occur less frequent, but
when it does occur, it is focused and prolonged. In con-
trast, lightweight communication is dispersed over time and
sometimes asynchronous, such as text-messaging, updating a
status on Facebook, or tweeting on Twitter [1].
A number of communication systems have been developed
that focus on establishing lightweight communication. These
lightweight communication systems can serve as an addition
to traditional heavyweight communication; contributing to
the connectedness between people who are separated by dis-
tance. This is the type of communication WallTalk aims to
establish as well, by including elderly in the social media
activities of their offspring. A selection of these lightweight
systems, focused on connecting elderly with their offspring,
will be discussed and evaluated.
2.1 The FamilyPlanter
Most people feel a strong need and desire to stay connected
with each other when they become separated by distance [8].
When people are separated by distance, situational cues
are one of the aspects of communication that are naturally
lost. These are cues such as the presence of others, hearing
someone’s footsteps, and observing the brightness of the
room. Situational cues provide information that serves as
a background-context in face-to-face communication. The
‘FamilyPlanter’ was developed to offer comfort and a feeling of
togetherness to people living apart from each other, helping
them to maintain their relationships [5]. The FamilyPlanter
is a terminal in the form of a plant, which facilitates the
exchange of data about people’s unconscious presence and
motion cues. The FamilyPlanter was not developed specifi-
cally to connect elderly with their distanced offspring, such as
WallTalk, but for all kinds of family relationships.
The information that is shared through the FamilyPlanter
differs from WallTalk for multiple reasons. First of all, the
information that is shared through the FamilyPlanter is am-
biguous and left up to the interpretation of the users them-
selves. The FamilyPlanter only provides sensor information
about the presence of people in their homes. This limits the
scope of the shared information, because the FamilyPlanter
only shares ‘presence’ of people, but not specifically which
people are present. Information that is shared using WallTalk
is less ambiguous, because the information is expressed in the
form of messages and photos.
Elderly users of the FamilyPlanter had more positive atti-
tudes towards the system than their children and grandchil-
dren. The elderly users stated in personal interviews that they
took comfort out of the gleaming of the FamilyPlanter, and
that it evoked thoughts about their families. These findings
suggest that elderly would potentially benefit more from the
feeling of closeness to their families elicited by the Family-
Planter, than their children and grandchildren. It is difficult
to generalize the findings from this research to a broader
population, since the FamilyPlanter was only installed in a
small number of households. However, the findings from this
research are in agreement with research pertaining to the
existing asymmetry in the relationship between grandparents
and their offspring. The relationship between grandparents
and their (grand)children seems more focused on the activ-
ities and accomplishments of the (grand)children, than vice
versa [1]. Additionally, according to Mynatt et al. (2001) [9],
elderly are often interested in what their offspring are doing
and want to ‘monitor’ this. In general, the lifestyle that comes
with seniority allows elderly more time for these monitoring
behaviors. Therefore, it is not surprising that the elderly in
this study responded more positively, and expressed more
interest in the FamilyPlanter, than their offspring.
2.2 The Digital Family Portrait
In the ‘Digital Family Portrait’, a digital photo frame was
used as a communication system to connect elderly with their
(grand)children [9]. The photo frame was chosen as medium,
because of elderly’s familiarity with it as a household object.
This idea has served as inspiration for WallTalk. Even though
many elderly have a positive attitude towards technology, they
often experience difficulties adapting to new technologies [10].
Therefore, to avoid this problem, WallTalk only adds visible
technology to existing household objects (photos in their
frames). While the Digital Family Portrait requires replace-
ment of existing photo frames with digital ones, WallTalk
preserves current photo frames, and even the placement of
the photo frames can be maintained.
Even though the Digital Family Portrait was positively
welcomed by the elderly and their offspring participating in
the study, there were some problems with the design of the
Digital Family Portrait that should be improved upon. The
photo frame displayed a history of a person’s activities, but it
turned out to be difficult to read off the photo frame when
activities had taken place. This caused erroneous interpre-
tations among the users. To avoid erroneous interpretations,
it is important that the communicated information is unam-
biguous. In contrast to the Digital Family Portrait, WallTalk
only displays information from the (grand)children to the
grandparents, and not vice versa. This is because WallTalk
has a different goal than the Digital Family Portrait. The
Digital Family Portrait focuses on providing (grand)children
2

with the ability to monitor the elderly, while WallTalk aims
at providing elderly inclusion in their offspring’s online lives.
When displaying information to elderly, it is even more impor-
tant that the information is not confusing or ambiguous. The
results from this research emphasize the importance of using
simple, unambiguous information, and limiting the amount of
information that is displayed. Therefore, WallTalk does not
provide elderly with a ‘history of information’, but instead,
will only display the most recent information.
2.3 The Family Window: a domestic media space
The previously discussed related works focused on creating
a feeling of connectedness by using abstracted information.
The ‘Family Window’ takes a completely opposite approach.
Judge et al. (2010) [11] argue that abstracted information does
not provide the same feeling of connectedness that is elicited
when seeing each other. Therefore, a video media space, the
Family Window, was developed to establish connectedness
between family members living apart. In the Family Window,
a continuous video connection was established between two
homes, allowing family members to monitor their distanced
family members continuously. Even though family members
had the option to turn the Family Window off, the participat-
ing families in a field experiment had many concerns about
their privacy. Sometimes family members felt uncomfortable
when watching the Family Window, while other times they
felt uncomfortable being watched.
An important lesson can be learned from the Family
Window: media spaces are not the type of systems that most
people feel comfortable with in their homes [12]. People are
likely to feel more comfortable sharing abstracted information
than sharing a continuous video connection. Even though
the researchers argue that abstracted information does not
provide a great feeling of connectedness, systems such as the
Family Window can lead to other dissatisfactions. WallTalk
aims at finding the sweet spot for the type of information
that is shared. The information should be less abstract than
systems such as the FamilyPlanter, but more abstract than
the Family Window. Therefore, extracting information from
Facebook seems a suitable way to acquire information with
the right amount of abstraction. Messages and photo’s on
Facebook are explicitly posted; the users know exactly which
information they are sharing. This prevents the type of pri-
vacy infringement that is more likely to occur with systems
such as the Family Window.
2.4 ePortrait
The ‘ePortrait’ is a similar system to WallTalk, because the
ePortrait also includes elderly in the online social network
activities of their (grand)children [13]. Social networks can
enhance the quality of life for elderly [14]. However, elderly
are a less represented group on online social networks than
younger generations [15]. An important barrier for elderly in
maintaining strong emotional ties with their distanced family,
is that elderly are often disassociated with events related
to their distanced family’s everyday lives [13]. Part of the
family’s everyday lives is shared on online social networks,
and not all elderly feel comfortable enough with technology
in order to engage in those [16]. Therefore, the ePortrait was
developed to include elderly in the social network activities
of their family, without having to face too many technological
barriers. The ePortrait is a digital photo frame, which displays
photographs from elderly’s (grand)children that are published
on Facebook.
A field experiment, in which the ePortrait was installed in
the homes of elderly, highlighted the importance of a feedback
possibility for the elderly. The elderly using the ePortrait
enjoyed being included in the social media activity of their
offspring, but there was no possibility for them to respond
to the pictures they were able to see on the ePortrait. This
highlights the importance for WallTalk to establish bidirec-
tional communication between the elderly and their offspring.
Even though the ePortrait includes elderly in the online social
media lives of their offspring, the ePortrait also excludes
elderly, because they do not have the opportunity to respond
to the photos. In order to foster bidirectional communication
and connectedness between elderly and their offspring, elderly
should have the possibility to provide feedback in a way that
is suitable for their generation. Therefore, WallTalk aims to
facilitate communication that is suitable for elderly: without
complicated user interfaces.
2.5 Building Bridges
Physical limitations are one reason why elderly stay at home
and avoid social interaction [17]. Moreover, older adults are
often not experienced with new technologies, such as smart
phones, to use as mediums for social interaction. ‘Building
Bridges’ is a touch screen device with communication software
especially developed for this target group to support social
connectedness [6]. The developers of Building Bridges believe
that a touch screen interface is more intuitive than a mouse
or keyboard, and thus easier for elderly to use. In contrast to
the previous discussed examples, this type of communication
establishes heavyweight communication, while WallTalk aims
to establish lightweight communication. Moreover, Jin et al.
(2007) [18] state that current research is mostly focused on
designing touch screen interfaces for younger generations, and
those interfaces are not optimal for elderly to use. Thus,
when focusing on developing innovative technologies for older
adults, we believe that these technologies can be more effort-
less than the touch screen device of Building Bridges. There-
fore, a touch screen interface is not integrated in WallTalk and
other technologies are explored to ensure effortless usability.
Furthermore, the developers of the Building Bridges touch
screen device believe that such a device hides the complex
technology behind the system. This vision overlaps with what
we aim to achieve with WallTalk, since older adults often
try to avoid new technologies due to computer anxiety [19].
Nonetheless, the presence of a digital screen is still an im-
plication of technology, while older adults are more attached
to physical non-technological objects [20]. Therefore, when
visualizing social media of the children and grandchildren of
elderly people, WallTalk takes one step further to hide the
integrated technology and to make use of familiar physical
non-technological objects.
2.6 Peek-A-Drawer: communication by furniture
The touch screen device from the Building Bridges project
clearly plays a technological role in the house of elderly. In con-
trast, ‘Peek-A-Drawer’ is a lightweight communication system
3

that makes use of a familiar household object [7]. Cameras are
placed in the drawers of the grandchildren’s cabinet, while the
bottom surfaces of the grandparents’ drawers contain digital
screens. When a child puts his or her items in the drawer,
a picture is taken which is displayed on the digital screen
in the grandparents drawer. Similar to WallTalk, this is a
simple communication method that enhances the feeling of
connectedness between grandparents and their grandchildren.
Forgani et al. (2014) [21] states that “grandparents like
to know nearly everything about their grandchildren”. They
found that grandparents sometimes feel self-conscious and
afraid to annoy their grandchildren by interfering too much
in their daily lives. Thus, Peek-A-Drawer can be a satisfying
solution for grandparents, since they do not have to disturb
the grandchildren, yet are still in contact with them. This is
an inspirational aspect for the development of WallTalk. The
pictures that are taken in the Peek-A-Drawer only provide
a small amount of information about the grandchildren’s life,
and no details such as school grades or outdoor activities. This
is what WallTalk aims to improve upon: establishing effortless
connectedness while providing more detailed communication.
Miyajima et al. (2005) [5] state that older adults have
enough time and interest to be actively involved in the com-
munication with their family members. Thus, when elderly
use Peek-A-Drawer, they might be frustrated, because they
can only receive information, and not interact or react on it.
Therefore, WallTalk aims for a bidirectional communication
flow, which is still effortless, to meet the needs of the older
adults.
2.7 Summary and Implications
The systems discussed in this section all have aspects that
can be learned from and improved upon. When develop-
ing communication technologies that connect elderly with
their (grand)children, such as WallTalk, the following aspects
should be taken into consideration.
First, the design of communication technologies should
be unambiguous and simple, and it is important to find
the right amount of information abstraction. Second, tech-
nology should be ‘hidden’ in the lives of elderly, preferably
by blending in with familiar physical objects, such as photo
frames. Third, elderly prefer communication technologies that
allow for bidirectional communication. Even though elderly
like to ‘monitor’ their offspring, they do not feel comfortable
using systems that enhance the feeling of disturbing their
children and grandchildren. At last, the privacy of both elderly
and their (grand)children should be assured. Facebook is a
suitable medium to extract privacy assured information from
children and grandchildren.
3 Contribution
There is currently no system that connects generations in a
lightweight manner while suitable for all its users, as con-
cluded from the studied related works. WallTalk aims to
establish this connection in order to enhance the health and
well being of both elderly and their (grand)children. This can
be referred to as the social relevance of WallTalk. Besides,
according to the discussed related work, no system exists that
combines the techniques of smile recognition, speech recogni-
tion and visualization of social media content into one system
that aims to achieve this goal. This combination of different
Artificial Intelligence techniques is our practical contribution
to the field of developing communication technologies. Most
importantly, by evaluating the usability and suitability of
WallTalk, we contribute to the academic field of Human-
Computer Interaction studies focused on developing commu-
nication technologies for elderly and their (grand)children.
4 System Description
A schematic synopsis of the usage of WallTalk is displayed in
Figure 2. The goal of WallTalk is to establish a connection
between elderly and their (grand)children, in a way that is
suitable for all generations involved. We aim to achieve this
connection by providing elderly with an easy and intuitive
communication method, while making use of familiar (physi-
cal) objects.
WallTalk is a system that projects (grand)children’s Face-
book content on the wall of elderly. The Facebook content
of each (grand)child is projected around a physical photo
frame that contains a picture of that (grand)child. Elderly
can ‘like’ and ‘comment’ on the latest Facebook status update
in an intuitive manner: by smiling and speaking respectively.
It should be noted, that grandparents can only comment
on the most recent posted status update of one of their
(grand)children. Even though this limits the communication
possibilities, it enhances the ease of use for elderly. This is
done purposely, to ensure that elderly do not have to make
use of complex user interfaces.
A technical system description of WallTalk can be found
in Appendix A, and the source code is available on GitHub 1
.
The flow of the system can be summarized as follows.
Figure 2. Schematic overview of the usage of WallTalk.
1) WallTalk extracts information from (grand)children via
a medium that is currently widely used among the generations
of the children and grandchildren of elderly: Facebook. This
information can appear in the form of written messages and
pictures. Section 4.2 goes further into detail about the rea-
soning behind the choice for this particular data source. The
communication from (grand)children to their grandparents
will be explained in Section 4.1.1.
1. Source code: https://github.com/postnathalie/WallTalk
4

2) Parsed by the system, the messages and/or pictures
are projected on the wall of the elderly. The projections of
the Facebook content are displayed around physical photo
frames of (grand)children on the wall. The visualization of
this content, together with WallTalk’s physical appearance in
the homes of elderly, will be discussed in Section 4.3.
3) Once the Facebook content is displayed on the grand-
parents’ wall, grandparents can communicate back to their
(grand)children. They can do this in two ways, namely by
‘liking’ and ‘commenting’ (in Facebook terminology). To ‘like’
certain content, grandparents simply have to smile in front of
a camera. To ‘comment’, grandparents need to hold an object
of choice and speak. In the prototype of WallTalk, a statuette
(that the grandparents already have in their homes) is con-
nected to WallTalk and used to initiate and stop recording
the grandparents’ comments (explained in detail in A.6). The
grandparent-(grand)child communication flow will be further
discussed in Section 4.1.2.
4.1 Communication flow and interaction
WallTalk is a bidirectional communication system, which
means that there are two communication flows: from
(grand)child to grandparent, and from grandparent to
(grand)child. Several studies [13][5] have highlighted the im-
portance of bidirectional communication for grandparent-
(grand)child communication technologies. However, not only
is it important that communication technologies facilitate
bidirectional communication, it is even more important
that both parties feel comfortable using the communica-
tion technology. In general, (grand)children and grandparents
have different preferences for communication technologies.
(Grand)children often use lightweight communication tech-
nologies, such as online social networks and text-messaging.
Grandparents are not always familiar or comfortable with
these communication technologies. They often prefer using so-
called heavyweight communication methods, such as calling
or letter writing, which are more focused and prolonged [1].
WallTalk addresses these different preferences of communica-
tion methods, by aiming to facilitate communication that is
tailored to the preferences of all generations involved.
4.1.1 (Grand)child to grandparent communication flow
The communication-flow from (grand)child to grandparent
comprises of messages and photos that are posted by the
(grand)children on Facebook. WallTalk facilitates sharing
these messages with the grandparents. The choice for this
communication-flow is based on two important character-
istics pertaining to the (grand)child in the (grand)child-
grandparent relationship, namely (1) the asymmetry in the
grandparent-grandchild relationship, and (2) the amount of
free time and effort.
1) The asymmetry of the grandparent-(grand)child rela-
tionship has been emphasized in several studies [22][1]. Often,
(grand)children are the focus of their grandparents’ affection.
This is observable in conversation topics in grandparent-
grandchild communication: conversations are often focused on
the (grand)children [23]. Therefore, WallTalk aims to incorpo-
rate the existing asymmetry of the (grand)child-grandparent
relationship, by facilitating communication that is focused on
the (grand)children.
2) The amount of ‘free time’ grandparents and their
(grand)children have is one of the main challenges of
grandparent-(grand)child communication. In contrast to el-
derly, younger generations often have busier lifestyles that in-
clude school, study, work and other social activities [5]. When
a system like WallTalk requires extra time and effort from
(grand)children, situations may arise where the (grand)chil-
dren forget to use the system due to their full agendas. This
can cause elderly to lose interest in using WallTalk. Therefore,
WallTalk should not require (grand)children to invest extra
time and effort. This implies that WallTalk should utilize
an information source from (grand)children that is already
integrated in their daily lives. Since a large proportion of these
younger generations is active on online social media platforms,
it is ideal to use social media as a communication source
(further explained in Section 4.2). This way, WallTalk aims
to include grandparents in a way that is suitable and effortless
for their (grand)children.
4.1.2 Grandparent to (grand)child communication flow
Grandparents have the opportunity to respond to the
messages and photos of their (grand)children, by ‘liking’
and ‘commenting’. Two important characteristics of the
grandparent are the rationale behind the choice for this
communication-flow, namely (1) ability to adapt to new
technologies, and (2) the asymmetry of the grandparent-
grandchild relationship.
1) Elderly are not always familiar with new technologies,
and can encounter great difficulties adapting to those [24].
They are by far the least represented age group that are active
on Facebook [13]. Therefore, WallTalk aims to include grand-
parents in the Facebook activities of their offspring, without
having to overcome technological barriers. In WallTalk, grand-
parents do not need a mouse or keyboard to ‘like’ and ‘com-
ment’ on their (grand)children’s Facebook content (as they
would be required to when accessing Facebook from a com-
puter). Instead, grandparents can express their affection and
interest the same way as they would do in face-to-face contact:
by smiling and talking. This method of communication seems
to comply with the needs of the older generation. However,
simplifying the communication also means that the possibil-
ities of communication become more limited. Grandparents
can therefore only ‘like’ or ‘comment’ on the most recently
posted message or photo of one of their (grand)children.
2) The communication flow from grandparent to
(grand)child is in agreement with the asymmetrical shape
of the grandparent-(grand)child relationship, as explained
previously in Section 4.1.1. (Grand)children’s messages and
activities are the focus of the communication, and grandpar-
ents can express their affection (by ‘liking’) or interest (by
‘commenting’).
As noted before, WallTalk is a lightweight communication
technology, while elderly often prefer more heavyweight com-
munication (such as calling). However, as Lindley (2012) [1]
states, “It seems that there may be an opportunity to per-
suade older adults to adopt lightweight technologies on the
grounds that they are more likely to connect them with their
(grand)children, and because they are more suitable for the
busy lifestyles of their offspring.” This is exactly the aim of
the communication flow from grandparent to (grand)child in
WallTalk.
5

4.2 Data Source
The social media platform Facebook is used as data source
for the (grand)child to grandparent communication flow in
WallTalk. The rationale behind this choice is described in this
section.
4.2.1 Choosing a suitable type of online social media
There are several different types of social media platforms,
of which an overview is displayed in Figure 3. Social media
platforms can be classified based on two key theoretical el-
ements from social media research [25]. The first element
is social presence (or media richness) and is defined as the
intimacy and immediacy of the medium. To establish a feeling
of connectedness, a reasonable amount of social presence is
needed. Thus, a type of social media with an intermediate
social presence and media richness would be most suitable
for WallTalk. The second element is self-presentation (or self-
disclosure), which is characterized by the proportion of per-
sonal information in social media content. Since grandparents
are often interested in their offspring’s lives [9], the social
media source should have a high level of self-presentation.
However, it is important that the (grand)children’s privacy
is not being invaded. In conclusion, WallTalk should extract
social media content from (grand)children with intermediate
social presence and high self-presentation characteristics. This
type of social media is classified as ‘social networking sites’.
Figure 3. Classification scheme of social media categorized by social
presence / media richness and self-presentation / disclosure. [25]
4.2.2 Facebook as most suitable social networking site
For the prototype version of WallTalk one social networking
site is used as a data source: Facebook. After considering
the landscape of popular social networking platforms, such as
Twitter, Facebook, Pinterest, Instagram and Tumblr, Face-
book seemed the most suitable data source for WallTalk. Four
reasons for the selection of Facebook as data source are listed
below.
1) Facebook is currently the most widely used social
media platform [26][27]. This increases the chance that for
each grandparent information can be extracted from all
(grand)children. However, McAndrew et al. (2012) [28] state
that “age is negatively related to the frequency of most
Facebook activities”. This might result in extracting more
(continuous) information from younger offspring and less from
the middle-aged.
2) The type of media people can share on Facebook is
diverse, including text, pictures, videos and web pages. This
diversity makes the information posted by (grand)children
interesting for grandparents. Most other platforms tend to
focus on one specific type of media. For example, Instagram is
centered around pictures.
3) The advantage of Facebook is that it gives the
(grand)children the opportunity to choose whether or not to
share the information with their grandparents. This ensures
that their privacy is not being invaded. However, it requires
slightly more effort from the (grand)children to prevent their
grandparents from viewing their content, which opposes the
goal of WallTalk.
4) Facebook’s media content is typically unambiguous and
understandable outside the scope of the platform. This can
make the (grand)children’s information understandable and
relevant for grandparents. Contrastingly, on the social net-
working site Twitter, content is posted using ‘Twitter syntax’;
containing abbreviations and symbols such as ‘hashtags’. This
syntax makes it difficult to understand the posted content
outside the scope of Twitter.
4.3 Physical setup and appearance
WallTalk is a system designed to be placed in the homes of
elderly people. For the purpose of this research, a prototype
version of WallTalk was developed.
For the final version of WallTalk, the appearance of the
system should be completely different: hiding all technolog-
ical components, in order to comply with the research goal.
However, only a prototype was developed, because further
development is beyond the scope of this research.
For the prototype version of WallTalk, the following com-
ponents are used, as illustrated in Figure 4:
• Statuette
• MakeyMakey microcomputer, for transferring a touch
signal [29]
• Microphone for recording spoken messages
• Webcam for smile detection
• Laptop with WallTalk web application connecting all
components and signals
• Projector for displaying WallTalk’s social media visu-
alization
Figure 4. An illustration of the physical setup of WallTalk’s prototype
version.
For the final version of WallTalk, the following choices for
the physical setup are made:
• Small box containing a:
– small short-throw projector
– camera
– microphone
– microcomputer, such as a Raspberry PI
• Wireless touch-sensor object of choice, for instance a
statuette
An illustration of the final physical setup can be found in
Figure 5.
6

Figure 5. An illustration of the physical setup of WallTalk’s final version.
Furthermore, the following items are needed in the homes
of the elderly:
• Table or similar furniture, positioned close to the wall,
to place the WallTalk box on
• Photo frames containing pictures of (grand)children on
the projected wall
• Power source close to WallTalk
In the final version of WallTalk, a small box is used to
‘hide’ the technology inside of WallTalk, which is placed on a
table or similar furniture close to the wall. This is important,
because introducing new technologies to elderly can cause
computer anxiety [19]. On the other hand, the presence and
interaction with physical objects enhances a sense of comfort
and control over the environment for elderly [30]. Therefore,
we believe that the action of holding a statuette in order to
comment on social media content is more suitable for elderly
than pressing a button, key or mouse device. Nonetheless,
when (grand)children install WallTalk in the home of their
(grand)parents, some technology, such as a keyboard and
mouse device, is temporarily necessary.
The advantage of a projector compared to other tech-
nologies, such as digital screens, is that the projector can be
combined with the (grand)parents’ familiar physical pictures
of their offspring. However, projectors are relatively expensive
and the lamp burns out after several years. Moreover, the
projection is less visible with bright daylight. The projector
needs to be a ‘short-throw’ projector, meaning it should be
placed very close to the wall it projects onto [31]. This way,
the projection will not be disturbed by shadow interference
while the grandparents are interacting with WallTalk.
An example visualization of a grandchild’s Facebook con-
tent is displayed in Figure 6. The physical photo frame on the
wall is surrounded by a ‘text cloud’ containing a Facebook
message. The photo frame of the (grand)child who has most
recently posted a picture or message lights up and blinks
slowly.
Elderly using this system can only ‘comment’ on, and
‘like’ the content of the (grand)child who posted the latest
status update. When the grandparent is ‘commenting’ or
‘liking’, visual feedback is displayed inside the photo frame
of the (grand)child with the latest Facebook status update
(Figure 7).
5 System Evaluation: Grandchildren
The overall goal of this evaluation was to determine whether
WallTalk is a suitable system for grandchildren. Therefore,
the following topics were evaluated through a questionnaire:
• The suitability of Facebook as a data source for
WallTalk
Figure 6. An example of a WallTalk projection.
Figure 7. Icon that is displayed while ‘liking’ (left), and icon that is
displayed while ‘commenting’. (right)
• Grandchildren’s willingness to share their Facebook
content with their grandparents
• The closeness of the relationship between grandchil-
dren and their grandparents
• The type of communication methods grandchildren
use
• Whether grandchildren are satisfied about the commu-
nication methods they use to communicate with their
grandparents
• Grandchildren’s opinion about WallTalk
5.1 Methods
5.1.1 Questionnaire
The questionnaire was comprised of two parts, and contained
28 questions in total (see Appendix B). The first part of
the questionnaire focused on evaluating the suitability of
Facebook as a data source for WallTalk, and was conducted
among all respondents. The second part of the questionnaire
was conducted among respondents with at least one living
grandparent (which will be referred to as ‘grandchildren’ in
the remainder of the questionnaire). For the second part of
the questionnaire, grandchildren were requested to answer the
questions about one of their grandparents (the grandparent
they see most often).
5.1.2 Population Sample
Respondents were recruited by convenience sampling through
Facebook student groups. In total, 254 respondents aged
between 18 and 35 filled in the questionnaire. All respondents
were university students from The Netherlands. From these
254 respondents, 198 had at least one grandparent who was
still alive, and 56 respondents did not have any living grand-
parents.
5.2 Results
5.2.1 Suitability of Facebook as data source for WallTalk
All 254 respondents were active on social media and Facebook.
The large majority of the respondents (87.8%) expressed
7

that Facebook was their most frequently used social media
platform, followed by Instagram (10.2%), Twitter (1.57%) and
WhatsApp (0.34%).
5.88% of the respondents never posts anything on Face-
book, 6.67% posts daily on Facebook, 12.16% posts weekly
on Facebook, 51.37% posts monthly on Facebook and 23.53%
posts yearly on Facebook.
From the respondents that post messages on Facebook,
the majority posts photo’s (79.5%) followed by messages
(40.9%). An overview of the posted content by respondents
on Facebook is displayed in Figure 8.
Figure 8. Bar chart representing the types of content that are posted on
Facebook by respondents.
5.2.2 Grandchildren’s willingness to share their Facebook con-
tent with their grandparents
25.8% of the grandchildren expressed that they are friends
with at least one of their grandparents on Facebook. They
were asked to rate on a 5-point Likert Scale how they felt
about their family members and grandparent(s) viewing their
Facebook content. On average, grandchildren expressed feel-
ing ‘neutral’ towards their family members viewing their
Facebook content. Likewise, grandchildren expressed feeling
‘neutral’ towards their grandparents viewing their Facebook
content.
Grandchildren expressed what type of messages they
would rather not share with their grandparent on Facebook.
37.9% of the grandchildren would share all the messages they
post on Facebook with their grandparent. 49.5% of the grand-
children preferred not to share messages about alcohol / drugs
with their grandparent, followed by messages about parties
(42.4%). A complete overview of message topics grandchildren
prefer not to share with their grandparent is displayed in
Figure 9.
5.2.3 Closeness of the relationship between grandchild and
grandparent
Grandchildren were asked to rate the closeness of the rela-
tionship with their grandparent on a 5-point Likert Scale.
Figure 9. Bar chart representing message topics that grandchildren prefer
not to share with their grandparent. Grandchildren could choose multiple
options.
On average, the relationship was rated between ‘neutral’ and
‘close’. Furthermore, grandchildren were asked how often they
had contact with their grandparent. Correlation between per-
ceived closeness of the relationship and frequency of contact
between grandchildren and their grandparent is displayed in
Figure 10.
Grandchildren were asked whether a lack of time caused
them to have less contact with their grandparent than they de-
sired. Almost half of the grandchildren (40.4%) either ‘agree’
or ‘completely agree’, that they have less contact than desired.
Figure 10. Bar chart representing the correlation between perceived
closeness of the relationship and frequency of contact between grand-
children and their grandparent.
8

5.2.4 Communication methods grandchildren use
Grandchildren were asked which communication methods
they use to communicate with their friends, and which com-
munication methods they use to communicate with their
grandparent. The large majority (97.47%) of grandchildren
expressed using WhatsApp to communicate with their friends.
Face-to-face communication was the most frequently used
communication method among grandchildren to communicate
with their grandparent (78.79%). An overview of grandchil-
dren’s communication method usage for communication with
friends and their grandparent is displayed in Figure 11.
Figure 11. Bar chart representing a comparison of the communica-
tion methods grandchildren use to communicate with friends, and the
communication methods grandchildren use to communicate with their
grandparent. Grandchildren were allowed to choose multiple options.
5.2.5 Grandchildren’s satisfaction about the communication
methods they use to communicate with their grandparent
Grandchildren were asked to rate (on a 5-point Likert Scale)
their satisfaction with the communication methods they use
to communicate with their grandparent. The majority of
the grandchildren expressed being ‘satisﬁed’ with the used
communication methods (51.0%) and ‘completely satisﬁed’
(24.2%). However, when asked which communication methods
they would like to use additionally to communicate with their
grandparent, 57.1% of the grandchildren expressed that they
would like to use one or more additional communication meth-
ods. An overview of the desired additional communication
methods is displayed in Figure 12.
Grandchildren’s opinion about WallTalk
The concept of WallTalk was explained to grandchildren, and
the opinion of grandchildren about several aspects of WallTalk
was evaluated.
5.2.6 Attitude towards grandparents viewing grandchildren’s
Facebook content
It was evaluated whether grandchildren would feel comfort-
able with their grandparent viewing their Facebook content
through WallTalk. The majority of grandchildren expressed
feeling ‘comfortable’ (28.3%) or ‘very comfortable’ (25.25%)
Figure 12. Bar chart representing the communication methods grand-
children desire to use to communicate with their grandparent
with their grandparent viewing their Facebook content. Ad-
ditionally, it was evaluated whether the frequency of con-
tact grandchildren have with their grandparent correlated
with grandchildren feeling comfortable with their grandparent
viewing their Facebook content (Figure 13). There appears to
be a correlation between the two: grandchildren with daily or
weekly contact with their grandparent feel more comfortable
with their grandparent viewing their Facebook content, com-
pared to grandchildren who have less frequent contact with
their grandparent.
Figure 13. Line chart representing a correlation between grandchildren
feeling comfortable with their grandparent viewing their Facebook con-
tent, and frequency of contact. ‘Feeling comfortable’ is rated on a
5-point Likert Scale (1 = ‘not comfortable at all’, 5 = ‘very comfort-
able’).
Additionally, it was evaluated whether closeness between
grandchildren and their grandparent correlates with grand-
children’s attitude towards their grandparent viewing their
Facebook content (Figure 14). These aspects seem to appear
in a synchronized manner. Grandchildren who responded ‘not
to be close at all’ with their grandparent, on average have
9

more negative attitudes (2.11 on average on a 5-point Likert
Scale) towards their grandparents viewing their Facebook
content. On the other hand, grandchildren that described
their bond with their grandparents as ‘very close’, expressed
feeling ‘neutral’ (3.18 on a 5-point Likert Scale) towards
grandparents viewing their Facebook content.
Figure 14. Line chart representing a correlation between grandchildren’s
affinity towards their grandparent viewing their Facebook content, and
the closeness of the relationship. ‘Affinity’ is rated on a 5-point Likert
Scale (1= ‘no affinity at all’, 5=‘a lot of affinity’).
5.2.7 Attitude towards reviewing Facebook content
Grandchildren were asked to rate (on a 5-point Likert Scale)
whether they would prefer to review their Facebook content
before the showing the content to their grandparent. Most
grandchildren either ‘agree’ (29.3%) or ‘completely agree’
(26.77%) to review their Facebook before showing it to their
grandparent.
5.2.8 Attitude towards grandparent ‘liking’ and ‘commenting’
on Facebook content
An overview of grandchildren’s attitude towards their grand-
parent ‘liking’ and ‘commenting’ on their Facebook content
is shown in Figure 15. On average, grandchildren show a
more positive attitude towards their grandparents ‘liking’
their Facebook content, than towards their grandparents
‘commenting’ on their Facebook content.
5.2.9 Grandchildren’s comments about WallTalk
Grandchildren were asked to express comments and/or feed-
back about WallTalk. The most important comments per-
taining to (1) hindrance of WallTalk for grandchildren, (2)
usefulness of WallTalk for elderly, and (3) usefulness for other
target groups, are listed below.
1) One grandchild expressed her concern about messages
that would be posted on Facebook by elderly. She stated that
elderly tend to post different types of messages than younger
generations are used to, which might lead to frustrations
among grandchildren.
2) Several grandchildren concerned themselves with the
usefulness of WallTalk for elderly. One participant speculates
that WallTalk might be too abstract for elderly: WallTalk
could be difficult for elderly to understand and use if they
Figure 15. Line chart representing grandchildren’s attitude towards their
grandparent commenting on their Facebook content and ‘liking’ their
Facebook content.
are not familiar with the concept of Facebook. Additionally, it
was suggested that WallTalk would be hard to use for elderly
that have difficulties with talking. Contrarily, one participant
commented that WallTalk might be superfluous for elderly
who are experienced with computer usage.
3) Grandchildren provided comments concerning the suit-
ability of WallTalk for more target groups, instead of only
focusing on elderly. For example, one participant expressed
that he would also be interested in using WallTalk, to project
his Facebook content on his wall, and interact with it using
his facial gestures and speech. Another participant suggested
that WallTalk could also be a beneficial system for mentally
disabled people.
6 System Evaluation: Grandparents
The goal of this evaluation was to determine whether
WallTalk is a suitable system for grandparents. Therefore, the
following topics were evaluated by conducting semi-structured
interviews:
• The current communication between grandparents
and their (grand)children
• Grandparents’ attitude towards technology
• Grandparents’ understanding of Facebook
• Grandparents’ attitude towards Facebook
• Grandparents’ attitude towards WallTalk
• Grandparents’ ability to use WallTalk
6.1 Methods
WallTalk was installed in the homes of the elderly partici-
pants, and semi-structured interviews were conducted. 2
Prior to the installation of WallTalk, the children and
grandchildren of the elderly were contacted and asked to sign
up for WallTalk.
2. Semi-structured interviews are frequently used in Human-
Computer Interaction studies to evaluate real-world technology usage
or future technology needs [32].
10

6.1.1 Semi-structured interview
The semi-structured interview consisted of two parts. The first
part of the interview was conducted prior to the installation of
WallTalk, and was focused on evaluating the grandparents’ at-
titude and understanding of technology and social media. The
second part of the interview was conducted after WallTalk
was installed. The concept of WallTalk was explained to the
elderly. The elderly had the opportunity to interact with the
system, after which they were asked about their opinion about
several aspects of WallTalk. The duration of the evaluation
process ranged from 3 to 5 hours.
6.1.2 Population Sample
Five grandparents (two males, three females) aged between 80
and 85 participated in the study, and were recruited through
convenience sampling. Among these grandparents, there were
two couples and one widow. Each grandparent had between 7
and 10 (grand)children.
6.2 Results
6.2.1 The current communication between grandparents and
their (grand)children
Grandparents were asked about the frequency of contact they
have with their children and grandchildren, and whether
they are satisfied with the amount of contact. All grand-
parents spoke on the phone weekly with at least one of
their (grand)children, and had monthly face-to-face contact
with at least one of their (grand)children. Four grandparents
expressed that they are satisfied with the frequency of contact,
because they respect that their offspring have busy lifestyles.
One grandparent expressed that she doesn’t see or hear much
of her (grand)children, because she thinks they are too busy.
She also stated that her own grandparents had more contact
with their (grand)children, than she has with her (grand)chil-
dren now, which she finds disappointing.
Furthermore, the five grandparents were asked about the
current communication methods they use to communicate
with their (grand)children. All the grandparents preferred
face-to-face contact and calling, because in their opinion these
methods are personal and easy. One grandparent expressed an
aversion towards text-messaging, because when she contacts
someone, she likes to hear a voice.
It was also evaluated whether grandparents feel aware
about the events in their (grand)children’s daily lives. All
interviewed grandparents expressed not to be completely
aware of the events in their (grand)children’s lives, because
they have too little contact. One grandparent stated that it is
normal that she is not completely aware of all the events in
her (grand)children’s lives.
6.2.2 Grandparents’ attitude towards technology
One of the five interviewed grandparents expressed that he
was interested in technology: he owns a mobile phone, a
tablet, and a computer, and keeps track of the latest news
about technological developments. The other four interviewed
grandparents expressed that they are not interested in tech-
nology. Two of these grandparents own mobile phones, but
they only use the mobile phone to make phone calls. When
asked to explain why they are not interested in technology,
one grandparent argued that technology seemed to progress
too fast to keep up with it. Another grandparent stated that it
is difficult for her to remember new information, which makes
it challenging for her to learn how to use technologies.
6.2.3 Grandparents’ understanding of Facebook
Grandparents were asked whether they understand the con-
cept of Facebook. One grandparent had a Facebook account,
but did not log in frequently (about once a month). All
other four grandparents were familiar with the existence of
Facebook, but they experienced difficulties explaining the
concept of Facebook. They seemed insecure talking about the
topic. However, they were all aware that Facebook could be
used to send messages and pictures to other people.
6.2.4 Grandparents’ attitude towards Facebook
Grandparents were asked whether they felt excluded from
their (grand)children’s Facebook activities. Two grandparents
expressed that it did not bother them that their (grand)chil-
dren were active on Facebook. However, one grandparent
expressed that her (grand)children were constantly occupied
with their phones in her presence, which did bother her.
One grandparent expressed her concerns about privacy
infringement on Facebook. She heard about a young girl who’s
sex photos were unwillingly circulating on Facebook. The
possibility of such thing occurring concerned her deeply. She
stated that this type of privacy infringement is one of the
reasons she has a negative attitude towards Facebook.
6.2.5 Grandparents’ attitude towards WallTalk
Even though grandparents expressed their interest and
seemed to understand the projected content, they had difficul-
ties understanding the full concept and features of WallTalk.
We had to explain to the grandparents that the projected
pictures and messages represented recent activities or opinions
of their (grand)children, and did not originate from an old
collection. Additionally, the grandparents were unsure about
the destination of their spoken ‘comments’ and smiled ‘likes’.
One grandparent revealed that she did not feel a connection
with the projected content, and that younger generations
would probably understand it better.
Grandparents were asked whether they are fond of
WallTalk and would like to have the final product perma-
nently installed in their homes. One grandparent responded
hesitantly, saying she would like to possess the system, but
would not want to use it every day. Another grandparent
stated that he would prefer phone calls over WallTalk to
communicate with his (grand)children.
Another aspect that was evaluated, was whether WallTalk
appears technologically complex to the elderly. The installa-
tion of WallTalk’s prototype in the homes of elderly seemed
rather intimidating to the elderly. There were a lot of cables
lying around that were connected to several technological
devices. However, we explained that, for the final version of
WallTalk, all these components would be fitted into a small
box, hiding the technology. It was difficult for the grandpar-
ents to imagine this final version of WallTalk, which left them
intimidated.
6.2.6 Grandparents’ ability to use WallTalk
Most grandparents had some difficulties using WallTalk. One
grandparent wanted to ‘like’ Facebook content of her grand-
child, but instead of smiling at the camera, she started talking
11

and said “Hey, here I am”. Besides that, three of the five inter-
viewed elderly did not have a (wireless) Internet connection in
their homes. For two of the five homes, it was difficult to install
WallTalk, because there was no wall with mounted photo
frames of all (grand)children. Therefore, the grandparents
had to mount pictures on the wall first, before WallTalk
could be installed. Finally, WallTalk allows grandparents to
‘comment’ on their (grand)children’s Facebook content, but
grandparents cannot not view replies to their comments or
others’ comments. Thus, grandparents were not able to have
conversations with their (grand)children through WallTalk.
One grandparent expressed some frustration with not being
able to do so.
7 Discussion
The results and limitations from the evaluation studies of the
grandchildren and grandparents are discussed in this section.
7.1 Grandchildren
7.1.1 The suitability of Facebook as a data source for WallTalk
At first, Facebook seems like a suitable data source for
WallTalk, because all participants were active on Facebook
and most of them regularly posted content on the platform.
This makes the presence of WallTalk in the homes of elderly
relevant and dynamic. Besides, the types of media most posted
by the respondents are text messages and photos, which corre-
spond to the content types WallTalk provides to grandparents.
However, further evaluation results revealed some issues
about Facebook as data source. Only about half the inter-
viewed grandchildren expressed feeling comfortable with the
idea that their grandparent can view their Facebook content.
They revealed that there are certain topics they do not want
to share with their grandparents through Facebook, such as
alcohol, drugs and parties. This has negative implications for
the suitability of Facebook as a data source for WallTalk.
7.1.2 The suitability of WallTalk
Grandchildren’s opinions and comments about WallTalk were
not always positive. When using WallTalk, most grandchil-
dren would like to review their content before showing it
to their grandparent. This is customizable in Facebook, but
making use of this feature conflicts with the goal of the system:
making it effortless for (grand)children. WallTalk should not
require grandchildren to invest extra time and effort. This
is because the evaluation results show that almost half of
the grandchildren have less contact with their grandparents
than desired, due to a lack of time. Moreover, grandchildren,
who are very close with their grandparents, on average feel
‘neutral’ towards their grandparents viewing their Facebook
content. On the other hand, grandchildren, who are not close
at all with their grandparents, are generally less positive
about their grandparents viewing their Facebook content.
This implies that the suitability of WallTalk for grandchil-
dren depends on the closeness of the grandparent-grandchild
relationship.
Even though most grandchildren admitted to be satisfied
with the current communication methods they use between
them and their grandparents, the majority of grandchildren
also desires the use of other methods. This has positive im-
plications for WallTalk and other innovative communication
technologies that focus on grandparent-grandchild communi-
cation.
7.2 Grandparents
7.2.1 The relevance of WallTalk
The majority of the interviewed grandparents was satisfied
with the frequency of contact they had with their offspring.
This seems to imply that WallTalk would be a superfluous
system for the interviewed grandparents. However, all inter-
viewed grandparents stated not to be fully aware of the events
in their (grand)children’s lives, because they had too little
contact with them. Therefore, there is a possibility that a
system such as WallTalk could contribute positively to the
relationship between grandparents and their (grand)children.
7.2.2 The suitability of WallTalk’s communication type
WallTalk establishes bidirectional communication between
grandparents and their offspring. Despite that, some grand-
parents felt limited in their communication options. WallTalk
allows grandparents to ‘comment’ on their (grand)children’s
messages, however, WallTalk does not allow grandparents to
receive their (grand)children’s replies to their comments, and
to continue the conversation. This disappointed some of the
interviewed grandparents. Therefore, the possibilities for more
extensive bidirectional communication should be explored.
7.2.3 The suitability of WallTalk
The goal of WallTalk was to develop a system that is suitable
for all generations involved. To make the system suitable for
elderly, it was expected that the system should not come
across as technologically complex. During the interviews, this
expectation was confirmed, as four out of five interviewed
grandparents expressed their disinterest in technology.
Our aim was to minimize the technological complexity of
WallTalk. Therefore, elderly can engage with the system using
speech and facial gestures, instead of a keyboard, mouse or
touchscreen. In spite of this construction, WallTalk still came
across as too technologically complex for the elderly. Part of
this can be attributed to the technological complexity of the
prototype version of WallTalk that was installed in the homes
of elderly, which consisted of many technological components
(as explained in Section 4.3). The final version of WallTalk, in
which all the technology is hidden in one small box, might be
less intimidating.
More importantly, in order to understand WallTalk, el-
derly have to understand what the displayed messages repre-
sent, the concept of ‘likes’ and ‘comments’, how their ‘smiles’
and voice messages are being converted, and where their
‘likes’ and ‘comments’ are sent to. WallTalk was developed
for elderly without a Facebook account, which implies that
the system should be suitable for elderly with a minimal
understanding of Facebook. However, even though all inter-
viewed elderly were aware that Facebook could be used to send
messages and photos to people, it turned out to be extremely
difficult for them to comprehend how WallTalk functions.
Therefore, WallTalk is most likely unsuitable for grandparents
that lack a deeper understanding of Facebook.
WallTalk was intended to be effortlessly installed in the
grandparents’ homes, by only adding visible technology to
photo frames on the wall. However, this installation is only
12

effortlessly if grandparents already have photo frames with
photos of their (grand)children on a wall. When this is not
the case, installing WallTalk requires the elderly to mount
photo frames on a wall, which does not make for a particularly
effortless installation.
Finally, since WallTalk is a web application, an Internet
connection is required to use the system. Grandparents with
a disinterest in technology are not likely to have access to
the Internet. In order to install WallTalk, grandparents would
be required to set up an Internet connection. However, this
problem is easily solved by including an Internet dongle in
WallTalk.
7.3 Limitations of the Study
A number of limitations can be pointed out in various aspects
of the evaluation methods. First of all, both grandchildren and
grandparents that participated in the study were recruited
using a non probabilistic sampling method, which does not
allow for generalizing the findings of this study to a broader
population. It should be noted that both populations of
‘grandchildren’ and ‘grandparents’ comprise of a large variety
of people, with many different ages, ethnicities and from
different social classes. This makes it nearly impossible to
generalize over those populations.
Furthermore, even though WallTalk is aimed at connecting
all generations, only evaluation studies among grandchildren
and grandparents were conducted, while the opinions of
grandparents’ children were not evaluated. Additionally, all
grandchildren who participated in the evaluation study were
recruited through Facebook; therefore, their opinion about
and presence on social media is most likely biased.
Finally, the prototype of WallTalk was only evaluated in a
small number of homes, and for a limited amount of time (3-5
hours). Additionally, the technological complexity of the pro-
totype of WallTalk might have influenced the grandparents’
attitudes towards WallTalk. In order to evaluate WallTalk
properly, a less technologically complex prototype should be
developed and installed in a larger number of households for
a greater amount of time.
8 Conclusion
This paper has described the design and evaluation of the
prototype of WallTalk. WallTalk is a lightweight communi-
cation system that aims to establish a connection between
elderly and their (grand)children, in a way that is suitable for
all generations involved. WallTalk projects (grand)children’s
Facebook content on the wall of elderly. The Facebook content
of each (grand)child is projected around a physical photo
frame, containing a picture of that (grand)child. Elderly can
‘like’ and ‘comment’ on this Facebook content in an intuitive
manner: by smiling and speaking. The findings of this research
indicate that the developed prototype of WallTalk is not
completely suitable for grandchildren and grandparents. For
grandchildren, the suitability of WallTalk is limited, mainly
because most grandchildren do not wish to share all their
Facebook content with their grandparents. WallTalk was also
not completely suitable for grandparents, primarily because
their lack of understanding about Facebook prevented them
from understanding how to use the system.
Future studies
Several aspects of WallTalk could be improved upon, in
order to make the system more suitable for all generations
involved. To make WallTalk more suitable for grandchildren,
future studies should focus on developing methods to filter
out content that grandchildren prefer not to share with their
grandparents. Natural Language Processing and Image Anal-
ysis could be useful techniques to determine which content
to share with grandparents. To improve the suitability of
WallTalk for elderly, future studies should either focus on
(a) making the concept of Facebook more comprehensible for
elderly, or (b) utilizing a data source from the (grand)children
that is more intuitive for elderly. Nevertheless, to produce
more generalizable results, additional evaluations need to
be conducted with probabilistic sampling methods among a
larger population. Besides that, future studies could also focus
on the potential relevance of WallTalk for other populations,
such as people with physical disabilities.
References
[1] S. E. Lindley, “Shades of lightweight: supporting cross-
generational communication through home messaging,” Univer-
sal Access in the Information Society, vol. 11, no. 1, pp. 31–43,
2012.
[2] A.-S. Melenhorst, W. A. Rogers, and E. C. Caylor, “The use
of communication technologies by older adults: exploring the
benefits from the user’s perspective,” in Proceedings of the Hu-
man Factors and Ergonomics Society Annual Meeting, vol. 45,
pp. 221–225, SAGE Publications, 2001.
[3] V. L. Bengtson, “Beyond the nuclear family: The increasing
importance of multigenerational bonds,” Journal of Marriage
and Family, vol. 63, no. 1, pp. 1–16, 2001.
[4] S. A. Ruiz and M. Silverstein, “Relationships with grandpar-
ents and the emotional well-being of late adolescent and young
adult grandchildren,” Journal of Social Issues, vol. 63, no. 4,
pp. 793–808, 2007.
[5] A. Miyajima, Y. Itoh, M. Itoh, and T. Watanabe, “Tsunagari-
kan communication: Design of a new telecommunication envi-
ronment and a field test with family members living apart,”
International Journal of Human-Computer Interaction, vol. 19,
no. 2, pp. 253–276, 2005.
[6] J. Doyle, Z. Skrba, R. McDonnell, and B. Arent, “Designing a
touch screen communication device to support social interaction
amongst older adults,” in Proceedings of the 24th bcs interac-
tion specialist group conference, pp. 177–185, British Computer
Society, 2010.
[7] I. Siio, J. Rowan, and E. Mynatt, “Peek-a-drawer: communi-
cation by furniture,” in CHI’02 extended abstracts on Human
factors in computing systems, pp. 582–583, ACM, 2002.
[8] N. Romero, P. Markopoulos, J. Van Baren, B. De Ruyter, W. Ijs-
selsteijn, and B. Farshchian, “Connecting the family with aware-
ness systems,” Personal and Ubiquitous Computing, vol. 11,
no. 4, pp. 299–312, 2007.
[9] E. D. Mynatt, J. Rowan, S. Craighill, and A. Jacobs, “Digital
family portraits: supporting peace of mind for extended family
members,” in Proceedings of the SIGCHI conference on Human
factors in computing systems, pp. 333–340, ACM, 2001.
[10] K. Chen and A. Chan, “A review of technology acceptance by
older adults,” Gerontechnology, vol. 10, no. 1, pp. 1–12, 2011.
[11] T. K. Judge, C. Neustaedter, and A. F. Kurtz, “The family
window: the design and evaluation of a domestic media space,”
in Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, pp. 2361–2370, ACM, 2010.
[12] N. Roussel, “From analog to digital, from the office to the living
room: why i happily worked in a media space but don’t live in
one,” in Media Space 20+ Years of Mediated Life, pp. 261–268,
Springer, 2009.
[13] R. Cornejo, J. Favela, and M. Tentori, “Ambient displays for
integrating older adults into social networking sites,” in Collab-
oration and Technology, pp. 321–336, Springer, 2010.
13

[14] L. C. Giles, G. F. Glonek, M. A. Luszcz, and G. R. Andrews,
“Effect of social networks on 10 year survival in very old aus-
tralians: the australian longitudinal study of aging,” Journal of
Epidemiology and Community Health, vol. 59, no. 7, pp. 574–579,
2005.
[15] A. Lenhart, “Adults and social network websites, pew internet
and american life project,” The Pew Center, Washington DC,
2009.
[16] S. J. Czaja, N. Charness, A. D. Fisk, C. Hertzog, S. N. Nair,
W. A. Rogers, and J. Sharit, “Factors predicting the use of
technology: findings from the center for research and education
on aging and technology enhancement (create).,” Psychology and
aging, vol. 21, no. 2, p. 333, 2006.
[17] B. W. Penninx, T. Van Tilburg, D. M. Kriegsman, D. J. Deeg,
A. J. P. Boeke, and J. T. M. van Eijk, “Effects of social sup-
port and personal coping resources on mortality in older age:
The longitudinal aging study amsterdam,” American journal of
epidemiology, vol. 146, no. 6, pp. 510–519, 1997.
[18] Z. X. Jin, T. Plocher, and L. Kiff, “Touch screen user interfaces
for older adults: button size and spacing,” in Universal acess in
human computer interaction. coping with diversity, pp. 933–941,
Springer, 2007.
[19] K. Laguna and R. L. Babcock, “Computer anxiety in young and
older adults: Implications for human-computer interactions in
older populations,” Computers in human behavior, vol. 13, no. 3,
pp. 317–326, 1997.
[20] M. M. Swenson and R. PhD, “The meaning of home to five
elderly women,” Health Care for Women International, vol. 19,
no. 5, pp. 381–393, 1998.
[21] A. Forghani and C. Neustaedter, “The routines and needs of
grandparents and parents for grandparent-grandchild conversa-
tions over distance,” in Proceedings of the 32nd annual ACM con-
ference on Human factors in computing systems, pp. 4177–4186,
ACM, 2014.
[22] J. Harwood, “Communication media use in the grandparent-
grandchild relationship,” Journal of Communication, vol. 50,
no. 4, pp. 56–78, 2000.
[23] M.-C. Lin, J. Harwood, and J. L. Bonnesen, “Conversation topics
and communication satisfaction in grandparent-grandchild rela-
tionships,” Journal of Language and Social Psychology, vol. 21,
no. 3, pp. 302–323, 2002.
[24] A. Savvopoulos and M. Virvou, “Tutoring the elderly on the use
of recommending systems,” Campus-Wide Information Systems,
vol. 27, no. 3, pp. 162–172, 2010.
[25] A. M. Kaplan and M. Haenlein, “Users of the world, unite. the
challenges and opportunities of social media,” Business hori-
zons, vol. 53, no. 1, pp. 59–68, 2010.
[26] M. Duggan and J. Brenner, The demographics of social media
users, 2012, vol. 14. Pew Research Center’s Internet & American
Life Project Washington, DC, 2013.
[27] E. M. Services, “Browser-based, excluding in-app, visits across
pc and mobile combined,” March 2015.
[28] F. T. McAndrew and H. S. Jeong, “Who does what on facebook?
age, sex, and relationship status as predictors of facebook use,”
Computers in Human Behavior, vol. 28, no. 6, pp. 2359–2365,
2012.
[29] J. LLC, Makey Makey, 2015. www.makeymakey.com (May 20,
2015).
[30] C. A. Cookman, “Older people and attachment to things, places,
pets, and ideas,” Image: The Journal of Nursing Scholarship,
vol. 28, no. 3, pp. 227–231, 1996.
[31] Z. S. Bassi, G. L. Smith, and L. Lee, “Short throw projection
system and method,” July 2007. US Patent 7,239,360.
[32] A. Blandford, “Semi-structured qualitative studies,” The Ency-
clopedia of Human-Computer Interaction, 2nd Ed., 2013.
[33] M. Rouse, Middleware, 2014. searchsoa.techtarget.com/defini-
tion/middleware (June 07, 2015).
[34] B. et al, Getting Started with Rails, 2015. guides.rubyonrails.org
(June 17, 2015).
[35] G. I. Glen Shires, Hans Wennborg, The Web Speech API, 2012.
https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html
(April 19 , 2015).
[36] J. Adorf, “Web speech api,” KTH Royal Institute of Technology,
2013.
[37] P. Kabal, “Tsp speech database,” McGill University, Database
Version, vol. 1, no. 0, pp. 09–02, 2002.
[38] A. M. Øygard, Clmtrackr, a Javascript library for precise
tracking of facial features via Constrained Local Models, 2014.
github.com/auduno/clmtrackr (April 15, 2015).
[39] D. Cristinacce and T. F. Cootes, “Feature detection and tracking
with constrained local models.,” in BMVC, vol. 1, p. 3, Citeseer,
2006.
[40] M. A. Hearst, S. T. Dumais, E. Osman, J. Platt, and
B. Scholkopf, “Support vector machines,” Intelligent Systems
and their Applications, IEEE, vol. 13, no. 4, pp. 18–28, 1998.
[41] A. M. Øygard, Clmtrackr, an emotion detection example,
2014. auduno.github.io/clmtrackr/examples/clm_emotionde-
tection.html (April 15, 2015).
Appendix A
Technical system description
A.1 Technical overview (architecture)
Several components shape the architectural layout of
WallTalk. An overview of these components (denoted by
rectangles) and belonging actions (denoted by ellipses) of
WallTalk is given in Figure 16.
Figure 16. Schematic overview of WallTalk’s architectural layout.
1) WallTalk completely relies on user data retrieved from
Facebook. Therefore, Facebook Login is used for authentica-
tion (Section A.2). Users signing up for WallTalk need to grant
WallTalk permissions to access their Facebook data; this will
be discussed in Section A.2.1.
2) The user data from Facebook needs to be stored in
a database. The database architecture of WallTalk will be
explained in Section A.3.
3) WallTalk is built in Ruby on Rails, and therefore
utilizes the Model-View-Controller architecture. Furthermore,
WallTalk needs to impose several usage restrictions in order
to function properly. The Model-View-Controller architecture
and the usage restrictions are discussed in Section A.4.
4) Several actions can take place when using WallTalk.
WallTalk polls the Facebook servers for data (Section A.5),
and WallTalk has the functionality of ‘commenting’ on Face-
book content (Section A.6) and ‘liking’ Facebook content
(Section A.7).
A.2 Facebook Login
(Grand)children and grandparents can sign up for WallTalk
through a web application. It is only possible to sign up for
WallTalk using ‘Facebook Login’, because WallTalk needs
to retrieve the Facebook data from the (grand)children’s
accounts, and needs to post Facebook data through the grand-
parent’s account. ‘Facebook Login’ allows people to sign in
to WallTalk by using their Facebook credentials; this means
that WallTalk does not require the user to supply additional
information (such as an application specific user name and
14

password). In the WallTalk web application, ‘Facebook Login’
is facilitated by OmniAuth, which is Rack Middleware 3
A.2.1 Facebook User Permissions
Facebook requires applications using the Facebook API to
request a user for permissions to access his/her data, in order
to protect the user’s privacy. The user is asked for his/her
permission to access certain data when he/she registers for the
application. When the user grants these permissions, the ap-
plication receives an access token to access data that requires
these permissions. WallTalk obligates its users (both grand-
parents and (grand)children) to grant ‘read stream’, ‘user
friends’, ‘user photos’ and ‘publish actions’ permissions. ‘Read
stream’ permission is needed to access the (grand)children’s
status updates, ‘user friends’ to determine which (grand)chil-
dren are friends with the grandparent, ‘user photos’ to access
the (grand)children’s photos they post in their status updates,
and ‘publish actions’ to allow the grandparent to ‘like’ and
‘comment’ on status updates of the (grand)children.
Without these permissions, WallTalk will not be able
to function. Therefore, users cannot register for WallTalk
without granting all these permissions. However, in order for
an application to be able to ask users for permissions, the
application has to be ‘Submitted to Facebook for Review’, in
which Facebook reviews how the application uses user data.
The prototype of WallTalk has not been submitted for review
yet, therefore only ‘Test Users’ can use the application for
now. 4
A.3 Database
It is necessary to store user data, because WallTalk needs
continuous access to the data from the Facebook accounts of
the grandparents and (grand)children. Moreover, grandpar-
ents need to be connected automatically to their (grand)chil-
dren in WallTalk, for which a relational schema is needed.
PostgreSQL, an open source database, is used to store the
user data. The database architecture of WallTalk is discussed
in this section.
A.3.1 User Model
Both (grand)children and grandparents are stored in the
database as a ‘User’. It is not necessary to distinguish between
grandparents and (grand)children in the User model; the only
consequence of this is that it enables (grand)children to use
the functionalities of WallTalk as well, if they would want to.
This way, (grand)children are able to see for themselves what
the application would look like for their grandparent.
A.3.2 Friend Model
For each user, the Facebook friends that also are registered for
WallTalk are stored in the database as a ‘Friend’ of this user.
Note that, due to Facebook privacy restrictions, WallTalk can
only access a user’s friends that have already registered for the
3. Middleware is the software layer in between the operating sys-
tem, and the applications on each side of a distributed computer
network [33]. In this case, on one side is the WallTalk application,
and on the other side Facebook.
4. ‘Test Users’ are users that have been added manually to use
WallTalk, while the application is in development mode. WallTalk
cannot be deployed until Facebook allows WallTalk to ask users for
permissions.
application. Therefore, each time a user signs in to WallTalk,
both the User model and the Friend model are updated.
WallTalk is aimed at grandparents without a Facebook
account; their (grand)children are expected to create an ac-
count for their grandparent on Facebook when signing up for
WallTalk. Therefore, it is assumed that the grandparents will
only have their (grand)children as friends on Facebook. How-
ever, if the grandparent is also friends on Facebook with other
users of WallTalk (who do not belong to their family), these
friends will also become part of the grandparent’s WallTalk
environment, which not desired. Since WallTalk is aimed at
grandparents who do not have a Facebook account yet, it
is unlikely that this situation will occur. Therefore, for the
prototype of WallTalk this database construction should be
sufficient.
A.3.3 User–Friend relationship
The cardinality of the User–Friend relationship is ‘one-to-
many’, meaning that one User can have zero to many Friends,
and each Friend belongs to exactly one User. A UML diagram
of the User-Friend relationship is displayed in Figure 17.
Figure 17. User-Friend relational schema.
User ID
The User ID is the primary key of the User model. In the
Friend model, the User ID is the foreign key, which points to
the primary key in the User model. This way, the relationship
between the tables can be identified.
Facebook User ID
It is necessary to store the Facebook User ID in order to
retrieve the status updates from the (grand)children’s Face-
book Wall, and to post the grandparents’ comments to the
(grand)children’s Facebook Wall.
Name
The full name of the user that registers for WallTalk, as stated
by the user on Facebook, is stored.
OAuth Access Token & OAuth Access Token Expires At
A crucial part of the User model is the OmniAuth Access
Token. This access token provides WallTalk secure access
to the Facebook Graph API, in order to read, write and/or
modify the user’s data. The OmniAuth Access Token expires
in approximately 60 days. This implies that (grand)children
need to be sent a notification to check into the application
every 60 days; otherwise WallTalk will no longer be able to
access their Facebook data.
15

Created at
The date and time when the record was created in the
database.
Updated at
The date and time when the record was updated for the last
time in the database.
A.4 Architecture and Restrictions
The software architecture of WallTalk, and usage restrictions
of WallTalk are discussed in this section.
A.4.1 Model-View-Controller Architecture
WallTalk is built in the Ruby on Rails (open source) web
framework. Ruby on Rails utilizes the ‘Model View Con-
troller’ principle, in which the application is divided into three
separate, cooperative subsystems. The Model represents the
database, the View is the user interface that presents data to
the user, and the Controller mediates between the Model and
the View: it queries the Model for specific data, and organizes
that data into a form that fits the needs of a given View [34].
A.4.2 Google Chrome
WallTalk by the Google Chrome browser, because Google
Chrome is the only browser that can access the Web Speech
API (which is needed to enable the ‘commenting’ functionality
of WallTalk).
A.4.3 Secure Server
WallTalk can only be accessed over HTTPS, because in HTTP
the user has to grant the application permission every time
the application wants to use the user’s camera or microphone.
When accessing the application over HTTPS, the user only
has to grant the application permission the first time.
A.5 Polling the Facebook Graph API
The Facebook Graph API is used in order to retrieve data
from Facebook. This is the only legal method to retrieve data
from Facebook, since Facebook does not allow scraping/crawl-
ing their website. WallTalk polls the Facebook server every
5 seconds, meaning that it ‘checks’ every 5 seconds whether
or not new status updates have been posted. This is not
the most ideal solution, since continuous polling can increase
the response latency. The Facebook Graph API does offer
a ‘Realtime Updates’ functionality, in which the Facebook
servers send the application’s servers a POST request when
certain (specified) fields have changed (e.g. when a new status
update is posted). Unfortunately, the Realtime Updates func-
tionality can take up to 10 minutes to send the application a
POST request. Therefore, even though polling is not the most
efficient solution, it ascertains a more reliable application than
when using the Facebook Realtime Updates functionality.
A.5.1 Technical overview
A technical overview of ‘Polling the Facebook Graph API’ is
displayed in Figure 18, and explained in more detail.
1. AJAX GET request
Every 5 seconds, an AJAX GET request is sent from the View
to the Controller. By making an AJAX request, WallTalk
Figure 18. Technical overview of ‘Polling the Facebook Graph API’.
can retrieve data from the server asynchronously, without
interfering with the display of the page.
2. Request user data
The Controller requests the necessary user data from the
Model in order to access the Facebook server: the Facebook
Omniauth Access Token and the Facebook User ID of the
grandparent and all (grand)children.
3. Pass user data
The Model passes the requested user data back to the Con-
troller.
4. GET request for status updates
A GET request is made to the Facebook server to retrieve the
status updates from the (grand)children’s profiles.
5. JSON response with status updates
The Facebook server returns the requested status updates in
JSON format.
6. Update view
The Controller parses the JSON data, and the grandchildren’s
status updates are displayed in the View. A blinking border
denotes the photo frame that belongs to the newest status
message.
A.6 Commenting (speech-to-text)
Grandparents can comment on the Facebook messages and
photos of their (grand)children, by recording a spoken mes-
sage. As mentioned in Section 4, the grandparents can only
‘comment’ on the latest message/photo that was posted by
one of their (grand)children. Grandparents can only comment
on the latest message/photo that was posted by one of their
(grand)children. In order not to infringe upon the privacy of
the grandparents, the recording is initiated when they pick up
an object of choice (such as a statuette), and the recording is
stopped when they put the object down.
In the prototype version of WallTalk, this statuette is
connected to a MakeyMakey device, which is connected to
a laptop (Figure 19). The MakeyMakey is a circuit board
that can be connected to everyday objects to send keyboard
and mouse events to a computer [29]. When the statuette is
picked up, the MakeyMakey signals a ‘key down’ event to the
WallTalk web application. This initiates the voice recording:
the grandparent can now comment on the message or photo of
a grandchild. When the grandparent puts down the statuette,
the MakeyMakey signals a ‘key up’ event to the WallTalk web
application. This stops the voice recording.
The recorded message is converted from speech to text, by
using the Web Speech API [35]. This is a JavaScript API,
16

Figure 19. An illustration of the physical setup for ‘commenting’.
which enables speech synthesis and speech recognition, of
which WallTalk only uses the latter. When the web applica-
tion receives the ‘key down’ event through the MakeyMakey,
the microphone is enabled, and speech recognition is started
simultaneously. When the web application receives the ‘key
up’ event through the MakeyMakey, the speech recognition is
stopped. The recorded speech is sent to the Google Chrome
servers, which convert the recorded speech to text. The tex-
tual output is returned to WallTalk’s server. Finally, WallTalk
sends a POST request to Facebook with the textual output of
the recorded message, upon which the textual output is posted
as a comment to the message of the (grand)child.
A.6.1 Accuracy measures of the Web Speech API
There are not many accuracy measures reported of the Web
Speech API, however, Adorf (2013) [36] measured sentence
correctness and word accuracy of the Web Speech API, by
using the TSP database5
. All sentences in the TSP database
were sent to the Web Speech API. The textual output of
the Web Speech API was compared to the annotation of the
speech recordings. The Web Speech API correctly converted
74% of the words from speech to text. There were no signifi-
cant differences in accuracy between males (74%) and females
(72%).
Though the accuracy of the Web Speech API is not out-
standing, it seems accurate enough to function in the WallTalk
prototype. It is important to note that the grandparent does
not get to see the textual output of the recorded message. This
is done purposely; to prevent the grandparent from getting
confused or frustrated when the speech is not converted 100%
accurately into text. This solution is not ideal, however, we
expect that (grand)children would be less frustrated with a
few incorrect words in their grandparent’s messages, than
the grandparent would be the other way around. Future
development of WallTalk would prefer speech recognition with
a higher accuracy, in which case the recorded message could
also be displayed to the grandparent.
A technical overview of ‘Commenting’ is displayed in Fig-
ure 20, and explained in more detail.
1. Pick up statuette
The grandparent picks up the statuette to which the Makey-
Makey is connected.
5. The TSP database is an annotated speech recognition dataset,
which contains more than 1400 annotated speech recordings by 24
speakers (half male, half female) [37].
Figure 20. Technical overview of ‘Commenting’.
2. Key down event
The picking up of the statuette triggers a ‘key down’ event
in the MakeyMakey, which is interpreted by the View and
initiates a new speech recording with JavaScript.
3. Display recording feedback
The View displays an icon that notifies the grandparent that
his/her speech is being recorded.
4. Put down statuette
The grandparent puts down the statuette to which the Makey-
Makey is connected.
5. Key up event
The putting down of the statuette triggers a ‘key up’ event in
the MakeyMakey, which is interpreted by the View and stops
the speech recording with JavaScript.
6. Stop displaying recording feedback
The icon that notifies the grandparent that his/her speech is
being recorded is no longer displayed.
7. AJAX GET request
An AJAX GET request is sent from the View to the Con-
troller, which passes the recorded message.
The Controller determines which status update is being com-
mented on, and requests the Facebook Omniauth Access
Token and the Facebook User ID of the grandparent and
the (grand)child to which the status update (that is being
commented on) belongs.
9. Pass user data
troller.
10. POST request for ‘Comment’
A POST request is made to the Facebook server to publish the
grandparent’s comment to the (grand)child’s status update.
A.7 Liking
Grandparents can ‘like’ Facebook content of their (grand)chil-
dren, by smiling at the camera that is near the projector.
As mentioned in Section 4, the grandparents can only ‘like’
the latest message/photo that was posted by one of their
(grand)children. This smile detection feature is implemented
using an open source facial detection JavaScript library called
‘clmtrackr’ [38], combined with smile detection. Both tech-
niques will be explained in this section.
17

A.7.1 Facial feature detection by the Constrained Local Models
method
Clmtrackr fits facial models onto images and videos making
use of the Constrained Local Models (CLM) method [39],
which is parsed by the emotion classifier afterwards. CLM
determines the positions and feature points of the mouth,
nose, eyes and eyebrows, by using models to search in a local
detected face region. These models provide the knowledge
about the shape of faces to constrain the search.
The general approach of CLM consists of two phases:
model-building and the search process. A CLM model incor-
porates a shape model (describing how face shapes can vary),
and a patch model (portraying how the pattern of images
around each feature point should be). The CLM search pro-
cess uses linear Support Vector Machines (SVM) 6
to search
through a local region around a feature point of the face, from
which a response image is obtained. To this response image, a
quadratic function is fitted to find new feature point positions.
The search is accomplished after repeating these steps until all
points reach stable position.
A.7.2 Smile detection
From the resulting constrained local model, WallTalk needs
to determine whether the person is smiling in order to send a
‘like’ on Facebook. The open source clmtrackr library provides
emotion classification to determine values for the emotions
‘angry’, ‘sad’, ‘happy’ and ‘surprised’. Logistic regression is
used to determine the mean value for each emotion parame-
ter [41]. Because WallTalk only needs to detect whether the
grandparent is smiling, the most relevant parameter is ‘happy’.
If the value for the ‘happy’ parameter is greater than the
other emotion values and greater than a certain threshold, this
is interpreted as a smile.
A.7.3 Accuracy measures of the CLM method and smile detec-
tion
According to Cristinacce and Cootes (2006) [39] the Con-
strained Local Models approach is a favorable approach,
because it outperforms many other facial feature tracking
methods. The CLM method is fairly computationally efficient
and able to track faces at approximately 25 frames per sec-
ond. Moreover, results of the experiments by Cristinacce and
Cootes (2006) [39] show an accuracy of 95%. The emotion
classifier used in WallTalk for smile detection does not pro-
vide accuracy and performance measures. However, for the
prototype of WallTalk the smile detection works sufficiently.
A technical overview of ‘Liking’ is displayed in Figure 21, and
explained in more detail.
1. Smile
The grandparent smiles into the webcam, which triggers the
Emotion Recognition to initiate in JavaScript.
2. Detects ‘Happy’
The Emotion Recognition detects that the grandparent is
smiling.
6. SVM is a supervised learning method within the area of machine
learning, used for classification and regression. Given a training set,
the SVM algorithm builds a linear model and assigns data points to
one category or another [40].
Figure 21. Technical overview of ‘Liking’.
3a. Display ‘Like’ feedback
The View displays an icon that notifies the grandparent that
his/her ‘Like’ has been detected.
3b. AJAX GET request
Asynchronously with displaying the ‘Like’ feedback, an AJAX
GET request is sent from the View to the Controller.
The Controller determines which status update is being liked,
and requests the Facebook OmniAuth Access Token and the
Facebook User ID of the grandparent and the (grand)child to
which the status update (that is being liked) belongs.
5. Pass user data
troller.
6. POST request for ‘Like’
A POST request is made to the Facebook server to publish
the grandparent’s ‘Like’ to the (grand)child’s status update.
A.8 Visualization
The Facebook content of the (grand)children is visualized in
the browser. The visualization algorithms are developed using
HTML, CSS, JavaScript and JQuery.
A.8.1 Initialization
In Figure 22 an initialization window of WallTalk is shown.
Manual initialization is needed for the system to determine the
positions of the physical photo frames of the (grand)children.
For correct initialization, the WallTalk page in the browser
should be set to full-screen and projected on the wall with
photo frames.
Figure 22. An example of WallTalk’s initialization window.
18

Initially, the HTML document object model (DOM) con-
tains only a few HTML elements. All elements and visuals
displayed on the page are the result of JavaScript functions.
For each (grand)child in the family, a square element is placed
within the projection area, representing the photo frames on
the wall (Figure 22).
These blocks contain the names of the (grand)children.
The projection of these elements can be dragged, resized and
rotated in order to place them onto the physical photo frames.
Furthermore, the borders of these blocks can be widened or
made smaller, depending on the size of the physical photo
frame. An example of a block projection that is fitted on the
physical photo frame is shown in Figure 23.
Figure 23. An example of a block projection that is fitted on the physical
photo frame.
After all block projections are placed correctly on the
physical photo frames, the user can save the initialization. Af-
ter saving, Facebook messages and pictures of the (grand)chil-
dren will be visualized.
A.8.2 Checking for new incoming media
Every 5 seconds, new Facebook content is retrieved on the
server-side and stored on the client side browser. Depending
on whether the content is new, the algorithms for placing the
content are called and the animations initiate.
A.8.3 Placing a message
When trying to place a message as ‘text cloud’ on a corner
of a photo frame, first it needs to be determined what the
length of the message will be in the DOM. The initial size of
the containing text cloud div depends on the message size.
Moreover, the width of the text container should be at least
the width of the widest word to prevent this word from flowing
out of the container.
After these initial steps, the algorithm searches for a suit-
able position for the text container, which should be placed
near one of the four corners of the photo frame. If the text
div does not overlap with any other element and/or is not
out of the window, it can stay there. If the element cannot
be placed near any corner of the frame, the size of the text
container should be adjusted until the element fits. When the
text container is positioned correctly, the visual appearance of
a text cloud is generated by making three of the four corners
rounded. Which corners are rounded depends on the location
of the text container relative to the photo frame. When the
width and height of the text container are reduced to an extent
that the text overflows the element vertically, the overflow is
animated. In this way, the text inside the text cloud moves up
slowly, enabling the user to read the text even though it was
difficult to place it around the photo frame.
The possibility exists that the algorithm is unable to find a
correct position for the text cloud. For example, this happens
when all photo frames are placed almost against each other.
Therefore, while initializing, the user should keep in mind that
some space should be left empty around the photo frames
to enable media content visualization. After initializing the
photo frames with this in mind, the algorithm is able to find a
good position for all media content.
A.8.4 Placing a picture
The process of placing a picture is similar to that of placing
messages. However, the initial size of the picture is set as
a certain proportion of the width of the window. After the
picture is loaded on the client side, the algorithm searches
for a suitable position on the page, as described in the
previous section. This picture is also re-sized until it fits on
the projection.
A.8.5 Animating frame of (grand)child with latest post
It is recorded which (grand)child was the last one to post con-
tent on Facebook. This is done with ‘time stamps’, which are
parsed on the server side and communicated to the client side.
The frame of this (grand)child blinks slowly, until another
(grand)child publishes content on Facebook.
Appendix B
Questionnaire Grandchildren
Part I: Questions about social media usage
1. Are you active on social media? (Yes / No)
2. Are you active on Facebook? (Yes / No)
3. Which social media website do you use most frequently?
(Facebook / Twitter / Instagram / Google Plus+ / Other,
specify)
4. How often do you post something on your Facebook
timeline? (Daily / Weekly / Monthly / Yearly / Never)
5. If you post something on your Facebook timeline, what
type of content do you post most often? (Text messages /
Photo’s / Video’s / Links to other websites / Facebook content
from others / Other, specify)
6. Are you friends on Facebook with any family members?
(Yes / No)
7. Which familymembers are you friends with on Face-
book? (Child(ren) / Brother(s) or sister(s) / Parent(s) /
Grandparent(s) / Nephew(s) or niece(s) / Uncle(s) and
aunt(s) / Other (such as stepfamily))
8. Do you have one or more grandparents that are still
alive? (Yes / No)
9. “I feel comfortable when my family members (would
be able to) see my Facebook content.” (Completely disagree /
Disagree / Neutral / Agree / Completely agree)
Part II: Questions about grandchild and grandparent
10. Do you have one or more grandparents that are active
on Facebook? (Yes / No)
11. “I feel comfortable when my grandparent can see my
Facebook content.” (Completely disagree / Disagree / Neutral
/ Agree / Completely agree)
12. Would you like your grandparents to be active on
Facebook? (Yes / No)
13. Which type of messages would you rather not to share
with your grandparent, that you would share with other
Facebook friends? (None, I would like to share everything with
my grandparent / Messages about politics / Messages about
19

WallTalk_K.Beard_N.Post

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (18)

Similar to WallTalk_K.Beard_N.Post

Similar to WallTalk_K.Beard_N.Post (20)

WallTalk_K.Beard_N.Post