1. ICT4AgeingWell 2015
International Conference on Information and
Communication Technologies for Ageing Well
and e-Health
Improving Activity Monitoring
through a Hierarchical Approach
Xavier Rafael-Palou, Eloisa Vargiu, Guillem Serra, Felip Miralles

2. - Motivation
- Sensor-based Home telemonitoring & support
- Hierarchical approach
- Upper lever
- Lower lever
- System Integration
- Conclusions
- Future work
2
Contents
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

3. Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
BackHome
FP-7 project which aims to assist people with disabilities back home after a discharge
Goals
- To study the transition from the hospital to the home
- To learn how different BNCIs and other assistive
technologies work together
- To learn how different BNCIs and other assistive
technologies can help in the transition from the
hospital to the home
- To reduce the cost and hassle of the transition from
the hospital to the home
Motivation
3

4. Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
BackHome
FP-7 project which aims to assist people with disabilities back home after a discharge
Main outcomes
- New and better integrated practical electrode systems
- Friendlier and more flexible BNCI software
- Better telemonitoring and home support tools
Motivation
4

5. The focus of this study is to improve performance and limitations of sensor based home
telemonitoring systems
Benefits:
- Quantify physical activity in daily life which is
an important predictor of risk of hospital
readmission and mortality in patients with chronic
diseases
- Enable therapists, caregivers, and relatives to
become aware of user context, infer user habits
and behaviour to provide an enhanced and
personalised assistance and support
- Allow to anticipate or identify if the person
requires a form of assistance since an unusual
activity has been recognized
5
Motivation
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

6. At home
- A mesh network of wireless sensors (e.g.
presence, door, mattress)
- A data collector to parse and transmit data
On the cloud
- A middleware to store data securely
- An intelligent monitoring system to
continuously and concurrently analyse new
sensor data according to a sliding window
approach
On a Healthcare center
- Therapist web application consumes
crunched data
6
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
SB-TMHSS - Sensor Based Home Telemonitoring and
support
Sensor-based Telemonitoring & Home Support
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
In Backhome we developed a sensor based home telemonitoring and support system:

7. Limitations:
- By requirements, sensor system should be
unobtrusive and cheap (e.g. open source,
wireless, minimum set of sensors, avoid
cameras)
- System is prone to sensor errors, sometimes
they lose or add noisy events (e.g. when
sensors remain with a low battery charge)
These conditions reduce the activities that the
system can recognize and makes more complex to
produce an accurate detection of user activities
Also makes crucial understanding the user
location, i.e. away, at home, alone or with visits,
before to proceed with the recognition of more
specific activities 7
Sensor-based Telemonitoring & Home Support
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
SB-TMHSS - Sensor Based Home Telemonitoring and
support

8. Examples of the importance of detecting the
user's location:
- The system may register a status door change
but the door was not opened, so the system
believes the user is away but sensor events
are being detected at user’s home
- On the contrary, the system may not register
that a door was opened so the system still
believes the user is at home
- Also, the system may fail to detect if a user is
away after a door change if multiple users
were at home or if a false positive door event
happened
8
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
Sensor-based Telemonitoring & Home Support
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
SB-TMHSS - Sensor Based Home Telemonitoring and
support

9. - Large literature on recognition of activities at home
- Great variability in the settings of the experiments either in the number of sensors
and their type, individuals involved or the duration
- Large amount of techniques for daily life activities (e.g. supervised, either generative
or discriminative; and unsupervised)
- This diversity makes it extremely difficult to compare performances and draw
conclusive findings from those studies
- Despite this we have not found extensive studies that analyze altogether the
detection of visits, presence or absence of users at home using wireless binary
sensors
9
Alternatives
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

10. To solve, or improve the user location ability in noisy
and limited sensorized environments:
- We partitioned the problem in two (upper and
lower classifiers) and aggregated their results
- The classifier at the upper level aimed at
augmenting performance of the door sensor,
improving its capability of recognizing if the user is
at home, or not
- The classifier at the lower level is aimed at
recognizing if the user is really alone or if s/he
received some visits
10
Hierarchical approach
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

11. Data preparation
- 4 month training & evaluation / 1 month test
- Extract relevant periods (door events)
- Compute features by counting motions before,
after and during door periods (total, avg, max,
min, ratios)
- Label data (0 home/ 1 away) contrasting with
a mobile activity tracker + user
Rule based approaches explored:
- Naive rule algorithm outperformed 77%
(accuracy score)
Can we improve this result?
11
Hierarchical approach: Upper level
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
PCA explained variance ratio for 5 features:
0.32 0.19 0.14 0.10 0.09

12. Methods
- Supervised algorithms: SVM, LogisticRegression, RandomForest, AdaBoost
- Search best learning parameters
- Use of 10-fold cv over evaluation set
- Study of the trade off between bias vs variance
12
Hierarchical approach: Upper level
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

13. Best results for high-level classifier during training (T) and evaluation (E) phase
Selected classifier SVM (gamma=1, C=0.452)
On testing phase: f1-score 0.97 and accuracy 0.968.
Rule
based
Best
ML
Improvement
correct 48 60 +12
errors 14 2 -12
accuracy 0.77 0.968 +0.19
13
Hierarchical approach: Upper level
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

14. Data preparation
- 4 month training & evaluation / 1 month test
- Extract relevant data periods (door events)
- Filter those when user is away
- Compute features by counting motions before, after and
during door periods (total, avg, max, min, ratios)
- Label data (0 alone / 1 multi-user) by contrasting them with
a mobile activity tracker + user
Rule based approach was explored unsatisfactorily:
● Num movements every 60 secs g.t. 6: 23 anomalies
● Num of movements every 60 secs g.t. 10: 1 anomaly
● Simultaneous movements: 20 anomalies
● Simultaneous movements in less 2 secs: 23 anomalies
● Max num of movements in 60 secs g.t. 6 or simultaneous
moves in less 2 secs: 28 anomalies
● Max num of movements in 60 secs g.t. 6 or simultaneous
moves: 21 anomalies
14
Hierarchical approach: Lower level
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

15. Eventually, we relied on a novelty detection approach, due to:
- Allows the identification of new or unknown data that a
machine learning system has not been trained with and
was not previously aware of
- Indicated when data is skewed i.e. has few positive cases
(i.e., anomalies) compared with the negatives (i.e., regular
cases)
Thus, we estimate a function f that is positive on the dataset and
negative on the complement.
The functional form of f is given by a kernel expansion in terms
of a potentially small subset of the training data;
f is regularized by controlling the length of the weight vector in
an associated feature space.
15
Hierarchical approach: Lower level
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

16. The Novelty approach was implemented by using a one-class SVM with RBF classifier
- The classifier was trained considering only the normal instances (person is alone) and
then evaluated introducing the anomalies (person receives a visit)
- According to the best evaluation results, we selected the regularization parameters
(n) =0.01 and g = 0.1
- The classifier was tested with the data coming from the 1-month window of monitored
events and results showed an accuracy of 0.94
16
Hierarchical approach: Lower level
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

17. Once both classifiers have been trained and chained we tested its performance with the
testing dataset corresponding to a window of 1 month
We compared the overall results with those using the rule-based approach in both levels of
the hierarchy
17
Hierarchical approach: Overall evaluation
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

18. The intelligent monitoring system is composed of 5
modules:
- PP, pre-processing module to encode the data for the
analysis
- ED, emergency detection module to notify, for
instance, in case of smoke and gas leakage
- AR, the activity recognition module to identify the
location, position, activity and sleeping-status of the
user
- EN, the event notification module to inform when a
new event has been detected
- ST, summary computation module to perform
summaries from the data
18
System integration: Intelligent monitoring system
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

19. The hierarchical approach was integrated in AR
- Once the current window recognizes a door event at
time tb it looks for the previous one in the window or
before (in the example ta)
- Then, the period from that door events (i.e, tb-ta) is
classified by the hierarchical classifier. Seemly, when
the event tc has been recognized, the period from tb
and tc is classified
- Finally, the period from tc to the end of the window is
classified
- In case of no door events have been recognized, the
period from ta to the end of the window is classified
19
System integration: Intelligent monitoring system
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

20. SB-TMHSS is currently running in 5 end user homes, 2 in Barcelona and 3 in Belfast
20
System integration: Therapist web application
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

21. 21
System integration: Therapist web application
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015
SB-TMHSS is currently running in 5 end user homes, 2 in Barcelona and 3 in Belfast

22. - Performance of sensor-based telemonitoring and home support systems depends,
among other issues, on the reliability of the sensors
- Binary sensors are quite adopted in the activity recognition literature and also in
commercial solutions, they are prone to noise and errors
- We presented a solution, based on machine learning techniques, aimed at reducing
error from the sensors and helping to discriminate when user is at home, away or
with a visit
- The proposed approach can be used as a preprocessing phase in every activity
recognition system, being completely independent from that
- Results showed an overall improvement of 15% in accuracy with respect to a rule-
based approach
- The system is part of the BackHome project and is currently running in 2-healthy-
users’ home and in 3-end-users’ home
22
Conclusions
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

23. We are currently setting-up new experiments aimed at comparing the hierarchical
approach with a multi-class classifier and with an ensemble (not hierarchical) of
classifiers
Moreover, we are improving the approach in order to be totally automatic, by using data
from Moves as feature instead of to validate the initial dataset
We are also interested in studying if we can generalize the proposed approach to use it
for all the users of the system, or if we have to use a personalized approach for each
different user
23
Future work
Improving Activity Monitoring through a Hierarchical Approach. ICT4AgeingWell, 2015

24. www.Backhome-FP7.eu
BackHome-FP7-Research-Innovation
@BackHomeFP7
BackHomeFP7