Seminar given for the SPHERE Interdisciplinary Research Collaboration at the University of Bristol on the 20th May 2014

1. Francisco Flórez-Revuelta
Digital Imaging Research Centre
Vision-based monitoring of
people in private spaces

2. Presentation
Architecture
Human action recognition
Optimisation
Privacy by the context
Discussion
Overview

3. Associate Professor (now on leave) at the University of Alicante (Spain)
Member of the "Domotics and Ambient Intelligence" research group
Multi-disciplinary research: computer vision, computational intelligence (artificial
neural networks, evolutionary computation), home automation…
Research on ICT for smart environments (services at home, system integration,
efficient use of energy, ambient assisted living, person-environment
interaction…)
Development of person-environment interfaces for the interaction of people with
their environment, paying particular attention to the support of the independent
living of elderly and disabled people
My background

5. Joined Kingston University in 2011 as Marie Curie fellow
The Robot Vision Team at the Digital Imaging Research Centre had a huge
experience on video-surveillance
Combine their expertise on video-surveillance with mine on ambient assisted
living
My background

6. A study on people activity in private spaces: towards a multisensor
network that meets privacy requirements
Project funded by the European Commission – 2011-2013
Objectives
Study, design and development of a distributed vision
system for human motion analysis in a private
environment
Development of appropriate techniques for maintaining
the privacy of the individuals being monitored
Integration of the vision system with other sensors
forming a perceptual, unobtrusive and interoperable
network
Integration and validation of the system in a smart home
environment

7. Senior Researcher since 2013
Strengthen research on ambient assisted living and intelligent environments at
Kingston University
Current position

8. Platform for self-assessment and efficient management for informal caregivers
Project funded by the European Commission
Ambient Assisted Living Joint Programme (call 5) – 2013-2015
Activities and events to be detected:
Recognition of activities of daily living (ADL): wake-up,
going to the toilet, sleeping…
Activity level: light, moderate, intensive
Location and tracking of the person inside the home
Risky situations: fall detection, dangerous objects; fire,
flood, gas alarms…
Social activity: leaving home, receiving visits, use of
telephone…
Detection of basic and advanced activities: cooking,
watching TV…

9. Architecture
Camera 1
Motion
Detection
Human
Behaviour
Analysis
Multi-view Human Behaviour Analysis
Privacy
Reasoning
System
Alarm Actuators
Camera 2
Motion
Detection
Human
Behaviour
Analysis
Camera N
Motion
Detection
Human
Behaviour
Analysis
...
Setup and Profiles
DB (Activities,
Inhabitants,
Objects, ...)
Log
Event
Long-term
analysis
...
...
Environmental
Sensor Information
Caregiver

10. Human behaviour analysis

11. Feature extraction

12. Bag of key poses

13. Multiple views

14. Results
Weizmann
MuHAVi-­‐8
MuHAVi-­‐14
IXMAS

15. Use of RGB-D cameras

16. Optimisation
We need to recognise the action/activity in order to apply an appropiate privacy
filter
A high success rate is compulsory

17. Evolutionary feature subset
selection
Some features are not needed or they introduce noise
Use of an evolutionary algorithm as a wrapper
Chromosomes are encoded as binary vectors, each gene representing whether
a feature is or is not used for recognition

18. Results
Weizmann
MuHAVi-­‐8
MuHAVi-­‐14
DHA
IXMAS

19. Evolutionary feature subset
selection
Chromosome
Normalisa=on
Crossover

20. Results

21. Results

22. Cooperative coevolutionary
optimisation
Simultaneous selection of the training set, feature set and parameters of the
clustering algorithm to improve the success rate

23. Cooperative coevolutionary
optimisation

24. Cooperative coevolutionary
optimisation

25. Cooperative coevolutionary
optimisation

26. Privacy preservation
Privacy by the context
Context:
identity of the subject
appearance
location
on-going activity
identity of the observer

27. Privacy filters

28. Current work
Fusion of multiple vision devices: multiple RGB-D devices, RGB and RGB-D
cameras…
Feature selection from a large bank of features
Incremental and adaptive learning
Continuous activity recognition
Validation in real scenarios
Monitoring people with all kind of sensors, fusion with environmental sensors
Anything related to intelligent environments

29. Papers
Human activity recognition
Chaaraoui, A.A.; Padilla-López, J.R.; Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; Flórez-Revuelta, F.: A
Vision-based System for Intelligent Monitoring: Human Behaviour Analysis and Privacy by Context,
Sensors, 14(5):8895-8925, 2014.
Chaaraoui, A.A.; Flórez-Revuelta, F.: Adaptive Human Action Recognition With an Evolving Bag of Key
Poses, IEEE Transactions on Autonomous Mental Development (accepted for publication)
Chaaraoui, A.A.; Padilla-López, J.R.; Flórez-Revuelta, F.: Fusion of Skeletal and Silhouette-based
Features for Human Action Recognition with RGB-D Devices, 3rd Workshop on Consumer Depth
Cameras for Computer Vision (CDC4CV), ICCV 2013 Workshop, Sydney (Australia), 2013.
Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F.: Silhouette-based Human Action Recognition
using Sequences of Key Poses, Pattern Recognition Letters, 34(15):1799–1807, 2013.
Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F.: An Efficient Approach for Multi-view Human
Action Recognition Based on Bag-of-Key-Poses, Lecture Notes in Computer Science, 7559:29-40,
2012.
Chaaraoui, A.A.; Climent-Pérez, P.; Flórez-Revuelta, F., A review on vision techniques applied to Human
Behaviour Analysis for Ambient-Assisted Living. Expert Systems with Applications, 39(12):
10873-10888, 2012.

30. Papers
Evolutionary optimisation
Chaaraoui, A.A.; Padilla-López, J.R.; Climent-Pérez, P.; Flórez-Revuelta, F.: Evolutionary
joint selection to improve human action recognition with RGB-D devices, Expert Systems
with Applications, 41(3):786-794, 2014.
Chaaraoui, A.A.; Flórez-Revuelta, F.: Optimizing human action recognition based on a
cooperative coevolutionary algorithm, Engineering Applications of Artificial Intelligence,
Volume 31:116–125, 2014.
Chaaraoui, A.A.; Flórez-Revuelta, F.: Human action recognition optimization based on
evolutionary feature subset selection, Proceeding of the 15th Annual Genetic and
Evolutionary Computation Conference, Amsterdam (The Netherlands), 2013.
Climent-Pérez, P.; Chaaraoui, A.A.; Padilla-López, J.R.; Flórez-Revuelta, F.: Optimal
Feature Selection for Skeletal Data from RGB-D Devices using a Genetic Algorithm, 11th
Mexican International Conference on Artificial Intelligence, San Luis Potosí (Mexico), 2012.

31. Papers
Privacy preservation
Padilla-López, J.R.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Visual privacy by context: A level-
based visualisation scheme (submitted to conference)
Padilla-López, J.R.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Visual privacy protection
methods: a survey (submitted to journal)
Padilla-López, J.R.; Flórez-Revuelta, F.; Monekosso, D.N.; Remagnino, P.: The "Good"
Brother: Monitoring People Activity in Private Spaces, International Symposium on
Distributed Computing and Artificial Intelligence (DCAI 2012), Salamanca (Spain), pp.
49-56, 2012.

32. Papers
Home automation and intelligent environments
Ferrández-Pastor, F.J.; García-Chamizo, J.M.; Romacho-Agud, V.; Flórez-Revuelta, F.:
Using the wavelet transform to disaggregate electrical power consumption into the major
end-uses (submitted to journal)
Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; García-Chamizo, J.M.; Flórez-Revuelta, F.: DAI
Virtual Lab: a Virtual Laboratory for Testing Ambient Intelligence Digital Services, V
Congreso Internacional de Diseño, Redes de Investigación y Tecnología para Todos
(DRT4ALL 2013), Madrid (Spain), 2013.
Climent-Pérez, P.; Chaaraoui, A.A.; Flórez-Revuelta, F.: Useful Research Tools for Human
Behaviour Understanding in the Context of Ambient Assisted Living, International
Symposium on Ambient Intelligence (ISAmI 2012), Salamanca (Spain), pp. 201-205, 2012.
Nieto-Hidalgo, M.; Limiñana-Benito, A.; Torregrosa-Jorqués, R.; Padilla-López, J.R.;
Cabo-Díez, M.; Ferrández-Pastor, F.J.; Romacho-Agud, V.; Climent-Pérez, P.; Chaaraoui,
A.; Flórez-Revuelta, F.: DAI Virtual Lab: a virtual lab for the co-design, development and
validation of AmI services and technologies, International Joint Conference on Ambient
Intelligence (AmI-11), Amsterdam (Nederlands), 2011.

33. Francisco (Paco) Flórez-Revuelta
F.Florez@kingston.ac.uk @fflorezrevuelta
www.dtic.ua.es/~florez franciscoflorezrevuelta