SlideShare a Scribd company logo
1 of 36
1
@orestibanos
Oresti Banos
October 9, 2020
oresti@ugr.es
@orestibanos
http://orestibanos.com/
Measuring human behaviour
by sensing everyday mobile
interactions
International Doctoral Summer School
in Conversational Systems for Mental
e-health
(ONLINE)
https://menhir-project.eu/index.php/summer-school-home/
2
@orestibanos
PRESENTATION
DO WE KNOW EACH OTHER?
Oresti Banos
Research Center for Information and
Communication Technologies
University of Granada
oresti@ugr.es
@orestibanos
http://orestibanos.com/
Research:
• smart mobile sensing
• holistic behaviour modelling
• virtual coaching systems
3
@orestibanos
INTRODUCTION
MOBILE PHONE PROSPECTS
Source: https://newsroom.cisco.com/press-release-content?articleId=1741352
4
@orestibanos
INTRODUCTION
SMARTPHONE PROSPECTS
Source: https://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/
5
@orestibanos
INTRODUCTION
SMARTPHONE EVOLUTION
2007
6
@orestibanos
INTRODUCTION
SMARTPHONE (EXPECTED) EVOLUTION
Projected Smartphones
Implantable
Smartphones
Wearable Smartphones
Foldable Smartphones
7
@orestibanos
SMARTPHONE AS A SENSOR (HUB)
HARDWARE SENSORS
accelerometer, gyro, proximity, compass,
barometer, heart rate, SpO2, humidity,
fingerprint, NFC, GPS, camera, mic, iris
scanner,...
8
@orestibanos
SMARTPHONE AS A SENSOR (HUB)
SOFTWARE SENSORS
phone call logs, sms logs, application usage,
battery, screen status, ...
9
@orestibanos
EVERYDAY (MOBILE) INTERACTIONS
“THE SILENT OBSERVERS”
Everyday “omnipresent”
companions/observers
10
@orestibanos
EVERYDAY (MOBILE) INTERACTIONS
“THE SILENT OBSERVERS”
(Deloitte, Global Mobile Survey 2016)
11
@orestibanos
EVERYDAY (MOBILE) INTERACTIONS
“THE SILENT OBSERVERS”
Explicit interactions Implicit interactions
12
@orestibanos
EVERYDAY (MOBILE) INTERACTIONS
“THE SILENT OBSERVERS”
Observe & Measure Human Behavior
In the wild – Naturalistic Sensing
Large groups – Crowd Sensing
Multiple dimensions – Holistic Sensing
13
@orestibanos
MEASURING HUMAN BEHAVIOUR
PHYSICAL ACTIVITY
(Nature, Althoff et al. 2017)
(Sensors, Hur et al. 2017)
(CHI, Min et al. 2014)
(Neurocomp., Reyes et al. 2016)
(Procedia Comp. Sci., Bayat et al. 2014)
14
@orestibanos
MEASURING HUMAN BEHAVIOUR
SOCIAL ACTIVITY
(Mobile Netw. Appl., Lane et al. 2014)
(Perv. & Mob. Comp., Vu et al. 2015)
15
@orestibanos
MEASURING HUMAN BEHAVIOUR
EMOTIONAL ACTIVITY
(Ubicomp, Pielot et al. 2015)
(MobileHCI, Gosh et al. 2017)(Sensors, Bailon et al. 2019)
16
@orestibanos
MEASURING HUMAN BEHAVIOUR
COGNITIVE ACTIVITY
(J. Amb. Intel. & Hum. Comp., Wohlfahrt-Laymann et al. 2019)
(Ubicomp, Abdullah et al. 2016)
Local time
Body time
17
@orestibanos
MEASURING HUMAN BEHAVIOUR
MISCELLANEOUS
(Intl. J. Hum.-Comp. Studies, Bevan & Stanton 2016)
Anthropological sensingMedical-selfies
Dietary
(BMJ, Ray et al. 2015)
(J. Diabetes Sci. and Tech., Zhang et al. 2015)
18
@orestibanos
MOBILE (BEHAVIOUR SENSING) FRAMEWORKS
(FOR) DATA COLLECTION
(Frontiers in ICT, Ferreira et al. 2015)
(Frontiers in Psychology, Piwek et al. 2016)
19
@orestibanos
MOBILE (BEHAVIOUR SENSING) FRAMEWORKS
(FOR) DATA COLLECTION
(Int. J. Distr. Sens. Netw., Felix et al. 2019)
20
@orestibanos
MOBILE (BEHAVIOUR SENSING) FRAMEWORKS
MHEALTHDROID
(Biomedical Engineering Online, Banos et al. 2015)
https://github.com/mHealthTechnologies/mHealthDroid
https://www.youtube.com/watch?v=AMdxw4osjCU
21
@orestibanos
MOBILE (BEHAVIOUR SENSING) FRAMEWORKS
INCENSE
(Int. J. Distr. Sens. Netw., Felix et al. 2019)
Conventional approach Component-based approach
22
@orestibanos
MOBILE (BEHAVIOUR SENSING) FRAMEWORKS
INCENSE
(Int. J. Distr. Sens. Netw., Felix et al. 2019)
Component-based approach
23
@orestibanos
MOBILE (BEHAVIOUR SENSING) FRAMEWORKS
AWARE
http://www.awareframework.com/
24
@orestibanos
AWARE
INFRASTRUCTURE
Platform
Sensing
Infrastructure
Accelerometer
Applications
Barometer
Bluetooth
Communicatio
n
ESM
Light
Gyroscope Magnetometer
Locations
Telephony Scheduler Temperature
Screen Text-2-Speech
Proximity Network Apps
+ PLUGINS/ PROBES!
Battery
AWARE
25
@orestibanos
AWARE
SETTING A DATA COLLECTION CAMPAIGN
Platform
Sensing
Infrastructure
AWARE
(1) Create new sensing campaign
(3) Users join the campaign
(informed consent)
(4) Digital data is collected,
uploaded and persisted into
the platform
(2) Select sensors of
interest (and/or plugins)
26
@orestibanos
AWARE
EXPERT VIEW: SERVER DASHBOARD
27
@orestibanos
AWARE
USER VIEW: CLIENT APP
28
@orestibanos
AWARE
SENSOR DATA (MODEL)
Timestamp (UNIX format)
Friday, 8 December 2017
12:30:34.877 (GMT+01:00)
Sensor(s)
Device_ID (User_ID)
Universally Unique Identifier (full anonymization)
Sensor values
Acceleration X-axis, Acceleration Y-axis, Acceleration Z-axis,
29
@orestibanos
AWARE
SENSOR DATA (EXAMPLES)
Bluetooth
Screen usage
Physical
Activity
30
@orestibanos
CHALLENGES
OPEN ISSUES
31
@orestibanos
CONCLUSIONS
OPEN ISSUES
Smartphones are possibly the richest devices in terms of sensing,
communication and processing capabilities
Smartphones provide an enormous set of opportunities to observe and
measure behavior unobtrusively, in the wild, at large-scale and in a
holistic fashion
Mobile sensor frameworks facilitate the realisation of experiments and
collection of multiple data types to measure behaviour holistically
32
@orestibanos
Oresti Baños
Room 26 (2nd floor), Faculty
ETSIIT, University of Granada,
E-18071 Granada, Spain
Phone
(+34) 958248598
Email / Web
oresti@ugr.es
http://orestibanos.com/
MANY THANKS!
CONTACT:
33
@orestibanos
REFERENCES
Abdullah, S., Murnane, E.L., Matthews, M., Kay, M., Kientz, J.A., Gay, G. and Choudhury, T., 2016, September.
Cognitive rhythms: Unobtrusive and continuous sensing of alertness using a mobile phone. In Proceedings of the
2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 178-189). ACM.
Althoff, T., Hicks, J.L., King, A.C., Delp, S.L. and Leskovec, J., 2017. Large-scale physical activity data reveal
worldwide activity inequality. Nature, 547(7663), p.336.
Bailon, C., Damas, M., Pomares, H., Sanabria, D., Perakakis, P., Goicoechea, C. and Banos, O., 2019. Smartphone-
Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods. Sensors, 19(15),
pp.1-23.
Banos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J.A., Lee, S., Pomares, H. and Rojas, I.,
2015. Design, implementation and validation of a novel open framework for agile development of mobile health
applications. Biomedical engineering online, 14(2), p.S6.
Bayat, A., Pomplun, M. and Tran, D.A., 2014. A study on human activity recognition using accelerometer data
from smartphones. Procedia Computer Science, 34, pp.450-457.
Bevan, C. and Fraser, D.S., 2016. Different strokes for different folks? Revealing the physical characteristics of
smartphone users from their swipe gestures. International Journal of Human-Computer Studies, 88, pp.51-61.
Felix, I.R., Castro, L.A., Rodriguez, L.F. and Banos, O., 2019. Mobile sensing for behavioral research: A component-
based approach for rapid deployment of sensing campaigns. International Journal of Distributed Sensor
Networks, 15(9), pp. 1-17
34
@orestibanos
REFERENCES
Ferreira, D., Kostakos, V. and Dey, A.K., 2015. AWARE: mobile context instrumentation framework. Frontiers in
ICT, 2, p.6.
Ghosh, S., Ganguly, N., Mitra, B. and De, P., 2017, September. TapSense: combining self-report patterns and
typing characteristics for smartphone based emotion detection. In Proceedings of the 19th International
Conference on Human-Computer Interaction with Mobile Devices and Services (p. 2). ACM.
Hur, T., Bang, J., Kim, D., Banos, O. and Lee, S., 2017. Smartphone location-independent physical activity
recognition based on transportation natural vibration analysis. Sensors, 17(4), p.931.
Lane, N.D., Lin, M., Mohammod, M., Yang, X., Lu, H., Cardone, G., Ali, S., Doryab, A., Berke, E., Campbell, A.T. and
Choudhury, T., 2014. Bewell: Sensing sleep, physical activities and social interactions to promote
wellbeing. Mobile Networks and Applications, 19(3), pp.345-359.
Madakam, S., Ramaswamy, R. and Tripathi, S., 2015. Internet of Things (IoT): A literature review. Journal of
Computer and Communications, 3(05), p.164.
Mehrotra, A., Tsapeli, F., Hendley, R. and Musolesi, M., 2017. MyTraces: investigating correlation and causation
between users’ emotional states and mobile phone interaction. Proceedings of the ACM on Interactive, Mobile,
Wearable and Ubiquitous Technologies, 1(3), p.83.
Min, J.K., Doryab, A., Wiese, J., Amini, S., Zimmerman, J. and Hong, J.I., 2014, April. Toss'n'turn: smartphone as
sleep and sleep quality detector. In Proceedings of the SIGCHI conference on human factors in computing
systems (pp. 477-486). ACM.
35
@orestibanos
REFERENCES
Pielot, M., Dingler, T., Pedro, J.S. and Oliver, N., 2015, September. When attention is not scarce-detecting
boredom from mobile phone usage. In Proceedings of the 2015 ACM international joint conference on pervasive
and ubiquitous computing (pp. 825-836). ACM.
Piwek, L., Ellis, D.A. and Sally, A., 2016. Can programming frameworks bring smartphones into the mainstream
of psychological science?. Frontiers in psychology, 7, p.1252.
Rachuri, K.K., Musolesi, M., Mascolo, C., Rentfrow, P.J., Longworth, C. and Aucinas, A., 2010, September.
EmotionSense: a mobile phones based adaptive platform for experimental social psychology research.
In Proceedings of the 12th ACM international conference on Ubiquitous computing (pp. 281-290). ACM.
Ray, A., Scott, A.D., Nikkhah, D. and Dheansa, B.S., 2015. The medical selfie. BMJ, 351, p.h3145.
Reyes-Ortiz, J.L., Oneto, L., Samà, A., Parra, X. and Anguita, D., 2016. Transition-aware human activity
recognition using smartphones. Neurocomputing, 171, pp.754-767.
Vu, L., Nguyen, P., Nahrstedt, K. and Richerzhagen, B., 2015. Characterizing and modeling people movement
from mobile phone sensing traces. Pervasive and Mobile Computing, 17, pp.220-235.
Wohlfahrt-Laymann, J., Hermens, H., Villalonga, C., Vollenbroek-Hutten, M., Banos, O., 2019.
MobileCogniTracker - A mobile experience sampling tool for tracking cognitive behaviour. Journal of Ambient
Intelligence and Humanized Computing, vol. 10, no. 6, pp. 2143-2160
36
@orestibanos
REFERENCES
Zhang, W., Yu, Q., Siddiquie, B., Divakaran, A. and Sawhney, H., 2015. “Snap-n-Eat” Food Recognition and
Nutrition Estimation on a Smartphone. Journal of diabetes science and technology, 9(3), pp.525-533.

More Related Content

Similar to Measuring human behaviour by sensing everyday mobile interactions

ARTIFICIAL INTELLIGENCE in Urban Planning.docx
ARTIFICIAL INTELLIGENCE in Urban Planning.docxARTIFICIAL INTELLIGENCE in Urban Planning.docx
ARTIFICIAL INTELLIGENCE in Urban Planning.docx
Ngoc Tuyen
 
Internet of things iot based real time gas leakage monitoring and controlling
Internet of things iot based real time gas leakage monitoring and controllingInternet of things iot based real time gas leakage monitoring and controlling
Internet of things iot based real time gas leakage monitoring and controlling
IAEME Publication
 
Smart Data - How you and I will exploit Big Data for personalized digital hea...
Smart Data - How you and I will exploit Big Data for personalized digital hea...Smart Data - How you and I will exploit Big Data for personalized digital hea...
Smart Data - How you and I will exploit Big Data for personalized digital hea...
Amit Sheth
 
The social aspect of Smart Wearable Systems in the era of Internet-of-Things
The social aspect of Smart Wearable Systems in the era of Internet-of-ThingsThe social aspect of Smart Wearable Systems in the era of Internet-of-Things
The social aspect of Smart Wearable Systems in the era of Internet-of-Things
Anax Fotopoulos
 
Future of Wearable Tech PSK
Future of Wearable Tech PSKFuture of Wearable Tech PSK
Future of Wearable Tech PSK
Josh Trent
 
Research Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docx
Research Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docxResearch Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docx
Research Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docx
ronak56
 

Similar to Measuring human behaviour by sensing everyday mobile interactions (20)

Top 5 most viewed articles from academia in 2019 -
Top 5 most viewed articles from academia in 2019 - Top 5 most viewed articles from academia in 2019 -
Top 5 most viewed articles from academia in 2019 -
 
ARTIFICIAL INTELLIGENCE in Urban Planning.docx
ARTIFICIAL INTELLIGENCE in Urban Planning.docxARTIFICIAL INTELLIGENCE in Urban Planning.docx
ARTIFICIAL INTELLIGENCE in Urban Planning.docx
 
IoT : Peluang Riset di Bidang Kesehatan
IoT : Peluang Riset di Bidang KesehatanIoT : Peluang Riset di Bidang Kesehatan
IoT : Peluang Riset di Bidang Kesehatan
 
Smart homes
Smart homesSmart homes
Smart homes
 
Internet of things iot based real time gas leakage monitoring and controlling
Internet of things iot based real time gas leakage monitoring and controllingInternet of things iot based real time gas leakage monitoring and controlling
Internet of things iot based real time gas leakage monitoring and controlling
 
Internet of things iot based real time gas leakage monitoring and controlling
Internet of things iot based real time gas leakage monitoring and controllingInternet of things iot based real time gas leakage monitoring and controlling
Internet of things iot based real time gas leakage monitoring and controlling
 
Keynote speech - Webmedia 2020
Keynote speech - Webmedia 2020Keynote speech - Webmedia 2020
Keynote speech - Webmedia 2020
 
IRJET- App Misbehaviour Check: Development of Virus Modeling, Propagation...
IRJET-  	  App Misbehaviour Check: Development of Virus Modeling, Propagation...IRJET-  	  App Misbehaviour Check: Development of Virus Modeling, Propagation...
IRJET- App Misbehaviour Check: Development of Virus Modeling, Propagation...
 
Trends in covolutional neural network in 2020 - International Journal of Arti...
Trends in covolutional neural network in 2020 - International Journal of Arti...Trends in covolutional neural network in 2020 - International Journal of Arti...
Trends in covolutional neural network in 2020 - International Journal of Arti...
 
Internet of Things
Internet of ThingsInternet of Things
Internet of Things
 
A43050104
A43050104A43050104
A43050104
 
IoT
IoTIoT
IoT
 
A Short Literature Review On The Internet Of Things Research And Development...
A Short Literature Review On The Internet Of Things  Research And Development...A Short Literature Review On The Internet Of Things  Research And Development...
A Short Literature Review On The Internet Of Things Research And Development...
 
Smart Data - How you and I will exploit Big Data for personalized digital hea...
Smart Data - How you and I will exploit Big Data for personalized digital hea...Smart Data - How you and I will exploit Big Data for personalized digital hea...
Smart Data - How you and I will exploit Big Data for personalized digital hea...
 
The social aspect of Smart Wearable Systems in the era of Internet-of-Things
The social aspect of Smart Wearable Systems in the era of Internet-of-ThingsThe social aspect of Smart Wearable Systems in the era of Internet-of-Things
The social aspect of Smart Wearable Systems in the era of Internet-of-Things
 
PSFK Future Of Wearable Tech Report
PSFK Future Of Wearable Tech ReportPSFK Future Of Wearable Tech Report
PSFK Future Of Wearable Tech Report
 
Future of Wearable Tech PSK
Future of Wearable Tech PSKFuture of Wearable Tech PSK
Future of Wearable Tech PSK
 
Research Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docx
Research Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docxResearch Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docx
Research Paper OutlineI. INTRODUCTIONa. Exploring mobile app.docx
 
April 2023-Top Cited Articles in International Journal of Ubiquitous Computin...
April 2023-Top Cited Articles in International Journal of Ubiquitous Computin...April 2023-Top Cited Articles in International Journal of Ubiquitous Computin...
April 2023-Top Cited Articles in International Journal of Ubiquitous Computin...
 
The Future Of Wearable Technology 2014
The Future Of Wearable Technology 2014The Future Of Wearable Technology 2014
The Future Of Wearable Technology 2014
 

More from Oresti Banos

Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...
Oresti Banos
 
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient MonitoringAnalysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
Oresti Banos
 
Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...
Oresti Banos
 
Diseño e implementación de técnicas de monitorización indoor en e-salud
Diseño e implementación de técnicas de monitorización indoor en e-saludDiseño e implementación de técnicas de monitorización indoor en e-salud
Diseño e implementación de técnicas de monitorización indoor en e-salud
Oresti Banos
 
Reconocimiento automático de la actividad física diaria aplicado a contextos ...
Reconocimiento automático de la actividad física diaria aplicado a contextos ...Reconocimiento automático de la actividad física diaria aplicado a contextos ...
Reconocimiento automático de la actividad física diaria aplicado a contextos ...
Oresti Banos
 
Handling displacement effects in on-body sensor-based activity recognition
Handling displacement effects in on-body sensor-based activity recognitionHandling displacement effects in on-body sensor-based activity recognition
Handling displacement effects in on-body sensor-based activity recognition
Oresti Banos
 
Activity recognition based on a multi-sensor meta-classifier
Activity recognition based on a multi-sensor meta-classifierActivity recognition based on a multi-sensor meta-classifier
Activity recognition based on a multi-sensor meta-classifier
Oresti Banos
 

More from Oresti Banos (20)

Biodata analysis
Biodata analysisBiodata analysis
Biodata analysis
 
Biosignal Processing
Biosignal ProcessingBiosignal Processing
Biosignal Processing
 
Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...Automatic mapping of motivational text messages into ontological entities for...
Automatic mapping of motivational text messages into ontological entities for...
 
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...
 
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient MonitoringAnalysis of the Innovation Outputs in mHealth for Patient Monitoring
Analysis of the Innovation Outputs in mHealth for Patient Monitoring
 
First Approach to Automatic Performance Status Evaluation and Physical Activi...
First Approach to Automatic Performance Status Evaluation and Physical Activi...First Approach to Automatic Performance Status Evaluation and Physical Activi...
First Approach to Automatic Performance Status Evaluation and Physical Activi...
 
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
First Approach to Automatic Measurement of Frontal Plane Projection Angle Dur...
 
High-Level Context Inference for Human Behavior Identi cation
High-Level Context Inference for Human Behavior IdenticationHigh-Level Context Inference for Human Behavior Identication
High-Level Context Inference for Human Behavior Identi cation
 
On the Development of A Real-Time Multi-Sensor Activity Recognition System
On the Development of A Real-Time Multi-Sensor Activity Recognition SystemOn the Development of A Real-Time Multi-Sensor Activity Recognition System
On the Development of A Real-Time Multi-Sensor Activity Recognition System
 
Facilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Facilitating Trunk Endurance Assessment by means of Mobile Health TechnologiesFacilitating Trunk Endurance Assessment by means of Mobile Health Technologies
Facilitating Trunk Endurance Assessment by means of Mobile Health Technologies
 
Mining Human Behavior for Health Promotion
Mining Human Behavior for Health PromotionMining Human Behavior for Health Promotion
Mining Human Behavior for Health Promotion
 
Multiwindow Fusion for Wearable Activity Recognition
Multiwindow Fusion for Wearable Activity RecognitionMultiwindow Fusion for Wearable Activity Recognition
Multiwindow Fusion for Wearable Activity Recognition
 
Mining Minds: an innovative framework for personalized health and wellness su...
Mining Minds: an innovative framework for personalized health and wellness su...Mining Minds: an innovative framework for personalized health and wellness su...
Mining Minds: an innovative framework for personalized health and wellness su...
 
A Novel Watermarking Scheme for Image Authentication in Social Networks
A Novel Watermarking Scheme for Image Authentication in Social NetworksA Novel Watermarking Scheme for Image Authentication in Social Networks
A Novel Watermarking Scheme for Image Authentication in Social Networks
 
mHealthDroid: a novel framework for agile development of mobile health appli...
mHealthDroid: a novel framework for agile development of mobile health appli...mHealthDroid: a novel framework for agile development of mobile health appli...
mHealthDroid: a novel framework for agile development of mobile health appli...
 
Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...Sistema automático para la estimación de la presión arterial a partir de pará...
Sistema automático para la estimación de la presión arterial a partir de pará...
 
Diseño e implementación de técnicas de monitorización indoor en e-salud
Diseño e implementación de técnicas de monitorización indoor en e-saludDiseño e implementación de técnicas de monitorización indoor en e-salud
Diseño e implementación de técnicas de monitorización indoor en e-salud
 
Reconocimiento automático de la actividad física diaria aplicado a contextos ...
Reconocimiento automático de la actividad física diaria aplicado a contextos ...Reconocimiento automático de la actividad física diaria aplicado a contextos ...
Reconocimiento automático de la actividad física diaria aplicado a contextos ...
 
Handling displacement effects in on-body sensor-based activity recognition
Handling displacement effects in on-body sensor-based activity recognitionHandling displacement effects in on-body sensor-based activity recognition
Handling displacement effects in on-body sensor-based activity recognition
 
Activity recognition based on a multi-sensor meta-classifier
Activity recognition based on a multi-sensor meta-classifierActivity recognition based on a multi-sensor meta-classifier
Activity recognition based on a multi-sensor meta-classifier
 

Recently uploaded

Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.
Cherry
 
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
US Environmental Protection Agency (EPA), Center for Computational Toxicology and Exposure
 
Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...
Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...
Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...
Cherry
 
Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.
Cherry
 
(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...
(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...
(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...
Scintica Instrumentation
 
PODOCARPUS...........................pptx
PODOCARPUS...........................pptxPODOCARPUS...........................pptx
PODOCARPUS...........................pptx
Cherry
 
Pteris : features, anatomy, morphology and lifecycle
Pteris : features, anatomy, morphology and lifecyclePteris : features, anatomy, morphology and lifecycle
Pteris : features, anatomy, morphology and lifecycle
Cherry
 

Recently uploaded (20)

Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.
 
Genome organization in virus,bacteria and eukaryotes.pptx
Genome organization in virus,bacteria and eukaryotes.pptxGenome organization in virus,bacteria and eukaryotes.pptx
Genome organization in virus,bacteria and eukaryotes.pptx
 
Efficient spin-up of Earth System Models usingsequence acceleration
Efficient spin-up of Earth System Models usingsequence accelerationEfficient spin-up of Earth System Models usingsequence acceleration
Efficient spin-up of Earth System Models usingsequence acceleration
 
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
Chemistry Data Delivery from the US-EPA Center for Computational Toxicology a...
 
CONTRIBUTION OF PANCHANAN MAHESHWARI.pptx
CONTRIBUTION OF PANCHANAN MAHESHWARI.pptxCONTRIBUTION OF PANCHANAN MAHESHWARI.pptx
CONTRIBUTION OF PANCHANAN MAHESHWARI.pptx
 
Sequence submission tools ............pptx
Sequence submission tools ............pptxSequence submission tools ............pptx
Sequence submission tools ............pptx
 
Molecular phylogeny, molecular clock hypothesis, molecular evolution, kimuras...
Molecular phylogeny, molecular clock hypothesis, molecular evolution, kimuras...Molecular phylogeny, molecular clock hypothesis, molecular evolution, kimuras...
Molecular phylogeny, molecular clock hypothesis, molecular evolution, kimuras...
 
EU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdfEU START PROJECT. START-Newsletter_Issue_4.pdf
EU START PROJECT. START-Newsletter_Issue_4.pdf
 
Site specific recombination and transposition.........pdf
Site specific recombination and transposition.........pdfSite specific recombination and transposition.........pdf
Site specific recombination and transposition.........pdf
 
Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...
Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...
Major groups of bacteria: Spirochetes, Chlamydia, Rickettsia, nanobes, mycopl...
 
Adaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte CarloAdaptive Restore algorithm & importance Monte Carlo
Adaptive Restore algorithm & importance Monte Carlo
 
Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.
 
Terpineol and it's characterization pptx
Terpineol and it's characterization pptxTerpineol and it's characterization pptx
Terpineol and it's characterization pptx
 
Genome sequencing,shotgun sequencing.pptx
Genome sequencing,shotgun sequencing.pptxGenome sequencing,shotgun sequencing.pptx
Genome sequencing,shotgun sequencing.pptx
 
Cyanide resistant respiration pathway.pptx
Cyanide resistant respiration pathway.pptxCyanide resistant respiration pathway.pptx
Cyanide resistant respiration pathway.pptx
 
(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...
(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...
(May 9, 2024) Enhanced Ultrafast Vector Flow Imaging (VFI) Using Multi-Angle ...
 
GBSN - Microbiology (Unit 4) Concept of Asepsis
GBSN - Microbiology (Unit 4) Concept of AsepsisGBSN - Microbiology (Unit 4) Concept of Asepsis
GBSN - Microbiology (Unit 4) Concept of Asepsis
 
PODOCARPUS...........................pptx
PODOCARPUS...........................pptxPODOCARPUS...........................pptx
PODOCARPUS...........................pptx
 
Pteris : features, anatomy, morphology and lifecycle
Pteris : features, anatomy, morphology and lifecyclePteris : features, anatomy, morphology and lifecycle
Pteris : features, anatomy, morphology and lifecycle
 
Taphonomy and Quality of the Fossil Record
Taphonomy and Quality of the  Fossil RecordTaphonomy and Quality of the  Fossil Record
Taphonomy and Quality of the Fossil Record
 

Measuring human behaviour by sensing everyday mobile interactions

  • 1. 1 @orestibanos Oresti Banos October 9, 2020 oresti@ugr.es @orestibanos http://orestibanos.com/ Measuring human behaviour by sensing everyday mobile interactions International Doctoral Summer School in Conversational Systems for Mental e-health (ONLINE) https://menhir-project.eu/index.php/summer-school-home/
  • 2. 2 @orestibanos PRESENTATION DO WE KNOW EACH OTHER? Oresti Banos Research Center for Information and Communication Technologies University of Granada oresti@ugr.es @orestibanos http://orestibanos.com/ Research: • smart mobile sensing • holistic behaviour modelling • virtual coaching systems
  • 3. 3 @orestibanos INTRODUCTION MOBILE PHONE PROSPECTS Source: https://newsroom.cisco.com/press-release-content?articleId=1741352
  • 6. 6 @orestibanos INTRODUCTION SMARTPHONE (EXPECTED) EVOLUTION Projected Smartphones Implantable Smartphones Wearable Smartphones Foldable Smartphones
  • 7. 7 @orestibanos SMARTPHONE AS A SENSOR (HUB) HARDWARE SENSORS accelerometer, gyro, proximity, compass, barometer, heart rate, SpO2, humidity, fingerprint, NFC, GPS, camera, mic, iris scanner,...
  • 8. 8 @orestibanos SMARTPHONE AS A SENSOR (HUB) SOFTWARE SENSORS phone call logs, sms logs, application usage, battery, screen status, ...
  • 9. 9 @orestibanos EVERYDAY (MOBILE) INTERACTIONS “THE SILENT OBSERVERS” Everyday “omnipresent” companions/observers
  • 10. 10 @orestibanos EVERYDAY (MOBILE) INTERACTIONS “THE SILENT OBSERVERS” (Deloitte, Global Mobile Survey 2016)
  • 11. 11 @orestibanos EVERYDAY (MOBILE) INTERACTIONS “THE SILENT OBSERVERS” Explicit interactions Implicit interactions
  • 12. 12 @orestibanos EVERYDAY (MOBILE) INTERACTIONS “THE SILENT OBSERVERS” Observe & Measure Human Behavior In the wild – Naturalistic Sensing Large groups – Crowd Sensing Multiple dimensions – Holistic Sensing
  • 13. 13 @orestibanos MEASURING HUMAN BEHAVIOUR PHYSICAL ACTIVITY (Nature, Althoff et al. 2017) (Sensors, Hur et al. 2017) (CHI, Min et al. 2014) (Neurocomp., Reyes et al. 2016) (Procedia Comp. Sci., Bayat et al. 2014)
  • 14. 14 @orestibanos MEASURING HUMAN BEHAVIOUR SOCIAL ACTIVITY (Mobile Netw. Appl., Lane et al. 2014) (Perv. & Mob. Comp., Vu et al. 2015)
  • 15. 15 @orestibanos MEASURING HUMAN BEHAVIOUR EMOTIONAL ACTIVITY (Ubicomp, Pielot et al. 2015) (MobileHCI, Gosh et al. 2017)(Sensors, Bailon et al. 2019)
  • 16. 16 @orestibanos MEASURING HUMAN BEHAVIOUR COGNITIVE ACTIVITY (J. Amb. Intel. & Hum. Comp., Wohlfahrt-Laymann et al. 2019) (Ubicomp, Abdullah et al. 2016) Local time Body time
  • 17. 17 @orestibanos MEASURING HUMAN BEHAVIOUR MISCELLANEOUS (Intl. J. Hum.-Comp. Studies, Bevan & Stanton 2016) Anthropological sensingMedical-selfies Dietary (BMJ, Ray et al. 2015) (J. Diabetes Sci. and Tech., Zhang et al. 2015)
  • 18. 18 @orestibanos MOBILE (BEHAVIOUR SENSING) FRAMEWORKS (FOR) DATA COLLECTION (Frontiers in ICT, Ferreira et al. 2015) (Frontiers in Psychology, Piwek et al. 2016)
  • 19. 19 @orestibanos MOBILE (BEHAVIOUR SENSING) FRAMEWORKS (FOR) DATA COLLECTION (Int. J. Distr. Sens. Netw., Felix et al. 2019)
  • 20. 20 @orestibanos MOBILE (BEHAVIOUR SENSING) FRAMEWORKS MHEALTHDROID (Biomedical Engineering Online, Banos et al. 2015) https://github.com/mHealthTechnologies/mHealthDroid https://www.youtube.com/watch?v=AMdxw4osjCU
  • 21. 21 @orestibanos MOBILE (BEHAVIOUR SENSING) FRAMEWORKS INCENSE (Int. J. Distr. Sens. Netw., Felix et al. 2019) Conventional approach Component-based approach
  • 22. 22 @orestibanos MOBILE (BEHAVIOUR SENSING) FRAMEWORKS INCENSE (Int. J. Distr. Sens. Netw., Felix et al. 2019) Component-based approach
  • 23. 23 @orestibanos MOBILE (BEHAVIOUR SENSING) FRAMEWORKS AWARE http://www.awareframework.com/
  • 25. 25 @orestibanos AWARE SETTING A DATA COLLECTION CAMPAIGN Platform Sensing Infrastructure AWARE (1) Create new sensing campaign (3) Users join the campaign (informed consent) (4) Digital data is collected, uploaded and persisted into the platform (2) Select sensors of interest (and/or plugins)
  • 28. 28 @orestibanos AWARE SENSOR DATA (MODEL) Timestamp (UNIX format) Friday, 8 December 2017 12:30:34.877 (GMT+01:00) Sensor(s) Device_ID (User_ID) Universally Unique Identifier (full anonymization) Sensor values Acceleration X-axis, Acceleration Y-axis, Acceleration Z-axis,
  • 31. 31 @orestibanos CONCLUSIONS OPEN ISSUES Smartphones are possibly the richest devices in terms of sensing, communication and processing capabilities Smartphones provide an enormous set of opportunities to observe and measure behavior unobtrusively, in the wild, at large-scale and in a holistic fashion Mobile sensor frameworks facilitate the realisation of experiments and collection of multiple data types to measure behaviour holistically
  • 32. 32 @orestibanos Oresti Baños Room 26 (2nd floor), Faculty ETSIIT, University of Granada, E-18071 Granada, Spain Phone (+34) 958248598 Email / Web oresti@ugr.es http://orestibanos.com/ MANY THANKS! CONTACT:
  • 33. 33 @orestibanos REFERENCES Abdullah, S., Murnane, E.L., Matthews, M., Kay, M., Kientz, J.A., Gay, G. and Choudhury, T., 2016, September. Cognitive rhythms: Unobtrusive and continuous sensing of alertness using a mobile phone. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 178-189). ACM. Althoff, T., Hicks, J.L., King, A.C., Delp, S.L. and Leskovec, J., 2017. Large-scale physical activity data reveal worldwide activity inequality. Nature, 547(7663), p.336. Bailon, C., Damas, M., Pomares, H., Sanabria, D., Perakakis, P., Goicoechea, C. and Banos, O., 2019. Smartphone- Based Platform for Affect Monitoring through Flexibly Managed Experience Sampling Methods. Sensors, 19(15), pp.1-23. Banos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J.A., Lee, S., Pomares, H. and Rojas, I., 2015. Design, implementation and validation of a novel open framework for agile development of mobile health applications. Biomedical engineering online, 14(2), p.S6. Bayat, A., Pomplun, M. and Tran, D.A., 2014. A study on human activity recognition using accelerometer data from smartphones. Procedia Computer Science, 34, pp.450-457. Bevan, C. and Fraser, D.S., 2016. Different strokes for different folks? Revealing the physical characteristics of smartphone users from their swipe gestures. International Journal of Human-Computer Studies, 88, pp.51-61. Felix, I.R., Castro, L.A., Rodriguez, L.F. and Banos, O., 2019. Mobile sensing for behavioral research: A component- based approach for rapid deployment of sensing campaigns. International Journal of Distributed Sensor Networks, 15(9), pp. 1-17
  • 34. 34 @orestibanos REFERENCES Ferreira, D., Kostakos, V. and Dey, A.K., 2015. AWARE: mobile context instrumentation framework. Frontiers in ICT, 2, p.6. Ghosh, S., Ganguly, N., Mitra, B. and De, P., 2017, September. TapSense: combining self-report patterns and typing characteristics for smartphone based emotion detection. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (p. 2). ACM. Hur, T., Bang, J., Kim, D., Banos, O. and Lee, S., 2017. Smartphone location-independent physical activity recognition based on transportation natural vibration analysis. Sensors, 17(4), p.931. Lane, N.D., Lin, M., Mohammod, M., Yang, X., Lu, H., Cardone, G., Ali, S., Doryab, A., Berke, E., Campbell, A.T. and Choudhury, T., 2014. Bewell: Sensing sleep, physical activities and social interactions to promote wellbeing. Mobile Networks and Applications, 19(3), pp.345-359. Madakam, S., Ramaswamy, R. and Tripathi, S., 2015. Internet of Things (IoT): A literature review. Journal of Computer and Communications, 3(05), p.164. Mehrotra, A., Tsapeli, F., Hendley, R. and Musolesi, M., 2017. MyTraces: investigating correlation and causation between users’ emotional states and mobile phone interaction. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(3), p.83. Min, J.K., Doryab, A., Wiese, J., Amini, S., Zimmerman, J. and Hong, J.I., 2014, April. Toss'n'turn: smartphone as sleep and sleep quality detector. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 477-486). ACM.
  • 35. 35 @orestibanos REFERENCES Pielot, M., Dingler, T., Pedro, J.S. and Oliver, N., 2015, September. When attention is not scarce-detecting boredom from mobile phone usage. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing (pp. 825-836). ACM. Piwek, L., Ellis, D.A. and Sally, A., 2016. Can programming frameworks bring smartphones into the mainstream of psychological science?. Frontiers in psychology, 7, p.1252. Rachuri, K.K., Musolesi, M., Mascolo, C., Rentfrow, P.J., Longworth, C. and Aucinas, A., 2010, September. EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In Proceedings of the 12th ACM international conference on Ubiquitous computing (pp. 281-290). ACM. Ray, A., Scott, A.D., Nikkhah, D. and Dheansa, B.S., 2015. The medical selfie. BMJ, 351, p.h3145. Reyes-Ortiz, J.L., Oneto, L., Samà, A., Parra, X. and Anguita, D., 2016. Transition-aware human activity recognition using smartphones. Neurocomputing, 171, pp.754-767. Vu, L., Nguyen, P., Nahrstedt, K. and Richerzhagen, B., 2015. Characterizing and modeling people movement from mobile phone sensing traces. Pervasive and Mobile Computing, 17, pp.220-235. Wohlfahrt-Laymann, J., Hermens, H., Villalonga, C., Vollenbroek-Hutten, M., Banos, O., 2019. MobileCogniTracker - A mobile experience sampling tool for tracking cognitive behaviour. Journal of Ambient Intelligence and Humanized Computing, vol. 10, no. 6, pp. 2143-2160
  • 36. 36 @orestibanos REFERENCES Zhang, W., Yu, Q., Siddiquie, B., Divakaran, A. and Sawhney, H., 2015. “Snap-n-Eat” Food Recognition and Nutrition Estimation on a Smartphone. Journal of diabetes science and technology, 9(3), pp.525-533.