Measuring human behaviour to inform e-coaching actions
Report
Oresti BanosFollow
Oct. 29, 2020•0 likes•256 views
Measuring human behaviour to inform e-coaching actions
Oct. 29, 2020•0 likes•256 views
Report
Science
Having a clear understanding of people’s behaviour is essential to characterise patient progress, make treatment decisions and elicit effective and relevant coaching actions. Hence, a great deal of research has been devoted in recent years to the automatic sensing and analysis of human behaviour. Sensing options are currently unparalleled due to the number of smart, ubiquitous sensor systems developed and deployed globally. Instrumented devices such as smartphones or wearables enable unobtrusive observation and detection of a wide variety of behaviours as we go about our physical and virtual interactions with the world. The vast amount of data generated by such sensing infrastructures can be then analysed by powerful machine-learning algorithms, which map the raw data into predictive trajectories of behaviour. The processed data is combined with computerised behaviour change frameworks and domain knowledge to dynamically generate tailored recommendations and guidelines through advanced reasoning. In view of the above, this keynote explores the recent advances in the automatic sensing and analysis of human behaviour to inform e-coaching actions.
Oresti BanosFollow
Recommended
Measuring human behaviour by sensing everyday mobile interactionsOresti Banos
Enabling remote assessment of cognitive behaviour through mobile experience s...Oresti Banos
Emotion AI: Concepts, Challenges and OpportunitiesOresti Banos
Ontological Modeling of Motivational Messages for Physical Activity CoachingOresti Banos
Rashed ieiceTateJohnson5
A Methodological Variation For Acceptance Evaluation Of Human-Robot Interacti...Heather Strinden
More Related Content
Similar to Measuring human behaviour to inform e-coaching actions
DGI_The_Smart_Object_AgeGianandrea Giacoma
The Smart Object Age - Design Group ItaliaGianandrea Giacoma
From Humanities to Metahumanities: Transhumanism and the Future of Education....eraser Juan José Calderón
Architecture and Agility with Lives at StakeInSTEDD
Agile & Architecture with lives at stake Eduardo Jezierski
How the Internet of Things and People can help improve our health, well-being...Maged N. Kamel Boulos
More from Oresti Banos
Biodata analysisOresti Banos
Biosignal ProcessingOresti Banos
Automatic mapping of motivational text messages into ontological entities for...Oresti Banos
Mobile Health System for Evaluation of Breast Cancer Patients During Treatmen...Oresti Banos
Analysis of the Innovation Outputs in mHealth for Patient MonitoringOresti Banos
First Approach to Automatic Performance Status Evaluation and Physical Activi...Oresti Banos
Recently uploaded
Imaging the Milky Way with Millihertz Gravitational WavesSérgio Sacani
g10science1stquartermodule1-200502160553 (2).pdfKhaoriVladir
Geography of Opportunity - Sansone.pptxnathancone5537
ECONOMIC IMPORTANCE OF BACTERIAshow.ppsxDrSandhya16
Global QCD analysis and dark photonsSérgio Sacani
Discussion of Jost et al. (2019): Input complexity affects long-term retentio...Pablo Bernabeu
Measuring human behaviour to inform e-coaching actions
- 1. 1 @orestibanos Oresti Banos October 29, 2020 oresti@ugr.es @orestibanos http://orestibanos.com/ Measuring human behavior to inform e-coaching actions Workshop on Multimodal e-Coaches ACM Conference on Multimodal Interaction (ICMI 2020) Utrecht, Netherlands
- 2. 2 @orestibanos PRESENTATION ABOUT ME Oresti Banos Research Center for Information and Communication Technologies University of Granada oresti@ugr.es @orestibanos http://orestibanos.com/ Research: • smart mobile sensing • holistic behaviour modelling • virtual coaching systems
- 3. 3 @orestibanos UNHEALTHY LIFESTYLEs A DARK PRESENT (AND FUTURE)
- 4. 4 @orestibanos COACHING WE ARE IN THE NEED OF HELP Multiple theoretical frameworks try to explain why humans break or follow established patterns of action (e.g. Social Cognitive Theory, Goal-Setting Theory, Transtheoretical Model). Transforming these theories into practice is difficult, especially in an era in which human attention span and ability to be self-reflective are diminished. Imperative to find effective new means for raising public awareness and changing behavior in order to improve our ability to remain active, healthy, and resilient (IEEE Computer Magazine, Banos & Nugent. 2018)
- 5. 5 @orestibanos E-COACHING DEALING WITH THE INHERENTLY COMPLEX NATURE OF COACHING HUMANS E-coaching is an emerging computing area in which intelligent systems are used to encourage progress toward specific health-related goals by providing tailored training and guidance. As intelligent systems, e-coaches exhibit abilities similar to those of their human counterparts • sensing (observing, listening, questioning) • learning (acquiring knowledge, understanding) • actuating (reasoning, clarifying, advising) (IEEE Computer Magazine, Banos & Nugent. 2018)
- 6. 6 @orestibanos HOW TO SENSE BEHAVIOUR? IOT AS A SENSOR (Blogspot, Kottolli. 2015)
- 7. 7 @orestibanos HOW TO SENSE BEHAVIOUR? SMARTPHONE AS A (GREAT) SENSOR Sensing Human Behavior In the wild – Naturalistic Sensing Large groups – Crowd Sensing Multiple dimensions – Holistic Sensing
- 8. 8 @orestibanos SENSING HUMAN BEHAVIOUR PHYSICAL ACTIVITY (Nature, Althoff et al. 2017) (Sensors, Hur et al. 2017) (CHI, Min et al. 2014) (Neurocomp., Reyes et al. 2016) (Procedia Comp. Sci., Bayat et al. 2014)
- 9. 9 @orestibanos SENSING HUMAN BEHAVIOUR SOCIAL ACTIVITY (Mobile Netw. Appl., Lane et al. 2014) (Perv. & Mob. Comp., Vu et al. 2015)
- 10. 10 @orestibanos SENSING HUMAN BEHAVIOUR EMOTIONAL ACTIVITY (Ubicomp, Pielot et al. 2015) (MobileHCI, Gosh et al. 2017)(Sensors, Bailon et al. 2019)
- 11. 11 @orestibanos SENSING HUMAN BEHAVIOUR COGNITIVE ACTIVITY (J. Amb. Intel. & Hum. Comp., Wohlfahrt-Laymann et al. 2019) (Ubicomp, Abdullah et al. 2016) Local time Body time
- 12. 12 @orestibanos BEHAVIOUR SENSING FRAMEWORKS MHEALTHDROID (Biomedical Engineering Online, Banos et al. 2015) https://github.com/mHealthTechnologies/mHealthDroid
- 13. 13 @orestibanos BEHAVIOUR SENSING FRAMEWORKS INCENSE https://github.com/incense-platform/incense (Int. J. Distr. Sens. Netw., Felix et al. 2019)
- 14. 14 @orestibanos BEHAVIOUR SENSING FRAMEWORKS MINING MINDS (BioMedical Eng. OnLine, Banos et al. 2016) https://github.com/ubiquitous-computing-lab/Mining-Minds
- 15. 15 @orestibanos BEHAVIOUR SENSING FRAMEWORKS MINING MINDS (Sensors, Banos et al. 2016) https://github.com/ubiquitous-computing-lab/Mining-Minds
- 16. 16 @orestibanos BEHAVIOUR SENSING FRAMEWORKS COUNCIL-OF-COACHES (ICT4AWE, op den Akker et al. 2018) https://github.com/AgentsUnited
- 17. 17 @orestibanos BEHAVIOUR SENSING FRAMEWORKS COUNCIL-OF-COACHES (UCAmI, Konsolakis et al. 2019) https://github.com/AgentsUnited Web Server Sensor Data Apache Ubuntu 14.04 Python 3.5 data acquisition raw data Behaviour DB processed data short-term & long term behaviour behaviour changes & statistics behaviour detector Shared Knowledge Base data acquisition virtual sensing information Holistic Behaviour Analysis Framework End User answers questions Coach-as-a-Sensor data acquisition Fitbit API
- 18. 18 @orestibanos CONCLUSIONS TAKE HOME MESSAGE Effective e-coaching systems require a solid understanding of the users’ behaviour. Measuring human behaviour is now more than ever possible as a result of an increasingly richer sensor ecosystems, the most powerful possibly being our own smartphones. Creative measurement approaches are nevertheless needed due to the complexity, randomness and diversity of behaviours. Behaviour sensing frameworks can facilitate the realisation of experiments and collection of multiple data types to measure behaviour holistically, continuously and opportunistically.
- 19. 19 @orestibanos Oresti Baños Room 26 (2nd floor), Faculty ETSIIT, University of Granada, E-18071 Granada, Spain Phone (+34) 958248598 Email / Web oresti@ugr.es http://orestibanos.com/ MANY THANKS! CONTACT:
Editor's Notes
- (IEEE Computer Magazine, Banos & Nugent. 2018) Banos, O., Nugent, C. E-Coaching for Health. Computer, vol. 51, no. 3, pp. 12-15 (2018)
- This is by far the most widely explored application of smartphones in what concerns human behaviour measurement: (Procedia Comp. Sci., Bayat et al. 2014) A Study on Human Activity Recognition Using Accelerometer Data from Smartphones 3D acceleration (Neurocomputing, Reyes et al. 2016) - Transition-Aware Human Activity Recognition Using Smartphones Detecting transitions like sit-to-stand, stand-to-sit, sit-to-bed, etc. (Sensors, Hur et al. 2017) Smartphone Location-Independent Physical Activity Recognition Based on Transportation Natural Vibration Analysis We recognize those activities by analyzing the vibrations of the vehicle in which the user is traveling. We extract natural vibration features of buses and subways to distinguish between them and address the confusion that can arise because the activities are both static in terms of user movement (CHI, Min et al. 2014) - Toss 'n' turn: smartphone as sleep and sleep quality detector - Data: one-month field trial with 27 participants sensor data that might be relevant to sleep and sleep quality including: sound amplitude (via the microphone), light (via the ambient light sensor), and movement (via the accelerometer). Light intensity may be less reliable as people keep phones in pockets and bags. Therefore, we also collected screen proximity sensor values. Device states, such as screen on/off, processes (apps running on the phone), and the battery-charging state are also potentially informative in detecting sleep. For example, screen on (using the phone) is a good signal that a person is probably not asleep, but the screen is also sometimes automatically turned on for incoming calls or text messages, and by notification alarms from apps. Thus, other data, such as motion, should be used with the screen state to detect people’s actual use of device. People often charge their phone before going to bed, and they often use the phone as an alarm clock. Both provide clues about bedtime and waketime Figure: Phone status values (the upper graph), sensor values (the lower graph), and sleep (the range in both upper and lower graph with the green shaded background) (Nature, Althoff et al. 2017) - Large-scale physical activity data reveal worldwide activity inequality Activity inequality Dataset consisting of 68 million days of physical activity for 717,527 people, giving us a window into activity in 111 countries across the globe. Argus app was used for data collection (step counts). We find inequality in how activity is distributed within countries and that this inequality is a better predictor of obesity prevalence in the population than average activity volume. Reduced activity in females contributes to a large portion of the observed activity inequality. Aspects of the built environment, such as the walkability of a city, are associated with a smaller gender gap in activity and lower activity inequality. In more walkable cities, activity is greater throughout the day and throughout the week, across age, gender, and body mass index (BMI) groups, with the greatest increases in activity found for females. Our findings have implications for global public health policy and urban planning and highlight the role of activity inequality and the built environment in improving physical activity and health.
- (Perv. & Mob. Comp., Vu et al. 2015) - Characterizing and modeling people movement from mobile phone sensing traces (Mobile Netw. Appl., Lane et al. 2014) - BeWell: Sensing Sleep, Physical Activities and Social Interactions to Promote Wellbeing (Perv. & Mob. Comp., Vu et al. 2015): from March 2010 to August 2010, we conducted three rounds of experiments with 123 participants at the University of Illinois. Our participants included grads, undergrads, faculties, and staffs. The first experiment lasted 19 days, the second was 38 days, and the third was 85 days. The number of scanned WiFi MACs and Bluetooth MACs of the third experiment were fewer than the second experiment (although the third experiment was much longer) since the third experiment was conducted during the summer break with fewer classes and students on campus. Fig. 7(a) shows the first contact pattern in which people usually have a considerably higher number of contacts during the weekdays than the weekends. This is the most common contact pattern found in our sensing traces since most people perform the casual routines at work for the weekdays when they make contacts with many more people. Working people usually spend time with family or for personal things at weekends and thus they meet fewer people. In contrast, Fig. 7(b) shows an opposite contact pattern in which people make more contacts during the weekends than the weekdays. This pattern does not appear frequently in our traces, however it exists. The third contact pattern exists for people who have the busiest schedule in the middle of the week as shown in Fig. 8(a). This figure looks similar to the bell shape and people with this pattern usually meet many more people during midweek. The last contact pattern is the most steady one as shown in Fig. 8(b). People, who belong to this pattern, make a similar number of contacts everyday, regardless of weekdays or weekends. (Mobile Netw. Appl., Lane et al. 2014) - Changes in social isolation based on the total duration of ambient speech during a day. This is estimated from the output of a speech/ nonspeech classifier using the phone’s microphone. (Audio isn’t recorded on the phone or the cloud for privacy reasons.). We rely on studies that connect social isolation and social support to psychological well-being, with low levels being linked with symptoms such as depression. We experimentally develop a scoring system using field trials to determine the typical daily quantities of speech encountered by people within the study. In addition to conversation, the social interaction dimension also considers the use of social applications on the phone (such as Facebook, voice calls, and email) when computing a composite sociability score for the user.
- (Ubicomp, Rachuri et al. ) EmotionSense: A Mobile Phones Based Adaptive Platform for Experimental Social Psychology Research (Ubicomp, Pielot et al. 2015) - When attention is not scarce - detecting boredom from mobile phone usage To identify, which usage patterns are indicative for boredom, we logged phone usage patterns of 54 volunteers for 2 weeks. At the same time, we asked them to frequently report how bored they felt. We found that patterns around the recency of communication activity, context (hour of the day and proximity sensor), demographics, and phone usage intensity were related to boredom. Boredom correlated with more time having passed since the last incoming communication, and less time passed since the user last initiated outgoing communication via calls, SMS, and messages. This finding suggests that being contacted by others is generally correlated with being less bored. Contacting others, however, is more likely to happen while being bored. Boredom further correlated with the intensity of mobile phone use. The higher the usage intensity, the higher the boredom. This confirms observations in previous works that people use their mobile phones when bored to kill time. Boredom positively correlated with the time of the day and darker ambient lighting conditions. This finding means that there are boredom levels vary throughout the day. Moreover, in contrast to Mark et al. [23] who found that boredom is lower during late working hours, our results include after-work hours, and indicate that boredom tends to increase as the day progresses. Boredom correlated with demographics. Boredom tended to be higher for male participants, and higher for participants in their 20s and 40s and lower in their 30s. This findings are in line with previous work which found that age [36] and gender [2] are significant predictors of experiencing boredom in leisure time In a follow-up study they monitored 22 participants including daily mobile phone usage patterns, such as, the average number of apps started in a day or the variance of the amount of notifications received per day Individuals with high boredom proneness were having more unstable daily phone usage patterns: they launched a higher number of apps per day, had strong peaks of social network activity, and turned on the phone a lot. However, surprisingly, the overall time of using the phone was not higher than for individuals with lower boredom proneness. Boredom proneness is related to a number of negative outcomes, such as depression, drug & alcohol consumption, or anxiety. (Ubicomp, Mehrotra et al. 2017) MyTraces: Investigating Correlation and Causation between Users’ Emotional States and Mobile Phone Interaction Performed a causality analysis between users’ behavior and mood and mobile phone interaction in terms of notification response, application usage and communication patterns. We collected 5,118 responses to questionnaires for logging users’ emotional states (namely, activeness, happiness and stress) from 28 users over a period of 20 days. Data: Notification Arrival time, seen time and removal time, alert type (sound, vibrate and flashing LED), user’s response (click or dismiss), sender application name and notification title. Context Physical activity and location. Communication Time, type and sender/recipient of calls and SMSs. Phone Usage Lock/unlock event, single click, long click, scrolls and usage time of all foreground applications (including home screen). Findings: - stressful situations people become more attentive: this results in a lower notification response time. people use their phone more when they are active and less when they are happy W.r.t. usage of specific apps and emotional states, we have shown that the increase in users’ activeness level reduces the usage of music app. as the stress level increases the usage of communication and lifestyle apps decreases. socializing makes people happier and traveling reduces their stress (MobileHCI, Gosh et al. 2017) TapSense: Combining Self-Report Patterns and Typing Characteristics for Smartphone based Emotion Detection Like speed (turns to be the most prevalent feature), typing session duration, number of mistakes, special characters used, are inferred from typing sessions as to identify happy, sad, stressed, and relaxed states
- (Ubicomp, Abdullah et al. 2016) Cognitive rhythms: Unobtrusive and continuous sensing of alertness using a mobile phone Validation with a Psychomotor Vigilance Task (which is the gold standard to measure sleepiness; smartphone validated version). It measures your alertness/sustained attention using a 2-minute task: shows a visual stimulus at random intervals to the user, who responds by touching the screen. Shows that alertness can oscillate approximately 30% depending on time and body clock type (circadian rhythms ) and that Daylight Saving Time (practice of advancing clocks during summer), hours slept, and stimulant intake can influence alertness. When more alert, participants checked their phones frequently but for shorter lengths of time, while during low alertness, participants engaged in more sustained use. (J. Amb. Intel. & Hum. Comp, Wohlfahrt-Laymann et al. 2018) MobileCogniTracker: A mobile experience sampling tool for tracking cognitive functioning Create digital version of cognitive test and schedule and assess these remotely. MMSE assesses: - spatial and temporal orientation by asking the patient to correctly identify the current time (year, season, date, day, and month) and location (state, county, town, hos- pital, and floor) Attention and calculation by asking participants to subtract 7 from 100 five times, or alternatively to spell the word “world” backwards Memory: recall three objects from the registration part Language: naming (subjects are asked to name two objects), reading (subject are asked to read and follow a command), writing (subjects are to write a sentence of their choosing), copying (subjects are asked to copy intersecting pentagons)
- (Biomedical Engineering Online, Banos et al. 2015) Banos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J.A., Lee, S., Pomares, H. and Rojas, I., 2015. Design, implementation and validation of a novel open framework for agile development of mobile health applications. Biomedical engineering online, 14(2), p.S6. https://github.com/RADAR-base
- (Int. J. Distr. Sens. Netw., Felix et al. 2019) Felix, I.-R., Castro, L.-A., Rodriguez, L.-F., Banos, O. Mobile sensing for behavioral research: A component-based approach for rapid deployment of sensing campaigns. International Journal of Distributed Sensor Networks, vol. 15, no. 9, pp. 1-17 (2019) Incluir
- (Int. J. Distr. Sens. Netw., Felix et al. 2019) Felix, I.-R., Castro, L.-A., Rodriguez, L.-F., Banos, O. Mobile sensing for behavioral research: A component-based approach for rapid deployment of sensing campaigns. International Journal of Distributed Sensor Networks, vol. 15, no. 9, pp. 1-17 (2019) Incluir
- (Sensors, Banos et al. 2016) Banos, O., Villalonga, C., Bang, J. H., Hur, T. H., Kang, D., Park., S.-B., Hyunh-The, T., Vui, L. B., Amin, M.-B., Razzaq, M.-A., Ali Khan, W., Hong, C.-S., Lee, S. Human Behavior Analysis by means of Multimodal Context Mining. Sensors, vol. 16, no. 8, pp. 1-19 (2016) Incluir
- (ICT4AWE, op den Akker et al. 2018) op den Akker, H., op den Akker, R., Beinema, T., Banos, O., Heylen, D., Bedsted, B., Pease, A., Pelachaud, C., Traver-Salcedo, V., Kyriazakos, S., Hermens, H. Council of Coaches - A Novel Holistic Behavior Change Coaching Approach. ICT for Ageing Well (ICT4AWE 2018), Funchal, Portugal, March 22-23, (2018).
- (UCAmI., Konsolakis et al. 2019) Konsolakis, K., Hermens, H., Banos, O. A Novel Framework for the Holistic Monitoring and Analysis of Human Behaviour. International Conference on Ubiquitous Computing and Ambient Intelligence (UCAmI 2019), Toledo, Spain, December 2-5, (2019)