VII Jornadas eMadrid "Education in exponential times". "Ongoing research about the use of commercial-off-the-shelf wrist wearables in educational contexts": Manuel Caeiro Rodríguez. 04/07/2017.
•Download as PPTX, PDF•
1 like•275 views
VII Jornadas eMadrid "Education in exponential times". "Ongoing research about the use of commercial-off-the-shelf wrist wearables in educational contexts": Manuel Caeiro Rodríguez. 04/07/2017.
Recommended
More Related Content
More from eMadrid network
More from eMadrid network (20)
Recently uploaded
Recently uploaded (20)
VII Jornadas eMadrid "Education in exponential times". "Ongoing research about the use of commercial-off-the-shelf wrist wearables in educational contexts": Manuel Caeiro Rodríguez. 04/07/2017.
- 1. Ongoing research about the use of commercial-off-the-shelf wearables in educational contexts Authors: Francisco de Arriba Pérez Manuel Caeiro Rodríguez, Juan Manuel Santos Gago UNIVERSITY OF VIGO TELEMATIC ENGINEERING DEPARTMENT This work is being supported by the Spanish State Research Agency, the European Regional Development Fund (ERDF) under the PALLAS (TIN2016-80515-R AEI/EFRD, EU) project and SNOLA (TIN2015-71669-REDT)
- 2. Outline ●Introduction ●COTS wearables ●Objectives ●Sleep Indicators ●Stress indicators ●Proposals for education ●Conclusions 2
- 4. Objectives ●Data collection from different COTS Wearables o Homogenization ●Propose indicators to feature students o Estimation of sleep indicators o Estimation of stress indicators ●Applicability of indicators in educational contexts 4
- 5. COTS Wearables 5 ● Devices tested: oFitbit Charge, Fitbit Charge 2, Fitbit Surge oAndroid wear devices LG Watch R Moto 360 v1 oMicrosoft Band, Microsoft Band 2 oJawbone UP move
- 6. Problems ●Data collection o Different configurations (an intermediate device is needed) o Authorization to third-parties (tokens) ●Data integration o Differences in the names o Difference related to the beginning of measurements o Differences related to the sensors available ●Other issues o Battery duration, memory capacity, etc. 6
- 7. Data collection configurations How to collect Data? ● 4 data transfer mode o Warehouse vs. wearable data source o Direct vs. indirect access 7
- 8. Data collection configurations How to collect Data? ● 4 data transfer mode o Warehouse vs. wearable transfer o Direct vs. indirect access 8
- 9. Data collection configurations How to collect Data? ● 4 data transfer mode o Warehouse vs. wearable transfer o Direct vs. indirect access 9
- 10. Data collection configurations How to collect Data? ● 4 data transfer mode o Warehouse vs. wearable transfer o Direct vs. indirect access 10
- 11. Indicators to feature students ● Sleep indicators o Sleep quality o Sleepiness o Chronotype o Sleep regularity ● Based on HR, Skin temperature and accelerometer 11
- 12. Sleep indicator ●Sleep Quality 12 C4.5 Prediction mean error 14.93% Vendors mean error 30.55% SQ
- 14. Sleep indicator ●Chronotype 14 Type Start Bed Time Values End Bed Time Values Extreme morning -/21:30 2 -/05:00 2 Moderately morning 21:30/22:45 1 05:00/06:30 1 Intermediate 22:45/00:45 0 06:30/08:30 0 Moderately evening 00:45/02:00 1 08:30/10:00 −1 Definitely evening 02:00/- 2 10:00/- −2
- 15. Indicators for Academic ● Stress indicators o Snapshot stress o Aggregated stress o Latent stress o Stress regularity ● Based on HR, accelerometer, Galvanic Skin Response (GSR) and Skin temperature 15
- 17. Stress indicator ●Latent stress Similar to Ebbinghaus forgetting curves and recovey status curves in sport 17
- 18. 18
- 19. Proposal for education ●Services for students: o To enrich the student profile and support student self-awareness o Time management o Recommendations/Alerts 19
- 22. Proposal for education ●Services for teachers/developers: o Schedule of activities and timetable management o Grouping students into working groups o Enhancing learning resources. 22
- 24. Proposal for education ●Services for teachers/developers 24 ● Moderately morning ● Traditional evaluation ● Most stressful activities: 3.2, 4.1, 5.2 ● No problems detected ● Moderately evening ● Continuous evaluation ● Most stressful activities: 2.2, 4.1, 5.2 ● Irregular sleep quality warning ● Latent stress high
- 25. Conclusions ●Problems in wearables oLow battery duration oInteroperability problems oSensors accuracy ●Opportunities oApplication in sport environments oFast evolution ●Challenges oGeneralize algorithms oCold start the algorithms 25
- 26. LEAP: Lean and Agile Practices linking Engineering Higher Education to Industry Universidad de Vigo, Instituto Superior de Engenharia do Porto, University of Central Lancashire, Tallin University, University of Thessaly
- 28. Thanks for your attention. 28 mcaeiro@gist.uvigo.es farriba@uvigo.com The images shown in this presentation have been used without any commercial purpose
Editor's Notes
- The main point in this presentation are Introduction COST wearables Objectives Sleep Indicators Stress indicators Proposals for education Conclusions
- Wearables are defined as electronic devices and systems incorporated in some part of our body or clothes. After the appearance of the first consumer electronics wearables in early 2014 (such as the Google Glasses), a boom in research related to the use of these devices in different domains were experienced. Companies like Google, Apple, Microsoft and new electronic developers such as Fitbit or Xiaomi, have created a variety of commercial wearable devices. Based on the features and benefits provided by these devices and the rapid adoption they are experiencing in the last years, some projects have also explored their use for educational purposes. These studies have two different kinds of works: (i) studies that use wearables as an educational tool to develop specific projects; (ii) analysis of the use of wearable sensors and capabilities to feature students and enrich their profile. All these works are at an early stage of development. Currently, any kind of physiological data or value provided by any wearable is being analyzed and processed in order to offer some kind of educational value. In this sense, we have been working towards the provision of sleep and stress related indicators Estos indicadores han demostrado estar muy relacionados con el rendimiento académico de los alumnos. Una baja calidad de sueño reduce el nivel de aprendizaje al igual que niveles de estrés altos que afectarían al alumnos hasta sentir el síndrome de Burnout. Por otro lado niveles de estrés medios favorecen la motivación en el desempeño de tareas.
- Our focus of study combines a total of 4 points of interest. Homogenization: The use of different devices is not direct, because they present very heterogeneous features, sensors, software platforms, protocols, etc. In order to work with COST wearables from different vendors, a platform is needed that unifies the incoming data and that offers a service of homogenization independent of the type of wearable and the data that it supplies. Propurse indicators for an academic environment We have been working on the development specific sleep indicators, such as: sleep quality, sleepiness, chronotype. We can detect these indicators using sensors like accelerometer and HR. Estimation of stress indicators. This is focused on estimating a subject stress levels through collecting data from wearable sensors on which useful indicators can be offered, such as: snapshot stress, latent stress Through a student enriched profile based on sleep and stress indicators, it could be possible to offer valuable information and services for education.
- Data processing problems The differences are not just about units of measure, or frequency of samples, in some case, they involve many other many issues. Each vendor names in a different way the same variables or similar ones. For example, the sleep states are named by Fitbit as (“asleep”, “awake” and “really awake”) and by Microsoft as (“awake”, “light” and “deep”). In some cases, vendors take measures such as calories consumed or steps walked from the start of each day. Nevertheless, other vendors never reset such counters, and they accumulate the values from all the days since the last reset or from the origin of the device When we consider devices with different sensors and try to integrate the data into a homogeneous model we have to decide what to do with the data items that cannot be collected. Finally the accuracy of current consumer electronic wearables has been questioned, some works have been performed to verify the fidelity of sensors to check the feasibility of using these wearables and it is generally accepted that sensors’ precision is enough for non-medical purposes. Estudios recientes ratifican un nivel de error de unas pocas pulsaciones, aunque esta precisión varía en función del dispositivo comparado.
- El primero de los estudios que realizamos nos enfocamos a la detección del sueño y de sus posibles indicadores. El indicador de calidad de sueño ofrece una estimación de lo bien que has dormido hoy. Su representación gráfica puede dar una idea de lo regular que eres durmiendo y si es necesario invertir más horas de sueño. Usando sleep duration, fall as sleep, awake, HR min and average Skin Temperature sobre un sistema de machine learning entrenado con la percepción diaria de los usuarios. Para validar este indicador realizamos un estudio piloto con 5 usuarios el resultado que nuestro sistema reduce en casi la mitad el error en la detección de la calidad de sueño de otros sistemas comerciales con fórmulas de cálculo más rígidas.
- En esta diapositiva mostramos dos aplicaciones realizadas por nuestro grupo de investigación para los experimentos pilotos. La primera muestra los datos del sueño y la calidad de sueño valorada por el usuario. Esta aplicación consulta diariamente que calidad de sueño ha percibido el usuario. La segunda fue utilizada para recopilar dos datos en tiempo real de los wearables. Cada segundo los dato son recopilados del dispositivo asociado, etiquetados con el valor de somnolencia percivido por el usuario y enviados al servidor donde serán procesados y almacenados.
- This indicator seeks to identify the student chronotype by observing the hours slept/raised and taking into account di erences between instructional days and holidays-week- ends. The working routines or the academic activities, in case of students, usually stand in the way of natural sleeping periods. In addition, during weekends people can vary the routines, making up for the sleep hours missed during the working week, or by the contrary, missing more sleeping hours because social activities. To calculate this indicator, we used an equivalence table relating the start and end sleeping times to a value in the morningness-eveningness scale, sumando los valores que aparecen en la tabla en función de cuándo haya dormido y luego realizando una media. The results obtained provide a full success of 100%.
- Nowadays we are working in stress indicators. This is the basic reference indicator providing the stress level value at each time point. This is based on the predictions of a classifier trained using a specific app.The app collects raw data from the HR, ST, GSR and accelerometer wearable sensors. While the Internet connection is available, the data from the sensors and from the tags is send to our server. Once it is trained, it provides an estimation of the user level from the new sensors’ raw data received. Of course, the more tagged samples used the better the precision of the classifier. Este estudio actualmente aún en desarrollo presenta resultados superiores al 90% de tasa de precisión sobre las muestras etiquetadas.
- This indicator is based on an idea used in other scientific areas, such as the declining curves using a roll-off rate . Some examples of these systems can be found in the educational field as the Ebbinghaus forgetting curves. The Ebbinghaus curves represent the loss of memory retention in time and how the review of the learning material contributes to extend it. For otherwise, in sport, the recovery status curves indicate that the recovering time after performing a physical exercise is extended on time.. Similarly, we consider the application of this idea to the stress, representing that it is extended on time in accordance to a declining curve. Como se puede ver en la figura el estrés decae hasta que un nuevo nivel de estrés es detectado y entonces vuelve a subir.
- Para realizar las pruebas de stress, diseñamos una aplicación que sirve como un juego de varios niveles que el usuario debe realizar. A través de esta aplicación y de la diseñada para el medir las señales fisiológicas de un wearable, podemos analizar las variaciones que se producen en diferentes estados de “relajación”, “concentración”, “estrés”. En la aplicación consta de: La presenta un video Un test de interferencia de color con tres niveles de dificultad, donde el usuario debe pulsar sobre un botón el color en el que está coloreada una palabra Un test adaptado de PASAT, donde el usuario debe sumar los dos últimos dígitos que oiga. Una tareas de hiperventilación, en la que el usuario debe inspirar y expirar de forma rápida.
- Our proposal is focused on allowing the student to get a good self-knowledge, supporting the identification of strengths, weakness and preference. In this way, and for educational purposes, it is very important that the student is aware about the personal sleep and stress levels and profiles. Issues like the best hours of the day to study alone, or the effects of different activities on his stress, should be well-known by the student himself. The student can also check if changes produce positive or negative results in these indicators. Level indicators can inform the student about the best time periods to perform academic activities, for example, showing the more active and restful times. This can support the student creating his own schedule, arranging his academic activities and free time. Besides, work agenda can be proposed to the student interleaving activities to maintain an adequate level of stress. The idea would be neither to generate too many monotonous and boring activities, nor to saturate students with difficult and costly activities that generate too much stress. Depending on the levels of stress and sleep indicators, notifications will be generated that the user can understand as suggestions of the system to improve some behaviours. Similarly, alerts that will show a worrying warning about some of the behaviours can be triggered, indicating how they could be tackled.
- Para poder recolectar todos estos datos y ofrecer a los alumnos parte de la información propuesta, hemos desarrollado varias aplicaciones android en donde el alumno puede consultar sus physiological signals, sus resúmenes de indicadores de sueño y stress, sirviendo como un dashboard de estudiante. En la diapositiva se presentan los datos del patrón de sueño y sus variaciones, además se representan las horas donde el usuario ha dormido más de forma habitualmente.
- En esta diapositiva presentamos los niveles de stress concentración y relajación diario/semanal y un ejemplo de “Alerta” producida por unos niveles de estrés elevados durante un periodo de tiempo
- Analysing the level of sleepiness and stress along with accumulated values (latent stress, sleep regularity) both outside and inside the classroom, and combining this information with the school and extracurricular hours could allow to discover specific difficulties for a student, higher workload day, or hours of the day at which arousal levels are higher. With this, the teacher could make better personalized work plans, or decide about the best days for the exams. The use of indicators such as the chronotype is the creation of working groups. Creating working groups with similar sleep profiles will allow students to agree on a broad range of hours. If people with different sleep profiles are joined this can introduce difficulties for arranging meetings and plan activities. In other scenario, if it is detected that the activities of oral presentation in front of the classroom are especially traumatic for a student, it could be wise to join him in a same group with students whose speech facility and level of stress in this type of activities is smaller. Detection of stress and mood states during exercises, exams or during lectures, allows to draw a map of the psychological profile for each student. In our case, we could use latent stress, variations in chronotype and regularity of sleep, to understand the mood of students. These indicators provide a snapshot of how the previous days have been and therefore of the accumulated fatigue and stress. As it has been said about the teacher, there is also the task of producing the educational content to be used by learners. A good way to know if this content is adequate would be to evaluate the levels of stress and sleepiness during their use. Those contents that generate very high levels of stress should be reconditioned, thus avoiding that these levels could lead to frustration and abandonment by the student. Likewise, activities that do not generate any stress and that raise the levels of sleepiness of the student should be eliminated or placed between tasks that generate a depletion above the average.
- Como en el caso de los estudiantes estamos trabajando en un dashboard para el profesor en el que pueda visualizar las variaciones de stress a lo largo de sus clases y ver los indicadores de cada alumno de forma individual. (Hablar de la figura. Como se puede ver, se podría seleccionar el día y la hora a analizar, el porcentaje de tiempo en cada estado (Relaxed, concentrated, stressed ) y la variación de estos estados durante esa hora)
- Además este dashboard sirve al profesor para detectar alumnos en potencial riesgo a través de los colores verde, naranja y rojo. Como se puede ver en la slide el primer alumno tiene un chronotype “moderately morning”, según su perfil de estrés en los exámenes recomendamos una evaluación tradicional, se muestran las actividades que más estrés le han causado y se indica que no tiene ningún aviso activo. Por el contrario el alumno “Fran”, presenta dos avisos y por eso se indica con un color naranja, el primero establece que se ha detectado un patrón de sueño muy irregular y el segundo que el stress acumulado es elevado es este alumno.