Download to read offline
Stroulia Nov27.2019
- 1. Sensors, Data, andHealth Eleni Stroulia stroulia@ualberta.ca https://hypatia.cs.ualberta.ca/wp6 Professor, Computing Science University of Alberta NSERC, AI, IBM, AGE-WELL, DITA
- 2. key scientific andtechnological advances are transforming our understanding of health and healthcare 4
- 3. https://experimentalwriting2014dg.wordpress.com/20 14/04/12/diary-entry-from-robert-shields-1918-2007/ Internet of Things1 • Collecting Data - Lifelogging the Quantified Self • Learning Data-driven Models of Health and Disease • Employing Game Elements to Incentivize Healthy Behaviors • Real-time Decision-Making Support Augmented with Intelligence
- 4. https://www.oreilly.com/ideas/ideas-on-interpreting-machine-learning Machine Learning 2 •Collecting Data - Lifelogging the Quantified Self • Learning Data-driven Models of Health and Disease • Employing Game Elements to Incentivize Healthy Behaviors • Real-time Decision-Making Support Augmented with Intelligence
- 5. http://renewableplus.blogspot.com/2014/12/2015-testing-major-challenge-for.html • Collecting Data- Lifelogging the Quantified Self • Learning Data-driven Models of Health and Disease • Real-time Decision-Making Support Augmented with Intelligence • Employing Game Elements to Incentivize Healthy Behaviors Ubiquitous Computing 3
- 6. Gamification https://www.behaviormodel.org/ 4 • Collecting Data- Lifelogging the Quantified Self • Learning Data-driven Models of Health and Disease • Real-time Decision-Making Support Augmented with Intelligence • Employing Game Elements to Incentivize Healthy Behaviors
- 7. Ingestible Sensors Mobile Phones and Wearables Social Networks Ambient Monitoring Epidemiological analytics A ProxemicsView of Sensing https://en.wikipedia.org/wiki/Proxemics
- 8.
- 9. Recognizing Depression From Voice Deep NeuralNetworks can be configured to accurately detect (the level of) depression in one’s voice. The method is language independent. Detecting Depression from Voice M Tasnim, E Stroulia; Canadian Conference on Artificial Intelligence, 2019
- 10. Serious Games for Cognitive Training Effectivecognitive assessment and training can be implemented in fun tablet- based games, such as whack-a-mole, word-search, bejewelled, and mahjong. Detecting Depression from Voice M Tasnim, E Stroulia; Canadian Conference on Artificial Intelligence, 2019 http://vibrant-minds.org
- 11.
- 12. Ambient Sensors In theSmart Condo, sensors unobtrusively observe the occupants’ activities. Subsequent analysis recognizes Activities of Daily Living, a key indicator of functional independence. https://www.youtube.com/watch?v=MWWDAZmO6Hg The smart condo project: services for independent living NM Boers, D Chodos, P Gburzynski, L Guirguis, J Huang, R Lederer, L Liu, I Nikolaidis, C Sadowski, E Stroulia; E-Health, assistive technologies and applications, 2011 Sensor-data fusion for multi-person indoor location estimation
- 13. Cameras for Functional Mobility Analysis The VirtualGym guides older adults through personalized exercise postures, specified by their therapists. It can be deployed on an external display or through a Virtual Reality mode. VirtualGym: A kinect-based system for seniors exercising at home V Fernandez-Cervantes, N Neubauer, B Hunter, E Stroulia, L Liu; Entertainment Computing, 2018 Sensor-enabled Functional-Mobility Assessment: An Exploratory Investigation S Golestan, DJD Romero, E Stroulia, A Miguel-Cruz, L Liu; 2019 IEEE 5th World Forum on Internet of Things
- 14. This requires that 1.we establish policies to ensure that everyone can afford data- collection devices, and they consent to the use of their data. 2. so that representative data is collected, 3. and high-quality models are constructed, 4. that individuals and health professionals can trust and use. Scientific and engineering advances present us today with many opportunities to lead healthier lives, longer. Our challenge is to thoughtfully take advantage of these opportunities to uplift everyone in our society.
Editor's Notes
- #2 It is my great pleasure to share with you some thoughts on the transformative role that technology is playing in redefining the way we understand our health, the healthcare system, and our role in managing our own health and that of our loved ones. I have been working for several years on technologies for supporting seniors to live independently longer, and I am excited to tell you a bit about my projects, the work and results of my team, and the general socio-technical context that has enabled and motivated this research. I have to start by acknowledging that this work has been a true interdisciplinary collaboration between Computing Science and Occupational Therapy, with substantial contributions from Medicine, Industrial Design and Education. This work has been supported by many funding organizations, including NSERC, Alberta Innovates, IBM, AGE-WELL and DITA.
- #3 Let’s start with an exploration of the key scientific and technological advances that underlie my work and much related work around the world. In the the next 4 slides, I will briefly review the 4 factors that are revolutionizing the health sector today.
- #4 The first key technology is our ability to collect, clean, curate, and store Big Data about ourselves! The most extreme lifelogger was Robert Shields, who manually recorded 25 years of his life from 1972 to 1997 at 5-minute intervals 37-million word diary is recording his body temperature, blood pressure, medications, urination and bowel movements; he slept for only two hours at a time so he could describe his dreams.[4] In modern times, likely the first person to capture continuous physiological data together with live first-person video from a wearable camera, was Steve Mann, the father of wearable computing. 25 million are active fitbit users, and fitbit has apparently fallen to third place after Apple and Xiaomi.
- #5 And today, this Big Data stream can actually generate insight and knowledge through machine learning, and more generally data science. My department is renowned for our ML expertise, third/fourth in international rankings in this area, one of the three Pancanadian AI strategy institutes. The overall workflow of learning from data involves the following steps: Data capture and cleanup raw data may be captured through devices but data records may be corrupted or lost before being stored due to network congestion or device failures, so cleanup and/or imputation may be needed Identify distinctive features in the data representation often the first pass of feature selection is done by domain experts with knowledge that enables them to eliminate data attributes that are not distinctive other methodologies adopt feature-selection steps, where features that correlate with other features are excluded Generate various different hypotheses that might “cover” the data it is useful to think of ML algorithms as searches over a space of possible hypotheses, using policies or heuristics to guide the exploration towards stronger hypotheses with more support for them Verify whether the data support the hypothesis with sufficient confidence In the end, the discovered hypotheses represent consistent patterns in the data that can be used in different reasoning tasks: prediction, action selection, realistic construction of new data instances, simulation. All algorithms require configuration of multiple parameters, often through trial-and-test Different algorithms perform better on different types of data sets and for different data classes, so we often employ ensembles of ML algorithms Adoption depends on trust, SO To build fairness (and by implication trust) through (a) representative data (b) ethical heuristics and optimization functions to integrate our knowledge in the process explainability
- #6 Having transformed data into knowledge the question becomes “how to use it to improve decision making and action” This is possible with the variety of (a) specialized computational devices we have at our disposal, and (b) networking protocols that we can deploy in different settings. Today we are developing software systems that are distributed and heterogeneous, including sensors on persons, buildings, vehicles, that establish ad-hoc networks opportunistically when they find themselves in the presence of each other communicate their data to the cloud, where different types of servers (multicore, GPUs,…) can efficiently analyze the data to extract knowledge, which can then be exposed end-user applications at the point of care, or study, or work (through mobile devices) So we can all be data producers and contribute our experiences to the model-learning process, as long as we have access to the sensing devices and to the internetwork; and on the other hand, we can benefit from this knowledge as long we have access to the internetwork, the mobile devices and the applications. We need to ensure democratic access to this infrastructure!
- #7 Finally, I think it is interesting to talk about another means of supporting good decision making: by embedding action into a game and making “good behaviors” necessary to win the game! This is the idea of gamification! In 2011 Jane McGonigal published her book Reality is Broken and the position of the book was that game play can have positive and in long-lasting affects on our mood and emotional health. Fogg’s Behavior Model (FBM) explains why and how game mechanics/dynamics are able to drive actions. FBM asserts that human behavior is a result of the precise temporal convergence of three factors: Motivation: the person wants desperately to perform the behavior (i.e. he is highly motivated) Ability: the person can easily carry out the behavior (i.e. he considers the behavior very simple) Trigger: the person is triggered to do the behavior (i.e. he is cued, reminded, asked, called to action, etc.) So if studies of accelerometer-collected data have led us to learn that sedentary behavior (too much sitting over the day) is bad, then when a wearable sensor observes that I am sitting for a long time the paired application on my smartphone may remind me to stand up and move, and may give me “points” if I actually act on the advice and through these points I may be able to win an award or maybe simply outperform my friends against whom I am competing.
- #8 So if Data is the fundamental ground on which whole this edifice relies, the question becomes where is data come from? And to answer this question I will adopt the theory of proxemics (Edward T. Hall, the cultural anthropologist who coined the term in 1963), which posits that our behaviour, communication, and social interaction can be observed and understood in different scales of space: intimate, personal, social and public. Based on this theory, I believe it makes sense to consider the data that we are collecting about ourselves as observations of our lives in these different space scales. At the intimate scale, we consider ingestible sensors; at the publics pace we are considering epidemiological data. I am working at the other two scales: Personal where wearables and mobile devices exist Social where ambient sensors are deployed.
- #9 Let’s start with wearables and mobile apps.
- #10 I will start with my most recent project! Key intuition: mood can be detected in one’s voice Experimental hypothesis: we can learn a model to distinguish between depressed and non-depressed individuals based on a short speech segment Study: Benchmark data in English and German; one speech example per person, with labels Towards Adoption: we envision the architecture depicted here
- #11 Key intuitions: The more time one devotes to a task, the better they become at it The more engaging a task is, the more time one is likely to spend on it Gamification of learning/training Experimental hypothesis: Individuals with dementia can learn new skills with extensive practice through games embedding these skills Study: We built games to embed various skills, with different levels to make them increasingly challenging whack-a-mole – attention and reaction Word-search – attention and language Bejeweled –pattern matching Model-driven engineering of product lines so that we can generate multiple levels, instrumented to collect performance data; the different levels play two roles: Assessment of skill Engagement and flow (too easy is boring and too difficult is demotivating) Towards Adoption: with LTC organizations
- #12 And on to the social space, with ambient sensing
- #13 My favorite project - the smart condo Key intuition: the degree to which one is able to perform ADLs indicates the degree of their functional independence The Barthel index is a list of ADLs that OTs use to score a person when they are assessing their ability to live alone Experimental hypothesis: We can learn a model to distinguish between independent and non-independent individuals based on the sensor-event trace that is generated by ambient sensors embedded in the person’s home We can infer a valid representation of a person’s ADLs from the sensor-event trace Study: 26 individuals (alone or in pairs) executed a daily-activity script in 2 hours in the smart condo, in which we have embedded motion sensor, proximity sensors, switches, pressure sensors, cameras We developed a variety of algorithms to fuse the sensor data and infer an activity trace Towards Adoption: difficult
- #14 Finally focusing on a very important aspect of functional independence, mobility, we are working on a project to evaluate and strengthen the physical abilities of individuals. This project is parallel to Vibrant Minds, but instead of training the brain we focus on the body.
- #16 So here is the recipe that I put forward for the new model of health!
- #17 Ingestible sensors are the most intimate of all – they observe our individual bodies https://www.wired.com/story/this-digital-pill-prototype-uses-bacteria-to-sense-stomach-bleeding/ Bacteria, you see, are microscopic sensing machines. Take Lactococcus lactis, a friendly little microbe that helps turn milk into cheese. It can do its curdling work even better if there’s some heme floating around. That’s the iron-containing molecule that transports oxygen in your blood (and the Impossible Burger’s secret ingredient). But taking up too much heme can be toxic. So the little buggers have a system to sense how much there is, complete with genetic switches to change up their metabolism. Stanford’s Lu’s team took L. lactis’s on-switch DNA, coupled it with some code for bacterial bioluminescence, and stuck the whole genetic circuit inside a gut-friendly strain of E. coli commonly sold as a probiotic. Those modified cells went into a body-safe capsule equipped with a semipermeable membrane on one side to let in liquid from the gut. Wireless semiconductors powered by a teeny battery were packed in the capsule too—separated from the cells by a tiny see-through window. The scientists tested their bacteria-on-a-chip prototype in mice with induced gastrointestinal bleeding and in pigs that had blood piped into their stomachs. When the bacteria hit the heme, they lit up. Not much, but enough for a custom phototransistor to capture it and relay that information to a microprocessor—which sent the signal to an Android app developed by an undergraduate engineering student.
- #18 We are all familiar with fitbit and smart watches etc… The wearables observe our person (and the functioning ofour bodies) from the outside Notice how important it is to consider all users! But there are far more interesting from a clinical diagnosis perspective wearables. As the diagram illustrates, according to a review of the wearables market by a Stanford team last year (Sep2018), there are many wearables with FDA approval for clinical purposes, substituting expensive devices typically available in offices and hospitals. can be mechanical, physiological and biochemical are targeted to consumers, to clinical practice, or to research
- #19 Zio: STEP 1: WEAR THE MONITOR Your doctor will apply a water-resistant Zio by iRhythm heart monitor to your chest. There are no wires, and it can be hidden by your clothes. STEP 2: NOTE ANY HEART SYMPTOMS If you feel anything that you think might be an unusual heart rhythm, press the top of the Zio patch. Then briefly describe it in the provided symptom-log booklet, on the myZioTM mobile app, or at www.myzio.com. STEP 3: RETURN THE MONITOR BY MAIL After your prescribed wear time, simply remove your monitor. Then place in the back of the provided Patient Instructions & Button Press Log and put the booklet into the pre-addressed return box and drop into any United States Postal Service mailbox. STEP 4: REVIEW THE RESULTS WITH YOUR DOCTOR Based on the clear and complete report we’ll provide, your doctor will have the information he or she needs to make a diagnosis.
- #20 Medtronic AUTO MODE* Automatically adjusts your basal (background) insulin every five minutes based on your CGM readings.†‡ Helps keep your sugar levels in your target range for fewer lows and highs — day and night.†‡1 SUSPEND BEFORE LOW§ Stops insulin up to 30 minutes before reaching your preset low limits. Automatically restarts insulin when your levels recover without bothersome alerts.‡ Helps you avoid lows and rebound highs.1
- #21 Owlet ~400 USD We need policy to support this cost through insurance!
- #22 And last but not least we have to remember two other sources from humanity at large: Genetics And Demographics The physical, social and economic environment in which people live, work and play influences the choices they make and how they live their lives. And of course we know that some diseases are linked to specific genes. These two facts seem at odds… The only possible strategy then is to include in our analyses data on the genetics of individuals or gene frequencies for populations. Adding genetic information to our analyses could reduce the amount of unobserved heterogeneity and produce estimates of the contribution of specific genes to variations among individuals or across populations. This is not yet a real option. As long as most of the genes that have been identified are associated with rare diseases (like Huntington's chorea or sickle cell anemia), the potential impact of genetics on demographic research is very limited. However, genetic epidemiologists are now searching for genes that have large effects on common conditions. During the next ten years this might lead to discoveries that will substantially alter demographic research.