Thesis Defense - Personal Informatics and Context: Using Context to Reveal Factors that Affect Behavior

P E R S O N A LI N F O R M A T I C S& C O N T E X TUSING CONTEXT TO REVEAL FACTORSTHAT AFFECT BEHAVIORIAN LIANIND DEY JODI FORLIZZI NIKI KITTUR JOHN STASKOCo-Chair Co-Chair HCII Georgia Tech

AliceJust entered collegeStarted gaining weightFamily history of heartdisease 2

AliceManage her time better,so she can ﬁndopportunities to be active. 3

Pedometer 4

Calendar 8

Calendar Location 9

Calendar Location Weight 10

Calendar Location WeightFood Consumption General Health Mood 11

Calendar Location Weight Food Consumption General Health Moodhttp://personalinformatics.org/tools 12

Dashboard 13

Opportunity!34% of U.S. adults are obese(National Health and ExaminationSurvey, 2010)27% of adult internet usershave tracked health dataonline (Pew Internet Report, TheSocial Life of Health Information, 2011) 14

ThesisA personal informatics systemthat allows users to associatecontext with behavioral informationcan betterreveal factors that affect behaviorcompared to systems that only showbehavioral information. 15

Model of Personal InformaticsCreated a model to guide the design ofpersonal informatics systems. 16

Model of Personal InformaticsField StudiesShowed evidence in ﬁeld studies that contextcan reveal factors that affect behavior. 17

Model of Personal InformaticsField StudiesVisualization SupportExplored what kinds of visualization supportpersonal informatics systems should provide. 18

Model of Personal InformaticsField StudiesVisualization SupportPersonal Informatics DashboardDeveloped a personal informatics dashboardthat makes it easier for users to associatedifferent kinds of data in a single interface. 19

Model of Personal InformaticsField StudiesVisualization SupportPersonal Informatics Dashboard 20

GoalCreate a model as a guide in designingpersonal informatics systems. 21

Survey and InterviewsRecruited 68 people who use personalinformatics toolsAsked participants what tools they use andproblems they’ve encountered. 22

Sample Questions•  How difﬁcult is it to collect this personal information?•  How do you explore this collected personal information?•  What patterns have you found?Transcript of the survey is at:http://personalinformatics.org/lab/survey 23

AnalysisIdentiﬁed problems that people experienced.Afﬁnity diagrams to identify themes.Derived a model composed of:•  5 stages 24

5 Stages PREPARATION COLLECTION INTEGRATION REFLECTION ACTION 25

PREPARATION COLLECTION INTEGRATION REFLECTION ACTION 26

PREPARATION COLLECTION INTEGRATION REFLECTION ACTION Alice Wanted to become active Decided to track her physical activity Chose to track step counts using a pedometer 27

PREPARATION COLLECTION INTEGRATION REFLECTION ACTION Pedometer 28

PREPARATION COLLECTION INTEGRATION REFLECTION ACTION Synchronize data to web site. 29

PREPARATION COLLECTION INTEGRATION REFLECTION ACTION ActiveInactive Inactive M! T! W! Th! F! Sa! Su! M! T! 30

PREPARATION COLLECTION INTEGRATION REFLECTION ACTIONThe stage when peoplechoose what they are going todo with their new-foundunderstanding of themselves. 31

Properties of the Stages1.  Problems cascade.2.  Stages are iterative.3.  User- vs. System-driven4.  Uni- vs. Multi-faceted 32

Properties of the Stages1.  Problems cascade.2.  Uni- vs. Multi-faceted.3.  Stages are iterative.4.  User- vs. System-driven 33

1. Problems cascade.Problems in the earlier stages can affect thelater stages. 34

1. Problems cascade. Active Inactive Inactive M! T! W! Th! F! Sa! Su! M! T! 35

1. Problems cascade. Active Inactive Inactive M! T! W! Th! F! Sa! Su! M! T! 36

1. Problems cascade.Problems in the earlier stages can affect thelater stages.Consider all the stages when buildingpersonal informatics tools. 37

2. Uni- vs. Multi-facetedUsers expressed desire to see associationsbetween different facets of their lives.“If it were easily collected, information on food intake, calories, fat, etc., would make an interesting starting point for analysis.” User who tracks medication intake 38

2. Uni- vs. Multi-facetedMost personal informatics are uni-faceted.Some personal informatics toolshave multi-faceted collection,but only support uni-faceted reﬂection. 39

2. Uni- vs. Multi-faceted Active Inactive Inactive M! T! W! Th! F! Sa! Su! M! T! 40

2. Uni- vs. Multi-faceted Calendar Location Weight Active Inactive Inactive M! T! W! Th! F! Sa! Su! M! T! 41

2. Uni- vs. Multi-facetedMost personal informatics are uni-faceted.Explore support for collecting dataon multiple facets of one’s life. 42

Beneﬁts of the ModelIdentiﬁed the problems with existing tools.Highlights the many challenges of buildingeffective personal informatics tools.A common framework for describing,comparing, and evaluating personalinformatics tools. 43

Field StudiesDiary StudyIMPACT 1.0IMPACT 2.0 46

Physical ActivityLack of physical activity is a common problemthat leads to obesity, diabetes, and highblood pressure.Lack of awareness of physical activity is onereason why people are not active. 47

Sedentary People & WalkingResearch suggests that they are less aware oftheir physical activity and how to becomeactive. (Sallis & Hovell 1990)Encourage walking because it is easier tointegrate into daily life. (Norman & Mills 2004) 48

application. This is shown in Fig. 2c and d. network. The network inputs are the sum of signal strength fluctuation across all monitored cells, and the number of 3.1 Sensing activity distinct cells monitored over a given time interval. The network consists of a single layer of eight hidden neurons; The current activity of the user is inferred using patterns of weights are learnt using back propagation. The network fluctuation in GSM signal strength and changes to the IDs outputs the currently sensed activity for the given input of detected cells. This method has been demonstrated as a values. The network is trained by repeatedly presenting dataPhysical Activity Awareness reliable and unobtrusive way of sensing current activity [2], and has the advantage over the more traditional approach of using an accelerometer in that it does not require additional sensor hardware as in Sensay [17] and the multimodal collected during each method of movement. The current activity of the user is conditionally depen- dent upon their previous activity. In order to provide instant feedback to the user interface, the neural network deliber- sensor board of [11]. Similarly, while the processing of ately does not model this behaviour. Instead, when deter- physiological and biometric data could complement our mining if any additional minutes have been earned, we approach, the benefits of encapsulating the system within a apply task knowledge based upon the output from the mobile phone would be lost. An alternative approach would neural network over the previous two and a half minutes. be to utilise the positioning information available from This enables noise to be filtered out and a more accurate some mobile phone networks, however this approach representation of the users’ activities achieved. For exam-Products frequently involves prohibitive cost, as well as depending upon much of the same technology as our client based monitoring. ple, periods of low signal strength fluctuation such as stopping at traffic lights whilst driving can be ignored when placed between periods of high fluctuation where many Rather like a traditional accelerometer, the levels of distinct neighbouring cells were monitored. It could be signal strength fluctuation change when a mobile phone is argued that activity would be more accurately inferred if a moved. For example, Fig. 3 shows the total signal strength longer rolling filter had been applied to the GSM data. fluctuation across all monitored cells during successive 30-s Introducing longer filters would have increased the likeli- time periods whilst walking, remaining still and travelling hood of active minutes ‘disappearing’ from the users’ Fish’n’Steps: Encouraging Physical Activity with an Interactive Computer Game 1 2Research 3 4 !! Fig. 1. One participant’s display after approximately two weeks into the trial in the Fishn team-condition, also the public kiosk and pedometer platform, which rotated through e the team fish-tanks. The components of the personal display include: 1) Fish Tank - Th tank contains the virtual pets belong to the participant and his/her team members, 2) Virtu Figure 2 The phone interface. Images a and b show screens for examining relative – The participant’s own fish in alevels:view on the right side next to the fish tank, 3) Ca and individual activity frontal compare Daily Activity and This Week’s Activity Images. c and d show two of the screens showing the estimated current activity level: Stationary and progress bar, personal and team ra tions and feedback - improvement, burned calories, Walking UbiFit Shakra Fish ‘n Steps etc., 4) Chat window for communicating with team members. To evaluate the effect of Fish’n’Steps, we recruited 19 participants from the Consolvo et al. ’08 Maitland et al. ‘06 Lin et al. ‘06 of Siemens Corporate Research to participate in a 14-week study. Two experim conditions were designed to separately assess the impact of the virtual pet an social influences. Application of the TTM to assess behavior that changed durin study demonstrated that Fish’n’Steps was a catalyst of a positive change for 14 o 19 participants. This effect was evident in either an increase in their daily step 49 (for 4 participants), a change in their attitudes towards physical activity (for 3 pa pants) or a combination of the two (for 7 participants). The greatest change in

Research on FactorsPhysical activity is affected by lack of time,choice of activities, the environment, andsocial inﬂuence. (Sallis & Hovell 1990)CDC suggests understanding of factors tocircumvent barriers to physical activity. 50

Research on FactorsDiabetes awareness of blood sugar level andfood consumption (Frost & Smith ’03)•  Images of food associated with blood sugar level.•  Used in a class where people discussed their images and blood sugar level.•  Made a prototype, but only tested with one person. 51

Research on FactorsAsthma patients videotaping daily routinesfound that they are in the presence of harmfulallergens more often than they realized(Rich et al. ‘00)•  Users videotaped daily routines, but a trained observer looked at the video for assessment. 52

Prototypes Step Counts 53

Prototypes Step Counts } Activity Contextual Location Information People 54

GoalBefore building a prototype,explore what people would do when theyhave access to both physical activity andcontextual information. 56

SenseWear Pedometer Booklet Date: Time How active were you? What? Where? With whom? Time How active were you? What? Where? With whom? 6a: 1p: : : : : : : 7a: 2p: : : : : : : 8a: 3p: : : : : : : 9a: 4p: : : : : : : 10a: 5p: : : : : : : 11a: 6p: : : : : : : 12p: 7p: : : : : : : Continue to the next page. 57

Takeaways“It was nice to see that I walked more than I did. There was one day when I was babysitting. I walked so much with the baby. I walked all over campus.” A1 Activity Location Person 58

Takeaways“Housework and walking to the bus stop cancontribute, really. I mean, I take that forgranted in terms of energy expenditure.” A4 Activity Location 59

Matching SenseWear graphs with booklet FRI DEC 8, 2:03 ... Start Time - Fri Dec 8, 2006 05:14 AM End Time entries. End Session end - Fri Dec 8, 2006 02:03 PM 2:03 PM, 2:16 ... FRI DEC 8, 2:03 ... Start Time - Fri Dec 8, 2006 05:14 AM End Time Session end - Fri Dec 8, 2006 02:03 PM Start End 5:14 AM 2:03 PM Active ... Physical Activity (2.5 ME... Step Count Lying Do... Sleep Duration of Vi...off-bodyof 4 cal 438 cal 1 hr 43 m... 11346 Not detect... Not detect... 8 hrs 49 m... 60

SummaryParticipants made associations between theirphysical activity and contextual informationhelping them become aware of factors thataffected their physical activity.Can a prototype support thisin a ﬁeld study with more people? 61

Pedometer Booklet 63

Plus-Context eDay withcontext labels fTable and chartof steps andcontext gSteps by hourand by periodof day Figure 3. a) Interface for recording steps. Steps-Only additions. 65 b) One day of steps. c) Week of steps by day. d) Week of steps for

Pedometer Booklet Dashboard Steps Baseline 1 week Visualization of Steps StepsSteps-Only 3 weeks Visualization of Steps & Context StepsIMPACT 1.0 Activity Location 3 weeks People 66

Participants30 participants (B1-B30)•  Sedentary. Pre-screened using Stages of Exercise Behavior Change (Marcus et al. 1998)Questionnaires at the end of each phase 67

Mentioned Context“It helped me realize which activities were more important. For example, I didn’t understand the importance of walking home versus taking the bus.” B8“It turns out I get the most walking done to and from work, which I cant say I wasnt expecting, but I also had no idea that walking around Squirrel Hill for just an hour or two made such a difference.” B24 68

Of the 30 participants… Mentioned Context (Activities, Location, People) IMPACT 1.0 13 participants Visualization of Steps and Context Steps-Only 7 participants Visualization of Steps Baseline 6 participants No Visualizations 69

IMPACT supportsreﬂection on context“The [visualization] I used the most was theone asking who I was with; I hadn’t realizedthat I was so sedentary most of the time Ispent with my friends.” B1 70

Possible Improvements“IMPACT gave a lot of cool information, buthaving to input all the various factors was ahassle.” B4 71

Possible Improvements“IMPACT gave a lot of cool information, but having to input all the various factors was a hassle.” B490% reported they would continue usingIMPACT if collection of context wasautomated. 72

Possible Improvements“IMPACT gave a lot of cool information, but having to input all the various factors was a hassle.” B490% reported they would continue usingIMPACT if collection of context wasautomated.Next: IMPACT 2.0 73

Automatic Collectionof Steps and Location Bluetooth GPS 75

Facilitated Collectionof Activities and People 76

Automated Integration Bluetooth Sync 77

Mobile Phone Dashboard Collected Baseline Steps Only Visualization of Steps CollectedSteps-Only Steps Only Collected Steps, Activity, Visualization Location, and People of Steps & ContextIMPACT 2.0 79

Baseline Phase Intervention Phase Control Baseline Steps-Only IMPACT 2.01 2 3 4 5 6 7 8 80

Participants35 participants (C1-C35)•  Sedentary. Pre-screened using Stages of Exercise Behavior Change (Marcus et al. 1998)Questionnaires at the end of each phase 81

ResultsNo complaints about inputting data.But people complained about carryingmultiple devices.•  “I would not like carrying two devices (GPS and phone), that was too much.” C30 82

Awareness of factors increased forall groups between the phases 32,./2*" 4.)5(67,*8" -9:;3<"=#>" $"!"#$%&%()*(+#,-)$( %#$" %" !#$" !" &()*+,)" -,.)/0),12," F[2,32] = 3.98, p = .0547 83

Mentioned Context Mentioned Context (Activities, Location, People) IMPACT 2.0 6 of 11 participants Visualization of Steps and Context Steps-Only 3 of 12 participants Visualization of Steps Control 5 of 12 participants No Visualizations 84

Short-Term Beneﬁts/Problems Short-term IMPACT 1.0 Harder to collect, Manual Collection but more engaged IMPACT 2.0 Easier to collect,Automated Collection but less engaged 85

Long-term reﬂectionWhat is the value of contextual information inthe long-term?6-months later when they were more likely tohave forgotten the data 86

Follow-Up InterviewsExpressed interest in comparing over longperiods of time.Curious about the peaks in physical activity.But only those who had visualizations ofcontextual information had reminders of whathappened during those peaks. 87

Long-Term Benefits/Problems Short-term Long Term IMPACT 1.0 Harder to collect, No reflection Manual Collection but more engaged opportunity IMPACT 2.0 Easier to collect, Has reflectionAutomated Collection but less engaged opportunity 88

Overall SummaryProvided some evidence that a system thatshows context can reveal factors that affectbehavior.But the value of the data is highlydependent on the type of support. 89

GoalDetermine what kinds of questions people askabout their data.Determine when contextual information isuseful. 93

Participants15 participants (P1-15) to interview. 94

Procedure1-hour interviews•  I observed participants using their personal informatics tool. 95

AnalysisIdentiﬁed the kinds of questions people askedabout their data.Afﬁnity diagrams to identify themes.Derived 6 kinds of questions. 96

Six Kinds of Questions Status What is my current status? History GoalsDiscrepancy Details Factors 97

Six Kinds of Questions Status History What happened in the past? GoalsDiscrepancy Details Factors 98

Six Kinds of Questions Status History Goals What goals should I pursue?Discrepancy Details Factors 99

Six Kinds of Questions Status History GoalsDiscrepancy How does my behavior compare Details to my goals? Factors 100

Six Kinds of Questions Status History GoalsDiscrepancy Details What other things happened Factors during a particular point in time? 101

Six Kinds of Questions Status History GoalsDiscrepancy Details Factors What inﬂuences my behavior over a long period of time? 102

Importance of the QuestionsNot all questions are important all the time.Some questions are more important thanothers as people’s information needs change. 103

Importance of the QuestionsNot all questions are important all the time.Some questions are more important thanothers as people’s information needs change. Maintenance Discovery & Phase Phase 104

Maintenance PhaseParticipants already know how differentfactors affect their behavior, so they just wantto know what their current status is.Participants have already identiﬁed theirgoals. They are only concerned with whetherthey are meeting their goal. 105

Maintenance PhaseCurrent StatusP13 just tracks the minutes that he spends onFacebook, Twitter, and other social mediasites, because he already know how theseaffects his productivity. 106

Maintenance PhaseDiscrepancyP1 uses Mint to keep track of herexpenditures to see whether she is meetingthe budget that she had set for herself. 107

Maintenance PhaseThese kinds of questions were the mostimportant:•  Status•  Discrepancy 108

Discovery PhaseParticipants collect several types ofinformation to ﬁnd out what factors affecttheir behavior.Participants are trying to ﬁgure outwhat their goals are. 109

Discovery PhaseFinding factors that affect their behavior.P3 has diabetes and she tracked her bloodglucose levels and her food consumption toﬁnd out their interaction. 110

Discovery PhaseFiguring out goalsP8 tracks the quality of her sleep so that she isbetter rested. She explores her sleep data to“spot trends for which I can take correctiveaction.” 111

Discovery PhaseThese kinds of questions were the mostimportant:•  History•  Goals•  Details•  Factors 112

Discovery PhaseThese kinds of questions were the mostimportant:•  History•  Goals•  Details•  Factors} Contextual information & Multiple types of data 113

Discovery PhaseThese kinds of questions were the mostimportant: }•  History Next Question•  Goals What visualization features would help users ﬁnd answers•  Details to these kinds of questions?•  Factors 114

Timeline SketchesHistory Goals GoalDetails Factors 115

ResultsHistory: Looking back in time.Participants generally agreed that the timelinesketches were the most appropriate for theDiscovery phase. 116

Results GoalGoals: Seeing goals information.“I like having the goal line...I always like being able to see what my baseline should be and if I am above or below.” P5 117

ResultsDetails: Seeing details to reason what happened.“When looking at exercise there are a couple of times where I really didn’t meet my goal, so it will be really nice to be able to say ‘why didn’t I meet my goal then?’” P4 118

ResultsFactors: Comparison of different kinds of data.“The most interesting thing here is the ability to compare two different time frames because I’m really interested in the relationship between data.” P6 119

SummaryContextual information and multiple types ofdata is important during the Discovery phase.Described visualization features to helppeople answer questions during the Discoveryphase. 120

SummaryContextual information and multiple types ofdata is important during the Discovery phase.Described visualization features to helppeople answer questions during the Discoveryphase.Next: Built a personal informaticsdashboard with the visualization features. 121

GoalBuild a personal informatics dashboard thatallows users to see multiple kinds of datatogether.Develop an approach that makes it easier tobuild. 123

Visualization FeaturesHistory Goals GoalDetails Factors 124

Data IntegrationData Sources Dashboard 125

Data IntegrationData Sources Dashboard 126

Problems with Data IntegrationDashboard has to:Access DataParse DataVisualize Data 127

Problems with Data IntegrationDashboard has to:Access Data Managing many data sources w/ different APIs.Parse Data The data source losesVisualize Data control of the data. 128

Problems with Data IntegrationDashboard has to:Access Data No standard format for the different types of dataParse Data that users collect.Visualize Data Dashboard has to create parsers for each format. 129

Problems with Data IntegrationDashboard has to:Access Data Dashboard has to create visualizations for eachParse Data type of data.Visualize Data Duplicates creation of the visualizations. 130

Visualization Integration 131

Visualization Integration Data Sources Dashboard 132

Visualization Integration Data Sources Widgets Dashboard 133

Visualization Integration Data Sources Widgets Dashboard 134

Beneﬁts of Viz IntegrationDashboard has to:Accessing Data Provide an API that data sources can use.Parsing Data Manage theVisualizing Data communication between widgets. 135

Beneﬁts of Viz IntegrationFor the perspective of data sources:Maintain control of the data.They can choose how the data is visualized.Create a widget and it can be used withwidgets that others have made. 136

INNERTUBE 137

ImplementationProgrammed in Javascript.1.  Innertube API2.  Innertube Widgets3.  Innertube Dashboard 138

Innertube APIData sources create visualization widgetsusing static images, Javascript, and/or Flash.Data sources use the API to communicatewith the dashboard and vice versa. 139

Innertube APIGet the date and range of visualizations todisplay.Get the currently highlighted data point.Change the appearance of the widget.•  Set height of the widget.•  Reload the widget. 140

Innertube WidgetsFitbit StepsGPS Location 141

Innertube WidgetsWeatherSleepBusynessEnergy LevelMoodNotes 142

Innertube Dashboard 143

Demo ofInnertube Dashboard 144

Field Study15 participants recruited via Craigslist.Were not tracking their physical activity. 145

Data Collection for 1 weekAutomatically Collected Manually CollectedStep Counts using Fitbit MoodGPS Location Amount of SleepWeather Information Busyness Energy Level People Notes 146

Returned to the LabUsed Innertube while thinking-aloud.•  What they were looking for•  What they were ﬁnding•  What problems they encounteredAnswered questionnaires about Innertube. 147

Results13 of the 15 participants agreed thatInnertube was useful. 148

Results“It allowed me to factor in location, times, and activity in order for me to assess where I may be able to increase physical activity.” P12“Seeing the temperatures of the times I went on my runs and knowing how well I did on them would allow me to determine the best condition for me to run in.” P14 149

Results“It gave me concrete contexts, in space and time, by which I could measure and evaluate my own physical activity. Interacting with that data gave me the opportunity to hypothesize about what factors inﬂuenced my own physical activity, and what speciﬁcally motivated me or discouraged me from exercising.” P9 150

Results“I thought certain widgets [factors] were less useful before I used the PI dashboard, and then I changed my mind after using it, because their usefulness became apparent to me.” P9 151

Future WorkImprove the usability of the InnertubeDashboard.Make the Innertube API available todevelopers. Coming soon!Create a directory of Innertube Widgets, sopeople can ﬁnd widgets easily. 152

SummaryDescribed visualization integration, an easierapproach to building personal informaticsdashboards.Implemented Innertube, an example ofvisualization integration. 153

Conclusion 154

ContributionsCreated a model to guide the design ofpersonal informatics systems.Showed evidence that contextual informationcan reveal factors that affect behavior. 155

ContributionsExplored what kinds of visualization supportpersonal informatics systems should provide.Developed an easier way to buildpersonal informatics dashboards to helpusers associate different kinds of data in asingle interface. 156

Future WorkDeploy longer ﬁeld studies.Conduct studies in other behavior domains.Explore how to convert awareness of factorsto changes in behavior (Action stage). 157

Thank you!To my committee, Anind Dey, Jodi Forlizzi, Niki Kittur, and John Stasko.To the many who have helped along the way: Gary Hsieh, Erin Walker,Karen Tang, Scott Davidoff, Amy Ogan, Ruth Wylie, Moira Burke, QueenieKravitz, Gabi Marcu, Rebecca Gulotta, Matt Lee, Turadg Aleahmad, ArunaBalakrishnan, Min Kyung Lee, Tawanna Dillahunt, Sunyoung Kim, Chloe Fan,Jenn Marlow, Jason Wiese, Stephen Oney, Chris Harrison, Julia Schwarz,Eliane Stampfer, Samantha Finkelstein, Aubrey Shick, Matt Easterday, BilgeMutlu, Andy Ko, Johnny Lee, Ido Roll, Jeff Nichols, Jeff Wong, Jennie Park,Sara Kiesler, Laura Dabbish, Scott Hudson, Tessa Lau, Fernanda Viegas,Jaime Teevan, Alexandra Carmichael, Gary Wolf.To HCII, QoLT, the Ubicomp Lab, and the Quantiﬁed Self.To my family, Papa, Mama, Robin, and Cassandra.This work is based on research supported by the National Science Foundation under Grant No.IIS-0325351 and EEEC-0540865. 158

http://tarciziosilva.com.br	1007
http://feeds2.feedburner.com	5
http://twitter.com	1
http://us-w1.rockmelt.com	1
http://a0.twimg.com	1
http://feeds.feedburner.com	1
http://pt.flavors.me	1

Thesis Defense - Personal Informatics and Context: Using Context to Reveal Factors that Affect Behavior

by Ian Li on Aug 22, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

7 Embeds 1,017

Statistics

Thesis Defense - Personal Informatics and Context: Using Context to Reveal Factors that Affect Behavior — Presentation Transcript