A Mobile Context-Aware Behavior Modification System for Healthy Lifestyle Management October 17, 2007 Taimur Hassan
Agenda
Introduction
Problem Statement
Research Questions
Background
HMSS Details
Evaluations
Research Answers
Contribution
Future Goals
Questions
Introduction
This dissertation emerged from collaboration with Kaiser Permanente’s preventive medicine division
The division conducts classes that aim at educating participants about weight management through:
Adopting a balanced diet in their daily life
Counting calories
Exercise
Reducing visits to fast-food restaurants
Problem Statement
Problems with the preventive medicine approach:
Maintaining motivation outside classroom is difficult because of the environment participants often face
Keeping track of activity and related parameters is tedious
Proposed idea: A mobile system that sends motivational messages based on analyses of a participant’s activity and location data. It has been named the Health Management and Support System (HMSS)
Research Questions
Before such a system could be designed, questions to be answered included:
How could activity and location information be captured?
Can these sources be feasibly used for choosing feedback related to the health choices of a user?
Could such feedback be effective in changing the behavior of a user?
Background
To understand how HMSS works, a brief introduction to pervasive and context-aware computing is needed
Pervasive computing refers to an arrangement where information processing has been integrated into everyday objects that surround a user
A common example given is a refrigerator that can sense what is placed inside and can warn the owner if something has expired or finished and put the item on an electronic shopping list
Background (cont.)
What is Context?
The set of facts or circumstances that surround a situation or event (Wordnet)
Context-aware computing is a sub-discipline of pervasive computing
Describes the idea of computer systems reacting to what is sensed in the user’s environment
An example is a room’s light dimming when the system senses the subject wants to lie down
Context can be sensed using information such as activity, location, time of day and schedule using a variety of sources
Why is context so important for HMSS?
Studies conducted suggest that contextualization of motivational messages reduces the chances of the message being ignored
HMSS uses context to select the most appropriate message
Background (cont.)
Some examples of context-aware applications:
A context-aware remote control developed at MIT which combines behavior modification theory and daily activity records to persuade users to increase their physical activity and reduce television viewing
Mitchell et al. describe an application which is wired with sensors to track users and allow them to summon each other without worrying about location
SenSay phone developed by MIT can provide remote callers with the ability to communicate the urgency of their calls, make call suggestions to users when they are idle, and provide the caller with feedback on the current status of another SenSay user based on sensor data
HMSS Overview
HMSS is based on knowledge from these main domains, as well as related fields such as mobile and sensor based computing
It is a prototype that sends just-in-time motivational messages based on a user’s context
Software packages include:
.NET Windows Mobile SDK
Java, Visual C#, PHP
MySQL database
Google Maps API
GT2K gesture recognition library
Hardware used:
Windows Mobile Verizon XV6700 smartphone
iTrek M5 Bluetooth GPS Receiver
Sparkfun Bluetooth Accelerometers
HMSS consists of two artifacts:
A mobile client application
A server
Mobile Client Application
Connected to server via a subscribed data service
Has two components:
Sensor Manager
Establishes connection with accelerometer and GPS receiver
Responsible for upload, compression and transmission of raw data to server
GUI
Controls sessions
Controls sensor data retrieval process
Contains menus for setting session properties
Feedback from the server is displayed in the GUI
Server
Has four components:
Session Manager
Handles data storage and processing for a session
Contains GT2k and other functions for handling raw data
A database
Stores data related to all aspects of HMSS
Message Selector
Selecting feedback based on context
Message Handler
Customizes feedback and responsible for dispatch to mobile client
Data Architecture
HMSS data architecture represents the steps taken from raw data to user context
Adopted from previous work by Dey et al. and Gellersen
Used for simplifying feedback selection
All data sources expressed as sensor types
Sensor Layer
Virtual Sensor
Source: User data (weight, gender, address etc)
Location Sensor
Source: A GPS receiver
External Physical State Sensor
Source: An accelerometer
Cue Layer
Abstracts raw sensor data into cues, or abstractions
Example can be the output of a light bulb which is expressed as a number from 0-255 into high, medium and low
Five cues derived from source layer:
Speed (outdoor walking pace)
Closest building (closest building to user)
Indoor/outdoor (is user inside the building)
Activity, as determined by GT2k
Calories burnt
Context Layer
Responsible for determining a users context by using cues to help minimize misidentification
Algorithms convert multiple cues into context at this layer, which is then used for feedback selection
Two contexts defined, with ability to add more
Evaluation
Developed using all domains as a reference
Eight unescorted subjects participating in mobile field experiments
The first evaluation was a pilot, followed by further development and seven subsequent evaluations
Each evaluation held in three phases
Pre-experiment training
Experiment
Post-experiment interview
Subjects were associated with SISAT
Three questions asked in all evaluations:
What features you would like to see?
Would you like to use such a device all day?
Did you alter your behavior during the course based on feedback?
Pilot
Held in March 2007
Instructions between three campus buildings, ACB, Sprague library and Hagelbarger’s lounge
Two activities were included for messages, walking and resting
Aimed at gathering data on technical and usability issues
Subject followed an instruction sheet, while carrying a smartphone, a GPS receiver, and wearing three accelerometers
Subject was female, in her twenties
Two sessions were held:
In first session, application crashed and session was reset
The second session proceeded normally
Subject was able to indicate via the smartphone whether the messages received were accurate or not
Course
Go downstairs and exit the building from the southern exit
Once outside, walk towards the Sprague library
Enter the library from the southern entrance and climb up the stairs to the fifth floor
Sit on the chairs available on the fifth floor for 2-4 minutes
Go downstairs and leave the building and proceed to Hagelbarger’s restaurant at CGU.
Once inside the restaurant, sit inside for one
Leave the restaurant through the southern exit, and walk to the ACB building
Sit on a bench outside the northern entrance of ACB for 2-4 minutes
Enter ACB from the northern entrance, go up the stairs and sit on the couch for 2 minutes
Pilot Results
Location did not update properly because the subject had put the GPS receiver in pant pocket
Subject received six messages
Subject found five of the messages inaccurate because of the GPS error
Subject indicated surprise that stair climbing was not recognized
Subject indicated less sensors were desirable
Resulted in several updates to system
Second Evaluation Phase
Seven sessions held in June 2007
Reliability and stability
Session restart
Improved sensor data handling
Accelerometers
Reduced to one
Subject training
Error recovery
Each evaluation issue led to iterative improvements
Instructions
Automatically received on smartphone based on subject location and specified time elapse period
Goals were attached to activity choices; outdoor walking pace and stairs vs. elevator choice
Inclusion of the Olin Science Building at Harvey Mudd
Subject were asked to acknowledge the receipt of an instruction via a smartphone button, otherwise session was suspended
Packaging
Easier to replace batteries
Improvement in wiring and the addition of an external power switch
Evaluation Overview
Seven subjects:
Five males, two females
Five in their twenties, one in forties, one in fifties
Experiment sessions lasted from 30 minutes to 1 hour
Each evaluation was remotely monitored by using the Google Maps API
A web page was created that had access to entire session data
Web page displayed subject location as well as feedback on a map in realtime
Allowed signaling of potential problems and correction notification to subject
Course
Evaluation #1
Subject was male in his twenties
Subject did not use stairs at Olin and Sprague, which was correctly identified by system
On the way back, subject used stairs at ACB, but only a small percentage of the sequence was correctly identified. Data delay transmission also played a factor in incorrect recognition
Interview revealed subject got lost inside Olin and had also thought Sprague Library stairs were off-limits
Subject training modified to emphasize architecture inside Olin and Sprague
Evaluation #2
Subject was female in her twenties
Subject was briefed in more details about Olin and Sprague, but lost way inside Olin
Extra walking inside Olin caused incorrect feedback
Subject had to wait a long time outside Sprague as she was standing closer to Olin
Inside Sprague, subject used elevator to go upstairs, but used the stairs to go down when she received her feedback
At instruction #9, the session had to be aborted as incorrect GPS data caused the system to not send the next instruction
Course was modified slightly to make it easier to trigger instruction #9
Evaluation #3
Subject was male in his twenties
Subject lost his way to the Olin building
Subject was guided back on course using text messaging to smartphone
Subject used stairs inside Olin and got correct feedback
Inside Sprague, subject was going to use stairs, but decided on sharing an elevator ride up
The subject received correct feedback at Sprague
At ACB, the subject’s stair walking sequence was correctly recognized
Exterior pictures of Olin were taken and shown to subjects to reduce chances of getting lost
Evaluation #4
Subject was male in his twenties
Subject could not find stairs inside Olin, extra activity caused incorrect feedback
Subject had to wait a long time for instruction #6 as he was not close enough to Sprague
Climbed stairs very quickly at both Sprague and ACB, which caused incorrect feedback
Took a route for Sprague-Hagelbarger’s that was unanticipated, which caused delay in next instruction dispatch
The instructions were modified to require the subject to wait outside Hagelbarger’s for the next instruction before proceeding inside
Video clips were incorporated into training
Evaluation #5
Subject was male in his twenties
This was the second session with this subject, the first one was aborted due to priority conflict
Subject used stairs in Olin, got correct feedback, but activity analysis showed low recognition of sequence
Subject had to wait a long time for instruction #6 because he was standing under a shade
Subject climbed stairs very quickly at Sprague which caused incorrect feedback
On way back, subject forgot to acknowledge instructions which caused the session to end
Subjects would now be escorted for a short while during training
Subjects would now be instructed to stand under open sky while waiting for a GPS fix
Evaluation #6
Subject was male in his fifties
There was high data transmission delay, causing several disconnections
One disconnection caused by subject forgetting to acknowledge instruction
Subject became conscientious inside Olin, tried to find stairs, but turned around and used elevator The extra activity caused an incorrect feedback
Subject used elevator at Sprague, and got correct feedback
The subject used stairs at ACB, but did not get correct feedback because of physical parameters
Evaluation #7
Subject was female in her forties
Session had no problems with getting lost, or losing server connection
Subject used elevator at Olin and got correct feedback
Outside Sprague, subject started conversing with someone, which caused some delay
Subject used elevator inside Sprague and got correct feedback
Subject used stairs inside ACB, which was correctly recognized
Location Recognition Summary
Eight instances of some delay in all sessions because of:
Subject near structures e.g. sunshade
Subject not close enough to building to trigger next instruction
Subject placing GPS receiver in pockets
One session had to be aborted because location misrecognition prevented new instruction from triggering
Activity Recognition Summary
Nine instances of stair climbing at different buildings
Out of nine instances of stair climbing, six were categorized as walking
Major reasons:
Subject running on stairs
Activity model based on one person
Single stair case used
More time needed to modify activity recognition capabilities
Subject Interview Summary
What features you would like to see?
Comparison, gaming, privacy, discrete packaging, a larger pool of messages
Would you like to use such a device all day?
Both older subjects expressed worry about using device at work
Younger group enthusiastic about using the device in their daily activities
Did you alter your behavior during the course based on feedback?
Two subjects attributed their changed pace on way back from course because of the feedback they had received
One subject consciously tried to look for the stairs for way down when feedback was received
Connectivity and Stability Summary
Eight instances of disconnection from server
Data transmission delay to server
Recoverable with session restart
Two instances where subject did not acknowledge new instruction. One was not recovered as he was not aware that session had closed
One instance of smartphone freezing
Session restarted
Connection stability affected by experiment time and location
Sessions taking place late afternoon had more disconnections than other timings
Research Answers
How could activity and location information be captured?
Accelerometers and a GPS receiver placed on the user were adequate sources to capture this information
Can these sources be feasibly used for choosing feedback related to the health choices of a user?
The development and evaluation of HMSS has shown that raw data from these sources can be used to determine a user’s context, which in turn can help the system select an appropriate feedback
Could feedback from the system be effective in changing behavior?
The results showed mixed result in changing behavior. Also the results were demonstrated for a short term. The study did uncover a clue to demographic factors in acceptance of such a feedback mechanism. Further study is required
Contributions
An understanding of user attitude towards real time health feedback devices
Knowledge gained about the subject and technical factors that can affect the outcome and results of mobile field experiments
An exploration of how customized real time reference to physical parameters such as activity and walking pace could affect short-term behavior
Future Goals
Context-awareness
Increase activity and location recognition accuracy
Use more data for context recognition e.g. weather
Implement user definable feedback parameters
Increase feedback message variety and quantity
Reduce data transmission to server
Enable customized activity modeling
Reduce power consumption
Publication: Article submitted to IEEE Pervasive Computing
Questions
References
S. Mitchell, M. D. Spiteri, J. Bates, and G. Coulouris, "Context-aware multimedia computing in the intelligent hospital," Proceedings of the 9th workshop on ACM SIGOPS European workshop: beyond the PC: new challenges for the operating system , pp. 13-18,2000.
D. Siewiorek, A. Smailagic, J. Furukawa, A. Krause, N. Moraveji, K. Reiger, and J. Shaffer, "SenSay: a context-aware mobile phone," Wearable Computers, 2003.
Proceedings. Seventh IEEE International Symposium on , pp. 248-249, 2003.
A. K. Dey, "Understanding and Using Context," Personal and Ubiquitous Computing , vol. 5, no. 1, pp. 4-7, 2001.
H. W. Gellersen, A. Schmidt, and M. Beigl, "Adding Some Smartness to Devices and Everyday Things," Proceedings of IEEE Workshop on Mobile Computing Systems and Applications 2000 (WMCSAA 2000) , 2000.
0 comments
Post a comment