Randomised Controlled Trials of full vs. Lite version of the app yielded confusing yet intriguing results. New questions were generated for qualitative research in the next phase of the study.Presented by Yitong Huang (Echo) on 25 June 2014 at Festival for Digital Health, London.
Evaluating a mobile app for healthier snacking behaviours
1. The role of regulatory
fit in using a mobile app
for healthier snacking
behaviours
25 June 2014, Festival of Digital Health
Yitong Huang
MSc Social Cognition: Research and Applications
UCL Division of Psychology and Language Sciences
Supervised by Prof. Anne Hsu & Prof. Ann Blandford
2. Conclusion
Traditional method like Randomised Controlled
Trials can provide a rigorous evaluation of the
overall effect of technology-based health
interventions;
However, it is important, both research-wise
and design-wise, to contextualise user data
through qualitative analysis and structural
modelling of individual differences;
The success of persuasive technologies requires
a regulatory fit between the user and the
application.
3. Introduction
iCrave - a mobile app designed in a
previous study (Hsu et al., 2014);
It helps users curtail snack cravings and
improve snack choices through imagery;
Allows the user to “save” cravings, or opt
for healthy snack;
The user can recall how many snacks
they have eaten or saved in the past.
My research focus:
A more rigorous method to evaluate iCrave
and other persuasive health technologies
4. Conventional Evaluation Method
–Randomised Controlled Trials
Weekly slots since 16 Apr.2014
81 pre-study questionnaire responses; 34 completed
the 1-week experiment and post-study
questionnaires
Control(N=17) iCrave (N=17)
Mean Std Mean Std
Age (years) 22.35 2.50 23.17 5.53
BMI 22.99 3.14 22.63 4.38
Crave for snacks (times/day) 2.88 1.69 2.71 1.26
Crave for unhealthy snacks
(times/day)
2.29 1.65 2.18 1.13
Eat snacks(times/day) 2.29 1.49 2.47 1.00
Eat unhealthy snacks 1.82 1.43 1.88 .99
Demographics & Baseline Snacking Behaviours
Control iCrave
5. Repeated Measures ANOVA of Snack Consumption
Source df SS MS F p
Between Subject
Condition 1 7.736 7.736 1.215 n.s.
Within Subject
Day
Day * Condition
Data source (survey vs. app)
Data source * Condition
Day * Data source
Day * Data source * Condition
6
6
1
1
6
6
30.004
16.913
5.403
9.282
2.026
3.238
5.001
2.819
5.403
9.282
.338
.540
4.364
2.460
4.490
7.714
.614
.982
.000
.026
.042
.009
n.s.
n.s.
Snack consumption decreased during the
week for both groups, with steeper slope in
control group
app data tended to underreport snacking,
especially for the experiment group
7. New Questions for Qualitative
Analysis
In what ways do users find the app useful or not useful?
Does it work in the same way for everyone?
What factors moderate and mediate the effectiveness
of the application?
Moderator: individual differences (e.g. motivational and
cognitive styles, personality etc.)
Mediator: subjective user experience
8. Promotion Focus
• Gain vs. Nongain
• Approach strategies
• Errors of omission
Prevention Focus
• Loss vs. Nonloss
• Avoidance strategies
• Error of commission
Regulatory Fit Theory (Higgins, 1997)
“Regulatory focus orientation”:
people’s tendency toward promotion
versus prevention focus when they
consider what goals to pursue and how
to pursue goals
Dispositional difference in regulatory
orientation can be operationalized by
BAS/BIS scales (Carver & White, 1994)
Regulatory fit occurs when the
regulatory focus of the individual is
sustained by that of the task framing
* BAS: Behavioural Activation System; BIS: Behavioural Inhibition System
e.g. eaten snacks;
imagery
e.g. saved cravings
9. •It was just recording
my snacks, without
changing my habits
•More creative
feature/ aesthetically
pleasing interface
•Remind me there is a
healthy/save snack
option
•Looking at “save”
history makes me feel
good
•Not strict enough
•Should automate
reminder
•The imagery doesn’t
stop me from craving
•Distract from craving
•Keep track of my junk
food habits
useless useful
Promotion Focus (Gain Framing)
Prevention Focus (Loss framing)
Qualitative Data Analysis Framework
10. Future direction
Structural equation modelling of the influence
of “fit” on the health efficacy of persuasive
technologies, mediated by the subjective user
experience.
Decreased Strength
of craving
Decreased
impulsive eating
Improved control
efficacy
App
Effectiveness
E
E
E
Regulatory
Fit
User’s motivational
style* perceived
regulatory focus of the
technology
Subjective User
Experience
D
D
DImproved snack
choices
E
11. Conclusion
Traditional method like Randomised Controlled
Trials can provide a rigorous evaluation of the
overall effect of technology-based health
interventions;
However, it is important, both research-wise
and design-wise, to contextualise user data
through qualitative analysis and structural
modelling of individual differences;
The success of persuasive technologies requires
a regulatory fit between the user and the
application.
12. Reference
Ainslie, G. (1992). Picoeconomics: The strategic interaction of successive motivational states
within the person. Cambridge, England: Cambridge University Press.
Carver, C. S., & White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective
responses to impending reward and punishment: The BIS/BAS Scales. Journal of Personality and
Social Psychology.
Cesario, J., Grant, H., & Higgins, E. T. (2004). Regulatory fit and persuasion: transfer from
“Feeling Right.”. Journal of personality and social psychology (Vol. 86, pp. 388–404).
doi:10.1037/0022-3514.86.3.388
Dholakia, U. M., Gopinath, M., Bagozzi, R. P., & Nataraajan, R. (2006). The Role of Regulatory
Focus in the Experience and Self-Control of Desire for Temptations. Journal of Consumer
Psychology.
Law, E. L.-C., & van Schaik, P. (2010). Modelling user experience – An agenda for research and
practice. Interacting with Computers, 22(5), 313–322.
Higgins, E. Tory. (1997). Beyond pleasure and pain. The American psychologist, 52, 1280–300.
Higgins, E. T., & Spiegel, S. (2004). Promotion and prevention strategies for self-regulation: A
motivated cognition perspective. In Handbook of self-regulation: Research, theory, and
applications (pp. 171–187).
Hsu, A. and Blandford, A. (2014） Designing for Psychological Change: Individuals' Reward and
Cost Valuations in Weight Management. Journal of Medical Internet Research
Hsu, A., Yang, J., Yilmaz, Y., Haque, M. S., Can, C., & Blandford, A. (2014). Persuasive
technology for overcoming food cravings and improving snack choices, Proceedings of CHI
Editor's Notes
Frame goals and outcomes in terms of gain vs. nongain;
Initiate acts to avoid missing out opportunities to accomplish something
Maintain status quo to avoid making mistakes
Both promotion and prevention focused individuals can have strong and weak craving for food, although the underlying mechanism might be different. Prevention – eat to get rid of negative feeling, negative reinforcement; Promotion – eat because of the positive reinforcement. But research shows reward sensitivity correlate with food craving postiviely
When users comment on app features, both usefulness and unusefulness of the app can be framed in promotion (gain) or prevention (loss) framing.
Informed by the qualitative analysis, the future direction would be use structural model to predict what intervention(s) work best for users of a certain personality
Axis: Time before, during;
Line: Condition: iCrave, Saver
