2. We want behavioral interventions that are…
• Evidence-based
• Cost-effective
• Easy to deliver/disseminate
• Promote maintenance
• Personalized
• Fit into a person’s daily life
• Financially self-sustaining/
reimbursable
@ehekler
4. Behavior is complex…
Hovell M, Wahlgren D, Adams M. The logical and empirical basis for the behavioral ecological model. Emerging
theories in health promotion practice and research. 2009;2:347-85.
5. Outline
• Target
– Precision behavior-change interventions
• Problem
– Our science is not matched to the target
• Reasons
– Subject-Matter – Behavior change is wicked
– Methods – Evaluation not granular enough
• (Possible) Solution
– Agile Science
Flickr -Espen_Faugstad@ehekler
6. Outline
• Target
– Precision behavior-change interventions
• Problem
– Our science is not matched to the target
• Reasons
– Subject-Matter – Behavior change is wicked
– Methods – Evaluation not granular enough
• (Possible) Solution
– Agile Science
Flickr -Espen_Faugstad@ehekler
7. Why now for precision behavior-change interventions?
Flickr – Stuck in Customs
Digital health technologies
are pervasive, personal,
and (potentially) powerful.
@ehekler
Health behaviors explain
the most in individual
health variations.
40
15
5
10
30
Sub-Optimal Health behaviors
Social Circumstances
Environmental Exposures
Healthcare
Genetics
McGinnis, et al. 2002 Health Affairs
8. What is a precise behavioral intervention?
@ehekler
Just in time Adaptive
Flickr -Casgen
9. What are the pieces of precision?
Flickr -Eldeem@ehekler
10. The pieces
• Technologies to sense & provide feedback
• What’s my goal right now?
• When is “now?”
• What will help right now?
• How do I choose the right option for now?
@ehekler
14. Decision rules
How to choose the right option for now?
@ehekler Flickr-Gerry Dincher
15. Precision behavior change spectrum
Individual/User
Controlled
System
Controlled
Individual/System
Balanced Control
@ehekler
16. System controlled
JITAI approach – “giving the fish”
NSF IIS-1449751: EAGER: Defining a Dynamical Behavioral
Model to Support a Just in Time Adaptive Intervention, PIs, Hekler & Rivera
@ehekler
17. Study designs to build controller
-100
100
300
500
700
900
1100
1300
1500
0
2000
4000
6000
8000
10000
12000
14000
1 8 15 22 29 36 43 50 57 64 71 78 85 92 99
Points
Stepsperday
Days
Points Provided (100, 300, 500)
Fictionalized actual steps per day
Daily step goal ((Baseline Median) to (Baseline Median+100% Baseline Median))
NSF IIS-1449751: EAGER: Defining a Dynamical Behavioral Model to Support a Just in Time
Adaptive Intervention, PIs, Hekler & Rivera
19. Individual controlled
DIY approach - “teach to fish”
Eric Hekler, Jisoo Lee, Erin Walker, Winslow Burleson,
Arizona State University; Bob Evans, Google Flickr Juhan Sonin
25. Outline
• Target
– Precision behavior-change interventions
• Problem
– Our science is not matched to the target
• Reasons
– Subject-Matter – Behavior Change is Wicked
– Methods – evaluation not granular enough
• (Possible) Solution
– Agile Science
Flickr -Espen_Faugstad@ehekler
26. Classic evidence pipeline
Phase 1
Define &
Design
Phase 2
Pilot and
Proof of
Concept
Phase 3
Efficacy
Trial
(RCT)
Phase 4
Effectiveness
Trial
@ehekler
27. Flickr – Metrix X
2005 2006 2007 2008 2009 2010 2011 2012
Conceive
of a study
Gather
Pilot Data
Submit
Grant
Receive
Funding
Conduct the study
Submit publications
for review
500,000th App
Accepted on
App Store
@ehekler
28. What are useful “products” from science?
Thursday, May 2, 14:00-15:20 | Mind the Theoretical Gap: Interpreting, Using, and Developing Behavioral Theory in HCI Research
@ehekler
29. What are useful “products” from science?
Thursday, May 2, 14:00-15:20 | Mind the Theoretical Gap: Interpreting, Using, and Developing Behavioral Theory in HCI Research
30. Outline
• Target
– Precision behavior-change interventions
• Problem
– Our science is not matched to the target
• Reasons
– Subject-Matter – Behavior change is wicked
– Methods – Evaluation not granular enough
• (Possible) Solution
– Agile Science
Flickr -Espen_Faugstad@ehekler
31. Horst Rittel’s “wicked problems”
Rith C, Dubberly H. Why Horst WJ Rittel matters. Design Issues. 2007;23(1):72-91.
@ehekler
35. “Taming” a wicked problem
• ill-defined problems
– Stakeholder interaction
• Mutating problems
– Rapid iteration
• Context-dependent problems
– “Match” between solution and context
Chambers et al. 2013; Riley, et al, 2013; Rittel & Webber, 1973; van Gemert-Pijnen et al. 2011; Hekler et al. under review
36. What do we get from RCTs?
Stead LF, Lancaster T. Group behaviour therapy programmes for smoking cessation. Cochrane Database of
Systematic Reviews 2005, Issue 2. Art. No.: CD001007. DOI: 10.1002/14651858.CD001007.pub2.@ehekler
37. What do we get from RCTs?
Mohr DC, Ho J, Hart TL, Baron KG, Berendsen M, Beckner V, et al. Translational Behavioral Medicine. 2014:1-17.@ehekler
45. Just in Time “Meaningful Moments”
• State of opportunity or vulnerability
• Receptive
• Behavior does not have to happen now
@ehekler Nahum-Shani, Hekler, & Spruijt-Metz, under review
52. What evidence do we need for
matchmaking algorithms?
@ehekler
“Perfect” intervention
package?
Pieces for meaningful moments
with insights on how to recombine?
Flickr - Paul Swansen Flickr - Benjamin Esham
www.agilescience.org
67. Thank you!
Dr. Eric Hekler
Arizona State University
ehekler@asu.edu
@ehekler
Flickr- Tony Fischer
Editor's Notes
Beyond just the difficulty of the complexity of behavior and the behavioral problems we are trying to solve, there are a lot of demands on behavior change technologies themselves and different points that tend to thought about from different disciplines. Indeed, we want behavior change technologies that are evidence-based, cost-effective, personalized, easy to disseminate, promote maintenance, fit into a person’s life, and can, hopefully be financially self-sustained in sustaining. As can be seen just from this list, this can’t be achieved through the class disciplinary silo model of creation.
Discuss the lack of understanding from behavioral scientists on how to really deal with big data and opportunities for setting up “in the wild” studies that could later be harnessed for A/B testing. Nice melding of behavioral science knowledge of randomized controlled trials and HCI’s knowledge on the systems to automate those types of systems in the real-world.
Research suggests effective interventions for behavior change have been developed but often the cumulative scientific knowledge base about behavioral interventions does not translate into actionable and repurposable insights for novel interventions in novel contexts. A plausible reason for this may have to do with the nature of the problems we are attempting to address. In brief, it is plausible that the development of behavior change interventions is a "wicked problem," meaning a type of problem that is not easily amenable to a linear translation of scientific knowledge into practice by practitioners. This is particularly the case with the increased interest in even more personalized, individualized, and precise behavior change interventions delivered in context via digital health technologies (i.e., interventions that can provide the most appropriate intervention option for a person in the exact moment when it is needed and not deliver anything when it is not needed). PURPOSE: The purpose of this talk is to present a reconceptualization Rittel’s “Wicked Problem” formulation (i.e., a class of problems that is not “solvable” using traditional scientific methods) and to then discuss how this reconceptualization influences the types of methods and processes needed to develop behavior change interventions, particularly individualized behavior change interventions. To illustrate this point, Dr. Hekler will highlight insights from his ongoing research including: his NSF-funded project focused on using control systems engineering methods for making decisions on exactly when, where, and how to intervene for promoting activity, his Google-funded project focused on developing a "DIY Behavior Change Self-Experimentation Toolkit", and his Robert Wood Johnson Foundation grant focused on developing a process he has labeled Agile Science, including the development of a prototyping platform for behavior change techniques that is being developed in partnership with Bob Evans at Google and Max Utter from Jawbone.
Research suggests effective interventions for behavior change have been developed but often the cumulative scientific knowledge base about behavioral interventions does not translate into actionable and repurposable insights for novel interventions in novel contexts. A plausible reason for this may have to do with the nature of the problems we are attempting to address. In brief, it is plausible that the development of behavior change interventions is a "wicked problem," meaning a type of problem that is not easily amenable to a linear translation of scientific knowledge into practice by practitioners. This is particularly the case with the increased interest in even more personalized, individualized, and precise behavior change interventions delivered in context via digital health technologies (i.e., interventions that can provide the most appropriate intervention option for a person in the exact moment when it is needed and not deliver anything when it is not needed). PURPOSE: The purpose of this talk is to present a reconceptualization Rittel’s “Wicked Problem” formulation (i.e., a class of problems that is not “solvable” using traditional scientific methods) and to then discuss how this reconceptualization influences the types of methods and processes needed to develop behavior change interventions, particularly individualized behavior change interventions. To illustrate this point, Dr. Hekler will highlight insights from his ongoing research including: his NSF-funded project focused on using control systems engineering methods for making decisions on exactly when, where, and how to intervene for promoting activity, his Google-funded project focused on developing a "DIY Behavior Change Self-Experimentation Toolkit", and his Robert Wood Johnson Foundation grant focused on developing a process he has labeled Agile Science, including the development of a prototyping platform for behavior change techniques that is being developed in partnership with Bob Evans at Google and Max Utter from Jawbone.
NOTE, this current draft is just to get a sense of timing and flow on key points to discuss. Formatting on almost all slides will not remain (e.g., likely will NOT have the titles at the top like that).
- OK, now you’re creating a plan for your problem. For a successful plan, you should set an appropriate goal, and come up with ways to apply behavior change techniques.
- OK, now you’re creating a plan for your problem. For a successful plan, you should set an appropriate goal, and come up with ways to apply behavior change techniques.
- OK, now you’re creating a plan for your problem. For a successful plan, you should set an appropriate goal, and come up with ways to apply behavior change techniques.
Research suggests effective interventions for behavior change have been developed but often the cumulative scientific knowledge base about behavioral interventions does not translate into actionable and repurposable insights for novel interventions in novel contexts. A plausible reason for this may have to do with the nature of the problems we are attempting to address. In brief, it is plausible that the development of behavior change interventions is a "wicked problem," meaning a type of problem that is not easily amenable to a linear translation of scientific knowledge into practice by practitioners. This is particularly the case with the increased interest in even more personalized, individualized, and precise behavior change interventions delivered in context via digital health technologies (i.e., interventions that can provide the most appropriate intervention option for a person in the exact moment when it is needed and not deliver anything when it is not needed). PURPOSE: The purpose of this talk is to present a reconceptualization Rittel’s “Wicked Problem” formulation (i.e., a class of problems that is not “solvable” using traditional scientific methods) and to then discuss how this reconceptualization influences the types of methods and processes needed to develop behavior change interventions, particularly individualized behavior change interventions. To illustrate this point, Dr. Hekler will highlight insights from his ongoing research including: his NSF-funded project focused on using control systems engineering methods for making decisions on exactly when, where, and how to intervene for promoting activity, his Google-funded project focused on developing a "DIY Behavior Change Self-Experimentation Toolkit", and his Robert Wood Johnson Foundation grant focused on developing a process he has labeled Agile Science, including the development of a prototyping platform for behavior change techniques that is being developed in partnership with Bob Evans at Google and Max Utter from Jawbone.
Research suggests effective interventions for behavior change have been developed but often the cumulative scientific knowledge base about behavioral interventions does not translate into actionable and repurposable insights for novel interventions in novel contexts. A plausible reason for this may have to do with the nature of the problems we are attempting to address. In brief, it is plausible that the development of behavior change interventions is a "wicked problem," meaning a type of problem that is not easily amenable to a linear translation of scientific knowledge into practice by practitioners. This is particularly the case with the increased interest in even more personalized, individualized, and precise behavior change interventions delivered in context via digital health technologies (i.e., interventions that can provide the most appropriate intervention option for a person in the exact moment when it is needed and not deliver anything when it is not needed). PURPOSE: The purpose of this talk is to present a reconceptualization Rittel’s “Wicked Problem” formulation (i.e., a class of problems that is not “solvable” using traditional scientific methods) and to then discuss how this reconceptualization influences the types of methods and processes needed to develop behavior change interventions, particularly individualized behavior change interventions. To illustrate this point, Dr. Hekler will highlight insights from his ongoing research including: his NSF-funded project focused on using control systems engineering methods for making decisions on exactly when, where, and how to intervene for promoting activity, his Google-funded project focused on developing a "DIY Behavior Change Self-Experimentation Toolkit", and his Robert Wood Johnson Foundation grant focused on developing a process he has labeled Agile Science, including the development of a prototyping platform for behavior change techniques that is being developed in partnership with Bob Evans at Google and Max Utter from Jawbone.
Rittel was a professor (mostly at UC Berkeley) of design and architecture (could be conceived as a design planner). Rittel functioned to transfer knowledge from sciences and engineering to design professions. He inspired work in urban design, and, more recently computer science. Key to his formulation was this idea of “Wicked” Problem.
(1) ill-defined formulations (i.e., successful formulations are subjectively defined and have multiple solutions with success determined on a continuum of good to bad); (2) context-dependent formulations (i.e., successful formulations are idiosyncratic to the context [e.g., time, place, cultural milieu] thus making replication and generalizability difficult); and (3) mutating formulations, often rapidly (i.e., successful formulations are “useful” for a limited time before the problem or platform the solution was built on has mutated).
We are facing complexity societal problems however.
(1) ill-defined formulations (i.e., successful formulations are subjectively defined and have multiple solutions with success determined on a continuum of good to bad); (2) context-dependent formulations (i.e., successful formulations are idiosyncratic to the context [e.g., time, place, cultural milieu] thus making replication and generalizability difficult); and (3) mutating formulations, often rapidly (i.e., successful formulations are “useful” for a limited time before the problem or platform the solution was built on has mutated).
Developing methods and processes for optimizing individual intervention components.
This is a method that helps you to test the decisions that you need to make when individuals are not responding to an intervention.
However, often many questions must be answered in order to develop a high-quality adaptive intervention:
Should treatment begin with CORE-DTT instead of JASP-EMT?
For improved responders, is parent training really helpful?
Do all slow responders require combined treatment?
Sequential Multiple Assignment Randomized Trial (SMART) designs are used to answer such questions.
This is an example SMART design for constructing a high-quality adaptive intervention for children with autism who are minimally verbal.
Background:
Earlier research has shown that even after early language-skills training, about one-third of school-aged children with Autism Spectrum Disorder remain minimally verbal.
Study Design:
The study utilizes a sequential multiple assignment randomized trial (SMART) design. All children with autism are randomized twice: once initially to understand which is the best first-stage treatment to improve spoken communication, and again at the end of week 6 to understand which is the best second-stage treatment depending on whether the child is a slow responder or an improved responder to first-stage treatment. The primary outcome is spontaneous spoken communication. Researchers plan to enroll 200 children in four cities: Los Angeles, Nashville, New York City, and Rochester (N.Y.)
Primary Aim:
This study will initially compare two types of intensive, daily instruction for children with autism spectrum disorder (ASD) who use only minimal verbal communication. The two interventions are JASP-EMT (a novel approach to improving communication) and CORE-DTT (the traditional approach to improving communication).
Secondary Aim 1:
Among children who are slow responders at week 6, the study will compare continued treatment versus an intervention that combines both types of instruction.
Secondary Aim 2:
Among children who are improved responders at week 6, the study will compare continued treatment versus augmenting the intervention with parent training.
Grant Information:
Funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), and collaborated by the National Institute on Deafness and Other Communication Disorders (NIDCD).
Grant #: RFA-HD-12-196
Additional Information:
Website: http://www.npistat.org/catherine/aim-asd-new
Clinical Trials #: NCT01751698
Lead PI: Connie Kasari, UCLA
Methods PIs: Daniel Almirall and Susan A Murphy, Univ of Michigan
Site PIs: Ann Kaiser (Vanderbilt, TN), Catherine Lord (New York, NY), Tristram Smith (Rochester, NY)
Research suggests effective interventions for behavior change have been developed but often the cumulative scientific knowledge base about behavioral interventions does not translate into actionable and repurposable insights for novel interventions in novel contexts. A plausible reason for this may have to do with the nature of the problems we are attempting to address. In brief, it is plausible that the development of behavior change interventions is a "wicked problem," meaning a type of problem that is not easily amenable to a linear translation of scientific knowledge into practice by practitioners. This is particularly the case with the increased interest in even more personalized, individualized, and precise behavior change interventions delivered in context via digital health technologies (i.e., interventions that can provide the most appropriate intervention option for a person in the exact moment when it is needed and not deliver anything when it is not needed). PURPOSE: The purpose of this talk is to present a reconceptualization Rittel’s “Wicked Problem” formulation (i.e., a class of problems that is not “solvable” using traditional scientific methods) and to then discuss how this reconceptualization influences the types of methods and processes needed to develop behavior change interventions, particularly individualized behavior change interventions. To illustrate this point, Dr. Hekler will highlight insights from his ongoing research including: his NSF-funded project focused on using control systems engineering methods for making decisions on exactly when, where, and how to intervene for promoting activity, his Google-funded project focused on developing a "DIY Behavior Change Self-Experimentation Toolkit", and his Robert Wood Johnson Foundation grant focused on developing a process he has labeled Agile Science, including the development of a prototyping platform for behavior change techniques that is being developed in partnership with Bob Evans at Google and Max Utter from Jawbone.
Based on this, we need to move more into an open discussion in which we explore lots and lots of different ideas if we really want to understand which ones are best.
Sadly, science, particularly behavioral science doesn’t really have the sort of “maker” culture that would allow us. As such, a key emphasis.
Based on this, we need to move more into an open discussion in which we explore lots and lots of different ideas if we really want to understand which ones are best.
Sadly, science, particularly behavioral science doesn’t really have the sort of “maker” culture that would allow us. As such, a key emphasis.
Based on this, we need to move more into an open discussion in which we explore lots and lots of different ideas if we really want to understand which ones are best.
Sadly, science, particularly behavioral science doesn’t really have the sort of “maker” culture that would allow us. As such, a key emphasis.
Based on this, we need to move more into an open discussion in which we explore lots and lots of different ideas if we really want to understand which ones are best.
Sadly, science, particularly behavioral science doesn’t really have the sort of “maker” culture that would allow us. As such, a key emphasis.
Based on this, we need to move more into an open discussion in which we explore lots and lots of different ideas if we really want to understand which ones are best.
Sadly, science, particularly behavioral science doesn’t really have the sort of “maker” culture that would allow us. As such, a key emphasis.
Developing methods and processes for optimizing individual intervention components.
Based on this, we need to move more into an open discussion in which we explore lots and lots of different ideas if we really want to understand which ones are best.
Sadly, science, particularly behavioral science doesn’t really have the sort of “maker” culture that would allow us. As such, a key emphasis.
