Recent advances in the evidence base for technology-based behavioral health applications have provided clinicians a better understanding and guidance on the integration of these tools into clinical care. Participants will learn about research findings on current technologies in use in clinical practice, such as audio conferencing, video conferencing, and virtual reality, in addition to tools available for use between patients, such as the use of websites and mobile applications and wearable sensors.

1. Evidence Base for Using Technology Solutions
in Behavioral Health Care
December 17, 2014, 1-2:30pm (EST)
Nancy A. Skopp, Ph.D.
Research Psychologist and Program Manager
National Center for Telehealth & Technology (T2)
Defense Centers of Excellence for Psychological
Health and Traumatic Brain Injury (DCoE)
Joint Base Lewis-McChord, Wash.
Christina M. Armstrong, Ph.D.
Clinical Psychologist, Program Lead for T2 Education & Training Program
National Center for Telehealth & Technology (T2)
Defense Centers of Excellence for Psychological
Health and Traumatic Brain Injury (DCoE)
Joint Base Lewis-McChord, Wash.
Janyce “Jae” Osenbach, Ph.D.
Research Psychologist and Psychometrician
National Center for Telehealth & Technology (T2)
Defense Centers of Excellence for Psychological
Health and Traumatic Brain Injury (DCoE)
Joint Base Lewis-McChord, Wash.
Presenters:
Moderator:

2. Webinar Overview
2
Recent advances in the evidence base for technology-based behavioral health
applications have provided clinicians a better understanding and guidance on the
integration of these tools into clinical care. Participants will learn about research
findings on current technologies in use in clinical practice, such as audio
conferencing, video conferencing, and virtual reality, in addition to tools available
for use between patients, such as the use of websites and mobile applications
and wearable sensors. Clinicians will leave this training with a review of the
evidence base for using technology solutions in behavioral health care that will
inform their clinical practice.
At the conclusion of this webinar, learners will be able to:
 Describe a theoretical perspective useful in the conceptualization and
application of technology-based interventions for clinical practice
 Differentiate between the concepts of synchronous and asynchronous
technologies with respect to behavioral health interventions
 Examine the current status of literature on technologically-supported
behavioral health interventions
 Assess potential gaps and recognize future trends in behavioral health
technology tools into clinical care
2

3. Disclosures
 The views expressed in this presentation are those of the
presenters and do not reflect the official policy of the
Department of Defense or the U.S. Government.
 We have no relevant financial relationships to disclose.
 We will be discussing web and mobile applications that have
been developed by the Defense Department, including those
developed by the National Center for Telehealth and
Technology (T2). Some of these applications may fall under
FDA device regulations, and in those cases we will consult
with the USAMRMC Division of Regulated Activities and
Compliance for guidance.
3

4. Overview
1. Evidence Base and Theory
– Theory to inform modality selection
– Clinical practice technologies
– “White space” technologies
2. Key Concepts
– Media Synchronicity Theory
– Synchronous
– Asynchronous
4

5. 5
 Synchronous - communication in real time
 Asynchronous - communication not in real time
Definitions

6.  Synchronous technologies support
convergence of understanding
 Asynchronous technologies maximize the
conveyance of information to support
individual level analysis
6
(Dennis, et.al., 2008)
Media Synchronicity Theory
(MST)

7. 7
Synchronous
Convergence
(e.g., video telehealth)
Asynchronous
Conveyance
(e.g., website)
Treatment
Outcomes
(Castonguay, Constantino, & Holtforth, 2006; Hatcher, Barends, Hansell, & Gutfreund,
1995; Norcross, 2011; Jarvis-Selinger, Chan, Payne, Plohman,& Kendall, 2008
Model to Guide Selection of Modality
Delivery
Success
• Therapeutic Alliance
• Technology
• Training
(Figure 1. Dennis, Fuller, Valacich, 2008)

8.  Telephone Interventions
 Video Telehealth
 Virtual Worlds
8
Evidence Base
Synchronous Technologies

9. Telephone Interventions
Cons
 No non-verbal cues
 Potential interruptions
 Confidentiality
 Safety
9
Pros
 Convenient
 Widely available
 Inexpensive
 Remote outreach
 Ease of operation

10. Overview: Telephone
Interventions
10
 Wide range of behavioral health applications1
 Ability to overcome barriers to conventional
care2
 Literature characterized by heterogeneity3
– Uses
– Type of treatment
– Research methods
1(Bee et al., 2008; Brenes, Ingram & Danhauer, 2012; Eakin, Lawler, Vandelanotte & Owen, 2007; Villanti, McKay,
Abrams, Holtgrave & Bowie, 2010; van Velthoven, Car, Car, & Atun, 2012)
2(Bee et al., 2008; Brenes et al., 2012; Eakin et al., 2007; Mohr, Vella, Hart, Heckman, & Simon, 2008)
3(Bee et al., 2008)

11. Telephone Interventions:
Applications
11
 Depression1
 Anxiety disorders2
 Smoking cessation3
 Alcohol abuse4
 Health behaviors – HIV, weight management, chronic
disease prevention5
1(Bee et al., 2008; Choi et al., 2014; Mohr et al., 2008)
2(Bee et al. 2008; Brenes et al., 2012; Lovell et al., 2006)
3(Villanti et al., 2010)
4(Lenaerts, Mathei, Matthys, Zeeuws, Pas, Anderson & Aertgeerts, 2014)
5(Eakin et al., 2007; Simek, McPhate, & Haines, 2012; vanVelthoven et al., 2012)

12. Video Telehealth
Pros
 Remote outreach
 Increased access
 Transportation $0.00
 Remote treatment
for low-base-rate
problems
Cons
 Technical issues
 Confidentiality issues
 No federal laws
 Start-up costs and
remuneration
12

13. Video Telehealth at a Glance
1996-2012
Empirical
Studies
47 (of 65 studies); 45% were controlled
Most Common
Psychotherapy
45% CBT
Session
Format
71% individual; 17% group; 10% family; 2%
other
Populations 86% adult; 10% child/adolescent; 5% unclear
74% Civilian; 21% Veteran; 5% Civilian + Military
Feasibility 38% positive contributions
Therapeutic
Relationship
34% examined; 88% strong alliance
13
(Table 1. Backhaus et al., 2012)

14. Video Telehealth: Clinical
Applications
 Anxiety and mood disorders1
 Eating disorders2
 Addiction2
 Physical ailments2
 Smoking Cessation3
 Parenting and child problems4
1(Backhaus et al., 2012; Gros et al., 2013)
2(Backhaus et al., 2012)
3(Carlson et al., 2012)
4(Backhaus et al., 2012; Comer et al., 2013; Himle et al, 2012; Reese, Slone, Soares, & Sprang, 2012)
14

15. Video Telehealth (VT) Clinical
Outcome Data
VT
= In-persona
VT
≥ In-personb
Depression
Anxiety Disorders
Eating Disorders
Anger
Physical Health Problems
Smoking Cessation
Child Problems
Substance Abuse
Disorder
15
Table 2. aNo significant between group differences found; Backhaus et al., 2012; Gros et al., 2013; bNelson,
Barnard, & Cain, 2003; Bouchard et al., 2004; Choi et al., 2014;

16. Video Telehealth (VT):
General Empirical Findings
 VT works
 Literature to support efficacy of VT
- Heterogeneous groups
- Diverse range of problems
 Some literature does not support VT
- Individual differences
- Nature of the disorder?
(Backhaus et al., 2012; Gros et al., 2013)
16

17. Video Telehealth: VA and DoD
17
 Driving force of video telehealth
 Ongoing and completed RCTs
 Preliminary evidence supports VT
 VT comparable to in-person therapy
 Majority of VT is facility-to-facility
(Gros et al., 2013)

18. Video Telehealth: Recent VA RCT
 Telemedicine Outreach for PTSD (TOP)
 11 outpatient units serving rural veterans
 Cognitive processing therapy (CPT) >
Usual care
18
(Fortney et al., 2014)

19. Video Telehealth: Research
Gaps
19
 Influence of demographics
 More non-inferiority trials needed
 Majority of research on PTSD
 Need research on process variables
(Backhaus et al., 2012; Gros et al., 2013)

20. Video Telehealth:
Additional Considerations
 Technology
 Ongoing support
 Training
 Room set-up
 Protocol development
20
(Jarvis-Selinger et al., 2008)

21. Virtual Worlds (VW)
 Immersive, 3D environments1
 Individualized
representations - “avatars” 1
 Research nascent
 Preliminary pilot – social
anxiety disorder2
 Future directions3
21
1(Pridmore & Phillips-Wren, 2011)
2(Yuen, Herbert, Forman, Goetter, Comer, & Bradley, 2013)
3(Morie, Haynes, & Chance, 2011; Riva, Wiederhold, Mantovani & Gaggioli, 2011)

22. Emerging Technologies
 Holographic Projection1
 Telepresence systems seamlessly approximate
a “true-to-life” workspace2
– Eye gaze tracking
– Life-sized imaging
– Visual continuity
22
1(Stefan & David, 2013)
2(O’Hara, Kjeldskov, & Paay, 2011)

23. Synchronous Technology:
Additional Considerations
 Technical Requirements1
– Facility with delivery
– Resolution, bandwidth, disconnection
 Interpersonal and Individual Factors2
– Therapeutic relationship
– Client preferences
– What works for whom?
1(Gros et al., 2013; Jarvis-Selinger et al., 2008)
2(Cavanagh & Millings, 2013; DeLucia, Harold, & Tang, 2013; Sucala, Schnur, Constantino, Miller, Brackman,
& Mongomery, 2012)
23

24. Poll Question #1
Do you use synchronous telehealth
with your patients?
24

25.  Web
 Mobile apps
 Emerging technologies
25
Evidence Base
Asynchronous Technologies

26. General Research Findings
26
(Lawlor & Kirakowski, 2014; Mohr, Burns, Schueller, Clarke, & Klinkman, 2013; Richards &
Richardson, 2012)

27. Poll Question #2
Do you use websites with your patients?
27

28. General Research Findings
28
(Bush, Skopp, Smolenski, Crumpton, & Fairall, 2013; Donker, Petrie, Proudfoot, Clarke, Birch, & Christensen, 2013; Gaggioli, & Riva,
2013; Seko, Kidd, & Wiljer, 2013)

29. App Marketplace Research
29
(Aguirre, McCoy, & Roan, 2013; Juarascio, Manasse, Goldstein, Forman, & Butryn, 2014)

30. The Paid-App Fallacy
30
(Boudreaux, Waring, Hayes, Sadasivam, Mullen, & Pagoto, 2014; West, Hall, Hanson, Barnes,
Giraud-Carrier, & Barrett, 2012)

31. Poll Question #3
Do you use mobile apps with your patients?
31

32. More Research Needed
32
(Ben-Zeev, Schueller, Begale, Duffecy, Kane, & Mohr, 2014)

33. Future of Devices
33

34. T2 Research
34
(Bush, et al., 2013a; Bush, et al., 2014a; Bush, et al., 2013b; Luxton, et al., 2014)

35. Poll Question #4
Have you used T2 mobile apps or websites?
35

36. Conclusion
In conclusion, during this webinar we:
 Described a theoretical perspective useful in the
conceptualization and application of technology-based
interventions for clinical practice
 Differentiated between the concepts of synchronous and
asynchronous technologies with respect to behavioral health
interventions
 Examined the current status of literature on technologically
supported behavioral health interventions
 Assessed potential gaps and recognized future trends in
using behavioral health technology tools in clinical care
36

37. References
Aguirre, R. T. P., McCoy, M. K., & Roan, M. (2013). Development guidelines from a study of suicide
prevention mobile applications (apps). Journal of Technology in Human Services, 31(3), 269-
293.
Backhaus, A., Agha, Z., Maglione, M.L., Repp, A. Ross, B.,…et al., Thorp, S.R. (2012).
Videoconferencing Psychotherapy: A systematic review. Psychological Services, 9, 111-131.
Bee, P.E., Bower, P. Lovell, K., Gilbody, S., Richards, D., Gask, L., & Roach, P. (2008). Psychotherapy
mediated by remote communication technologies: A meta-analytic review. BioMed Central
Psychiatry, 8, 60.
Ben-Zeev, D., Schueller, S. M., Begale, M., Duffecy, J., Kane, J. M., & Mohr, D. C. (2014). Strategies
for mHealth research: Lessons from 3 mobile intervention studies. Administration and Policy in
Mental Health and Mental Health Services Research, Advance online publication.
37

38. References
Brenes, G.A., Ingram, C.W., & Danhauer, S.C. (2011). Benefits and challenges of conducting
psychotherapy by telephone. Professional Psychology: Research and Practice, 42, 543-549.
Bouchard, S., Paquin, B., Payeur, R., Allard, M., Rivard,V., Fournier, T.,…Lapierre, J. (2004).
Delivering cognitive-behavior therapy for panic disorder with agoraphobia in videoconference.
Telemedicine Journal and e-health, 10, 13-25.
Boudreaux, E. D., Waring, M. E., Hayes, R. B., Sadasivam, R. S., Mullen, S., & Pagoto, S. (2014).
Evaluating and selecting mobile health apps: Strategies for healthcare providers and healthcare
organizations. Translational Behavioral Medicine, Advanced online publication
38

39. References
Bush, N. E., Dobscha, S. K., Crumpton, R., Denneson, L. M., Hoffman, J. E., Crain, A., ... & Kinn, J. T.
(2014a). A virtual hope box smartphone app as an accessory to therapy: Proof‐of‐concept in a
clinical sample of veterans. Suicide and Life-Threatening Behavior, Advanced online publication.
Bush, N. E., Ouelette, G., & Kinn, J. (2014b). Utility of the T2 Mood Tracker mobile application among
Army Warrior Transition Unit service members. Military Medicine, In press.
Bush, N. E., Prins, A., Laraway , S., O’Brien, K., Ruzek, J., & Ciulla, R. (2013a). A pilot evaluation of
the afterdeployment.org online posttraumatic stress workshop for military service members and
veterans. Psychological Trauma: Theory, Research, Practice, and Policy, Advance online
publication.
Bush, N. E., Skopp, N. A., Smolenski, D., Crumpton, R., & Fairall, J. (2013b). Behavioral screening
measures delivered with a smartphone ‘app’: Psychometric properties and user preference.
Journal of Nervous and Mental Disease, 201(11), 991-995.
39

40. References
Castonguay, L.G., Constantino, M.J., & Holtforth, M.G. (2006). The working alliance: Where we are and where should
we go? Psychotherapy, 43, 271-279.
Carlson, L.E., Lounsberry, J.J., Maciejewski, O., Wright, K., Collacutt, V., & Taenzer, P. (2012). Telehealth-delivered
group smoking cessation for rual and urban participants: Feasibility and cessation rates. Addictive Behaviors, 27,
108-114.
Cavanagh, K., & Millings, A. (2013). (Inter) personal computing: The role of the therapeutic relationship in e-mental
health. Journal Contemporary Psychotherapy, 4, 197-206.
Choi, N.G., Marti, C.N., Bruce, M.L., Hegel, M.T., Wilson, M.A., & Kunik, M.E. (2014). Six-month postintervention for
depression and disability outcome of in-home telehealth problem-solving therapy for depressed, low-income
homebound older adults. Depression and Anxiety, 31, 653-661.
Comer, J.S., Furr, J.M., Cooper-Vince, C.E., Kerns, C.E., Chan, P.T., Edson, A.L., …et al., Freeman, J.B. (2013).
Internet-delivered, family-based treatment for early onset OCD: A preliminary case series. Journal of Clinical
Child & Adolescent Psychology, 43, 74-87.
40

41. References
DeLucia, P., Harold, S.A., & Tang, Y. (2013). Innovation in Technology-aided psychotherapy through
human factors/ergonomics: Toward a collaborative approach. Journal of Contemporary
Psychotherapy, 43, 253-260.
Dennis, A.R., Fuller, R.M., & Valacich, J.S. (2008). Media, tasks, and communication processes: A
theory of media synchronicity. MIS Quarterly, 32, 575-600.
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M. R., & Christensen, H. (2013). Smartphones
for smarter delivery of mental health programs: A systematic review. Journal of Medical Internet
Research, 15(11), e247.
41

42. References
Eakin, E.G., Lawler, S.P., Vandelanotte, C., & Owen, N. (2007). Telephone interventions for physical
activity and dietary behavior change: A systematic review. American Journal of Preventive
Medicine, 32, 419-434.
Fortney, J.C., Pyne, J.M., Kimbrell, T.A., Hudson, T.J., Robinson, D.E., Schneider, R., Moore, W.M,
…et al., Schnurr, P.P. (2014). Telemedicine-based collaborative care for posttraumatic stress
disorder: a randomized clinical trial. JAMA Psychiatry. Epub ahead of print.
Gaggioli, A., & Riva, G. (2013). From mobile mental health to mobile wellbeing: Opportunities and
challenges. Studies in Health Technology and Informatics, 184, 141-147.
Gros, D.F., Morland, L.A., Greene, C.J., Acierno, R., Strachan, M., Egede, L.E., …et al., Frueh, B.C.
(2013). Delivery of evidence-based psychotherapy via video telehealth. Journal of
Psychopathology and Behavioral Assessment, 35, 506-521.
42

43. References
Hatcher, R.L., Barends, A., Hansell, J., & Gutfreund, M.J. (1995). Patients’ and therapists’ shared and
unique views of the therapeutic alliance: An investigation using confirmatory factor analysis in a
nested design. Journal Consulting and Clinical Psychology, 63, 636-643.
Himle, M.B., Freitag, M., Walther, M., Franklin, S.A., Ely, L., & Woods, D.W. (2012). A randomized pilot
trial comparing video conference versus face-to-face delivery of behavior therapy for childhood
tic disorders, 50, 565-570.
Jarvis-Selinger, S., Chan, E., Payne, R., Polhman, L.L.M., & Kendall, H. (2008). Clinical telehealth
across the disciplines: Lessons learned. Telemedicine and e-Health, 14, 720-725.
Juarascio, A. S., Manasse, S. M., Goldstein, S. P., Forman, E. M., & Butryn, M. L. (2014). Review of
smartphone applications for the treatment of eating disorders. European Eating Disorders
Review, Advanced online publication.
Lawlor, A., & Kirakowski, J. (2014). Online support groups for mental health: A space for challenging43

44. References
Lenaerts, E., Mathei, C., Matthys, F., Zeeuws, D., Pas, L., Anderson, P., & Aertgeerts, B. (2014).
Continuing care for patients with alcohol use disorders: A systematic review. Drug and Alcohol
Dependence, 135, 9-21.
Lovell, K., Cox, D., Haddock, G., Jones, C., Raines, D., Garvey, R., Roberts, C., & Hadley, S. (2006).
Telephone administered cognitive behaviour therapy for treatment of obessive compulsive
disorder: randomized controlled non-inferiority trial. British Medical Journal, 333, 883-886.
Luxton, D. D., Hansen, R. N., & Stanfill, K. (2014). Mobile app self-care versus in-office care for stress
reduction: A cost-minimization analysis. Journal of Telemedicine and Telecare, Advanced online
publication.
44

45. References
Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013). Behavioral intervention
technologies: Evidence review and recommendations for future research in mental health.
General Hospital Psychiatry, 35(4), 332-338.
Mohr, D.C., Vella, L., Hart, S., Heckman, T., & Simon, G. (2008). The effect of telephone-administered
psychotherapy on symptoms of depression and attrition: A meta-analysis. Clinical Psychology:
Science and Practice, 15, 243-253.
Morie, J.G., Haynes, E., & Chance, E. (2011). Warriors’ journey: A path to healing through narrative
exploration. International Journal of Disability and Human Development, 10, 17-23.
Nelson, E.L., Barnard, M., & Cain, S. (2003). Treating childhood depression over videoconferencing.
Telemedicine Journal ande-Health, 9, 49-55.
Norcross, J.C. (2011). Psychotherapy Relationships that Work: Evidence-Based Responsiveness. 2nd
45

46. References
Pridmore, J., & Phillips-Wren, G. (2011). Assessing decision making quality in face-to-face teams
versus virtual teams in a virtual world. Journal of Decision Systems, 3, 283-308.
Reese, R.J., Slone, N.C., Soares, N., & Sprang, R. (2012). Telehealth for underserved families: An
evidence-based parenting program. Psychological Services, 9, 320-322.
Richards, D., & Richardson, T. (2012). Computer-based psychological treatments for depression: A
systematic review and meta-analysis. Clinical Psychology Review, 32(4), 329-342.
Riva, G., Wiederhold, B.K., Mantovani, F., & Gaggioli, A. (2011). Interreality: The experiential use of
technology in the treatment of obesity. Clinical Practice and Epidemiology in Mental Health, 7,
51-61.
Seko, Y., Kidd, S., & Wiljer, D. (2013). Apps for those who help themselves: Mobile self-guided
interventions for adolescent mental health. Selected Papers Of Internet Research, 3. Retrieved
from http://spir.aoir.org/index.php/spir/article/view/833
46

47. References
Simek, E.M., McPhate, L., & Haines, T.P. (2012). Adherence to and efficacy of home exercise
programs to prevent falls: A systematic review and meta-analysis of the impact of exercise
program characteristics. Preventive Medicine: An International Journal Devoted to Practice and
Theory, 55, 262-275.
Stefan, S., & David, D. (2013). Face-to-face counseling versus high definition holographic projection
system. Efficacy and therapeutic alliance. A brief report. Journal of Cognitive and Behavioral
Psychotherapies, 13, 299-307.
Sucala, M., Schnur, J. B., Constantino, M. J., Miller, S. J., Brackman, E. H., Montgomery, G.H. (2012).
The therapeutic relationship in E-therapy for mental health: A systematic review. Journal of
Medical Internet Research, 14:e110
West, J. H., Hall, P. C., Hanson, C. L., Barnes, M. D., Giraud-Carrier, C., & Barrett, J. (2012). There’s
an app for that: Content analysis of paid health and fitness apps. Journal of Medical Internet
47

48. References
vanVelthoven, M.H.M.T., Car, L.R., Car, J., & Atun, R. (2012). Telephone consultation for improving
health of people living with or at risk of HIV: A systematic review. PLoSONE, 7, e36105-e36105
Villanti, A.C., McKay, H.S., Abrams, D.B., Holtgrave, D.R., Bowie, J.V. (2010). Smoking-cessation
interventions for U.S. young adults: A systematic review. American Journal of Preventive
Medicine, 39, 564-574.
Yuen, E.K., Herbert, J.D., Forman, E.M., Goetter, E.M., Comer, R., & Bradley, J. (2013). Treatment of
social anxiety disorder using online virtual environments in second life. Behavior Therapy, 44,
51-61.
48

49. Nancy A. Skopp, Ph.D.
49
 Dr. Nancy Skopp is a Research Psychologist and
Program Manager in the Research, Outcomes
and Investigations Division at the National Center
for Telehealth & Technology (T2), U.S.
Department of Defense.
 She is also an Affiliate Associate Professor at the
University of Washington’s Department of
Psychiatry & Behavioral Sciences.
 She is a Clinical Psychologist currently licensed
in the states of Texas and Washington.
 She holds doctoral and master’s degrees in
Clinical Psychology from the University of
Houston.
Nancy Skopp, Ph.D.

50. Janyce “Jae” Osenbach,
Ph.D.
50
Jae Osenbach, Ph.D.
 Dr. Jae Osenbach is a Research Psychologist
and Psychometrician in the Mobile Health
Program at the National Center for Telehealth &
Technology (T2), U.S. Department of Defense.
 She serves as a subject matter expert on mobile
apps and websites related to issues involving
military and veteran psychological health and
traumatic brain injury.
 She is also the lead for the Mobile Health
Program’s evaluations, surveys and
assessments, and serious games for behavioral
health.
 She holds a doctoral degree in Psychometrics
and Quantitative Psychology and a master’s
degree in Psychology from Fordham University.

51. Jae Osenbach, Ph.D.
janyce.e.osenbach.ctr@mail.mil
51
Nancy A. Skopp, Ph.D.
nancy.a.skopp.civ@mail.mil
http://t2health.dcoe.mil/

52. DCoE Contact Info
DCoE Outreach Center
1-866-966-1020 (toll-free)
http://www.dcoe.mil/
resources@dcoeoutreach.org
5252