Methodological Issues and Challenges in eHealth Research Panel [Aud 900 Proudfoot (Panel)]


Published on

Published in: Health & Medicine, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Methodological Issues and Challenges in eHealth Research Panel [Aud 900 Proudfoot (Panel)]

    1. 1. Proudfoot, J. et al.: Methodological Issues and Challenges in eHealth Research Panel <ul><li>This slideshow, presented at Medicine 2.0’08 , Sept 4/5 th , 2008, in Toronto, was uploaded on behalf of the presenter by the Medicine 2.0 team </li></ul><ul><li>Do not miss the next Medicine 2.0 congress on 17/18th Sept 2009 ( ) </li></ul><ul><li>Order Audio Recordings (mp3) of Medicine 2.0’08 presentations at </li></ul>
    2. 2. Methodological Issues and Challenges in eHealth Research Dr Judy Proudfoot , Black Dog Institute & School of Psychiatry, University of New South Wales, Sydney, Australia. Dr Lisa Whitehead , University of Otago, Christchurch, New Zealand. Dr Caryl Barnes , School of Psychiatry, University of New South Wales, Sydney, and Sentiens, Perth, Australia
    3. 4. Session Outline <ul><li>Introduction </li></ul><ul><li>Systematic review of methodological and ethical issues in Internet research </li></ul><ul><li>Sampling issues </li></ul><ul><li>Ethical issues </li></ul><ul><li>Measurement issues </li></ul><ul><li>Discussion </li></ul><ul><li>Conclusion </li></ul>
    4. 5. Systematic review of methodological and ethical issues in Internet research Dr Lisa Whitehead
    5. 6. Search strategy <ul><li>8 databases (EBSCO,EMBASE, Medline, PsycInfo, CINAHL, PubMed, Cochrane and TRIP). </li></ul><ul><li>Keywords: Internet, research, quality, credibility, reliability and validity. </li></ul><ul><li>Quality appraisal (Dixon-Woods et al. 2005). </li></ul><ul><li>Whitehead, L. (2007) Methodological and ethical issues in Internet-mediated research in the field of health: An integrated review of the literature. Social Science and Medicine 65: 782-791 </li></ul>
    6. 7. 3 Key Issues Sampling issues Ethical issues Reliability and validity <ul><li>3 key issues </li></ul><ul><ul><li>Potential for sampling bias </li></ul></ul><ul><ul><li>Ethical issues in Internet research? </li></ul></ul><ul><ul><li>Validity of the data- impact on the established psychometric properties of measures </li></ul></ul>
    7. 8. Sampling Issues <ul><li>Two approaches (open and selective) </li></ul><ul><ul><li>Open (convenience sampling) </li></ul></ul><ul><li>Enhance representation? </li></ul><ul><li>Can large samples override potential bias? </li></ul><ul><li>Presence and impact of self-selection can be traced to some degree </li></ul>
    8. 9. Sampling issues <ul><li>Selective (if able to define population) </li></ul><ul><ul><li>Few studies based on a randomly selected sample </li></ul></ul><ul><ul><li>Convenience sample random allocation </li></ul></ul><ul><ul><li>Propensity scoring an option </li></ul></ul>
    9. 10. Response rates <ul><li>Unsolicited e-mail (with electronic return) generates lowest response rate, even when compared to unsolicited postal survey. </li></ul><ul><li>Impact of presentation- browser type, download time, password, security issues, computer anxiety, computer self-efficacy </li></ul>
    10. 11. Ethical Issues <ul><li>Seeking/obtaining consent </li></ul><ul><li>Protecting participants’ anonymity </li></ul><ul><li>Security of site </li></ul><ul><li>Opportunistic research </li></ul>
    11. 12. Ethical issues in planned research <ul><li>Developing informed consent </li></ul><ul><li>Ability to ensure a level of anonymity </li></ul><ul><li>Security of site </li></ul>
    12. 13. Opportunistic research <ul><li>When and how is material posted on the web classed as public or private? </li></ul><ul><ul><li>Level of “disguise” in presenting results? </li></ul></ul><ul><ul><li>Issue of perceived privacy </li></ul></ul><ul><ul><li>Are researchers “lurkers”? </li></ul></ul><ul><ul><li>Is the process of “non-alienation” acceptable? </li></ul></ul>
    13. 14. Reliability and validity <ul><li>Equivalence must be demonstrated not assumed </li></ul><ul><ul><li>Some evidence of differences in the distribution of scores and psychometric properties (e.g. factor loading) </li></ul></ul>
    14. 15. Reliability and credibility <ul><li>Layout of pages </li></ul><ul><li>Linguistic competence </li></ul><ul><li>Culturally driven interpretations </li></ul><ul><li>Low attentiveness </li></ul><ul><li>Repeat participation </li></ul><ul><li>Incomplete submissions </li></ul><ul><li>Misrepresentation of self </li></ul>
    15. 16. Construct validity <ul><li>Psychometric properties </li></ul><ul><ul><li>Mixed evidence </li></ul></ul><ul><ul><li>How critical are the differences? </li></ul></ul><ul><ul><li>Difficult to disentangle issues around sampling biases and possible effects of completing measures on-line. </li></ul></ul>
    16. 17. Sampling Issues in Internet-Mediated Research Dr Judy Proudfoot
    17. 18. Online Psychoeducation for Bipolar Disorder: Randomised Controlled Trial <ul><li>Dr Judy Proudfoot – Black Dog Institute & School of Psychiatry, University of New South Wales </li></ul><ul><li>Amisha Jawawant - Black Dog Institute & School of Psychiatry, University of New South Wales </li></ul><ul><li>Prof. Gordon Parker – Black Dog Institute & School of Psychiatry, University of New South Wales </li></ul><ul><li>A/Prof Vijaya Manicavasagar – Black Dog Institute & School of Psychiatry, University of New South Wales </li></ul><ul><li>A/Prof Meg Smith – University of Western Sydney </li></ul>
    18. 19. Background <ul><li>Bipolar Disorder is difficult to detect. Accurate diagnosis can take 10-20 years. </li></ul><ul><li>Poor treatment adherence, inadequate self management, increased levels of disability are common. </li></ul><ul><li>40% of patients relapse in 1 year, 60% over 2 years and 73% over 5 years (Gitlin et al, Am J Psych, 1995) . </li></ul><ul><li>Psycho-education reduces relapses and hospitalisations. Not previously delivered online. </li></ul>
    19. 20. Research Study <ul><li>RANDOMISED CONTROLLED TRIAL </li></ul><ul><li>Does an 8-module online education program + ‘treatment as usual’ for Bipolar Disorder improve psycho-social adjustment in adults with newly-diagnosed BD </li></ul><ul><li>With and without online support from an apomediary (“informed supporter”). </li></ul><ul><li>In comparison with online information about bipolar disorder + monitoring. </li></ul>
    20. 21. Bipolar Disorder Education Program
    21. 22. Topics <ul><li>Diagnosing Bipolar Disorder </li></ul><ul><li>Causes of Bipolar disorder </li></ul><ul><li>Medications </li></ul><ul><li>Psychological Treatments </li></ul><ul><li>Stay-Well Plans (identifying triggers and early warning signs, and designing preemptive actions involving pt and family) </li></ul><ul><li>Support Networks and Carers </li></ul><ul><li>Omega-3 as a treatment for depression </li></ul><ul><li>Person First Illness Last </li></ul>
    22. 23. Random Allocation Bipolar Disorder Education Program Email support from an apomediary + lnformation + Monitoring
    23. 24. Online interventions <ul><li>One online module per week over 8 weeks </li></ul><ul><li>Workbook activities to assist participants to apply the information and strategies to their situation </li></ul><ul><li>Self-monitoring between sessions </li></ul>
    24. 25. “Informed Supporters” <ul><li>People with bipolar disorder </li></ul><ul><li>Effectively managing their condition for >2 years </li></ul><ul><li>Trained to provide email support to participants under supervision of our clinical team. </li></ul>
    25. 26. Informed Support <ul><li>At least one and maximum two emails per week sent to each participant (max 300 words per email), with invitation to respond. </li></ul><ul><li>Practical advice about how to set up a “well-being plan” (strategies to minimise risks and illness triggers and a contingency plan for times when unwell) as well as general issues about bipolar disorder. </li></ul><ul><li>Posts from Informed Supporters are checked for adherence and safety. </li></ul><ul><li>Posts from participants to Informed Supporters are monitored for adverse events. </li></ul>
    26. 27. Inclusion Criteria: <ul><li>18+ years old </li></ul><ul><li>Diagnosed with BD in previous 12 months </li></ul><ul><li>Currently receiving regular care from psychiatrist or GP </li></ul><ul><li>Has access to internet and printer, and is computer literate </li></ul><ul><li>Able to read and write English </li></ul><ul><li>Living in Australia </li></ul>
    27. 28. Screening <ul><li>Bipolar Disorder Assessment Test (Parker et al., 2006) Sensitivity: Specificity: </li></ul><ul><li>Potential for over-ride by Black Dog Institute Psychiatrists (who validated the BDAT) </li></ul>
    28. 29. Sampling <ul><li>Open (convenience) sampling: - Via Internet Black Dog Institute website, websites of professional organisations, websites of mental health organisations in Australia - Via print media Newspaper advertisements and articles </li></ul>
    29. 30. Sampling (cont) <ul><li>Selective Sampling: </li></ul><ul><li>- Via Black Dog Institute Mood Disorders Clinic – invited participation - Via leaflets at special events , eg Fun Runs, Flower Festivals, Training programs for GPs, School Counsellors </li></ul>
    30. 31. Sampling (cont) <ul><li>Variety of recruitment methods to minimise sampling bias and maximise response rates. </li></ul><ul><li>But still not a random sample (ie each member of the population does not have equal probability of being selected). </li></ul><ul><li>No recruitment by unsolicited email (lower response rate). </li></ul><ul><li>But still low response rate (3-4 per week) - due to tight inclusion criteria? </li></ul><ul><li>- due to phenomenology of BD diagnosis? </li></ul>
    31. 32. A Solution?
    32. 33. Methodological Issues and Ethical Challenges in e-Health Research: Panel Discussion Dr Caryl Barnes MBBS,FRANZCP Doctoral fellow University of New South Wales, Sydney, Australia ‘ HealthSteps (formally Recovery Road) for Bipolar Disorder’: an on-line relapse prevention program
    33. 34. Overview <ul><li>Bipolar disorder: role of internet in treatment </li></ul><ul><li>Aims of RCT: ‘ Taking control of Bipolar Disorder: Staying well, Learning more’-12 month online relapse prevention program </li></ul><ul><li>Ethical issues considered </li></ul><ul><ul><ul><li>Privacy/Confidentiality/Anonymity </li></ul></ul></ul><ul><ul><ul><li>Informed consent </li></ul></ul></ul><ul><ul><ul><li>Analysis of potential risk to participants/ benefits </li></ul></ul></ul><ul><ul><ul><li>Beneficence/ Duty-of-care </li></ul></ul></ul><ul><li>Discussion </li></ul>
    34. 35. Bipolar Disorder: role of internet in treatment <ul><li>Common 1-2% of the general population </li></ul><ul><li>Chronic relapsing remitting illness </li></ul><ul><li>6th highest cause of disability world wide (WHO figures) </li></ul><ul><li>Frequent relapses despite good adherence to medication </li></ul><ul><li>Psychological interventions vital part of treatment e.g. psycho-education / CBT </li></ul><ul><li>Disadvantages </li></ul><ul><ul><li>time consuming, </li></ul></ul><ul><ul><li>expensive </li></ul></ul><ul><ul><li>depending on area, difficult to access </li></ul></ul>
    35. 36. Research Team: collaboration between public/private <ul><li>University of New South Wales </li></ul><ul><li>Dr Caryl Barnes-Chief Investigator, Consultant Psychiatrist/ Doctoral fellow </li></ul><ul><li>Dusan Hadiz-Pavlovic, Senior Hospital Scientist </li></ul><ul><li>Professor Philip Mitchell, Head of School of Psychiatry </li></ul><ul><li>Assoc Professor Kay Wilhelm </li></ul><ul><li>Sentiens Pty Ltd </li></ul><ul><li>Dr Dennis Tannenbaum, CEO </li></ul><ul><li>Dr Caroline Spiranovic, Research and Development Associate </li></ul><ul><li>Gavin Pinto, IT Architect </li></ul><ul><li>Michael Bosley-Smith, Research Officer </li></ul><ul><li>Stacey Bosley-Smith, Research Officer </li></ul><ul><li>Dr Robin Harvey, Research Manager </li></ul>Perth Western Australia Sydney, NSW
    36. 37. Aims of 12-month RCT <ul><li>To evaluate on-line adjunctive relapse prevention program ‘HealthSteps for Bipolar Disorder’ vs. placebo/control program ‘Virtual Highway for Bipolar Disorder’ </li></ul><ul><li>Outcome measures: </li></ul><ul><ul><li>Time to relapse- Survival analysis </li></ul></ul><ul><ul><li>Measures: BDI-II, ISS, Sheehan disability scale, Hospitalization, </li></ul></ul>
    37. 38. Study Inclusion Criteria <ul><li>Over 18 years of age </li></ul><ul><li>Meets criteria for bipolar disorder (BDSQ Confirmed) </li></ul><ul><li>Diagnosis made by a mental health professional </li></ul><ul><li>Currently under care of psychiatrist /GP </li></ul><ul><li>Taking medication </li></ul><ul><li>Access to internet and printer- at home/ work. </li></ul><ul><li>Has an email account/ has email address. </li></ul><ul><li>Able to understand written English </li></ul><ul><li>Interested in using web based program </li></ul>
    38. 39. Study Inclusion Criteria <ul><li>Over 18 years of age </li></ul><ul><li>Meets criteria for bipolar disorder (BDSQ Confirmed) </li></ul><ul><li>Diagnosis made by a mental health professional </li></ul><ul><li>Currently under care of psychiatrist /GP </li></ul><ul><li>Taking medication </li></ul><ul><li>Access to internet and printer- at home/ work. </li></ul><ul><li>Has an email account/ has email address. </li></ul><ul><li>Able to understand written English </li></ul><ul><li>Interested in using web based program </li></ul>
    39. 41. Ethical Considerations: Getting started <ul><li>UNSW Human Research Ethics Committee (HREC) </li></ul><ul><ul><li>Formal application process </li></ul></ul><ul><ul><li>Consist of a diverse panel includes ’lay’ members </li></ul></ul><ul><ul><li>Issues </li></ul></ul><ul><ul><ul><li>Privacy/Confidentiality/Anonymity </li></ul></ul></ul><ul><ul><ul><li>Informed consent </li></ul></ul></ul><ul><ul><ul><li>Analysis of potential risk to participants/ benefits </li></ul></ul></ul><ul><ul><ul><li>Beneficence/ Duty-of-care </li></ul></ul></ul>
    40. 42. Privacy/ Confidentiality <ul><li>Screening/Recruitment-BDSQ </li></ul><ul><ul><li>Only contact email required </li></ul></ul><ul><ul><li>Minimal identifying details collected </li></ul></ul><ul><ul><li>Contact participants via email only </li></ul></ul><ul><ul><li>Secure database/server </li></ul></ul><ul><ul><li>Professional standards for confidentiality </li></ul></ul><ul><li>Online Enrolment/Access to study </li></ul><ul><ul><li>Allocation 8 digit individual ID number/password </li></ul></ul><ul><li>Data entered during study </li></ul><ul><ul><li>De-identification </li></ul></ul><ul><ul><li>Secure data storage -fire walled </li></ul></ul><ul><ul><li>Limited access </li></ul></ul>
    41. 44. Privacy / Confidentiality
    42. 45. Informed Consent <ul><li>Consent for Study: Two-way process </li></ul><ul><ul><li>Provide information about requirements, risks and benefits of the study </li></ul></ul><ul><ul><li>Ensure level of common understanding about study is reached </li></ul></ul><ul><li>Information via online ‘Participation Information statement’ start of BDSQ </li></ul><ul><li>Used ‘click-button’ option. </li></ul><ul><li>Consent to inform treating clinician optional </li></ul><ul><li>Participant able to withdraw consent at any point </li></ul><ul><li>Technical support given via s [email_address] .au </li></ul>
    43. 46. <ul><li>Screen shots in here </li></ul>
    44. 48. Risks/Benefit Ratio <ul><li>Low risk of inducing psychological distress </li></ul><ul><li>No risk of physical harm </li></ul><ul><li>Vulnerable population recognized; inclusion criteria had to be under care treating mental health professional </li></ul><ul><li>Program adjunctive only </li></ul><ul><li>Belief participation would benefit participants </li></ul>
    45. 49. Duty of Care <ul><li>Need to recognize worsening/ life threatening symptoms/ suicidal ideation </li></ul><ul><li>Developed a RED FLAG system </li></ul><ul><li>Outcome measures set to trigger flag within system </li></ul><ul><li>Individual data entry reviewed, if required de-identification participant, review of consents </li></ul><ul><li>Email to participant, if applicable to treating clinician </li></ul>
    46. 51. Discussion <ul><li>Restraints caused by a research collaboration-private and public entities </li></ul><ul><li>Privacy- not enough identifying details collected </li></ul><ul><li>Informed consent process – was this sufficient? </li></ul><ul><li>Risk/Benefit </li></ul><ul><ul><li>email </li></ul></ul><ul><ul><li>Ongoing clinician contact? </li></ul></ul>
    47. 52. Discussion <ul><li>Duty of care </li></ul><ul><ul><li>Management of participants outside of Australia </li></ul></ul><ul><ul><li>Red flag system appreciated-feedback received rated this most useful components of study </li></ul></ul><ul><ul><li>Consent to contact clinician optional, is this sufficient? </li></ul></ul><ul><ul><li>Control group going into HealthSteps lack of resources limited retention some features e.g. Red Flags </li></ul></ul>
    48. 53. Acknowledgements <ul><li>Graduate Research School and Faculty of Medicine, University of New South Wales, Sydney, Australia </li></ul><ul><ul><li>Travel Scholarship and Doctoral Scholarship </li></ul></ul><ul><li>Dr Dennis Tannebaum, CEO and all Research Staff at Sentiens Pty Ltd </li></ul><ul><li>Professor Philip Mitchell : Doctorate Supervisor </li></ul>For a sustainable future in research The 5.31 tonnes of Co2 produced by the author’s flights from Australia to Pittsburgh to present this poster for the 7 th ICBD have been offset with
    49. 54. Carbon Neutral: for a sustainable future in research The 6.09 tonnes of Co2 produced by the presenter’s flights from Perth, Western Australia to Toronto for the Medicine 2.0 Conference has been offset with
    50. 55. The health of tertiary students: A mixed method study Dr Lisa Whitehead, Dr Shelagh Dawson, Avin Panckhurst Centre for Postgraduate Nursing Studies, University of Otago, Christchurch, New Zealand
    51. 56. Methods <ul><ul><li>Student population of 20,665 </li></ul></ul><ul><ul><li>A random sample of 2000 students was taken and these were randomly assigned to either a postal (N=1000) or an on-line survey (N=1000). </li></ul></ul><ul><ul><li>The survey was about physical and emotional health, anxiety, depression and fatigue </li></ul></ul>
    52. 57. Sampling Issues <ul><li>Response rates </li></ul>
    53. 58. Pattern of response
    54. 59. Sampling issues
    55. 60. Correlations between measures
    56. 61. Distribution of scores- Single fatigue item Table 4. Single item fatigue measure (I feel tired for no reason) X 2 =0.07 df=1 p=0.79 (Armitage-Cochran test for trend) 969 39 90 424 416 Total 497 19 (3.8%) 52 (10.4%) 212 (42.7%) 214 (43.1%) Postal 472 20 (4.2%) 38 (8.1%) 212 (44.9%) 202 (42.8%) On-line Total 3 (Most of the time) 2 (A good part of the time) 1 (Some of the time) 0 (none or a little of the time)
    57. 62. Mean differences
    58. 63. Construct validity FSI- Mean differences in fatigue between the on-line and postal group
    59. 64. Mean differences on FSI items on-line, postal and scale development group
    60. 65. Correlations among the fatigue intensity and duration ratings as an indication of the construct validity of the FSI
    61. 66. Reach Out Central Evaluation Swinburne eTherapy Unit
    62. 67.
    63. 68. <ul><li>Online game-based youth mental health education program </li></ul><ul><li>Target audience: </li></ul><ul><ul><li>young people aged 16-25, particularly young men. </li></ul></ul><ul><li>Objectives: </li></ul><ul><ul><li>Increase mental health literacy </li></ul></ul><ul><ul><li>Increase skills to change thoughts and solve problems </li></ul></ul><ul><ul><li>Application of learned skills to improve mental health </li></ul></ul>ROC Overview
    64. 69. Methodology <ul><li>Single group, quasi-experimental repeated measures design (pre-, post-program and follow-up) </li></ul><ul><li>N = 266 (176 females; 88 males) </li></ul><ul><li>Questionnaires included: </li></ul><ul><ul><li>K10 (depression and anxiety) </li></ul></ul><ul><ul><li>AUDIT (alcohol use) </li></ul></ul><ul><ul><li>Coping Strategy Indicator – short form (subscales – problem solving , seeking support, avoidance) </li></ul></ul><ul><ul><li>Resilience Scale – short form (RS – SF) </li></ul></ul><ul><ul><li>Satisfaction with Life Scale (SWLS) </li></ul></ul><ul><li>Additional items examined drug use, mental health stigma, help seeking attitudes, program satisfaction and application of skills in day-to-day life. </li></ul>
    65. 70. Issues for Discussion <ul><li>What is the minimal process considered sufficient for informed consent? </li></ul><ul><li>Are we overconcerned with privacy? </li></ul><ul><li>What are the minimal requirements for duty of care? </li></ul><ul><li>What is your experience in the transfer of measures to the online setting? </li></ul><ul><li>Should standard measures be robust enough to cope with minor variance? </li></ul><ul><li>How generalisable are the results from Internet studies (convenience samples)? </li></ul><ul><li>How can the generalisability be improved? </li></ul><ul><li>Can inferences be drawn from randomised controlled trials within convenience samples? </li></ul>