Your SlideShare is downloading. ×
CATCH-IT Journal Club presentation Shamsa Jiwani
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

CATCH-IT Journal Club presentation Shamsa Jiwani


Published on

Published in: Technology, Health & Medicine

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Background and context/setting
  • The next four slides borrowed from Dr. Eysenbach’s previous course in consumer/public health informatics
  • Generate themselves = accessible
  • Enables individuals to own, manage, secure, digital copy of their health and wellness information.
  • Enables individuals to own, manage, secure, digital copy of their health and wellness information.
  • How did they study? Go to methodology….
  • I think this is the point where I start critically appraising the study and the way it was reported in the article. Although not explicitly stated, it implies framework analysis. Formative research occurs before a program is designed and implemented or while a program is being conducted. As well, can be conducted during the intervention (known as process evaluation).
  • This research was supported by the Centers for Disease Control and Prevention and the National Institutes of Health / National Institute on Alcohol Abuse and Alcoholism .
  • Blog comments/discussion
  • Blog comments/discussion
  • Blog comments/discussion
  • Blog comments/discussion
  • Blog comments/discussion
  • Blog comments/discussion
  • Transcript

    • 1. CATCH-IT Journal Club Shamsa Jiwani November 9, 2009 HAD5726 MHI Program University of Toronto Weitzman ER, Kaci L, Mandl KD Acceptability of a Personally Controlled Health Record in a Community-Based Setting: Implications for Policy and Design J Med Internet Res 2009;11(2):e14 URL: doi:10.2196/jmir.1187 PMID:19403467 (Critically Appraised Topics in Communication, Health Informatics and Technology)
    • 2. Agenda
      • Background
      • Objective and aims of the study
      • Intervention, setting, outcome measures
      • Methodology
      • Results and conclusions of authors
      • Limitations
      • Blog comments, questions and discussion
      Weitzman et al. JMIR 2009
    • 3. Weitzman et al. JMIR 2009 1. BACKGROUND
    • 4. Why this paper?
      • Increase in patient-centered care
      • User-centeredness; human factors
      • Consumer / Public Health Informatics on the rise
      • Intervention not much researched- first report that investigates consumers’ response to a platform PCHR and subsequent implications for policy and design
      Weitzman et al. JMIR 2009
    • 5. Weitzman et al. JMIR 2009
    • 6. Weitzman et al. JMIR 2009
    • 7. Weitzman et al. JMIR 2009 PubMed search for “Personally controlled health record” and “usability” results ONLY two studies!!!
    • 8. Weitzman et al. JMIR 2009 Power
    • 9. Weitzman et al. JMIR 2009 Shifting
    • 10. Weitzman et al. JMIR 2009 Autonomy
    • 11. Weitzman et al. JMIR 2009 And now ...
    • 12. For whom and why?
      • Students and Researchers
      • Health Informaticians
      • Clinicians
      • Vendors / Developers
      • Policy analysts / Policy-makers
      Weitzman et al. JMIR 2009
    • 13. Lead author: Weitzman Elissa R
      • Doctorate in Health and Social Behavior and Psychiatric Epidemiology and Masters in HPM from Harvard School of Public Health
      • Research Faculty with CHIP, Harvard School of Public Health
      • Principal Investigator developing tests of CHR-based systems
      • Pioneering the field of Public Health Informatics through developing tests of acceptability, use and impacts of survey-enabled PHR systems
      Weitzman et al. JMIR 2009 Source:
    • 14. Lead author’s comments:
      • "As we progress toward large-scale adoption of PCHRs, we need to proceed with a critical eye for how to advance the technology but also continuously assess the impact it is having on the existing landscape of health behaviors, knowledge, understanding and relationships"
      • "When you drop a new technology like this into an already complex system of behaviors, knowledge and assumptions, we can expect it to create a ripple effect that will resonate out from an individual to their care providers and beyond."
      Weitzman et al. JMIR 2009 Source:
    • 15. Co-author: Mandl Kenneth D
      • Doctorate in Medicine from Harvard Medical School (‘89), MPH from Harvard School of Public Health (‘95)
      • Asst. Prof. of Pediatric at HMS, Affiliated Faculty at Harvard-MIT
      • Pioneer in Consumer-focused health informatics
      • Primary investigator in the Indivo project
      • Authored more than 50 peer-reviewed journal including NEJM, JAMA, proceedings of National Academy of Sciences, BMJ, JAMIA, Annals of Internal Medicine and the Lancet.
      Weitzman et al. JMIR 2009 Source:
    • 16. Co-author Mandl’ s comments:
      • "This research comes at an important time.“
      • "The Obama administration has made health information technology a priority and we have a unique window of opportunity now, before widespread adoption of PCHRs is realized, to do research - in this case, to see how people in a community access and interact with their health information independent of a clinician or administrator - and determine what changes need to be made and policies put in place , and get it right."
      Weitzman et al. JMIR 2009 Source:
    • 17. Co-author: Kaci Liljana
      • Acceptability of a personally controlled health record in a community-based setting: implications for policy and design (2009)
      • Assessing residual confounding of the association between antipsychotic medications and risk of death using survey data (2009)
      • Potential causes of higher mortality in elderly users of conventional and atypical antipsychotic medications (2008)
      • Prior authorization for biologic disease-modifying antirheumatic drugs: a description of US Medicaid programs (2008)
      Weitzman et al. JMIR 2009 Source:
    • 18. Weitzman et al. JMIR 2009 2. OBJECTIVES AND AIMS OF THE STUDY
    • 19. Study Objectives
      • To understand-
        • Acceptability
        • Early impacts
        • Policy
        • Design requirements
        • of PCHRs in a community-based setting
        • Primary aim- understand the acceptability of PCHRs
        • Secondary aim- identify design and policy issues
      Weitzman et al. JMIR 2009
    • 20. gene Weitzman et al. JMIR 2009 PHR defined by Markle Foundation Connecting for Health Initiative Group- “ An internet-based set of tools that allows people to access and coordinate their lifelong health information and make appropriate parts available to those who need it. PHRs offer an integrated and comprehensive view of health information, including information people generate themselves " and "The PHR is a single, person-centered system designed to track and support health activities across one’s entire life experience; it is not limited to a single organization or a single health care provider.” Source: The Personal Health Working Group Final Report. July 1, 2003. Markle Foundation. Available at: . Source:
    • 22.
      • Original PCHR System
      • Free, open-source, uses open, unencumbered standards
      Indivo PCHR Weitzman et al. JMIR 2009
      • Deployed at Children’s Hospital Boston and the Dossia Consortium (including- AT&T, Walmart, Intel, BP America)
        • secure storage, categorization and aggregation of health data.
        • single sign-on and standards-based data-access delegation.
        • a simple, open, web-based Application Programming Interface (API).
        • unified user notification.
    • 23. Weitzman et al. JMIR 2009 Indivo Architecture Source:
    • 24. Setting Weitzman et al. JMIR 2009
      • Urban area within the northeastern region of US
      • Community-based, university health maintenance organization
      • Duration: 2 yr-study (May/June 06 – April 08)
    • 25. Sample
      • Persons affiliated with the site and setting
      • Pre-deployment [n=20 administrators, clinicians, institutional stakeholders]
        • Focus Group, One-to-one interviews
        • 35-60 years
        • Trained in medicine and/or health care
      • Usability testers [n=12 community members]
        • Undergraduate, graduate, employee, retiree
        • 25-65 years
      • Pilot participants [n=40 community members]
      • Full Demonstration study users [n=250]
        • 18-83 years (average age 53)
        • 81 email communications
      Weitzman et al. JMIR 2009 n=52 More than 300 participants
    • 26. Outcome studied Weitzman et al. JMIR 2009
      • Beliefs
      • Reactions
      • Through 3 formative research assessment activities:
      • Administrative
      • Clinical
      • Public health stakeholders
    • 27. Weitzman et al. JMIR 2009 4. METHODOLOGY
    • 28. Methodology
      • Formative evaluation
      • Data collection
        • Observational
        • Narrative
          • Transcribed audio-tape
          • Written observational notes
          • Text from email communication
      Weitzman et al. JMIR 2009
      • A priori defined major
      • codes
      • Inductive iterative
      • process for subthemes
      • Framework Analysis
    • 29. Weitzman et al. JMIR 2009 5. RESULTS & CONCLUSIONS
    • 30. Results
      • Level of awareness of PCHRs: LOW
      • Privacy: MODERATE
      • Autonomy: HIGH
      • Barriers and Facilitators to adoption:
        • Societal level factors
        • Interpersonal level factors
        • Individual level factors
      Weitzman et al. JMIR 2009
    • 31. Implications for Policy and Design Weitzman et al. JMIR 2009
      • Facilitates discussion on
        • Design options
        • Literacy issues
        • Guidelines on risk and safety mechanisms
        • Creation on family-focused health record system
        • Protocols and models in human subject participation in PCHR evaluations
    • 32. Are the results valid?
        • Does the ‘conclusion’ answer the purpose set in the
        • ‘ objective (s)’?
      Weitzman et al. JMIR 2009
    • 33. STARE-HI recommendations
        • (Statement on reporting evaluation studies in Health Informatics)
      • Title
      • Abstract
      • Keywords
      • Introduction
        • Scientific background
        • Rationale for the study
        • Objectives of the study (???)
      • Study Context
        • Organizational setting
        • System details and system in use
      • Methods
        • Study design
        • Theoretical background
        • Participants (???)
        • Study flow
        • Outcome measures or evaluation criteria (???)
        • Methods for data acquisition and measurement (???)
        • Methods for data analysis (???)
      • Results
        • Demographic and other study coverage data
        • Unexpected events during the study (??)
        • Study findings and outcome data (???)
        • Unexpected observations (???)
      • Discussion
        • Answers to study questions (???)
        • Strengths and weaknesses of the study
        • Results in relation to other studies
        • Meaning and generalizability of the study (???)
        • Unanswered and new questions
      • Conclusion
      • Author’s contribution
      • Competing interests
      • Acknowledgement
      • References (some missing)
      • Appendices (???)
      Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykanen P, Rigby M. STARE-HI -Statement on Reporting of Evaluation Studies in Health Informatics. Yearb.Med.Inform. 2009:23-31.
    • 34. Weitzman et al. JMIR 2009 6. LIMITATIONS
    • 35. What was good /bad about the study?
      • Optimized user involvement
      • Included diverse users
      • Users’ experience in interacting with the web-based system
      Weitzman et al. JMIR 2009
    • 36. Weitzman et al. JMIR 2009 7. BLOG COMMENTS, QUESTIONS AND DISCUSSION
    • 37. STROBE- Strengthening The Reporting of Observational Studies in Epidemiology Weitzman et al. JMIR 2009 Vandenbroucke JP, Von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. Gac.Sanit. 2009 Mar-Apr;23(2):158.
      • Title and Abstract
      • Results
      • Descriptive data
      • Outcome data
      • Main results
      • Other analysis
      • Introduction
      • Background/rationale
      • Objectives
      • Methods
      • Study design
      • Setting
      • Participants
      • Variables
      • Data sources/measurement
      • Bias
      • Study size
      • Quantitative variables
      • Statistical methods
      • Discussion
      • Key results
      • Limitations
      • Interpretation
      • Generalizability
      • Other Information
      • Funding
    • 38. Issues raised in the Blog Weitzman et al. JMIR 2009
      • Study Objective
        • Far too many objectives for one study
        • No particular focus
    • 39. Issues raised on the Blog Weitzman et al. JMIR 2009
      • Sample Selection / Sampling Technique
        • Who actually constituted the sample and how the sample was chosen? Not sufficient information. How did the researchers recruit participants - what was the process involved with the recruitment, who did they choose, who did they not choose?
        • How representative is the sample of the general population since all the participants in the study had training in medicine and/or health administration, and were highly proficient in technology.
        • Objective of the study was to understand the acceptability of PCHRs—the fact that the participants were all volunteers may suggest their inherent willingness to accept PCHRs as part of a health care model, causing the results of the study to be biased towards higher degrees of acceptance.
        • How many participated in focus groups and one-to-one interview?
    • 40. Issues raised on the Blog Weitzman et al. JMIR 2009
      • Methodology
        • The author did not address what methodological approach they used; was it ethnography or grounded theory. In addition, the analysis section did not provide a transparent description of how the coding was performed.
        • Research method is not clearly described. Did they use narrative inquiry?
        • Why were only the focus group interviews transcribed? Did they not use the data from the one-to-one interviews?
        • How were the written observational notes of usability testing analyzed? And were notes taken for all 12 testers and 40 pilot participants?
        • How many users are represented in these emails? Did the other users that did not communicate with the study team have any issues? Did they even use the PCHR?
    • 41. Issues raised in the Blog Weitzman et al. JMIR 2009
      • Privacy
        • Indivo PCHR being open source, what long-term implications of Indivo’s relationship with Dossia consortium?
        • What level of privacy education provided to participants / consent choices offered?
          • Did the IRB approval stated how to dispose of the research data?
          • Why did participants include non-clinical information? Was there a visible privacy policy on the PCHR application?
          • Was the individual record encryption both ways (patient/PCHR)?
    • 42. Issues raised in the Blog Weitzman et al. JMIR 2009
      • Intervention
        • Unclear on the nature of intervention
        • The study included 3 disparate PCHR systems. What was the utilization rate of each system and did this have an effect on the qualitative findings?
      • Reporting
        • Could authors have provided a figure or table with an overview of 3 research activities summarizing key information such as, Activity, Number of Participants, Characteristics of Participants, etc.?
      Pg 2. We studied patient-centered research…to the Indivo PCHR… Absolutely agree!
    • 43. Not mentioned in the paper
        • Schematic overview of the architecture
        • Reference for formal usability protocol
        • Potential biases
        • Some participants share passwords and account information; Was this addressed in the analysis? How?
        • Medico-legal concerns; who owns the records were not brought out by clinicians who were interviewed (blog comment)
        • Quantitative evidence to back up Table 1. Example: Uncertainty about appropriate and safe read/edit access policies was evident among young adults/students- how do we verify? Was this evident amongst all young adults and students that responded? (blog comment)
    • 44. Additional two cents... Weitzman et al. JMIR 2009
        • A much needed research
        • Labor-intensive and time-consuming study
        • However, the reporting needs improvements with respect to-
        • Study methodology
        • Results
        • Appendices (for validity and context)
        • Food for thought :
        • Could it have taken some other form of evaluation?
    • 45. Thank you CATCH-IT Journal Club Members HAD5726 for comments on the blog- Weitzman et al. JMIR 2009