Your SlideShare is downloading. ×
0
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
PCORI at Academy Health
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

PCORI at Academy Health

210

Published on

Slides from PCORI's presentation at Academy Health on June 4, 2013.

Slides from PCORI's presentation at Academy Health on June 4, 2013.

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
210
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Using the PCORI Methodology Standards to Generate Robust, Relevant, and Timely Evidence for Patient-Centered Outcomes Research Joe V. Selby, David Hickam, Brian Mittman, and Sean Tunis June 4, 2013
  • 2. The audio and slide presentation will be delivered directly to your computer  Speakers or headphones are required to hear the audio portion of the webinar.  If you do not hear any audio now, check your computer’s speaker settings and volume.  If you need an alternate method of accessing audio, please submit a question through the Q&A pod.
  • 3. Technical Assistance  Live technical assistance: – Call Adobe Connect at (800) 422-3623  Refer to the ‘Technical Assistance’ box in the bottom left corner for tips to resolve common technical difficulties.
  • 4. To submit a question: 1. Click in the Q&A box on the left side of your screen 2. Type your question into the dialog box and click the Send button Questions may be submitted at any time during the presentation
  • 5. Welcome  Erin Holve, Ph.D., M.P.H., M.P.P. – Senior Director of Research & Education, AcademyHealth
  • 6. A Collaborative Effort to Improve Training in PCOR
  • 7. PCORI Mini-Course  Two morning panels – Developing a Program of Comparative Effectiveness Research at the Patient Centered Outcomes Research Institute” (9:45-11:15am) • Chair: Joe Selby, PCORI • Faculty: Chad Boult, Rachael Fleurence, Romana Hasnain- Wynia, and David Hickam, PCORI – “PCORI Methods Pilot Projects: Learning from Engagement” (11:30am-1:00pm) • Chair: Sue Sheridan, PCORI • Faculty: Katherne Bevans, Children’s Hospital of Philadelphia; Renee Robinson, Southcentral Foundation; and Tiffany Haynes, University of Arkansas for Medical Sciences
  • 8. PCORI Mini-Course  Afternoon workshop (June 25, 2:00-6:00pm ET) – Heterogeneity of Treatment Effects • Chair: Brian Mittman, PhD, Department of Veterans Affairs • Faculty: Ravi Varadhan, PhD, Jodi Segal, MD, MPH, Johns Hopkins University School of Medicine – Adaptive and Bayesian Trial Designs • Chair: Sally Morton, PhD, University of Pittsburgh • Faculty: Jason Connor, PhD, Berry Consultants – Data Registries • Chair: Robin Newhouse, PhD, RN, NEA-BC, FAAN, University of Maryland • Faculty: Priscilla Velentgas, Ph.D., Quintiles Outcome
  • 9. Learning Objectives  Provide an overview of the PCORI Methodology Standards and the process used to develop the standards;  Provide general guidance on using the standards when responding to PCORI funding announcements; and  Review plans for implementing the standards and updating them in the future.
  • 10. Webinar Faculty Joe V. Selby, MD, MPH David Hickam, MD, MPH Brian Mittman, PhD Sean Tunis, MD, MSc
  • 11. Academy Health Webinar June 4, 2013 Joe V. Selby Executive Director, PCORI Using the PCORI Methodology Standards to Generate Robust, Relevant, and Timely Evidence for Patient-Centered Outcomes Research
  • 12. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE About PCORI • An independent, non-profit organization authorized by Congress. • Mission is to fund research that will provide patients, their caregivers and clinicians with the evidence-based information needed to make better-informed health care decisions. • Committed to continuously seeking input from patients and a broad range of stakeholders to guide its work.
  • 13. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE What is Patient Centered Outcomes Research? Patient-Centered Outcomes Research (PCOR) helps people and their caregivers communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options. PCOR has the following characteristics: • Actively engages patients and key stakeholders throughout the research process. • Compares important clinical management options. • Evaluates the outcomes that are most important to patients. • Addresses implementation of the research finings in clinical care environments.
  • 14. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Approximate PCORI Funding * 20% of this amount goes to AHRQ/HHS each year for dissemination and capacity-building
  • 15. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Current Funding Opportunities and Events http://www.pcori.org/funding-opportunities/funding-announcements/funding-center/ Announcement Letter of Intent Due Assessment of Prevention, Diagnosis, and Treatment Options 06/15/13 Improving Healthcare Systems 06/15/13 Communication and Dissemination Research 06/15/13 Addressing Disparities 06/15/13 Improving Methods for Conducting Patient-Centered Outcomes Research 06/15/13 Improving Infrastructure for Conducting Patient Centered Outcomes Research: Phase One —Clinical Data Research Networks 06/19/13 Improving Infrastructure for Conducting Patient Centered Outcomes Research: Phase One —Patient-Powered Research Networks 06/19/13
  • 16. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE David Hickam, MD, MPH, PCORI
  • 17. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Developing the PCORI Methodology Standards • Congressional Requirements- Patient Protection and Affordable Care Act, Subtitle D, Paragraph (6)(C)(i)
  • 18. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE PCORI Methodology Committee • Sherine Gabriel, MD, MSc (chair), Mayo Clinic • Robin Newhouse, PhD, RN (vice chair), University of Maryland • Naomi Aronson, PhD, BlueCross BlueShield • Ethan Basch, MD, MSc, Memorial Sloan-Kettering Cancer Center • Alfred O. Berg, MD, MPH, University of Washington • David Flum, MD, MPH, University of Washington • Steven Goodman, MD, MHS, PhD, Johns Hopkins School of Public Health • Mark Helfand, MD, MS, MPH, Dept. of Veterans Affairs • John Ioannidis, MD, DSc, Stanford University • Michael S Lauer, MD, National Heart, Lung, and Blood Institute • David O. Meltzer MD, PhD, University of Chicago • Brian S. Mittman, PhD, Dept. of Veterans Affairs • Sebastian Schneeweiss, MD, ScD, Harvard Medical School • Jean R. Slutsky, Agency for Healthcare Research and Quality • Mary Tinetti, MD, Yale University School of Medicine • Clyde Yancy, MD, MSc, Northwestern University Feinberg School of Medicine
  • 19. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE PCORI Methodology Standards http://www.pcori.org/research-we-support/methodology/ PCORI Methodology Committee Formed (2011) Draft Methodology Report Posted (July 2012) Public Comment Period (July- September 2012) Proposed Revisions to Methodology Standards (November 2012) Revised PCORI Methodology Standards (December 2012) Final Methodology Report (Spring 2013)
  • 20. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE 11 Categories of Methodology Standards 47 Individual Standards *Bold indicates topics to be discussed in detail at the June 25 in-person workshop at the AcademyHealth Annual Research • Formulating Research Questions • Patient-Centeredness • Systematic Reviews • General and Cross-cutting Methods for all PCOR • Causal Inference • Heterogeneity of Treatment Effects • Preventing/Handling Missing Data • Data Networks • Adaptive and Bayesian Trial Designs • Data Registries • Studies of Diagnostic Tests
  • 21. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Characteristics of the Methodology Standards • Are minimal standards for performing comparative effectiveness research. • Are intended to provide helpful guidance to researchers and those who use research results. • Reflect generally accepted best practices. • Provide guidance for both project protocols and reporting of results. • Are used to assess the scientific rigor of funding applications. • Context of research should drive use of the standards.
  • 22. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Standards that Apply to Most Projects • Formulating research questions (6 standards) • Patient-centeredness (4 standards) • General and cross-cutting (6 standards) • Missing data (5 standards)
  • 23. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Formulating Research Questions  Identify affected populations and health decision(s), and how study results will inform the health decision (RQ-3)  Identify and assess participant subgroups (RQ-4)  Select appropriate interventions and comparators and make explicit how each were selected (RQ-5)
  • 24. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Patient-Centeredness  Engage the population of interest and other relevant stakeholders as appropriate and necessary in a given research context (PC-1)  Identify, select, recruit, and retain study participants representative of the population of interest (PC-2)  Use patient-reported outcomes when patients or people at risk of a condition are the best sources of information. (PC-3)  Support dissemination and implementation of study results (PC-4)
  • 25. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE General and Cross-cutting Methods • Assess the suitability of the data source (GM-1) • Document data analysis plans for major aims (GM-3) • Document validated scales and tests • Include characteristics of the scales and psychometric properties (GM-4)
  • 26. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Missing Data • Use Validated Methods to Deal with Missing Data that Properly Account for Statistical Uncertainty Due to Missingness (MD-3) • Record and Report All Reasons for Dropout and Missing Data, and Account for All Patients in Reports (MD-4)
  • 27. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Standards that Apply to Specific Data Sources and Study Designs • Systematic reviews (1 standard) • Causal inference (6 standards) • Heterogeneity of treatment effects (4 standards) • Data networks (2 standards) • Adaptive trials (5 standards) • Registries (3 standards) • Diagnostic tests (5 standards)
  • 28. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Causal Inference • Precisely Define the Timing of the Outcome Assessment Relative to the Initiation and Duration of Exposure (CI-3) • Measure Confounders before Start of Exposure. Report data on confounders with study results (CI-4)
  • 29. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Diagnostic Tests • Assess the Effect of Factors Known to Affect Diagnostic Performance and Outcomes (DT-3)
  • 30. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Brian Mittman, PhD, Department of Veterans Affairs
  • 31. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE How to Use the Standards • The standards represent “best practice” recommendations for improving the quality and value of PCOR and CER. • The standards aim to achieve greater transparency, rigor and other improvements to strengthen the science and methods of PCOR and CER.
  • 32. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Implementing the standards: improving PCOR research practices • “Research practices” include decisions and actions by numerous stakeholders (funders, reviewers, researchers, etc.) • Changing these practices is challenging – Researchers value independence and autonomy – Researcher actions are guided by knowledge, experience, judgment • Analogue: the challenge of improving research practices through standards is comparable to the challenge of improving clinical practices through evidence-based clinical practice guidelines.
  • 33. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Implementing research standards • Professional practices are influenced by multiple factors operating at multiple levels • Implementation gaps must be thoroughly diagnosed to identify “root causes” • Implementation campaigns must include multiple, coordinated components targeting the full spectrum of determinants of gaps • Influences on research practices – and points of leverage to change practices – should be depicted in a causal, conceptual model There are “no magic bullets”: examples of dramatic change in research practices (e.g., trial registration) are rare and exceptional.
  • 34. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Conceptual model of the research process • Research question identification and prioritization (researchers, funding agencies) • Study design and methods selection (researcher judgment guided by training, professional norms, published standards) • Peer review (reviewer judgment guided by training, professional norms, published standards) • Study governance and conduct (researcher judgment, possibly guided by reporting requirements, IRB review, DSMB review) • Documentation and reporting (researcher judgment guided by reporting guidelines, journal requirements, peer review)
  • 35. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Dissemination and Future Directions • Initial set of standards released in December 2012. • PCORI Methodology Report will be released in Summer 2013. • No change in the set of standards. • Provides further guidance on interpretation and use. • Responding to future PCORI Funding Announcements (PFAs). • All future submissions must demonstrate adherence to/adoption of/consideration of the standards. • PCORI will continue to develop tools and resources for researchers, reviewers and other stakeholders.
  • 36. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Sean Tunis MD, MSc - CMTP
  • 37. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Purpose of Standards: PMC draft report 7/12 “Departures from basic good research practices are partially responsible for the mismatch between the quality and relevance of the information research provides and the information patients need to make informed clinical decisions.” “The initial range of topics was chosen to reflect areas in which the Committee believed that there were either substantial deficiencies or inconsistencies in how available methods were applied in practice, or for which there was specialized knowledge in how best to conduct research that had not been effectively disseminated.”
  • 38. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Required by Law! (ACA) “Methodological standards shall provide specific criteria for”: • Internal validity (robust) • Generalizability (relevant) • Timeliness (timely) • Feasibility
  • 39. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Where to Focus • Craft an important, relevant research question • Informed by gaps in systematic reviews and meaningful input from patients and other decision makers • Patient-centered care requires attention to subgroups and PROs • Apply best practices in application of selected methods • Provide full transparency Clear, detailed protocol – explain deviations from standards Follow existing reporting standards
  • 40. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Asking the Right Question “To produce information that is meaningful and useful to people when making specific health decisions, research proposals and protocols should describe: 1) the specific health decision the research is intended to inform; 2) the specific population for whom the health decision is pertinent; and 3) how study results will inform the health decision.” • Requires dialogue with decision makers (more is better) • Methods should serve research question; not vice versa
  • 41. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Use of Patient Reported Outcomes “Select outcomes based on input directly elicited from patient informants, people representative of the population of interest, either in previous studies or in the proposed research.” • Use existing core outcomes measures sets when available • COMET (Core Outcomes Measures in Effectiveness Trials) • Effectiveness Guidance Documents (Green Park Collaborative)
  • 42. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Feasibility: An Unresolved Issue Standard for Studies of Dx Tests: “Studies of clinical outcomes after diagnostic testing should use a prospective randomized study design when possible. If a non- randomized design is proposed, the reason for using an observational study (or modeling and simulation) should be addressed and efforts to minimize confounding documented.”
  • 43. PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE Looking Forward • Specifics of implementation of standards not yet clear • Initial emphasis of methods standards on patients and caregivers as decision makers. Intent to expand scope to clinicians, payers/purchasers, policy makers, others. • PCORI thinking continues to evolve based on experience and public feedback from researchers and other stakeholders • How best to achieve optimal balance of internal validity, generalizability, timeliness and feasibility remains an important challenge
  • 44. To submit a question: 1. Click in the Q&A box on the left side of your screen 2. Type your question into the dialog box and click the Send button Submitting Questions
  • 45. Expert Q&A  Erin Holve (facilitator)  Dave Hickam  Brian Mittman  Sean Tunis
  • 46. Still time to sign up for the PCORI Mini-Course • In-person events at AcademyHealth’s Annual Research Meeting in Baltimore, MD (June 25) Two morning panels • “Developing a Program of Comparative Effectiveness Research at the Patient Centered Outcomes Research Institute” (9:45-11:15am) • “PCORI Methods Pilot Projects: Learning from Engagement” (11:30am-1:00pm) • Afternoon workshop (June 25, 2:00-6:00pm ET) • Heterogeneity of Treatment Effects • Chair: Brian Mittman, PhD, Department of Veterans Affairs • Faculty: Ravi Varadhan, PhD, Jodi Segal, MD, MPH, Johns Hopkins University School of Medicine • Adaptive and Bayesian Trial Designs • Chair: Sally Morton, PhD, University of Pittsburgh • Faculty: Jason Connor, PhD, Berry Consultants • Data Registries • Chair: Robin Newhouse, PhD, RN, NEA-BC, FAAN, University of Maryland • Faculty: Priscilla Velentgas, Ph.D., Quintiles Outcome • Click here to register: $50 for AcademyHealth members, $100 for nonmembers
  • 47. Thank You Please take a moment to fill out the brief evaluation which will appear in your browser.

×