Setting Standards for Research Methods
•Download as PPTX, PDF•
2 likes•453 views
PCORI held a special webinar on Friday, August 3, 2012, from 1:00 p.m.− 2:00 p.m. ET, to brief stakeholders on PCORI’s draft Methodology Report and encourage public comment. The draft report, released July 23, 2012, offers standards to guide health care stakeholders towards the best and most appropriate methods for conducting patient-centered outcomes research.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Setting Standards for Research Methods
Similar to Setting Standards for Research Methods (20)
More from Patient-Centered Outcomes Research Institute
More from Patient-Centered Outcomes Research Institute (20)
Recently uploaded
Recently uploaded (20)
Setting Standards for Research Methods
- 1. Patient-Centered Outcomes Research Institute PCORI Methodology Committee Report Setting Standards for Research Methods August 3, 2012
- 2. Presenters Joe Selby, MD, MPH Robin Newhouse, PhD, RN Executive Director Member, PCORI Methodology PCORI Committee Chair and Professor, Organizational Systems and Adult Health at University of Maryland School of Nursing Steven Goodman, MD, PhD Lori Frank, PhD Member, PCORI Methodology Director, Committee Engagement Research Associate Dean for Clinical and PCORI Translational Research, School of Medicine, Stanford University 2
- 3. Webinar Agenda 1. Introduction to PCORI 1:00pm – 1:10pm ET 2. Methodology Committee Mission & Report 1:10pm – 1:20pm ET 3. Research Methods Standards 1:20pm – 1:30pm ET 4. Questions and Answers 1:30pm – 2:00pm ET Please submit questions for the Q&A portion of today‟s webinar to methodswebinar@pcori.org Formal public comments can be submitted at pcori.org/survey/methodology-report/ 3
- 4. Poll Questions 1 - 4 1. Are you familiar with the contents of the Methodology Committee Report? (Y/N) 2. Are you a researcher? (Y/N) 3. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 4. Which response most closely matches your opinion of the Standards in the draft Report? a) The Standards largely cover the main areas important to patient-centered outcomes research. b) Several important areas are not covered and additional Standards should be considered. c) Don‟t know/Not sure 4
- 5. About PCORI • An independent, non-profit organization authorized by Congress. • Committed to continuously seeking input from patients and a broad range of stakeholders to guide its work. • Mission − To help people make informed health care decisions and improve health care delivery and outcomes by: Producing and promoting high integrity, evidence- based information that comes from research guided by patients, caregivers and the broader health care community. 5
- 6. Defining Patient-Centered Outcomes Research (PCOR) Helps people and their caregivers communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options. This research answers patient-centered questions such as: Expectations Options Outcomes Decisions “Given my “What are my “What can I do “How can personal options and what to improve the clinicians and the characteristics, c are the potential outcomes that care delivery onditions and benefits and are most systems help me preferences, wha harms of those important to make the best t should I expect options?” me?” decisions about will happen to my health and me?” healthcare?” 6
- 7. Webinar Agenda Introduction to PCORI 1:00pm – 1:10pm ET Methodology Committee Mission & Report 1:10pm – 1:20pm ET Research Methods Standards 1:20pm – 1:30pm ET Questions and Answers 1:30pm – 2:00pm ET Please submit questions for the Q&A portion of today‟s webinar to methodswebinar@pcori.org Formal public comments can be submitted at pcori.org/survey/methodology-report/ 7
- 8. PCORI Methodology Committee MEMBER TITLE Sherine Gabriel, MD, MSc Professor of Medicine and of Epidemiology , William J. and Charles H. Mayo Professor at Mayo Clinic (Chair) Sharon-Lise Normand, MSc, Professor of Health Care Policy (Biostatistics) in the Department of Health Care Policy at Harvard Medical PhD (Vice Chair) School and Professor in the Department of Biostatistics at the Harvard School of Public Health Naomi Aronson, PhD Executive Director of the Blue Cross and Blue Shield Association Technology Evaluation Center Ethan Basch, MD, MSc Associate Attending Physician and Outcomes Scientist at Memorial Sloan-Kettering Cancer Center Alfred Berg, MD, MPH Professor in the Department of Family Medicine at the University of Washington in Seattle David Flum, MD, MPH Professor in the Department of Surgery and Adjunct Professor in Health Services and Pharmacy at the University of Washington Schools of Medicine, Public Health and Pharmacy Steven Goodman, MD, PhD Associate Dean for Clinical and Translational Research, School of Medicine , Stanford University Mark Helfand, MD, MS, MPH Professor of Medicine and Professor of Medical Informatics and Clinical Epidemiology at the Oregon Health & Science University John Ioannidis, MD, DSc C.F. Rehnborg Chair in Disease Prevention, Professor of Medicine, Professor of Health Research and Policy, and Director of the Stanford Prevention Research Center at Stanford University Michael Lauer, MD Director of the Division of Cardiovascular Sciences at the National Heart, Lung, and Blood Institute David Meltzer, MD, PhD Chief of the Section of Hospital Medicine, The University of Chicago Brian Mittman, PhD Director, VA Center for Implementation Practice and Research Support, Department of Veterans Affairs Greater Los Angeles VA Healthcare System Robin Newhouse, PhD, RN Chair and Professor, Organizational Systems and Adult Health at University of Maryland School of Nursing Sebastian Schneeweiss, MD, Associate Professor of Medicine and Epidemiology at Harvard Medical School and Vice Chief of the Division of ScD Pharmacoepidemiology and Pharmacoeconomics at the Brigham and Women’s Hospital Jean Slutsky, PA, MSPH Director of the Center for Outcomes and Evidence , Agency for Healthcare Research and Quality Mary Tinetti, MD Gladdys Phillips Crofoot Professor of Medicine, Epidemiology, and Public Health in the Division of Geriatrics at Yale University School of Medicine Clyde Yancy, MD, MSc Chief, Cardiology, Northwestern University Feinberg School of Medicine
- 9. Methodology Report • The mandate for PCORI‟s Methodology Committee is to define methodological standards and a translation table to guide health care stakeholders towards the best methods for patient- centered outcomes research (PCOR). • Rigorous methods are essential to building trust in research findings. • The report is the necessary catalyst for scientifically rigorous, patient-centered outcomes research that can inform decision-making. • Once Report is revised and accepted by the PCORI Board of Governors, future PCORI funding applicants will be expected to reference the Standards in their applications and use the Standards in their PCORI funded research. 9 9
- 10. Methodology Report – Methods Selection Building on the work of the Institute of Medicine*, the Methodology Committee defined a standard as… • A process, action, or procedure for performing PCOR that is deemed essential to producing scientifically valid, transparent, and reproducible results; a standard may be supported by scientific evidence, reasonable expectation that the standard helps achieve the anticipated level of quality in PCOR, or by broad acceptance of the practice in PCOR • The recommendation is actionable, feasible, and implementable • Proposed standards are intended for use by the PCORI Board, in PCORI policies and procedures, and by PCORI researchers *Reference: National Research Council. Find What Works in Health Care: Standards for Systematic Reviews. 10 Washington, DC: The National Academies Press; 2011.
- 11. Methodology Report – Methods Selection The MC sought to address selected topics in 4 broad phases of activities in the first Methodology Report: What study How do we How do we What should designs carry out and enable people we study? should we govern the to apply the use? study? study results? 11
- 12. Methodology Report Development 1 Methods • Working groups identified and prioritized major research methods Selection questions to be addressed 2 • Researchers contracted to address selected topics • Contractors developed research materials (e.g., reports, summary Committee Expertise Information templates for proposed standard) Gathering • MC solicited for external feedback on the translation table (RFI) • Workshops held to discuss contractor findings, with invited experts in attendance 3 • MC conducted in-depth internal review of materials developed by contractors, and support staff • MC independently submitted preliminary votes on proposed Internal Review standards • MC deliberated to reach consensus on recommendations to be endorsed in the report 4 Report • Refined recommendations and report content per committee evaluations and discussions Generation 12
- 13. Methodology Report – Methods Selection 13
- 14. Methodology Report – Internal Review The MC deliberated and agreed upon standards based on the following: Patient- Respect for and responsiveness to individual Centeredness patient preferences, needs, and values Objectivity, minimizing bias, improving Scientific Rigor reproducibility, complete reporting Explicit methods, consistent application, public Transparency review Empirical/ Information upon which a proposed standard is Theoretical Basis based Other Practicality, feasibility, barriers to Considerations implementation, and cost 14
- 15. Methodology Report Submitted to the PCORI Board of Governors on May 10, 2012 Accepted by the PCORI Board of Governors on May 21, 2012 A public comment period on the draft report: Through September 14 2012 Revised Report goes to the Board of Governors November 2012 15
- 16. Webinar Agenda Introduction to PCORI 1:00pm – 1:10pm ET Methodology Committee Mission & Report 1:10pm – 1:20pm ET Research Methods Standards 1:20pm – 1:30pm ET Questions and Answers 1:30pm – 2:00pm ET Please submit questions for the Q&A portion of today‟s webinar to methodswebinar@pcori.org Formal public comments can be submitted at pcori.org/survey/methodology-report/ 16
- 17. Methodology Report – Research Domains Formulating Research Patient Research Prioritization Centeredness Questions Heterogeneity Causal General and Missing of Treatment Inference Crosscutting Data Effects Data Adaptive Data Diagnostic Networks Trials Registries Testing 17
- 18. General Research Standards 3.1.3 Identify and Assess Participant Subgroups 3.1.4 Select Appropriate Interventions and Comparators 7.1.1 Assess Data Source Adequacy 7.1.2 A Priori, Specify Plans for Data Analysis that Correspond to Major Aims 7.1.3 Document Validated Scales and Tests 7.1.4 Use Sensitivity Analyses to Determine the Impact of Key Assumptions 7.1.5 Provide Sufficient Information in Reports to Allow for Assessments of the Study‟s Internal and External Validity 18
- 19. Causal Inference Standards 7.2.1 Define Analysis Population Using Information Available at Study Entry 7.2.2 Describe Population that Gave Rise to the Effect Estimate(s) 7.2.3 Precisely Define the Timing of the Outcome Assessment Relative to the Initiation and Duration of Intervention 7.2.4 Measure Confounders before Start of Exposure 7.2.5 Assess Propensity Score Balance 7.2.6 Assess Instrumental Variable Assumptions 19
- 20. Heterogeneity of Treatment Effects • People react differently to treatment • Problems with summarizing/ averages – Answers across lots of types of people are not useful for decisions – Do not answer “what will happen to people like me” • Challenges in dividing patients in „right‟ groups 20
- 21. Heterogeneity (HTE) Standards 7.3.1 State the Goals of HTE Analyses 7.3.2 For Confirmatory and Descriptive HTE Analyses, Pre-specify Subgroups and Outcomes; for Confirmatory HTE Analyses, Pre-specify Hypotheses for Each Subgroup Effect 7.3.3 For Confirmatory HTE Analyses, Report a priori Statistical Power 7.3.4 For Any HTE Analysis, Perform an Interaction Test and Report Sufficient Information on Treatment Effect Estimates 7.3.5 For Exploratory HTE Analyses, Discuss Findings in the Context of Study Design and Prior Evidence 7.3.6 For Any HTE Analysis, Report All Pre-specified Analyses and, at Minimum, the Number of Post-hoc Analyses, Including Number of Subgroups and Outcomes Analyzed 21
- 22. Missing Data Standards 7.4.1 Describe in Protocol Methods to Prevent and Monitor Missing Data 7.4.2 Describe Statistical Methods to Handle Missing Data in Protocol 7.4.3 Use Validated Methods to Deal with Missing Data that Properly Account for Statistical Uncertainty Due to Missingness, Such as Multiple Imputation. All Forms of Single Imputation Are Discouraged 7.4.4 Record and Report All Reasons for Dropout and Missing Data, and Account for All Patients in Reports 7.4.5 Examine Sensitivity of Inferences to Missing Data Methods and Assumptions, and Incorporate into Interpretation. 22
- 23. Data Networks • Explosion of new data – Electronic Medical Records (EMRs) – Linking data sets – New data collection technology • Need to assure – Patient Privacy – Data quality – Consistency 23
- 24. Data Network Standards 7.5.1 Data Integration Strategy 7.5.2 Risk Assessment Strategy 7.5.3 Identity Management and Authentication of Individual Researchers 7.5.4 Intellectual Property Policies 7.5.5 Standardized Terminology Encoding of Data Content 7.5.6 Metadata Annotation of Data Content 7.5.7 Common Data Model 24
- 25. Adaptive Trials • Flexible not fixed – Adjust based on results that are monitored during study period • Advantages – More relevant – Faster results – Less expensive (sometimes) • Challenges – Complex to conduct – Need to be careful not to introduce bias into the study 25
- 26. Adaptive Trial Standards 8.1.1 Specify Planned Adaptations and Primary Analysis 8.1.2 Evaluate Statistical Properties of Adaptive Design 8.1.3 Specify Structure and Analysis Plan for Bayesian Adaptive Randomized Clinical Trial Designs 8.1.4 Ensure Clinical Trial Infrastructure Is Adequate to Support Planned Adaptation(s) 8.1.5 Use the CONSORT Statement, with Modifications, to Report Adaptive Randomized Clinical Trials 26
- 27. Registries • Database – Information generated during normal care – Focused on a disease or treatment – Data from multiple sources • Challenges – Privacy – Data Quality and Consistency – Sorting out cause and effect 27
- 28. Registry Standards 8.2.1 Describe Data Linkage Plans, if Applicable 8.2.2 Plan Follow-up Based on the Registry Objective(s) 8.2.3 Describe Data Safety and Security 8.2.4 Take Appropriate Steps to Ensure Data Quality 8.2.5 Document and Explain Any Modifications to the Protocol 8.2.6 Collect Data Consistently 8.2.7 Enroll and Follow Patients Systematically 8.2.8 Monitor and Take Actions to Keep Loss to Follow-up to an Acceptable Minimum 8.2.9 Use Appropriate Statistical Techniques to Address Confounding 28
- 29. Diagnostic Tests Standards 8.3.1 Specify Clinical Context and Key Elements of Diagnostic Test Study Design 8.3.2 Study Design Should Be Informed by Investigations of the Clinical Context of Testing 8.3.3 Assess the Effect of Factors Known to Affect Diagnostic Performance and Outcomes 8.3.4 Structured Reporting of Diagnostic Comparative Effectiveness Study Results 8.3.5 Give Preference to Randomized Designs of Studies of Test Outcomes 29
- 30. Webinar Agenda Introduction to PCORI 1:00pm – 1:10pm ET Methodology Committee Mission & Report 1:10pm – 1:20pm ET Research Methods Standards 1:20pm – 1:30pm ET Questions and Answers 1:30pm – 2:00pm ET Please submit questions for the Q&A portion of today‟s webinar to methodswebinar@pcori.org Formal public comments can be submitted at pcori.org/survey/methodology-report/ 30
- 31. Questions and Answers Please submit questions for the Q&A portion of today‟s webinar to methodswebinar@pcori.org Formal public comments can be submitted at pcori.org/survey/methodology-report/ 31
- 32. Poll Questions 5 - 8 5. Have you ever submitted an application for funding to PCORI?? (Y/N) 6. Do you plan to submit an application for funding to PCORI in the future? (Y/N) 7. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 8. Do you plan to submit comments on the Report through the PCORI website? (Y/N) 32
- 33. We look forward to your comments on the Draft Methodology Report Visit us at www.pcori.org (today‟s webinar will be archived there) Subscribe to PCORI updates at pcori.org/subscribe Follow @PCORI on Twitter Watch our YouTube channel PCORINews 33
Editor's Notes
- Lori to introduce webinar and welcome listeners.
- Lori to describe agenda for the hour, with explanation of how to submit questions and first of many encouragements for people to submit comments through website. Lori to introduce first poll and give instructions for completing. Poll questions:1. Are you familiar with the contents of the Methodology Committee Report? (Y/N)2. Are you a researcher? (Y/N)3. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 4. Which response most closely matches your opinion of the Standards in the draft Report? a) The Standards largely cover the main areas important to patient-centered outcomes research. (Agree/Disagree) b) Several important areas are not covered and additional Standards should be considered. (Agree/Disagree)
- Lori will introduce Joe. Joe to speak to this slide.Before we get to the specifics about the Methodology Report, I want provide some background on the mission of PCORI and of the Methodology Committee.PCORI is an independent, non-profit organization committed to seeking input from patients and other stakeholders on all aspects of our work. Why was PCORI created?For all of the research conducted; for all of the therapies created; for all of the advances in care that have been made – Patients and clinicians do not always have the information they need to make choices that reflect their unique situation, priorities or personal preferences. In some areas, research has yet to ask some of the most important questions patients have. PCORI is committed to helping people make informed health decisions b producing and promoting high quality evidence-based information to guide patients, caregivers, and clinicians.
- Joe’s slideHere is a definition of patient-centered outcomes research that guides the work PCORI does.
- Lori’s slideThank you Joe. Now we’ll provide a brief overview of the Mission of the MC and the process that led to creation of this first draft report. More detail can be found within the report itself, available on the PCORI website: pcori.orgWe remind everyone to submit questions that you have for Robin and Steven through xxx.pcori.org.We also want to encourage your submission of comments and suggestions related to the Methodology Report and you can do so through the PCORI website at xxx.pcori.org.Now I’ll turn it over to Dr. Newhouse.
- Robin: Dr. Selby introduced you to the overall mission of PCORI, and I will speak directly about the MC and introduce you to the Methodology Report, now posted for public comment. The Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI) was appointed by the Comptroller General of the U.S. Government Accountability Office in January 2011, after appointment of the PCORI Board in September 2010. The MC has a specific function outlined in the statute.To achieve its goals, The Methodology Committee consists of 17 experts: You can see the diversity of scientific expertise on this slide, which includes:Health services researchClinical researchComparative clinical effectiveness researchBiostatistics Genomics Research methodologies.This slide is included as reference and will be available in the archived slideset, and details about the membership of the Committee is available in the Report.
- Robin: The mandate for the Methodology Committee of PCORI is to define methodological standards and develop a translation table to guide health care stakeholders towards the best methods for patient-centered outcomes research (PCOR). The MC has four specific roles:Provide guidance about the appropriate use of methods in patient-centered outcomes research Establish priorities to address gaps in research methods or their application Recommend actions to support standards Map research methods to specific research questions via Translation TableThe methodology report is the first deliverable to address PCOR methods. The report includes the first set of methodological standards and a translation table to guide health care stakeholders towards the best methods for patient-centered outcomes research (PCOR). Use of these standards will promote Rigorous methods that are essential to building trust in research findings. Using scientifically rigorous methods will produce patient-centered outcomes research results that can inform decision-making. Once the Report is revised and accepted by the PCORI Board of Governors, future PCORI funding applicants will be expected to reference the Standards in their applications and use the Standards in their PCORI funded research.
- RobinThe methodology report includes the recommended standards for PCOR. Building on the work of the IOM*, the MC defined a standard as…A process, action, or procedure for performing PCOR that is deemed essential to producing scientifically valid, transparent, and reproducible results. Standards can be supported by scientific evidence, a reasonable expectation that the standard helps achieve the anticipated level of quality in PCOR, or by broad acceptance of the practice in PCORThe recommendation is actionable, feasible, and implementableProposed standards are intended for use by the PCORI Board, in PCORI policies and procedures, and by PCORI researchers
- RobinFor the first report, the methodology committee sought to address selected topics in four broad phases of activities. Those four phases were:What should we study? (research priorities)What study design should we use, and how do we carry out and govern the study? (research designs and methods)How do we enable people to apply the study results? (patient-centerdness)
- RobinThe process for developing the first methodology report over our first year together as the methodology committee involved four phases: method selection, information gathering, internal review, and report generation.Method selection involve working in groups to prioritize the major research methods questions to be addressed.Second, we entered anInformation gathering phase in which request for proposals were publicly solicited for selected topics, contractors were selected and completed their reports with summary templates of proposed standards. In addition to methodology committee solicited externalfeedback on the translation table through a request for information. Workshops were held to discuss contractor findings with invited experts in attendance.Third, the methodology committee began a phase of internal review. The committee conducted an in-depth internal review of the materials that were developed by the contractors and support staff. We then independently submitted are pulmonary votes on the proposed standards and then met in person to deliberate and reach consensus on the recommendations to be endorsed in the report.The report was then generated and refined based on committee of evaluations and discussions.
- Robin In the course of creating the MC report, PCORI, through contractors, spoke with a lot of people all over the country. This map shows some of the parts of the country where researchers spoke with patients, caregivers, and other stakeholders in patient-centered outcomes research. In addition to input from patients, caregivers and other stakeholders, input was also received from 17 groups representing 100 individuals on research teams, 15 invited experts that provided input on the standards at workshops in March 2012, 24 submissions to provide input on the translation table after a Request for Information, and 57 stakeholders were interviewed to understand CER use in health records and informatics.
- Steve
- SteveThe first methodology report was specifically required by the legislation that created PCORI, as we stated earlier. The MC submitted a draft report to the Board of Governors in May. The board accepted the draft and we are now asking for public comment. We want and need input….and based on that as well as ongoing review by the Methods Committee the report will be revised and a final version will be submitted to the Board in November.
- SteveNow we’ll spend a few minutes briefly discussing the topics covered in the Research Methods standards, and we continue to collect your questions which we will address for the remainder of the hour.
- SteveAs we mentioned earlier, today’s webinar covers many of the standards related specifically to research methods. The next webinar, on August 14, covers standards related to patient-centeredness, patient engagement, and research prioritization.Time check – if close to 1:30pm ET, announce that we will move to questions.If between 1:25 and 1:30, your call if you and Robin want to take the time to show slides 25-34.
- These standards can be found in chapters 3 and 7 and the numbers of the standards correspond with their location in the report. The first step is to establish some general standards for research.A big part of good research, is planning. These general standards are designed to promote planning that including documenting key decisions and testing the assumptions that underlie the analysisThese standards are about being clear about WHO is going to be studied, HOW the research will be conducted, and WHY this approach was taken.Being clear---standard 7.1.5, and the rest are the WHO, HOW and WHYAs we go through more specific standards you will see these themes of planning, documentation and clarity repeated.
- As researchers we want to produce research that will help people make decisions and this requires attempting to understand what cause and effect.Since we cannot change the complexity of people and health, the standards focus on a few key approaches that can INCREASE OUR CONFIDENCE that the outcome is caused by the treatmentFirst, we need to be clear about who is studied---this is the first two standardsThen we need to specify key components of the relationship between the treatment and oucome7.2.3 is about making sure the treatment comes before the outcome….but that their relationship in time is such that you can be pretty sure that the treatment is causing the effect.The other issues are about thinking about other causes, besides treatment.---that is what confounders are…so this standard is saying you should think about what else might be the cause and measure it as part of the study…and do so before treatment starts, so that you know that it was not affected by treatment at that point.Propensity scores and Instrumental Variables are just tools used in studies where you don’t randomize patients to conditions like “treatment condition” and “control group.” They help researchers account for differences in the groups of patients and adjust the results for these differences. These methods are increasing in popularity and for this reason the MC felt it was important that researchers be clear when using propensity scores and when they address instrumental variables.
- [Someone on the MC has a better image…multiple people merged into one….I can not remember who]One challenge to doing this type of research is that all of us are different; we are usually not “typical.” If we study a treatment across a large group of people we may not be able to tell how it actually works for specific people…”people like me.”Figuring out what different groups to study and how to study them is important to PCOR.
- The standards about HTE are about being clear, having a plan, examining your assumptions about what is important for what group of people and using statistical methods to strength any conclusions the study can make about subgroups.This is important because AFTER you do a study it is often possible to find some group of patients in which the treatment has an effect…..(to be ridiculous---women who have had 2 children born on a Tuesday)…When they are silly like this it is easy to see the problem….the challenge is that some subgroups might seem entirely reasonable…For studies to provide useful information about ‘people like me’ different groups need to be carefully defined and justified.
- The MC provides some standards relating to missing data issues specifically. It is important that Researchers[Stay on here or move to next slide—up to presenter]Try to prevent it…..7.4.1Monitor and document it 7.4.1 and 7.4.4Describe and use the best statistical methods possible to address missing data 7.4.2, 7.4.3Check your work---figure out what the impact of the missing data is 7.4.5
- One of the very important trends that supports the potential of PCOR is the explosion of data….More data are available….through electronic patient records, through merged data sets—for example linking hospital data, cancer registries and ER data to follow people for years after chemotherapy.. through innovation technologies for data collection…like using smart phones to have patient record symptoms or GPS trackers to see when and where patients need to use their asthma inhalers.We have more data then we have ever had before. This is VERY exciting.Using this for PCOR requires that we work to assure that privacy is protected, that the data are of high quality and that data are consistent across sources---
- The standards for data networks are designed to ensure that theses concerns are addressed. This is an area where more standards and refinement of standards are likely to be needed as the field is changing quickly.
- Adaptive Trials are those in which changes are made during the study. This could be the number of people, the dose of the drug being tested, how long people take it.These have huge potential, but they are difficult to do. The reasons studies do not normally change things part way through is to reduce the risk of bias and to have clarity about cause and effect. Changing things as the study progresses makes this harder. Adaptive trials can get us answers more quickly, though.
- Here again, you can see that the standards focus onPlanningUsing Statistical Tools to evaluation key elementsAnd Reporting that makes the approach transparent.
- Registries are essential databases set up to collect information on patients from multiple sources as they receive ‘usual care’. This means there is not an intervention or a trial---the researcher isn’t determining who gets which treatment. Data can come from hospitals, laboratories, doctors offices The patient and clinician may not need to do any thing….it is automated and the data are identified and merged or there may be a system to trigger the patient or doctor to provide more information. For example a cancer registry might ask that the patient complete a standardized questionnaire about chemotherapy side effects.The challenges with registries are simliar to some of the challenged for data networks (privacy and data quality and consistency) and are some of the same challenges we face in causal inference and issues of confounding seen across other study designs.
- This is reflected the Standards….and the themes should be looking familiar by now.
- Basically the standards ask for details….both about the text and context. As in other areas, the standards call for reporting that is standardized and complete…in order to make what the research involved more transparent to all users of the research.Finally in this area the final standard recommends that RCTs be considered…in part because these are not often done and they may be better able to address selection bias and confounding which are often common in studies of tests.
- Thank you both. We have been collecting questions as those listening in have been submitting them so now we’ll read some questions and I’ll Robin and Steve to address them.We also would like to draw your attention to another poll we would like to you to complete…1. Have you submitted an application for funding to PCORI in the past? (Y/N)2. Do you plan to submit an application for funding to PCORI in the future? (Y/N)3. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 4. Do you plan to submit comments on the Report through the PCORI website? (Y/N)
- Thank you both. We have been collecting questions as those listening in have been submitting them so now we’ll read some questions and I’ll Robin and Steve to address them.We also would like to draw your attention to another poll we would like to you to complete…1. Have you submitted an application for funding to PCORI in the past? (Y/N)2. Do you plan to submit an application for funding to PCORI in the future? (Y/N)3. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 4. Do you plan to submit comments on the Report through the PCORI website? (Y/N)
- Lori1:59pm ETThank you for your time today and thank you for your interest in the work of the the MC and PCORI. A copy of these slides are available on the pcori website, and you can submit all comments on the MC Report through the website as well. This concludes our webinar today. Thank you again.