Selecting studies - Korean


Published on

After you have run your literature search, the next step is to carefully screen the results to identify those studies that will be included in your review. This presentation gives guidance on how to go about selecting your included studies.

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Once you have conducted your search, the next step is to go through the search results to identify those studies that meet your inclusion criteria.
  • Selecting which studies are included in and excluded from your review is perhaps the most important decision you will make, determining the results you will find to answer your review question. It’s essential to minimise any possible bias in making these judgements. Each record identified through your search should be carefully compared against your inclusion criteria, and must meet all of them to be included in the review. This is why it is crucial to have clear, pre-specified criteria from your protocol to help you make these decisions as objectively as possible, and avoid ad hoc, biased decision making about inclusion. Having a written summary or checklist for authors to have on hand when screening the search results can be helpful – whether on paper or electronically as a form or spreadsheet. As soon as a study fails to meet one of the criteria, it can be excluded and there’s no need to consider it further. HANDOUT: Sample checklist for eligibility criteria. To minimise the risk of bias, two authors should independently select studies. Having a second perspective is helpful especially where it isn’t clear whether a study meets the inclusion criteria or not. The process of discussion around the decision ensures that the criteria are applied fairly and consistently. It may be helpful to have one of the authors selecting studies not be a content expert, to avoid any preconceptions about particular studies or interventions. Discussion about difficult decisions can identify gaps where your eligibility criteria are not clear, or where different authors interpret the criteria differently. It’s helpful to pilot the selection process on a few papers first, to identify any issues and perhaps clarify the checklist or explanation of eligibility criteria for authors. Although you should avoid make any changes to your criteria at this stage, it may be necessary to make an addition or change to address an issue not anticipated at the protocol stage. If that happens, you should be transparent in your review and declare any changes made. Any changes affecting the scope of your review should be agreed with your Review Group. When both authors have looked at all the included studies, they should come together to discuss their findings and identify any remaining disagreements, before making final decisions about the included studies. [Note if asked: It is possible to conduct selection process with blinding, e.g. by editing copies of the articles to removed information about authors, locations, journals and results. This is time-consuming and benefit has not been conclusively shown.] [Note if asked: Agreement can be measured using the kappa statistic, but this is not required (see Handbook section 7.2.6 for method of calculation).]
  • A typical selection process begins with the search results. The majority of your search results will be in the form of titles and abstracts retrieved from electronic databases, although sometimes you will only have the title and keywords, and other times you will immediately have the full text available. The first stage is to go through the titles and abstracts, removing any records where it is clear that the study does not meet one or more of the eligibility criteria. It’s important to be inclusive at this stage, and to give records the benefit of any doubt, as only limited information is available in the abstract, and you might be surprised by what you find in the full text. In some cases, obviously irrelevant records can be removed at this stage by a single author, although best practice is to have two authors complete the whole selection process. Once you have removed any clearly excluded studies, then you can proceed to retrieve the full text of all the remaining reports. Studies must meet all your criteria to be included in the review, but make sure you have all the information you need. Some studies are reported in more than one paper, but studies – not individual papers – are the primary units of your review. You will need to find and bring together all the reports of a study for your review before you have all the information about that study – not forgetting to include any errata, comments or retractions that may have been printed. In some cases, the information you have may not be enough to make a decision about the study, in which case you should contact the study’s authors if possible to obtain further details.
  • Detailed information on which participants met your inclusion criteria may not be available – so you may not be able to just extract those participants of interest, or even to determine whether a study meets a threshold of the majority of participants meeting your criteria. Although post hoc decisions should be avoided, it may be better to consider the overall aims and rationale for defining the scope of your review, and considering studies on a case by case basis, than to stick to an arbitrary rule. Any decisions you make about inclusion of particular studies that aren’t clearly covered by your eligibility criteria should be documented in your review, and you may wish to conduct sensitivity analysis on your results to assess the impact of your decisions on the review’s findings.
  • Throughout the selection process, you should keep careful records about the studies excluded from the review. In the Results section of the review, you will be required to give a summary of the total number of records identified in your search, and the number excluded at each stage of the screening process (i.e. the number excluded at the title and abstract stage, the number retrieved in full text, and the final number included in the review). You may wish to give a brief summary here of the reasons for exclusion (e.g. so many excluded because they were not the right intervention, so many because they were not the right population, etc.). You will also include in your review a table of ‘Characteristics of excluded studies’. Here you will have to list excluded studies that readers may think might meet your inclusion criteria, but which have been excluded. Each study should be listed along with the specific reason for it exclusion. Depending on the size of your search results, this list might include, for example all the studies you retrieved in full text, that could not be excluded on the basis of the abstract, or it may include a more limited list. This ensures the transparency of your decisions to exclude studies – any reader who wonders about a particular study or disagrees with your decision can see exactly why the study was excluded.
  • Thinking back to the protocol stage, you should provide a brief description of this selection process. Describe whether two authors will independently assess each record identified from your search, the process you will use (e.g. starting with abstracts, then retrieving the full text of potentially relevant studies), how you will manage disagreements (e.g. by discussion, or using a third author), and any other details of your planned selection process.
  • This is a PRISMA flow chart which can be generated in RevMan for your review, which outlines the flow of information through the different phases of the systematic review , that is, the number of records identified by database searching, the number of records excluded and the number studies included in the systematic review and in a meta-analysis. It is recommended that you include a PRISMA flowchart in your review so that your readers can clearly see the methods you used in selecting studies for inclusion.
  • Selecting studies - Korean

    1. 1. 분석대상 연구자료 선 정 Selecting studies
    2. 2. 코크란 체계적 문헌고찰의 과정 Steps of a Cochrane systematic review1. 연구질문 설정2. 선정 기준 계획3. 연구 방법 계획4. 연구자료 ( 원자료 ) 검색5. 선정 기준 적용6. 자료 추출7. 분석 대상 연구들의 비뚤림 위험 평가8. 결과 분석 및 제시9. 분석 결과 해석 및 결론 도출10. 문헌고찰 개선 및 주기적 갱신See Chapter 7 of the Handbook cochrane training
    3. 3. 분석대상 선정 중 비뚤림 위험의 최소 화 Minimising bias in selection• 분석 대상 선정 시 리뷰어의 판단이 개입되며 , 이 판단 은 문헌고찰 결과에 큰 영향을 미침• 분석 대상으로 선정된 모든 연구자료들이 사전에 정의된 선정기준에 부합하는지 비교 • 요약본 및 체크리스트 작성이 도움이 될 수 있음• 두 명의 저자가 독립적으로 각각 분석대상을 선정해야 함 • 저자 간 불일치는 보통 토론이나 제 3 저자의 중재로 해결함 • 토론을 통해 분석대상 분류 혹은 선정기준 적용 상 개선점을 발견할 수도 있음 • 예비연구 차원으로 , 처음 몇 개 논문에 선정기준을 적용 , 검토해 볼 것 cochrane training
    4. 4. 선정 기준의 적용 Putting it into practice1. 제목 (title) 및 초록 (abstract) 검토 • 해당 연구는 모든 선정기준을 충족시킬 수 있는가 ? • 명백하게 무관한 연구는 배제시키되 , 본 단계에서는 일단 선정시킨 후 추가 검토를 통해 배제할 수 있음 (be inclusive)2. 연구 논문 확보 및 검토 • 해당 연구가 모든 선정 기준을 충족시키는가 ? • 동일 연구에 대한 여러 논문이 발표되었을 경우 , 관련 정 보를 모두 취합할 것 • 최종 판단을 위해 모든 자료가 필요할 수도 있음 • 저자 정보 , 연구 이름 , 연구 시행 장소 , 치료 / 중재 방법 , 연구참여자 , 기저점 baseline 정보 , 날짜 , 연구등록번호 등 의 정보를 취합할 것 • 논문 정오표 , 논평 및 논문 철회 정보 등을 취합할 것 • 추가 정보 필요 시 , 교신저자에게 연락할 것 cochrane training
    5. 5. 연구대상 집단이 혼재된 경우 Mixed populations• 연구자료 내 연구참여자 중 일부만이 선정기준에 부합할 경우 • 예 ; 연구자의 선정기준 상으로는 16 세 이하의 청소년을 포함하 지만 , 어떤 연구자료는 18 세 이하의 청소년을 포함할 수 있음• 어떤 방법이 해당 연구의 목적에 가장 부합하는가 ? • 해당 연구자료의 내용 전체를 포함 • 최소 기준을 정할 수 있음 ( 예 ; 80% 혹은 대다수 연구참여자가 선 정기준에 부합할 경우 ) • 선정기준에 부합하는 일부 연구참여자들의 정보만 포함 • 그러나 일부 환자들만의 정보는 제공되지 않을 수도 있음 • 해당 연구자료 내용 전체를 배제• 항상 합리적 기준을 제시하고 판단 근거를 명기할 것 cochrane training
    6. 6. 배제된 연구자료 내용의 보고 Reporting excluded studies• 결과 (Results) 항목 • 각 단계 별로 확인 ( 검색 ) 및 배제된 연구자료의 개수를 포함한 , 검색 결과 보고• ‘ 배제된 연구자료의 특성’ 목록 ‘ Characteristics of excluded studies’ table • 주요 배제된 연구자료 목록 및 주요 배제 원인 기입 • 독자들이 선정기준에 부합할 수도 있다고 여길 수 있지만 , 면밀히 조사할 경우 그렇지 않은 연구들을 포함시킬 것 • 선정기준에 부합하지 않음이 명백할 경우 포함시키지 말 것 cochrane training
    7. 7. 연구계획서에 포함시켜야 할 내용 What to include in your protocol• 두 명의 저자가 독립적으로 연구자료 선정을 위한 평가과정에 참여할 것인지 여부• 평가 과정 ( 예 ; 초록 및 전문 사용 )• 저자 간 의견 불일치 발생 시 해결 방안• 기타 사용된 연구방법 cochrane training
    8. 8. cochrane training
    9. 9. cochrane training
    10. 10. 핵심 전달 사항 Take home message• 선정 기준에 근거하여 초록 및 논문 전문을 평가할 때 체계적 방법론을 견지할 것• 상기 과정은 오류 및 비뚤림을 최소화하기 위해 독 립적인 두 명의 저자에 의해 수행되어야 함 . cochrane training
    11. 11. References• Higgins JPT, Deeks JJ (editors). Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008). The Cochrane Collaboration, 2008. Available from Acknowledgements • 편집 Miranda Cumpston • 호주 코크란 센터 (Australasian Cochrane Centre) 교육자료를 원자료로 함 • 본 교육자료는 Cochrane Methods Board 가 승인하였음 • Translated by Kun Hyung Kim, Myeong Soo Lee and Byung-Cheul Shin cochrane training