Review of information retrieval – first experiences inGermany with manufacturer dossiers on new drugs9th HTAi Annual Meeti...
Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossie...
Drug assessment according to AMNOG                                                                                        ...
Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossie...
Requirements for information retrieval in dossier templates –Module 4   4.2.3 Methods of information retrieval      Bibl...
Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossie...
Process in Information Management Unit: Check the search in bibliographic                                                 ...
Checking the search in bibliographic databases (formal check)Methods of information retrievalCheck whether requirements f...
Checking the search in bibliographic databases (content check)   Is the number of hits plausible?   Checking the search ...
Process in Information Management Unit: Check the search in bibliographic                                                 ...
Potential literature searches by IQWiG1. Search in public trial registries- ClinicalTrials.gov- ICTRP Search Portal and- p...
Process in Information Management Unit: Check the search in bibliographic                                                 ...
Documentation                Informationsbeschaffung in Dossiers   13
Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossie...
Examples of shortcomings potentially resulting in a check of thecompleteness of the study pool by IQWIG   Search strategy...
Syntax errors → Search not sensitive enough                         Informationsbeschaffung in Dossiers   16
EMBASEInterface: Elsevier                                                            Ovid Interface                       ...
Lack of plausibility of the number of hits                           Informationsbeschaffung in Dossiers   18
Search checked on 08.07.2011: n= 61New citations since 01.05.2011: n=3                         Informationsbeschaffung in ...
Experiences so far   additional relevant studies have only been identified from the list of studies    provided by the ma...
Conclusions   Our approach to the review of information retrieval in manufacturer dossiers    seems to be workable.   Be...
Institute for Quality and Efficiency in Health Care (IQWiG)Dr. Siw WaffenschmidtPotential Conflicts of Interest: NONE   I...
Search in bibliographic databasesChoice of search techniques is based on the work by Sampson 2009 (“Updating   searches fo...
Example – IQWiG’s search in PubMedPubmed: 13.01.2012Total number of hits without duplicates: 70Screening result: no releva...
Upcoming SlideShare
Loading in …5
×

Review of information retrieval – first experiences in Germany with manufacturer dossiers on new drugs

587 views

Published on

Review of information retrieval – first experiences in Germany with manufacturer dossiers on new drugs

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
587
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Review of information retrieval – first experiences in Germany with manufacturer dossiers on new drugs

  1. 1. Review of information retrieval – first experiences inGermany with manufacturer dossiers on new drugs9th HTAi Annual Meeting BILBAO26 June 2012Dr. Siw WaffenschmidtInformation Management Unit, IQWiG
  2. 2. Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossier template3) Review of information retrieval by IQWiG4) Examples Informationsbeschaffung in Dossiers 2
  3. 3. Drug assessment according to AMNOG Market Submission of a dossier Price set by entry by the manufacturer manufacturer Assessment (by IQWiG) Month 3 Decision on added benefit by the Federal Joint Committee Month 6 no Added benefit? yes Reference Price negotiations Permissible for price yes no SHI and reference price? manufacturer Price at no added cost versus Discount on Agreement? yes comparator sales price Month 12 treatment no Decision by arbitration body Month 15 On request of SHI or manufacturer: health3 economic evaluation
  4. 4. Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossier template3) Review of information retrieval by IQWiG4) Examples Informationsbeschaffung in Dossiers 4
  5. 5. Requirements for information retrieval in dossier templates –Module 4 4.2.3 Methods of information retrieval  Bibliographic literature search  Search in trial registries  Selection of relevant studies 4.3.1 Results of information retrieval  RCTs of the drug to be assessed  Studies for indirect comparisons  Indirect comparisons on the basis of RCTs  Non-randomized comparative studies  Further studies Appendix  4-A: Search strategies – bibliographic literature search  4-B: Search strategies – search in trial registries  4-C: List of studies screened as full texts and excluded (with reason for exclusion) Attachments  RIS files Informationsbeschaffung in Dossiers 5
  6. 6. Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossier template3) Review of information retrieval by IQWiG4) Examples Informationsbeschaffung in Dossiers 6
  7. 7. Process in Information Management Unit: Check the search in bibliographic Check the search in trial registries databases Evaluate information retrievalDepending on results of evaluation: (potentially) conduct own search in Depending on results of evaluation: bibliographic databases and/or trial No further action registries Documentation Informationsbeschaffung in Dossiers 7
  8. 8. Checking the search in bibliographic databases (formal check)Methods of information retrievalCheck whether requirements for the search procedure and the presentation ofresults have been fulfilled as specified in the dossier templateScreeningWas the procedure for study selection described?Are deviations from the above requirements justified?Result of the information retrievalCheck consistency of retrieved hitsDo the individual sections of the dossier contain consistent information oninclusion and exclusion criteria? Documentation of the above steps in checklists developed for this purpose. Informationsbeschaffung in Dossiers 8
  9. 9. Checking the search in bibliographic databases (content check) Is the number of hits plausible? Checking the search strategy presented in Appendix 4-A  Were search filters used and correctly transferred?  Syntax acc. to PRESS Checklist Was the study pool identified via the search strategy?  Is the search sensitive enough? Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J. PRESS: Peer Review of Electronic Search Strategies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2008. URL: http://www.msmandhiv.org/documents/globaltopics/PRESS.pdf. Informationsbeschaffung in Dossiers 9
  10. 10. Process in Information Management Unit: Check the search in bibliographic Check the search in trial registries databases Evaluate information retrievalDepending on results of evaluation: (potentially) conduct own search in Depending on results of evaluation: bibliographic databases and/or trial No further action registries Documentation Informationsbeschaffung in Dossiers 10
  11. 11. Potential literature searches by IQWiG1. Search in public trial registries- ClinicalTrials.gov- ICTRP Search Portal and- potentially in industry registries2. Search in bibliographic databases- Simple Structured Boolean Search (SSBS)- combined with the first 20 Related Citations in PubMed Sampson M. Updating searches for systematic reviews. Aberystwyth: University; 2009. URL: http://cadair.aber.ac.uk/dspace/bitstream/2160/3760/1/Sampson%20- %20Updating%20Searches%20for%20Systematic%20Reviews%20- %20PhD.pdf. Waffenschmidt S, Janzen T, Hausner E, Kaiser T. Assessing the completeness of systematic reviews via the “related articles” function and/or a simple structured Boolean search in PubMed – a pilot study.2011. (Scientific evidence for healthcare quality and patient safety: 19th Cochrane Colloquium; 19.-22.10.2011; Madrid, Spanien). Informationsbeschaffung in Dossiers 11
  12. 12. Process in Information Management Unit: Check the search in bibliographic Check the search in trial registries databases Evaluate information retrievalDepending on results of evaluation: (potentially) conduct own search in Depending on results of evaluation: bibliographic databases and/or trial No further action registries Documentation Informationsbeschaffung in Dossiers 12
  13. 13. Documentation Informationsbeschaffung in Dossiers 13
  14. 14. Content1) Introduction to the early benefit assessment of new drugs2) Requirements for information retrieval in the dossier template3) Review of information retrieval by IQWiG4) Examples Informationsbeschaffung in Dossiers 14
  15. 15. Examples of shortcomings potentially resulting in a check of thecompleteness of the study pool by IQWIG Search strategy errors Lack of sensitivity Lack of plausibility of the number of hits Inconsistencies in search documentation Informationsbeschaffung in Dossiers 15
  16. 16. Syntax errors → Search not sensitive enough Informationsbeschaffung in Dossiers 16
  17. 17. EMBASEInterface: Elsevier Ovid Interface results in 64.508 Informationsbeschaffung in Dossiers 17
  18. 18. Lack of plausibility of the number of hits Informationsbeschaffung in Dossiers 18
  19. 19. Search checked on 08.07.2011: n= 61New citations since 01.05.2011: n=3 Informationsbeschaffung in Dossiers 19
  20. 20. Experiences so far additional relevant studies have only been identified from the list of studies provided by the manufacturer (e.g. study listed but excluded by manufacturer), not by own searches Challenges in the checking and evaluation of information retrieval:  Assessment of the sensitivity of the searches  Indirect comparisons and further studies Informationsbeschaffung in Dossiers 20
  21. 21. Conclusions Our approach to the review of information retrieval in manufacturer dossiers seems to be workable. Besides checking the search strategies, peer review of information retrieval should cover all aspects of a search. Ultimately, the in(completeness) can only be verified by doing a focused search yourself . Informationsbeschaffung in Dossiers 21
  22. 22. Institute for Quality and Efficiency in Health Care (IQWiG)Dr. Siw WaffenschmidtPotential Conflicts of Interest: NONE Im Mediapark 8 50670 Cologne, Germany Tel: +49-221/3 56 85-0 Fax: +49-221/3 56 85-1 Siw.Waffenschmidt@iqwig.de www.iqwig.de Informationsbeschaffung in Dossiers 22
  23. 23. Search in bibliographic databasesChoice of search techniques is based on the work by Sampson 2009 (“Updating searches for systematic reviews” [PhD])On the basis of Sampson’s approach, we conducted our own analysis based on systematic reviews of drugs: Main result:The combination of the first 20 Related Citations and the SSBS seems to be the mostappropriate technique, with high completeness (sensitivity: 97.1%), good reliability(sensitivity range: 80-100%) and an efficient use of resources (precision: 5.0%, NNR:24). (manuscript submitted) Informationsbeschaffung in Dossiers 23
  24. 24. Example – IQWiG’s search in PubMedPubmed: 13.01.2012Total number of hits without duplicates: 70Screening result: no relevant additional studies found ItemsSearch Query found#1 Search Sativex*[tiab] or Nabiximol*[tiab] or Cannab*[tiab] or 20478 GW*1000*[tiab] or *THC*[tiab] or Marinol*[tiab] or Tetrahydrocannabinol*[tiab] or Nabilon*[tiab] Simple Structered#2 Search multiple sclerosis 51363 Boolean Search#3 Search #1 and #2 355#4 Search (Therapy/Narrow[filter]) AND (#3) 33#5 Search 14615106[pmid] 1 The first 20 Related#6 Related Citations for PubMed (Select 14615106) 106 Citations of the 5#7 Select 19 document(s) 19 relevant citations#8 …… …. included in the dossier Informationsbeschaffung in Dossiers 24

×