Presenter: Ronan Madden Unit 3 Evaluating your Search

Graduate Skills Programme Unit 1: Information Literacy & Your Research Unit 2: Research Resource Discovery Unit 3:Evaluating Resource Results Unit 4:Tracking Down Results & Keeping Up-to-date Unit 6: Ethics in Using Information Unit 5: Managing Information Publishing & Disseminating Research

Aim and Learning Outcomes Aims To examine the importance of critical thinking skills and to consider the use of an analytical framework To look at methods of identifying and interpreting research performance data as a measure of information quality and relevance Learning Outcomes Recognise the significance of critical appraisal Adopt an analytical approach to evaluating information Use citation analysis tools when evaluating research Place information in the context of your own research area

Aims

To examine the importance of critical thinking skills and to consider the use of an analytical framework

To look at methods of identifying and interpreting research performance data as a measure of information quality and relevance

Learning Outcomes

Recognise the significance of critical appraisal

Adopt an analytical approach to evaluating information

Use citation analysis tools when evaluating research

Place information in the context of your own research area

Topics for Today Overview: - significance of evaluation Critical Analysis/Appraisal Analytical Framework: - checklist (6 items) Focus on scholarly communication Citation analysis: - How do I evaluate a paper? Including citation impact example - How do I evaluate a Journal? including impact factor examples Final Word

Overview: - significance of evaluation

Critical Analysis/Appraisal

Analytical Framework: - checklist (6 items)

Focus on scholarly communication

Citation analysis:

- How do I evaluate a paper? Including citation impact example

- How do I evaluate a Journal? including impact factor examples

Final Word

Overview: Evaluating Information Sources (1) Critically important for 4 th level research. Quality sources: fundamental to developing a review of the literature that can be used as a foundation on which new research can be built. A critical approach helps determine relevance and value to your field. Researchers have traditionally drawn on journals, monographs, conference proceedings, (Cochrane Reviews in Health Sciences). Now many new information formats e.g. blogs, wikis, discussion lists, open access journals, open repositories How can you decide if material is of sufficient quality , suitable for inclusion in your literature review?

Critically important for 4 th level research. Quality sources: fundamental to developing a review of the literature that can be used as a foundation on which new research can be built.

A critical approach helps determine relevance and value to your field.

Researchers have traditionally drawn on journals, monographs, conference proceedings, (Cochrane Reviews in Health Sciences). Now many new information formats e.g. blogs, wikis, discussion lists, open access journals, open repositories

How can you decide if material is of sufficient quality , suitable for inclusion in your literature review?

Overview: Evaluating Information Sources (2) Generally we think of: Scholarly resources : aimed at those within the field; disseminate research within that discipline. Scholarly methods used in order to make claims about the world that are valid and trustworthy. Popular resources: aimed at a wider public/mass audience; entertain, inform, promote viewpoints, sell products and services (vested interests). So at basic level, is it: Factual, methodical, scientific, based on clearly referenced sources and documentation? OR subjective, journalistic, personal accounts/impressions, opinion?

Generally we think of:

Scholarly resources : aimed at those within the field; disseminate research within that discipline. Scholarly methods used in order to make claims about the world that are valid and trustworthy.

Popular resources: aimed at a wider public/mass audience; entertain, inform, promote viewpoints, sell products and services (vested interests).

So at basic level, is it:

Factual, methodical, scientific, based on clearly referenced sources and documentation?

OR

subjective, journalistic, personal accounts/impressions, opinion?

But: even material accepted for publication is not always reliable: E.G. 1. Research of Dr Hwang Woo-suk - creating human embryonic stem cells by cloning - no scientific evidence that Hwang had established stem cells Science 17/06/05, Vol. 308 Issue 5729, p1777-1783 Science 12/03/04, Vol. 303 Issue 5664, p1669-1674 E.G. 2. Research of Andrew Wakefield (MMR controversy) The Lancet 02/28/98, Vol. 351 Issue 9103, p637 The Lancet 06/03/04, Volume 363 Issue 9411, p750

But: even material accepted for publication is not always reliable:

E.G. 1. Research of Dr Hwang Woo-suk

- creating human embryonic stem cells by cloning

- no scientific evidence that Hwang had established stem cells

Science 17/06/05, Vol. 308 Issue 5729, p1777-1783 Science 12/03/04, Vol. 303 Issue 5664, p1669-1674

E.G. 2. Research of Andrew Wakefield (MMR controversy)

The Lancet 02/28/98, Vol. 351 Issue 9103, p637

The Lancet 06/03/04, Volume 363 Issue 9411, p750

Critical Analysis/Appraisal A focussed approach: process of reflective, reasonable, rational thinking to gather, interpret and evaluate information Adopting an analytical and reflective mindset at every stage of the research process will help you to - find the best sources - evaluate the information you find there - make decisions about your research methodology - discuss new findings - weigh up evidence and form conclusions - make recommendations and contribute effectively to your field i.e. your literature review becomes a solid foundation on which you can build new research, ideas, theories etc

A focussed approach: process of reflective, reasonable, rational thinking to gather, interpret and evaluate information

Adopting an analytical and reflective mindset at every stage of the research process will help you to

- find the best sources

- evaluate the information you find there

- make decisions about your research methodology

- discuss new findings

- weigh up evidence and form conclusions

- make recommendations and contribute effectively to your field

i.e. your literature review becomes a solid foundation on which you can build new research, ideas, theories etc

An Analytical Framework An analytical approach to assessing quality/value of information: look: - Relevance - Authority - Method - Objectivity - Presentation - Currency of information Apply this checklist to all information sources that you use

An analytical approach to assessing quality/value of information: look:

- Relevance

- Authority

- Method

- Objectivity

- Presentation

- Currency of information

Apply this checklist to all information sources that you use

1. Relevance W hy you wish to include a source; appropriate in the context of your research needs? Screen the content and look for general clues before deciding whether to digest or reject: Books: look at title, keywords, contents, index Journal: look at abstract, keywords, descriptors Websites look at title bar, document title, links to and from the site, author

W hy you wish to include a source; appropriate in the context of your research needs? Screen the content and look for general clues before deciding whether to digest or reject:

Books: look at title, keywords, contents, index

Journal: look at abstract, keywords, descriptors

Websites look at title bar, document title, links to and from the site, author

Relevance (contd.) Focus on: Level: detailed/general/simple? Emphasis: not always obvious from keywords/abstract Geographical: what countries/regions are included? Context of your own research: a unique insight into an aspect of your own research? Are ideas confirmed/refuted by the research? Note: ‘ Inclusion criteria ’ are often used in health sciences research to assess suitability. The inclusion criteria are clearly identified in advance with a focus for e.g. on who was allowed to participate in a trial/study, sample size, control groups, etc. Are they appropriate?

Focus on:

Level: detailed/general/simple?

Emphasis: not always obvious from keywords/abstract

Geographical: what countries/regions are included?

Context of your own research: a unique insight into an aspect of your own research? Are ideas confirmed/refuted by the research?

Note: ‘ Inclusion criteria ’ are often used in health sciences research to assess suitability. The inclusion criteria are clearly identified in advance with a focus for e.g. on who was allowed to participate in a trial/study, sample size, control groups, etc. Are they appropriate?

2. Authority Author: someone whose opinion or testimony is accepted; affiliated, qualified expert, academic credentials OR journalist/other author/commentator. Has their research been frequently cited? Is there a ‘ H index ’ ? In what publications? Peer-reviewed? Where indexed? Impact factor? (see Citation Analysis) Organisation: is it commercial, non-profit, government, research/educational? Vested interest? Contactable?

Author: someone whose opinion or testimony is accepted; affiliated, qualified expert, academic credentials OR journalist/other author/commentator. Has their research been frequently cited? Is there a ‘ H index ’ ? In what publications? Peer-reviewed? Where indexed? Impact factor?

(see Citation Analysis)

Organisation: is it commercial, non-profit, government, research/educational? Vested interest? Contactable?

Authority (contd.) Web Sites: (see also the ‘ Internet Detective ’) URL will often give a clue to type of organisation/country of origin: - Commercial company? (.com or .co.uk or .biz) - A non-profit organisation? (.org) - A government body? (.gov) - A research/educational organisation? (.org or .ac.uk or .edu) Look at the ‘about’ or ‘who are we’ section of the site Use Google Advanced Search to ‘find pages that link to the page’ How well established is it and when was the site last updated? Evidence of sponsorship? Is it clear who wrote the content and is that person contactable?

Web Sites: (see also the ‘ Internet Detective ’)

URL will often give a clue to type of organisation/country of origin:

- Commercial company? (.com or .co.uk or .biz)

- A non-profit organisation? (.org)

- A government body? (.gov)

- A research/educational organisation? (.org or .ac.uk or .edu)

Look at the ‘about’ or ‘who are we’ section of the site

Use Google Advanced Search to ‘find pages that link to the page’

How well established is it and when was the site last updated?

Evidence of sponsorship?

Is it clear who wrote the content and is that person contactable?

3. Method of Production & Methodology Type of Publication (Remember: anyone can publish on the web) - Peer-review procedure and instructions for authors e.g. BMJ peer reviewers , Information for Authors Peer review/Refereed/Juried: process by which an academic journal passes a paper submitted for publication to independent experts for comments on its suitability and worth; refereeing. - Editorial board members information should be available on the journal website or on print version if available e.g. BMJ Editorial Advisory Board

Type of Publication (Remember: anyone can publish on the web)

- Peer-review procedure and instructions for authors

e.g. BMJ peer reviewers , Information for Authors

Peer review/Refereed/Juried:

process by which an academic journal passes a paper submitted for publication to independent experts for comments on its suitability and worth; refereeing.

- Editorial board members information should be available on the journal website or on print version if available

e.g. BMJ Editorial Advisory Board

Method of Production & Methodology (contd.) Research Methodology: - Validity : tests what it aims to test? claims made that are trustworthy? Applicable to the case or circumstances? - Reliability : E xtent to which a measurement made repeatedly in identical circumstances will yield concordant results, consistent. Methods appropriate in terms of sample size, use of control groups, quantitative or qualitative survey design etc. to ensure reproducibility?

Research Methodology:

- Validity : tests what it aims to test? claims made that are trustworthy? Applicable to the case or circumstances?

- Reliability : E xtent to which a measurement made repeatedly in identical circumstances will yield concordant results, consistent. Methods appropriate in terms of sample size, use of control groups, quantitative or qualitative survey design etc. to ensure reproducibility?

4. Objectivity: What is the objective?: - to further understanding? To contribute to the field OR news, entertainment, opinion; editorial agenda/viewpoint - sponsors? bias/vested interests? Hidden bias, whether deliberate or not (GM foods, Stem cell research). - Backed up with evidence? Clearly referenced in a bibliography? Has the evidence been interpreted in a balanced/unbiased manner?

What is the objective?:

- to further understanding? To contribute to the field OR news, entertainment, opinion; editorial agenda/viewpoint

- sponsors? bias/vested interests? Hidden bias, whether deliberate or not (GM foods, Stem cell research).

- Backed up with evidence? Clearly referenced in a bibliography? Has the evidence been interpreted in a balanced/unbiased manner?

How is the information presented?: - Colour & font, general appearance - Language and writing style serious/sober OR glossy appearance - Use of diagrams and images - Structure and layout; logical? - Quality of reproduction - Advertising: minimal/substantial; target audience 5. Presentation

How is the information presented?:

- Colour & font, general appearance

- Language and writing style serious/sober OR glossy appearance

- Use of diagrams and images

- Structure and layout; logical?

- Quality of reproduction

- Advertising: minimal/substantial; target audience

6. Currency How up to date is the information/research? The importance of currency depends on the context within which you plan to refer to that research in your literature review. Factors to consider include: - Is it clear when the information was produced? - Does the date of the information meet my requirements? - Is it obsolete or superseded?

How up to date is the information/research?

The importance of currency depends on the context within which you plan to refer to that research in your literature review.

Factors to consider include:

- Is it clear when the information was produced?

- Does the date of the information meet my requirements?

- Is it obsolete or superseded?

‘ Scholarly ’ Communication Traditional Approach : from informal to formal with communication of research at every stage: - Informal: meetings, discussions, seminars, emails, blogs, social networking sites - Report ongoing research at conference (conference proceedings) - Publication in an academic journal - Indexed in research/bibliographic databases A ‘ periodical ’ published regularly/ at set intervals (includes magazines, newspapers, newsletters). A n ‘ academic journal ’ is a peer-reviewed scholarly periodical aimed at specialists and researchers. ‘ Scholarly communication ’ : dissemination of new ideas, theories, research in academic journals

Traditional Approach : from informal to formal with communication of research at every stage:

- Informal: meetings, discussions, seminars, emails, blogs, social networking sites

- Report ongoing research at conference (conference proceedings)

- Publication in an academic journal

- Indexed in research/bibliographic databases

A ‘ periodical ’ published regularly/ at set intervals (includes magazines, newspapers, newsletters). A n ‘ academic journal ’ is a peer-reviewed scholarly periodical aimed at specialists and researchers. ‘ Scholarly communication ’ : dissemination of new ideas, theories, research in academic journals

Other Periodicals (i.e. not peer-reviewed): 1. Popular Magazines : (articles written by journalists) - Newspapers - Substantive news e.g. ‘ The Economist ’ , ‘ National Geographic ’ , New Scientist ’ - Journal of opinion e.g. ‘ Magill ’ , ‘ Phoenix ’ - Other Magazines 2. Trade Journals (specific industry, enable practitioners share market and product information within an industry) e.g. ‘ Macworld ’ , ‘ Restaurant Business ’ , ‘ Chemical Week ’ . ‘ Beverage World ’ , ‘ Computerworld ’ etc.

(i.e. not peer-reviewed):

1. Popular Magazines : (articles written by journalists)

- Newspapers

- Substantive news e.g. ‘ The Economist ’ , ‘ National Geographic ’ , New Scientist ’

- Journal of opinion e.g. ‘ Magill ’ , ‘ Phoenix ’

- Other Magazines

2. Trade Journals (specific industry, enable practitioners share market and product information within an industry) e.g. ‘ Macworld ’ , ‘ Restaurant Business ’ , ‘ Chemical Week ’ . ‘ Beverage World ’ , ‘ Computerworld ’ etc.

Citation Analysis (Bibliometrics) How many times a paper or researcher is cited by other scholars in the field; assumes influential researchers/authors and important works cited more often than others. How do you find out? 1. Web of Knowledge (Thomson Reuters) Covers 9,000+ peer-reviewed journals 2. Scopus (Elsevier) Similar to Web of Science; covers 16,000 peer-reviewed journals; more than 4,000 international publishers; 1996 on (not yet available in UCC, IReL are considering) 3. Google Scholar provides links to ‘ cited by ’ information 4. Scimago Country & Journal Rank Database details journals and country-specific scientific indicators developed from the information contained in ‘ Scopus . ’

How many times a paper or researcher is cited by other scholars in the field; assumes influential researchers/authors and important works cited more often than others.

How do you find out?

1. Web of Knowledge (Thomson Reuters) Covers 9,000+ peer-reviewed journals

2. Scopus (Elsevier) Similar to Web of Science; covers 16,000 peer-reviewed journals; more than 4,000 international publishers; 1996 on (not yet available in UCC, IReL are considering)

3. Google Scholar provides links to ‘ cited by ’ information

4. Scimago Country & Journal Rank Database details journals and country-specific scientific indicators developed from the information contained in ‘ Scopus . ’

Citation Impact The impact of a paper or group of papers i.e. Number of times an article is cited by others: Check: - Web of Science (part of ISI Web of Knowledge) ‘ Times cited ’ , also includes ‘ Cited Reference Searching ’ to find articles that cite a person ’ s work. - Google Scholar: gives ‘ cited by ’ information - H index of the researcher: a new measure - highest number of papers a scientist has that have each received at least that number of citations e.g. someone with a h-index of 50 has written 50 papers that have each had at least 50 citations. Can find this in Web of Science; ‘ Create Citation Report ’ .

The impact of a paper or group of papers i.e. Number of times an article is cited by others:

Check:

- Web of Science (part of ISI Web of Knowledge) ‘ Times cited ’ , also includes ‘ Cited Reference Searching ’ to find articles that cite a person ’ s work.

- Google Scholar: gives ‘ cited by ’ information

- H index of the researcher: a new measure - highest number of papers a scientist has that have each received at least that number of citations e.g. someone with a h-index of 50 has written 50 papers that have each had at least 50 citations. Can find this in Web of Science; ‘ Create Citation Report ’ .

Example: Number of times cited Article Title: Identification of pancreatic cancer stem cells Author(s): Li CW, Heidt DG, Dalerba P, et al. Source: CANCER RESEARCH   Volume: 67   Issue: 3   Pages: 1030-1037   Published: FEB 1 2007 1. Go to Web of Knowledge 2. Pick ‘ Web of Science ’ 3. Enter the Title to locate article 4. Note Number of ‘ Times Cited ’ 5. Go to Google Scholar 6. Type article title in box 7. Note ‘ Cited by - ’ N.B Number of cites can only provide an indication of ‘ quality ’ . See final note below.

Article Title: Identification of pancreatic cancer stem cells

Author(s): Li CW, Heidt DG, Dalerba P, et al.

Source: CANCER RESEARCH Â Â Volume: 67 Â Â Issue: 3 Â Â Pages: 1030-1037 Â Â

Published: FEB 1 2007

1. Go to Web of Knowledge

2. Pick ‘ Web of Science ’

3. Enter the Title to locate article

4. Note Number of ‘ Times Cited ’

5. Go to Google Scholar

6. Type article title in box

7. Note ‘ Cited by - ’

N.B Number of cites can only provide an indication of ‘ quality ’ . See final note below.

Password: Qfloor1

Password: Qfloor1

How many times has the following article been cited in Web of Science: ‘Evaluation of RAD50 in Familial Breast Cancer’? 58 8 9 0 0 of 30 10

58

8

9

0

Journal Impact Journal Impact Factor: the average number of times articles from the journal published in the past two years have been cited in the JCR year Journal Immediacy Index : average number of times an article is cited in the year it is published Check: ‘ Journal Citation Reports ’ ( Journal Performance Metrics ) includes impact factors. See also ‘ Essential Science Indicators ’ . Part of Thomson ’ s ISI Web of Knowledge Scimago Country & Journal Rank Database : a new resource (free on the web) detailing journals and country scientific indicators developed from the information contained in ‘ Scopus ’

Journal Impact Factor: the average number of times articles from the journal published in the past two years have been cited in the JCR year

Journal Immediacy Index : average number of times an article is cited in the year it is published

Check:

‘ Journal Citation Reports ’ ( Journal Performance Metrics ) includes impact factors. See also ‘ Essential Science Indicators ’ . Part of Thomson ’ s ISI Web of Knowledge

Scimago Country & Journal Rank Database : a new resource (free on the web) detailing journals and country scientific indicators developed from the information contained in ‘ Scopus ’

Examples: Impact Factor 1. What is the impact factor of the Journal ‘ The Lancet ’ ? 1. Go to ISI Web of Knowledge 2. Click on ‘ Select a database ’ 3. Pick ‘ Journal Citation Reports ’ 4. Choose ‘ JCR Science Edition ’ and ‘ Search for Specific Journal ’ 5. Click on ‘ submit ’ 6. Type ‘ Lancet ’ in box and search for ‘ Full Journal Title ’ Impact factor 2007 = 28.638 (extremely high) 2. What are the highest impact journals for Oncology? 1. Go to ISI Web of Knowledge 2. Click on ‘ Select a database ’ 3. Pick ‘ Journal Citation Reports ’ 4. Choose ‘ JCR Science Edition ’ and ‘ View a Group of Journals by Subject Category ’ 6. Click on ‘ Submit ’ 7. Scroll down to ‘ Oncology ’ 8. Click on ‘ Submit ’ 9. Sort by Impact factor

1. What is the impact factor of the Journal ‘ The Lancet ’ ?

1. Go to ISI Web of Knowledge

2. Click on ‘ Select a database ’

3. Pick ‘ Journal Citation Reports ’

4. Choose ‘ JCR Science Edition ’ and ‘ Search for Specific Journal ’

5. Click on ‘ submit ’

6. Type ‘ Lancet ’ in box and search for ‘ Full Journal Title ’

Impact factor 2007 = 28.638 (extremely high)

2. What are the highest impact journals for Oncology?

1. Go to ISI Web of Knowledge

2. Click on ‘ Select a database ’

3. Pick ‘ Journal Citation Reports ’

4. Choose ‘ JCR Science Edition ’ and ‘ View a Group of Journals by Subject Category ’

6. Click on ‘ Submit ’

7. Scroll down to ‘ Oncology ’

8. Click on ‘ Submit ’

9. Sort by Impact factor

What is the 2007 Impact Factor for ‘Cancer Research’ journal? 2.139 0.452 17.824 7.672 0 of 30 :10

2.139

0.452

17.824

7.672

What journal had the highest 2007 impact factor in the field of Biochemistry & Molecular Biology? International Journal of Cancer Annual Review of Biochemistry ACS Chemical Biology BMJ 0 of 30 :09

International Journal of Cancer

Annual Review of Biochemistry

ACS Chemical Biology

BMJ

Impact Factor- some limitations Very useful tools, but IF not a measure of true quality of a journal, e.g.: current popularity of topic and availability journal may give a higher IF based on an average over all articles: underestimates the citations of the top cited article, exaggerates the number of citations of the average article comparison of impact factors between different fields is invalid e.g. not relevant for literature, where the most important publications are books citing other books only the ISI database journals are used; undercounts the number of citations from other journals e.g. in ‘ less-developed ’ countries, other languages (American bias) 50% of papers are never cited

Very useful tools, but IF not a measure of true quality of a journal, e.g.:

current popularity of topic and availability journal may give a higher IF

based on an average over all articles: underestimates the citations of the top cited article, exaggerates the number of citations of the average article

comparison of impact factors between different fields is invalid e.g.

not relevant for literature, where the most important publications are books citing other books

only the ISI database journals are used; undercounts the number of citations from other journals e.g. in ‘ less-developed ’ countries, other languages (American bias)

50% of papers are never cited

Impact Factor- some limitations (contd.) ‘ Like nuclear energy, the impact factor is a mixed blessing.’ ‘ The use of journal impacts in evaluating individuals has its inherent dangers. In an ideal world, evaluators would read each article and make personal judgments ’ . Garfield, E. (2006) ‘ The History and Meaning of the Impact Factor ’ . JAMA, 295(1): pp 90-93 [Garfield invented scholarly citation in the 1960 ’ s] N.B. See also Unit 7: Section 7.6 ‘ Deciding Where to Publish ’

‘ Like nuclear energy, the impact factor is a mixed blessing.’

‘ The use of journal impacts in evaluating individuals has its inherent dangers. In an ideal world, evaluators would read each article and make personal judgments ’ .

Garfield, E. (2006) ‘ The History and Meaning of the Impact Factor ’ . JAMA, 295(1): pp 90-93 [Garfield invented scholarly citation in the 1960 ’ s]

N.B. See also Unit 7: Section 7.6 ‘ Deciding Where to Publish ’

Final word: Cannot always rely on impact factors, times cited etc. Each researcher must make their own judgment about the quality and suitability of any article or information source. Depends a lot on the context i.e. how the research is to be used or referred to in your literature review. You may need to draw on material from newspapers, conferences, web material etc. with no peer review available. Refer to the framework/checklist outlined above and check with your supervisor when in doubt. Evaluation is an art; there no perfect indicator of quality. You need to look for clues, and ultimately judge on the basis of usefulness for your research question

Cannot always rely on impact factors, times cited etc.

Each researcher must make their own judgment about the quality and suitability of any article or information source.

Depends a lot on the context i.e. how the research is to be used or referred to in your literature review. You may need to draw on material from newspapers, conferences, web material etc. with no peer review available.

Refer to the framework/checklist outlined above and check with your supervisor when in doubt.

Evaluation is an art; there no perfect indicator of quality. You need to look for clues, and ultimately judge on the basis of usefulness for your research question

Appendix Critical Appraisal for EBM: (some specific questions): Is there a clear research question? Was the assignment of treatments to patients randomised? Were all participants accounted for in the conclusion? Is the literature review appropriate? Were participants ‘ blind ’ to the treatment? Were the groups similar at the start of the trial? Were the groups treated equally (aside from the intervention)? Were ethical issues considered? Were data collection methods described adequately? Were methods of analysis appropriate? What are the key findings? How significant and precise are the results? Can the results be applied to my research/the local population? Were all important outcomes considered? Were the benefits greater than harms and costs? (Compiled from links on the final slide)

Critical Appraisal for EBM: (some specific questions):

Is there a clear research question?

Was the assignment of treatments to patients randomised?

Were all participants accounted for in the conclusion?

Is the literature review appropriate?

Were participants ‘ blind ’ to the treatment?

Were the groups similar at the start of the trial?

Were the groups treated equally (aside from the intervention)?

Were ethical issues considered?

Were data collection methods described adequately?

Were methods of analysis appropriate?

What are the key findings?

How significant and precise are the results?

Can the results be applied to my research/the local population?

Were all important outcomes considered?

Were the benefits greater than harms and costs?

(Compiled from links on the final slide)

Links for Further Information on Critical Appraisal in Health Sciences: What is critical Appraisal? How to read a paper: Assessing the methodological quality of published papers (Trisha Greenhalgh) Critical appraisal and using the literature (University of Sheffield) (includes worked examples) Netting the Evidence Oxford Centre for Evidence Based Medicine NHS Center for Reviews and Dissemination

What is critical Appraisal?

How to read a paper: Assessing the methodological quality of published papers (Trisha Greenhalgh)

Critical appraisal and using the literature (University of Sheffield) (includes worked examples)

Netting the Evidence Oxford Centre for Evidence Based Medicine

NHS Center for Reviews and Dissemination

Exercises: Check how many times the following article has been cited in a. Web of Knowledge b. Google Scholar: ‘ Influence of a family history of cancer….’ by Poole 2. What is the h-index for Tommiska, author of the following article: ‘Evaluation of RAD50 in Familial Breast Cancer’ Find information about ‘Cancer Research’ journal in Scimago Country & Journal Rank Database 4. What are the top two multidisciplinary journals in Web of Knowledge (ranked by impact factor)?

Check how many times the following article has been cited in

a. Web of Knowledge

b. Google Scholar:

‘ Influence of a family history of cancer….’ by Poole

2. What is the h-index for Tommiska, author of the following article: ‘Evaluation of RAD50 in Familial Breast Cancer’

Find information about ‘Cancer Research’ journal in Scimago Country & Journal Rank Database

4. What are the top two multidisciplinary journals in Web of Knowledge (ranked by impact factor)?