Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Who can you really trust in science

36 views

Published on

Critical assessment session for researchers

Published in: Education
  • Be the first to comment

  • Be the first to like this

Who can you really trust in science

  1. 1. Who can you really trust in science? George Cronin Deputy Librarian (Biological Sciences) [they/them]
  2. 2. Where do you get your scientific knowledge from?
  3. 3. Sources of scientific knowledge • Published books • Academic journals • Conferences • Teaching • Research (original and secondary) • PIs • Newsletters • Mailing lists • Mainstream news • Blogs • Social media
  4. 4. How do you evaluate the reliability of your sources?
  5. 5. Reputation/credentials of author Research is in a good publication Research has been peer-reviewed
  6. 6. Case studies Why would you trust this person? Why might you not trust them? Discuss in groups…
  7. 7. Example 1 – The researcher Researcher works for leading research institution in Japan with an annual budget of around $760 million Is promoted to head of their lab and develops innovative new technique which is a leap forward for their field Publishes ground-breaking research in leading peer-reviewed journal Nature with multiple co-authors
  8. 8. Example 2 – The revolutionary Significant research career spanning several decades and helped redefine their field of research in multiple ways Recipient of multiple awards including French Légion d'honneur Prolific writer with multiple titles to their name both within academia and in the public eye
  9. 9. Example 3 – The blogger Prolific blogger writing extensively online about their field of research Lots of external links to secondary sources plus promotion of author’s own book Collaborates with other authors on the blog and covers a wide range of topics and issues with no paywalls
  10. 10. Example 4 – the TV expert Medical consultant with relevant qualifications as well as membership of multiple professional bodies Holds lectureship position at UK Higher Education institution Regularly consults and presents on national television programming and has written multiple books aimed as a public audience
  11. 11. Example 1 - Haruko Obokata • Appointed Head of Lab for Cellular Reprogramming in 2013 • Developed a quick and easy technique of converting body cells into a form similar to embryonic stem cells • Published findings with multiple co-authors in Nature • Findings were quickly questioned by scientific community • Research company Riken concluded scientific misconduct • Nature articles retracted • Obokata also has her PhD thesis removed due to plagiarism • One co-author (and mentor) committed suicide
  12. 12. Example 2 – Jane Goodall • In 2014, Goodall’s Seeds of Hope book had its publication delayed after a Washington Post editor spotted replication of other work • At least 12 sections of the book had material taken from a multiple of sources, including Wikipedia, without due credit • Goodall explained: “In some cases, you look at my notebooks, there's no way you can tell whether this is from talking to somebody or whether it was something I read on the internet.”
  13. 13. Example 3 – Vaughan Bell • Lead writer of the blog Mind Hacks, which focuses on neuroscience and psychology • Neuroscience researcher at UCL and clinician in London hospital • Blog is very vague as to who authors are but they have expertise • Be open with your identity online as a researcher • But anonymity can be useful too…
  14. 14. Example 4 – Dr Raj Persaud • Well-known TV psychologist with lots of medical experience and credentials to back him up • But he was caught out for plagiarism in a book and several articles that he wrote • He was disciplined by the GMC and was struck off for 3 months • Continues to work in TV and is still practicing
  15. 15. Traditional publishing What helps you decide where to publish? •Prestige of journal? •Impact Factor score? •Recommendation from PI? •Best coverage of discipline? •Open Access compliant?
  16. 16. Predatory journals and conferences Further guidance on the Office of Scholarly Communication website
  17. 17. Is High Impact all that important?
  18. 18. Reasons to reconsider the Impact Factor • The data is skewed • High Impact Factor journals skew the data even more • Not all journals are represented or counted • Methodology behind Impact Factor calculations is not transparent • Impact Factor does not just apply to original research • The numbers can be cooked by self-citations • No correlation between citations and quality of research • Impact Factor does not support interdisciplinary research • Impact Factor scores have huge time lags Thanks to Jeroen Bosman for some of this information: https://im2punt0.wordpress.com/2013/11/03/nine-reasons-why-impact-factors-fail-and-using-them-may-harm-science/
  19. 19. Pre-prints and specialist publishing • Lots of discipline-specific pre-print sites (arXiv, BioRxiv etc.) • Overlay journals that sit on top of pre-print services • Allow democratisation of content publishing • Support very niche subject areas
  20. 20. Retraction Watch Top 3 cited retracted papers Primary Prevention of Cardiovascular Disease with a Mediterranean Diet New England Journal of Medicine (2013) Retracted 2018. 1845 citations before retraction Visfatin: A protein secreted by visceral fat that mimics the effects of insulin Science (2005) Retracted 2007. 243,915 citations before retraction Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children Lancet (1998) Retracted 2010. 640 citations before retraction
  21. 21. Peer review challenges Peer review • Often anonymous • Can be useful process • But can also be flawed • Very time consuming • Things get missed • People are not infallible
  22. 22. What do you think about closed peer review?
  23. 23. What about open sharing?
  24. 24. Working reproducibly • Helps to avoid disaster • Makes it easier to write papers • Helps reviewers see it your way • Enables continuity of your work • Helps to build your reputation Florian Markowetz, CRUK
  25. 25. Reporting standards Transparency improves reproducibility www.equator-network.org/ Slide provided by Isla Kuhn
  26. 26. Social media • Gives people an equal platform • Allows Early Career Researchers to develop their voice • Reach multiple audiences • But also deal with problematic challenges • Use feedback and interactions to drive research • Block those who don’t help • Engage with those who do
  27. 27. Contribute to Wikipedia Image: xkcd CC BY
  28. 28. What is fake news? ‘Used to defend an illogical position, or a way to avoid using critical thinking skills when a news source does not match a personal or ideological bias’ (Urban Dictionary)
  29. 29. How can researchers make a difference?
  30. 30. Communicate your science
  31. 31. To conclude… • Reputation and reliability of science is as fallible as the people who do it • Critically evaluate your sources • Take advantage of different publishing and sharing options • Use metrics with an awareness of their biases • Share, contribute, communicate!

×