Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bias and the Data Lifecycle


Published on

Presentation to CRC Mental Health Early Career Researcher Workshop, Melbourne 29.11.17 for @andsdata.
Workshop title: A by-product of scientific training: We're all a little bit biased.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Bias and the Data Lifecycle

  1. 1. Richard Ferrers Bias and the Data Life Cyle Research Data Specialist, ANDS 29 Nov 2017 @valuemgmt v.4
  2. 2. Outline  Bias: What is the problem? Where do I come from?  The Code of Responsible Conduct for Research  Types of Bias; cognitive, memory, emotional, cultural  Data Approaches to engage with Bias  Data in the research lifecycle  Interesting notes on Bias; positivism vs pragmatism  An exercise – qualitative research and bias
  3. 3. Introduction / Background My perspective, My preferences / biases My Data – google: Ferrers Figshare My works: Topics: NBN, Value, Adoption / Diffusion of Innovation My PhD: An intepretive pragmatist social constructionist Science/Bias is socially constructed. Science as a contested discursive space.
  4. 4. NCRIS Research Infrastructure •Data Storage for Communities •Data (Cloud) Compute •Research Data Management “Massive datasets… [can] be assembled and explored [to] reveal… unsuspected relationships… This data-led science is a promising source of knowledge” Royal Society 2012 p.3
  5. 5. ANDS-Nectar-RDS In 2017/18, we are working on four transformations for Australian research:  A world leading data advantage  Accelerated innovation  Collaboration for borderless research  Enhanced translation of research
  6. 6. The Australian National Data Service (ANDS) makes Australia’s research data assets more valuable for researchers, research institutions and the nation. people skills services
  7. 7. The problem of bias Community views, Scientific views
  8. 8. Quotes on Bias – a lay perspective “It is useless to attempt to reason a man out of a thing he was never reasoned into” Jonathan Swift “It’s not at all hard to understand a person; it’s only hard to listen without bias” Criss Jami, Killosophy “We all see only what we are trained to see.” Robert Anton Wilson, Masks of the Illuminati “But I think that no matter how smart, people usually see what they’re already looking for, that’s all.” Veronica Roth, Allegiant (Source: GoodReads)
  9. 9. The answers you get depend on the questions you ask.p.139 “Normal science, the activity in which most scientists inevitably spend almost all their time, is predicated on the assumption that the scientific community know what the world is like.” p.5 There are significant limits to what proponents of different theories can communicate to one another; incommensurability (p.198) For example, flat vs round earth; helio or geo-centric. The Structure of Scientific Revolutions (1962/70) Citations: >80k Excerpt Thomas Kuhn - Paradigms
  10. 10. What’s the problem?  Why most published research findings are false? PLoS Medicine, Iaonnidis 2005, 8, 2, e124. (vs Goodman & Greenland)  Publishing exciting, counter-intuitive findings  How biased is science, really? Washington Post, 31.3.17  Replicate 100 psychology studies, 36 get same results  Let’s stop pretending like Science is perfect. Washington Post, 13.1.17  Science 2015, 349, 6251, aac4716 DOI: 10.1126/science.aac4716  Does Science have a replication crisis? (Maybe not ...) Washington Post 9.3.16
  11. 11. Aust Code for Responsible Research (1.6)  Intellectual honesty and integrity  Scholarly and scientific rigour  Instn climate for responsible and ethical behavior (1.1)  Institutions provide induction, formal training and continuing education for all research staff, inc. trainees (1.3)  Adopt methods appropriate for achieving the aims of each research proprosal  Conform to policies – institution -funder NHMRC 2007
  12. 12. Types of Bias My perspective, My preferences / biases Keen to speak to NBN customers on HFC, get in touch if you are not happy with your service #NBN (The Aust). 180,000 plus complaints about @TurnbullMalcolm second rate #nbn means $30 billion waste and no jobs for #gennbn #broadband #telecom @NBNCo Assoc. Prof Mark Gregory, RMIT
  13. 13. What is bias?  Iaonnidis (2005). PLoS Medicine “let us define bias as the combination of various design, data, analysis and presentation factors that tend to produce research findings when they should not have been produced.”  Fanelli et al. (2017) PNAS Bias patterns include: small study effects, gray literature bias, decline effects, early extreme, citation bias, industry bias  Wikipedia – list of cognitive biases; heuristics (too much; too fast), emotion, social influence (Nobel Prize Economics – Kahneman) Royal Society. Video
  14. 14. List of Cognitive Biases •Too much information •Need to act fast •Not enough meaning •What to keep/delete Wikipedia
  15. 15. Data approaches to engage with bias
  16. 16. Data in the Research Life Cycle Data Cite data Data Publish data Data Collect, store, analyse data Data Find data Data Plan for data Source: Bournemouth University RDA – the national data catalogue Re3data– the int’l data catalogue Unimelb checklist Datacite stats FAIR data checklist eg Figshare CRC Mental Health
  17. 17. Data driven solutions to bias  TOP Guidelines “Reproducibility of research can be improved by increasing transparency of the research process and products… eight standards: citation, data, analytic methods, research materials, design and analysis, preregistration (study, analysis plan), replication.” (Transparency and Openness)  Publishing and describing your datasets – subject to ethics, community consent, legal, privacy  “Open enquiry is at the heart of the scientific enterprise… permits others to identify errors” Royal Soc.2012
  18. 18. ANDS resources on managing medical data  Health and Medical webinar series  Guide to publishing and safely sharing sensitive data  Including informed consent  De-identifying medical data  Training – 10 Health and Medical Data Things “I know the data is anonymised, but I want you to remember That it’s my story – it’s about me, my life, my family Researchers should honour that by making information available about what the data is used for and what is found” A patient Patient Views on Data Sharing – Anne McKenzie WA CCHRN
  19. 19. Other solutions to bias  Peer Review  Working in a team; induction, training, review  Understanding your own assumptions – ontology and epistemology; incommensurability  “Open enquiry is at the heart of the scientific enterprise… permits others to identify errors” Royal Soc.2012  Documenting your process, assumptions, approaches
  20. 20. Notes on bias How UQ Business trains management PhDs – my experience
  21. 21. How the UQ Business School trains PhDs  Positivism vs Interpretivism; social construction of reality; ontology, epistemology, incommensurability  Paradigm wars; qual vs quant; hard vs soft science  Foucault 1969 – The Archaeology of Knowledge  Berger and Luckmann 1966 – Social Construction of Reality (citations: >39k)  Glaser and Strauss 1967 - The Discovery of Grounded Theory (citations: >78k) | Interpretive validity Sandberg (2005)  Kuhn 1962 – The Structure of Scientific Revolutions
  22. 22. Data approaches to the Rsch Lifecycle Activity – Review a study critically – find the biases - review list of cognitive biases; believeable? - review Washington Post articles; is there bias? - review Iaonnidis 2005; is he credible? - review ANDS Guides on Sensitive data sharing; Doable? 20mins, then 10min to discuss
  23. 23. With the exception of third party images or where otherwise indicated, this work is licensed under the Creative Commons 4.0 International Attribution Licence. ANDS, Nectar and RDS are supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program (NCRIS). Research Data Specialist, CRC Mental Health ANDS Liaison +61 3 9902 0569 @valuemgmt Richard Ferrers