Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

neuropredict: a proposal and a tool towards standardized and easy assessment of biomarkers

293 views

Published on

Proper application of machine learning to accurately evaluate the accuracy of biomarkers is challenging and error-prone for those without expertise in machine learning or programming. We offer an easy to use tool which implements the best practices and produces a comprehensive yet clinically-relevant report when comparing several biomarkers or different methods/studies. It is called neuropredict, which is open source and applicable to any domain whose biomarkers can be represented by numbers.

Published in: Healthcare
  • Be the first to comment

  • Be the first to like this

neuropredict: a proposal and a tool towards standardized and easy assessment of biomarkers

  1. 1. neuropredict: a proposal and a tool towards standardized and easy assessment of biomarkers github.com/raamana Pradeep Reddy Raamana, PhD crossinvalidation.com
  2. 2. What are biomarkers? 2 [1]. Strimbu, K., & Tavel, J. A. (2010). What are Biomarkers? Current Opinion in HIV and AIDS, 5(6), 463–466.
  3. 3. What are biomarkers? • “The term “biomarker”, a portmanteau of “biological marker”, refers to a broad subcategory of medical signs – that is, objective indications of medical state observed from outside the patient – which can be measured accurately and reproducibly. ”1 2 [1]. Strimbu, K., & Tavel, J. A. (2010). What are Biomarkers? Current Opinion in HIV and AIDS, 5(6), 463–466.
  4. 4. What are biomarkers? • “The term “biomarker”, a portmanteau of “biological marker”, refers to a broad subcategory of medical signs – that is, objective indications of medical state observed from outside the patient – which can be measured accurately and reproducibly. ”1 • simplified: “set of numbers predicting label(s)” 2 [1]. Strimbu, K., & Tavel, J. A. (2010). What are Biomarkers? Current Opinion in HIV and AIDS, 5(6), 463–466.
  5. 5. What are biomarkers? • “The term “biomarker”, a portmanteau of “biological marker”, refers to a broad subcategory of medical signs – that is, objective indications of medical state observed from outside the patient – which can be measured accurately and reproducibly. ”1 • simplified: “set of numbers predicting label(s)” • biomarkers are essential for computer-aided diagnosis: 1) detection of disease and staging their severity, and 2) monitoring response to treatment. 2 [1]. Strimbu, K., & Tavel, J. A. (2010). What are Biomarkers? Current Opinion in HIV and AIDS, 5(6), 463–466.
  6. 6. Measuring biomarkers accuracy is hard and error-prone! 3 • As proper application of ML requires • training in linear algebra and statistics • training in programming and engineering • It only gets harder in biomarker domain: • blind application is not enough • interpretability/limitations are important • Too many black-boxes and knobs -->
  7. 7. Measuring biomarkers accuracy is hard and error-prone! 3 • As proper application of ML requires • training in linear algebra and statistics • training in programming and engineering • It only gets harder in biomarker domain: • blind application is not enough • interpretability/limitations are important • Too many black-boxes and knobs -->
  8. 8. Typical ML/biomarker workflow 4 Raw data Preproce ssing Feature extraction Cross- validation (CV) Analysis of CV results Visualize and compare
  9. 9. Typical ML/biomarker workflow 4 Raw data Preproce ssing Feature extraction Cross- validation (CV) Analysis of CV results Visualize and compare Tools exist to do many of the small tasks individually, 
 but not as a whole!
  10. 10. Typical ML/biomarker workflow 4 Raw data Preproce ssing Feature extraction Cross- validation (CV) Analysis of CV results Visualize and compare Tools exist to do many of the small tasks individually, 
 but not as a whole! To those without machine learning or 
 programming experience, this is incredibly hard.
  11. 11. Typical ML/biomarker workflow 4 Raw data Preproce ssing Feature extraction Cross- validation (CV) Analysis of CV results Visualize and compare neuropredict covers 
 these parts Tools exist to do many of the small tasks individually, 
 but not as a whole! To those without machine learning or 
 programming experience, this is incredibly hard.
  12. 12. neuropredict : easy and comprehensive predictive analysis
  13. 13. Accuracy distributions neuropredict : easy and comprehensive predictive analysis
  14. 14. Confusion Matrices Accuracy distributions neuropredict : easy and comprehensive predictive analysis
  15. 15. Confusion Matrices Accuracy distributions Intuitive comparison of misclassification rates neuropredict : easy and comprehensive predictive analysis
  16. 16. Confusion Matrices Feature Importance Accuracy distributions Intuitive comparison of misclassification rates neuropredict : easy and comprehensive predictive analysis
  17. 17. Billions of dollars and decades of research, but not much insight into biomarkers! 6Woo, CW., et al.. (2017). Nature Neuroscience, 20(3), 365-377.
  18. 18. Standardized measurement and reports are necessary! • Research studies do not report all the information necessary • to assess biomarker performance well, and • to engage in statistical comparison with previous studies/biomarkers • Standardization of performance measurement and reports is needed! 7
  19. 19. neuropredict is an attempt to standardize and learn from each other! 8 This is NOT specific to neuroscience. Ideas and tools are generic!
  20. 20. I have a plan 9 Consensus on standards of analysis Consensus on significance tests! Standardize report format Open validation of neuro-predict Cloud repo and web portals Release, test, improve and iterate! but I need your support!
  21. 21. Come, join me! 
 let’s improve biomarker science. 
 one commit at a time! 10 github.com/raamana

×