SlideShare a Scribd company logo
The current state of
prediction in neuroimaging
Saige Rutherford
@being_saige
www.beingsaige.com
Road Map
• Quick review of typical ML workflow + my favorite predictive
model
• Which traits and behaviors can we predict from brain
images?
• What do various successful predictive models have in
common?
• What does a “successful” predictive model look like?
• How does big data fit in, is there hope for smaller datasets?
• Where is there room to improve brain-behavior predictive
models?
Dream Workflow
Reality Workflow
Favorite predictive model: Brain Basis Set
Basis Set = Chosen # of top components from PCA decomposition of subjects x features matrix
aka principle component regression
Phenotype BBS CPM
General Executive 0.44 0.42
Processing Speed 0.39 0.23
Penn Progressive
Matrices
0.30 0.32
ASR Externalizing 0.24 0.03
ASR Internalizing 0.20 0.04
ASR Attention 0.21 0.00
NEO-Openness 0.18 0.11
NEO-
Conscientiousness
0.19 0.15
NEO-Extroversion 0.13 0.04
NEO-Agreeableness 0.19 0.10
NEO-Neuroticism 0.00 0.05Number of Components Used to Predict
MeanCorrelationbetweenPredicted&
ObservedPhenotype
Sripada et al. Scientific Reports (2019)
100 held out unrelated subjects10-fold Cross Validation
r = 0.31
r = 0.06
r = 0.15
Successful Predictive Modeling
Test your prediction model in “the wild”
Sripada et. al Molecular Psychiatry (2019)
Ex. controlling for confounds (motion, demographics,
medication), different cross validation splits.
This shows more believable and realistic results!
Rozycki et. al Schizophrenia Bulletin (2017)
Successful Predictive Modeling
Impact of region-definition method on
prediction accuracy
Impact of connectivity
parameterization on prediction
accuracy
Impact of classifier choice on
prediction accuracy
https://www.sciencedirect.com/science/article/pii/S1053811919301594Dadi et al. Neuroimage (2019)
Successful Predictive Modeling
https://www.sciencedirect.com/science/article/pii/S1053811919301594Dadi et al. Neuroimage (2019)
What not to do
don’t be this guy
1. Be a research troll
Research Troll: Someone who is overly protective of their
data, unwilling to share data and well-documented code.
2. No out of sample test set or cross validation
What not to do
Make bold claims about one model/method being the best…
You know what they say when you assume…
You’re probably wrong, and someone will publicly prove this to
you in a Twitter thread
Big Datasets are taking over…
Where does my “small” data fit in?
Big Datasets are taking over…
Where does my “small” data fit in?
Big data can be act as a “discovery” data
set.
Use HCP, ABCD, or UKBiobank to find a
brain basis set then get expression
scores of these components in your
dataset.
Use pretrained models from big data,
treat your dataset as a true out of sample
test set.
Externalizing
Internalizing
Attention
Model
Externalizing*
Multi-Task Learning, Transfer Learning
Contributing to Big Data
Federated Learning: allows us to train models on distributed datasets that you cannot
directly access.
https://blog.openmined.org/federated-learning-differential-privacy-and-encrypted-computation-for-medical-imaging/
https://arxiv.org/pdf/1610.05492.pdf
https://ai.googleblog.com/2017/04/federated-learning-collaborative.html
Federated Learning tutorial using brain age prediction model coming soon
How can we improve prediction?
Put in the (hard) work to prepare your data properly…
Tangential point about preprocessing fMRI data
Haak, Marquand, Beckman, Neuroimage 2017
Lots of papers pointing to this same idea…
Don’t use a fixed atlas!
https://cdn.elifesciences.org/articles/44890/elife-44890-v2.pdf
https://cdn.elifesciences.org/articles/32992/elife-32992-v1.pdf
https://www.ncbi.nlm.nih.gov/pubmed/25598050
https://www.sciencedirect.com/science/article/pii/S1053811917305463
https://www.biorxiv.org/content/10.1101/431833v2https://www.ncbi.nlm.nih.gov/pubmed/29878084
How can we improve prediction?
Most of machine learning is about good data hygiene.
UNDERSTAND YOUR DATA!
https://twitter.com/justmarkham/status/1155840938356432896
pip install pandas_profiling
import pandas_profiling
df.profile_report()
Patient or
healthy
control?
Think deeply before you turn a continuous
trait into a categorical trait.
Dimensional neuroimaging: our ability to
place a brain scan into a succinct, yet highly
comprehensive and informative reference
system, dimensions of which will reflect
patterns associated with normal or pathologic
brain structure or function.
How can we improve prediction?
Bias in neuroimaging data…we need to do better at acknowledging it.
Big Data != Population Data
Does ML reveal the true nature of relationships, unconstrained by any bias or human influence?
The answer is an unequivocal No.
https://www.cs.mcgill.ca/~jpineau/ReproducibilityChecklist.pdf
Take Home Messages
There is not one perfect prediction framework to rule them all.
Machine Learning No Free Lunch theorem: no machine learning
method is better than the others, on average, over a broad family of
problems.
Embrace and collaborate with big data.
Big data: multi-task learning, share your models
Small data: transfer learning, use pre-trained models
Focus on transparency and reproducibility
Learning Resources
This is a research process, not a final offering.
OHBM 2019 talks on ML:
https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138295
https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138032
https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138231
https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138291
https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138219
Gael Varoquaux talks:
https://www.slideshare.net/GaelVaroquaux/functionalconnectome-biomarkers-to-meet-clinical-needs
https://www.slideshare.net/GaelVaroquaux/machine-learning-on-non-curated-data-154905090
Machine learning in neuroimaging: Progress and challenges. Neuroimage. 2019 August 15.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6499712/pdf/nihms-1025732.pdf
Learn a new Pandas trick everyday: https://www.dataschool.io/python-pandas-tips-and-tricks/
Thank you!
All who have supported/inspired me on my learning journey.
Mike Angstadt, Chandra Sripada, Jenna Wiens, Daniel Kessler, Aman
Taxali, Bennet Fauber, Marlena Duda, GirlsWhoCode Organization, Ivy Tso,
Soo-Eun Chang, Steve Taylor, the entire University of Michigan community!
@being_saige
www.beingsaige.com

More Related Content

What's hot

Barga Data Science lecture 5
Barga Data Science lecture 5Barga Data Science lecture 5
Barga Data Science lecture 5
Roger Barga
 
Virtual Worlds And Real World
Virtual Worlds And Real WorldVirtual Worlds And Real World
Virtual Worlds And Real World
KanavKahol
 
Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...
Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...
Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...
Edureka!
 
Azure machine learning 101 - Part 1
Azure machine learning 101 - Part 1Azure machine learning 101 - Part 1
Azure machine learning 101 - Part 1
Setu Chokshi
 
introduction to machine learning
introduction to machine learningintroduction to machine learning
introduction to machine learning
Johnson Ubah
 
Azure machine learning 101 Parts 1 & 2 - Classification Algorithms
Azure machine learning 101  Parts 1 & 2  -  Classification Algorithms Azure machine learning 101  Parts 1 & 2  -  Classification Algorithms
Azure machine learning 101 Parts 1 & 2 - Classification Algorithms
Setu Chokshi
 
Azure Boot Camp 2017 getting started with azure machine learning
Azure Boot Camp 2017 getting started with azure machine learningAzure Boot Camp 2017 getting started with azure machine learning
Azure Boot Camp 2017 getting started with azure machine learning
Setu Chokshi
 
Barga Data Science lecture 7
Barga Data Science lecture 7Barga Data Science lecture 7
Barga Data Science lecture 7
Roger Barga
 
Machine Learning for Developers - Pop-up Loft Tel Aviv
Machine Learning for Developers - Pop-up Loft Tel AvivMachine Learning for Developers - Pop-up Loft Tel Aviv
Machine Learning for Developers - Pop-up Loft Tel Aviv
Amazon Web Services
 
Prediction Analysis in Clinical and Basic Neuroscience
Prediction Analysis in Clinical and Basic NeurosciencePrediction Analysis in Clinical and Basic Neuroscience
Prediction Analysis in Clinical and Basic Neuroscience
Cameron Craddock
 
Using Bioinformatics Data to inform Therapeutics discovery and development
Using Bioinformatics Data to inform Therapeutics discovery and developmentUsing Bioinformatics Data to inform Therapeutics discovery and development
Using Bioinformatics Data to inform Therapeutics discovery and development
Eleanor Howe
 
Managing Data Science by David Martínez Rego
Managing Data Science by David Martínez RegoManaging Data Science by David Martínez Rego
Managing Data Science by David Martínez Rego
Big Data Spain
 
BrainLift Presentation
BrainLift PresentationBrainLift Presentation
BrainLift PresentationaiWorker.com
 
PyGotham 2016
PyGotham 2016PyGotham 2016
PyGotham 2016
Manojit Nandi
 
Credit Card Fraud Detection - Anomaly Detection
Credit Card Fraud Detection - Anomaly DetectionCredit Card Fraud Detection - Anomaly Detection
Credit Card Fraud Detection - Anomaly Detection
Lalit Jain
 
Bayesian reasoning
Bayesian reasoningBayesian reasoning
Bayesian reasoning
Marta Fajlhauer
 
Demystifying Recommendation Systems
Demystifying Recommendation SystemsDemystifying Recommendation Systems
Demystifying Recommendation Systems
Rumman Chowdhury
 
Introduction to machine learning
Introduction to machine learningIntroduction to machine learning
Introduction to machine learning
Adetimehin Oluwasegun Matthew
 
Data Mining in Market Research
Data Mining in Market ResearchData Mining in Market Research
Data Mining in Market Researchbutest
 
Data Mining In Market Research
Data Mining In Market ResearchData Mining In Market Research
Data Mining In Market Research
jim
 

What's hot (20)

Barga Data Science lecture 5
Barga Data Science lecture 5Barga Data Science lecture 5
Barga Data Science lecture 5
 
Virtual Worlds And Real World
Virtual Worlds And Real WorldVirtual Worlds And Real World
Virtual Worlds And Real World
 
Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...
Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...
Sentiment Analysis | Machine Learning Algorithms | Data Science Tutorial | Ed...
 
Azure machine learning 101 - Part 1
Azure machine learning 101 - Part 1Azure machine learning 101 - Part 1
Azure machine learning 101 - Part 1
 
introduction to machine learning
introduction to machine learningintroduction to machine learning
introduction to machine learning
 
Azure machine learning 101 Parts 1 & 2 - Classification Algorithms
Azure machine learning 101  Parts 1 & 2  -  Classification Algorithms Azure machine learning 101  Parts 1 & 2  -  Classification Algorithms
Azure machine learning 101 Parts 1 & 2 - Classification Algorithms
 
Azure Boot Camp 2017 getting started with azure machine learning
Azure Boot Camp 2017 getting started with azure machine learningAzure Boot Camp 2017 getting started with azure machine learning
Azure Boot Camp 2017 getting started with azure machine learning
 
Barga Data Science lecture 7
Barga Data Science lecture 7Barga Data Science lecture 7
Barga Data Science lecture 7
 
Machine Learning for Developers - Pop-up Loft Tel Aviv
Machine Learning for Developers - Pop-up Loft Tel AvivMachine Learning for Developers - Pop-up Loft Tel Aviv
Machine Learning for Developers - Pop-up Loft Tel Aviv
 
Prediction Analysis in Clinical and Basic Neuroscience
Prediction Analysis in Clinical and Basic NeurosciencePrediction Analysis in Clinical and Basic Neuroscience
Prediction Analysis in Clinical and Basic Neuroscience
 
Using Bioinformatics Data to inform Therapeutics discovery and development
Using Bioinformatics Data to inform Therapeutics discovery and developmentUsing Bioinformatics Data to inform Therapeutics discovery and development
Using Bioinformatics Data to inform Therapeutics discovery and development
 
Managing Data Science by David Martínez Rego
Managing Data Science by David Martínez RegoManaging Data Science by David Martínez Rego
Managing Data Science by David Martínez Rego
 
BrainLift Presentation
BrainLift PresentationBrainLift Presentation
BrainLift Presentation
 
PyGotham 2016
PyGotham 2016PyGotham 2016
PyGotham 2016
 
Credit Card Fraud Detection - Anomaly Detection
Credit Card Fraud Detection - Anomaly DetectionCredit Card Fraud Detection - Anomaly Detection
Credit Card Fraud Detection - Anomaly Detection
 
Bayesian reasoning
Bayesian reasoningBayesian reasoning
Bayesian reasoning
 
Demystifying Recommendation Systems
Demystifying Recommendation SystemsDemystifying Recommendation Systems
Demystifying Recommendation Systems
 
Introduction to machine learning
Introduction to machine learningIntroduction to machine learning
Introduction to machine learning
 
Data Mining in Market Research
Data Mining in Market ResearchData Mining in Market Research
Data Mining in Market Research
 
Data Mining In Market Research
Data Mining In Market ResearchData Mining In Market Research
Data Mining In Market Research
 

Similar to The current state of prediction in neuroimaging

ML crash course
ML crash courseML crash course
ML crash course
mikaelhuss
 
Barga Data Science lecture 2
Barga Data Science lecture 2Barga Data Science lecture 2
Barga Data Science lecture 2
Roger Barga
 
Module 5: Decision Trees
Module 5: Decision TreesModule 5: Decision Trees
Module 5: Decision Trees
Sara Hooker
 
Human in the loop: Bayesian Rules Enabling Explainable AI
Human in the loop: Bayesian Rules Enabling Explainable AIHuman in the loop: Bayesian Rules Enabling Explainable AI
Human in the loop: Bayesian Rules Enabling Explainable AI
Pramit Choudhary
 
Rutherford_MiCHAMP2020.pptx
Rutherford_MiCHAMP2020.pptxRutherford_MiCHAMP2020.pptx
Rutherford_MiCHAMP2020.pptx
SaigeRutherford
 
LearningAG.ppt
LearningAG.pptLearningAG.ppt
LearningAG.pptbutest
 
Twitter sentiment classifications 1
Twitter sentiment classifications 1Twitter sentiment classifications 1
Twitter sentiment classifications 1
eshtiyak
 
Machine Learning.pptx
Machine Learning.pptxMachine Learning.pptx
Machine Learning.pptx
NitinSharma134320
 
Ms 07 information systems for managers
Ms   07 information systems for managersMs   07 information systems for managers
Ms 07 information systems for managers
smumbahelp
 
Screening of Mental Health in Adolescence.pptx
Screening of Mental Health in Adolescence.pptxScreening of Mental Health in Adolescence.pptx
Screening of Mental Health in Adolescence.pptx
NitishChoudhary23
 
How to Start Doing Data Science
How to Start Doing Data ScienceHow to Start Doing Data Science
How to Start Doing Data Science
Ayodele Odubela
 
Melissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AIMelissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AI
melissadata
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
Knowledge And Skill Forum
 
Machine Learning
Machine LearningMachine Learning
Machine Learning
Joshua Robinson
 
BIG DATA AND MACHINE LEARNING
BIG DATA AND MACHINE LEARNINGBIG DATA AND MACHINE LEARNING
BIG DATA AND MACHINE LEARNING
Umair Shafique
 
Unit 2.pptx
Unit 2.pptxUnit 2.pptx
Big Data & ML for Clinical Data
Big Data & ML for Clinical DataBig Data & ML for Clinical Data
Big Data & ML for Clinical Data
Paul Agapow
 
Data science presentation
Data science presentationData science presentation
Data science presentation
MSDEVMTL
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"
Aalto University
 
Hacking Predictive Modeling - RoadSec 2018
Hacking Predictive Modeling - RoadSec 2018Hacking Predictive Modeling - RoadSec 2018
Hacking Predictive Modeling - RoadSec 2018
HJ van Veen
 

Similar to The current state of prediction in neuroimaging (20)

ML crash course
ML crash courseML crash course
ML crash course
 
Barga Data Science lecture 2
Barga Data Science lecture 2Barga Data Science lecture 2
Barga Data Science lecture 2
 
Module 5: Decision Trees
Module 5: Decision TreesModule 5: Decision Trees
Module 5: Decision Trees
 
Human in the loop: Bayesian Rules Enabling Explainable AI
Human in the loop: Bayesian Rules Enabling Explainable AIHuman in the loop: Bayesian Rules Enabling Explainable AI
Human in the loop: Bayesian Rules Enabling Explainable AI
 
Rutherford_MiCHAMP2020.pptx
Rutherford_MiCHAMP2020.pptxRutherford_MiCHAMP2020.pptx
Rutherford_MiCHAMP2020.pptx
 
LearningAG.ppt
LearningAG.pptLearningAG.ppt
LearningAG.ppt
 
Twitter sentiment classifications 1
Twitter sentiment classifications 1Twitter sentiment classifications 1
Twitter sentiment classifications 1
 
Machine Learning.pptx
Machine Learning.pptxMachine Learning.pptx
Machine Learning.pptx
 
Ms 07 information systems for managers
Ms   07 information systems for managersMs   07 information systems for managers
Ms 07 information systems for managers
 
Screening of Mental Health in Adolescence.pptx
Screening of Mental Health in Adolescence.pptxScreening of Mental Health in Adolescence.pptx
Screening of Mental Health in Adolescence.pptx
 
How to Start Doing Data Science
How to Start Doing Data ScienceHow to Start Doing Data Science
How to Start Doing Data Science
 
Melissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AIMelissa Informatics - Data Quality and AI
Melissa Informatics - Data Quality and AI
 
Introduction to Machine Learning
Introduction to Machine LearningIntroduction to Machine Learning
Introduction to Machine Learning
 
Machine Learning
Machine LearningMachine Learning
Machine Learning
 
BIG DATA AND MACHINE LEARNING
BIG DATA AND MACHINE LEARNINGBIG DATA AND MACHINE LEARNING
BIG DATA AND MACHINE LEARNING
 
Unit 2.pptx
Unit 2.pptxUnit 2.pptx
Unit 2.pptx
 
Big Data & ML for Clinical Data
Big Data & ML for Clinical DataBig Data & ML for Clinical Data
Big Data & ML for Clinical Data
 
Data science presentation
Data science presentationData science presentation
Data science presentation
 
Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"Inverse Modeling for Cognitive Science "in the Wild"
Inverse Modeling for Cognitive Science "in the Wild"
 
Hacking Predictive Modeling - RoadSec 2018
Hacking Predictive Modeling - RoadSec 2018Hacking Predictive Modeling - RoadSec 2018
Hacking Predictive Modeling - RoadSec 2018
 

More from SaigeRutherford

Whistler2023_Saige.pptx
Whistler2023_Saige.pptxWhistler2023_Saige.pptx
Whistler2023_Saige.pptx
SaigeRutherford
 
Oxford_15-03-22.pptx
Oxford_15-03-22.pptxOxford_15-03-22.pptx
Oxford_15-03-22.pptx
SaigeRutherford
 
Basic Units of Inter-Individual Variation in fMRI Data
Basic Units of Inter-Individual Variation in fMRI DataBasic Units of Inter-Individual Variation in fMRI Data
Basic Units of Inter-Individual Variation in fMRI Data
SaigeRutherford
 
Isdp2019 fetal slides
Isdp2019 fetal slidesIsdp2019 fetal slides
Isdp2019 fetal slides
SaigeRutherford
 
Revamped CNNs for brains
Revamped CNNs for brainsRevamped CNNs for brains
Revamped CNNs for brains
SaigeRutherford
 
Ismrm 2018 e-poster
Ismrm 2018 e-posterIsmrm 2018 e-poster
Ismrm 2018 e-poster
SaigeRutherford
 
Developmental Mega Sample: Exploring Inter-Individual Variation
Developmental Mega Sample: Exploring Inter-Individual VariationDevelopmental Mega Sample: Exploring Inter-Individual Variation
Developmental Mega Sample: Exploring Inter-Individual Variation
SaigeRutherford
 

More from SaigeRutherford (7)

Whistler2023_Saige.pptx
Whistler2023_Saige.pptxWhistler2023_Saige.pptx
Whistler2023_Saige.pptx
 
Oxford_15-03-22.pptx
Oxford_15-03-22.pptxOxford_15-03-22.pptx
Oxford_15-03-22.pptx
 
Basic Units of Inter-Individual Variation in fMRI Data
Basic Units of Inter-Individual Variation in fMRI DataBasic Units of Inter-Individual Variation in fMRI Data
Basic Units of Inter-Individual Variation in fMRI Data
 
Isdp2019 fetal slides
Isdp2019 fetal slidesIsdp2019 fetal slides
Isdp2019 fetal slides
 
Revamped CNNs for brains
Revamped CNNs for brainsRevamped CNNs for brains
Revamped CNNs for brains
 
Ismrm 2018 e-poster
Ismrm 2018 e-posterIsmrm 2018 e-poster
Ismrm 2018 e-poster
 
Developmental Mega Sample: Exploring Inter-Individual Variation
Developmental Mega Sample: Exploring Inter-Individual VariationDevelopmental Mega Sample: Exploring Inter-Individual Variation
Developmental Mega Sample: Exploring Inter-Individual Variation
 

Recently uploaded

SCHIZOPHRENIA Disorder/ Brain Disorder.pdf
SCHIZOPHRENIA Disorder/ Brain Disorder.pdfSCHIZOPHRENIA Disorder/ Brain Disorder.pdf
SCHIZOPHRENIA Disorder/ Brain Disorder.pdf
SELF-EXPLANATORY
 
plant biotechnology Lecture note ppt.pptx
plant biotechnology Lecture note ppt.pptxplant biotechnology Lecture note ppt.pptx
plant biotechnology Lecture note ppt.pptx
yusufzako14
 
EY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptxEY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptx
AlguinaldoKong
 
general properties of oerganologametal.ppt
general properties of oerganologametal.pptgeneral properties of oerganologametal.ppt
general properties of oerganologametal.ppt
IqrimaNabilatulhusni
 
Large scale production of streptomycin.pptx
Large scale production of streptomycin.pptxLarge scale production of streptomycin.pptx
Large scale production of streptomycin.pptx
Cherry
 
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
Sérgio Sacani
 
filosofia boliviana introducción jsjdjd.pptx
filosofia boliviana introducción jsjdjd.pptxfilosofia boliviana introducción jsjdjd.pptx
filosofia boliviana introducción jsjdjd.pptx
IvanMallco1
 
Hemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptxHemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptx
muralinath2
 
platelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptxplatelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptx
muralinath2
 
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...
Scintica Instrumentation
 
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
ChetanK57
 
insect taxonomy importance systematics and classification
insect taxonomy importance systematics and classificationinsect taxonomy importance systematics and classification
insect taxonomy importance systematics and classification
anitaento25
 
Richard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlandsRichard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlands
Richard Gill
 
Seminar of U.V. Spectroscopy by SAMIR PANDA
 Seminar of U.V. Spectroscopy by SAMIR PANDA Seminar of U.V. Spectroscopy by SAMIR PANDA
Seminar of U.V. Spectroscopy by SAMIR PANDA
SAMIR PANDA
 
Orion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWSOrion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWS
Columbia Weather Systems
 
GBSN - Biochemistry (Unit 5) Chemistry of Lipids
GBSN - Biochemistry (Unit 5) Chemistry of LipidsGBSN - Biochemistry (Unit 5) Chemistry of Lipids
GBSN - Biochemistry (Unit 5) Chemistry of Lipids
Areesha Ahmad
 
Lateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensiveLateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensive
silvermistyshot
 
justice-and-fairness-ethics with example
justice-and-fairness-ethics with examplejustice-and-fairness-ethics with example
justice-and-fairness-ethics with example
azzyixes
 
Lab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerinLab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerin
ossaicprecious19
 
Mammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also FunctionsMammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also Functions
YOGESH DOGRA
 

Recently uploaded (20)

SCHIZOPHRENIA Disorder/ Brain Disorder.pdf
SCHIZOPHRENIA Disorder/ Brain Disorder.pdfSCHIZOPHRENIA Disorder/ Brain Disorder.pdf
SCHIZOPHRENIA Disorder/ Brain Disorder.pdf
 
plant biotechnology Lecture note ppt.pptx
plant biotechnology Lecture note ppt.pptxplant biotechnology Lecture note ppt.pptx
plant biotechnology Lecture note ppt.pptx
 
EY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptxEY - Supply Chain Services 2018_template.pptx
EY - Supply Chain Services 2018_template.pptx
 
general properties of oerganologametal.ppt
general properties of oerganologametal.pptgeneral properties of oerganologametal.ppt
general properties of oerganologametal.ppt
 
Large scale production of streptomycin.pptx
Large scale production of streptomycin.pptxLarge scale production of streptomycin.pptx
Large scale production of streptomycin.pptx
 
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
THE IMPORTANCE OF MARTIAN ATMOSPHERE SAMPLE RETURN.
 
filosofia boliviana introducción jsjdjd.pptx
filosofia boliviana introducción jsjdjd.pptxfilosofia boliviana introducción jsjdjd.pptx
filosofia boliviana introducción jsjdjd.pptx
 
Hemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptxHemoglobin metabolism_pathophysiology.pptx
Hemoglobin metabolism_pathophysiology.pptx
 
platelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptxplatelets_clotting_biogenesis.clot retractionpptx
platelets_clotting_biogenesis.clot retractionpptx
 
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...
(May 29th, 2024) Advancements in Intravital Microscopy- Insights for Preclini...
 
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATIONPRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
PRESENTATION ABOUT PRINCIPLE OF COSMATIC EVALUATION
 
insect taxonomy importance systematics and classification
insect taxonomy importance systematics and classificationinsect taxonomy importance systematics and classification
insect taxonomy importance systematics and classification
 
Richard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlandsRichard's aventures in two entangled wonderlands
Richard's aventures in two entangled wonderlands
 
Seminar of U.V. Spectroscopy by SAMIR PANDA
 Seminar of U.V. Spectroscopy by SAMIR PANDA Seminar of U.V. Spectroscopy by SAMIR PANDA
Seminar of U.V. Spectroscopy by SAMIR PANDA
 
Orion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWSOrion Air Quality Monitoring Systems - CWS
Orion Air Quality Monitoring Systems - CWS
 
GBSN - Biochemistry (Unit 5) Chemistry of Lipids
GBSN - Biochemistry (Unit 5) Chemistry of LipidsGBSN - Biochemistry (Unit 5) Chemistry of Lipids
GBSN - Biochemistry (Unit 5) Chemistry of Lipids
 
Lateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensiveLateral Ventricles.pdf very easy good diagrams comprehensive
Lateral Ventricles.pdf very easy good diagrams comprehensive
 
justice-and-fairness-ethics with example
justice-and-fairness-ethics with examplejustice-and-fairness-ethics with example
justice-and-fairness-ethics with example
 
Lab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerinLab report on liquid viscosity of glycerin
Lab report on liquid viscosity of glycerin
 
Mammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also FunctionsMammalian Pineal Body Structure and Also Functions
Mammalian Pineal Body Structure and Also Functions
 

The current state of prediction in neuroimaging

  • 1. The current state of prediction in neuroimaging Saige Rutherford @being_saige www.beingsaige.com
  • 2. Road Map • Quick review of typical ML workflow + my favorite predictive model • Which traits and behaviors can we predict from brain images? • What do various successful predictive models have in common? • What does a “successful” predictive model look like? • How does big data fit in, is there hope for smaller datasets? • Where is there room to improve brain-behavior predictive models?
  • 5.
  • 6. Favorite predictive model: Brain Basis Set Basis Set = Chosen # of top components from PCA decomposition of subjects x features matrix aka principle component regression
  • 7. Phenotype BBS CPM General Executive 0.44 0.42 Processing Speed 0.39 0.23 Penn Progressive Matrices 0.30 0.32 ASR Externalizing 0.24 0.03 ASR Internalizing 0.20 0.04 ASR Attention 0.21 0.00 NEO-Openness 0.18 0.11 NEO- Conscientiousness 0.19 0.15 NEO-Extroversion 0.13 0.04 NEO-Agreeableness 0.19 0.10 NEO-Neuroticism 0.00 0.05Number of Components Used to Predict MeanCorrelationbetweenPredicted& ObservedPhenotype Sripada et al. Scientific Reports (2019) 100 held out unrelated subjects10-fold Cross Validation
  • 8. r = 0.31 r = 0.06 r = 0.15
  • 9. Successful Predictive Modeling Test your prediction model in “the wild” Sripada et. al Molecular Psychiatry (2019) Ex. controlling for confounds (motion, demographics, medication), different cross validation splits. This shows more believable and realistic results! Rozycki et. al Schizophrenia Bulletin (2017)
  • 10. Successful Predictive Modeling Impact of region-definition method on prediction accuracy Impact of connectivity parameterization on prediction accuracy Impact of classifier choice on prediction accuracy https://www.sciencedirect.com/science/article/pii/S1053811919301594Dadi et al. Neuroimage (2019)
  • 12. What not to do don’t be this guy 1. Be a research troll Research Troll: Someone who is overly protective of their data, unwilling to share data and well-documented code. 2. No out of sample test set or cross validation
  • 13. What not to do Make bold claims about one model/method being the best… You know what they say when you assume… You’re probably wrong, and someone will publicly prove this to you in a Twitter thread
  • 14. Big Datasets are taking over… Where does my “small” data fit in?
  • 15. Big Datasets are taking over… Where does my “small” data fit in? Big data can be act as a “discovery” data set. Use HCP, ABCD, or UKBiobank to find a brain basis set then get expression scores of these components in your dataset. Use pretrained models from big data, treat your dataset as a true out of sample test set. Externalizing Internalizing Attention Model Externalizing* Multi-Task Learning, Transfer Learning
  • 16. Contributing to Big Data Federated Learning: allows us to train models on distributed datasets that you cannot directly access. https://blog.openmined.org/federated-learning-differential-privacy-and-encrypted-computation-for-medical-imaging/ https://arxiv.org/pdf/1610.05492.pdf https://ai.googleblog.com/2017/04/federated-learning-collaborative.html Federated Learning tutorial using brain age prediction model coming soon
  • 17. How can we improve prediction? Put in the (hard) work to prepare your data properly… Tangential point about preprocessing fMRI data Haak, Marquand, Beckman, Neuroimage 2017
  • 18. Lots of papers pointing to this same idea… Don’t use a fixed atlas! https://cdn.elifesciences.org/articles/44890/elife-44890-v2.pdf https://cdn.elifesciences.org/articles/32992/elife-32992-v1.pdf https://www.ncbi.nlm.nih.gov/pubmed/25598050 https://www.sciencedirect.com/science/article/pii/S1053811917305463 https://www.biorxiv.org/content/10.1101/431833v2https://www.ncbi.nlm.nih.gov/pubmed/29878084
  • 19. How can we improve prediction? Most of machine learning is about good data hygiene. UNDERSTAND YOUR DATA! https://twitter.com/justmarkham/status/1155840938356432896 pip install pandas_profiling import pandas_profiling df.profile_report()
  • 20. Patient or healthy control? Think deeply before you turn a continuous trait into a categorical trait. Dimensional neuroimaging: our ability to place a brain scan into a succinct, yet highly comprehensive and informative reference system, dimensions of which will reflect patterns associated with normal or pathologic brain structure or function.
  • 21. How can we improve prediction? Bias in neuroimaging data…we need to do better at acknowledging it. Big Data != Population Data Does ML reveal the true nature of relationships, unconstrained by any bias or human influence? The answer is an unequivocal No.
  • 23. Take Home Messages There is not one perfect prediction framework to rule them all. Machine Learning No Free Lunch theorem: no machine learning method is better than the others, on average, over a broad family of problems. Embrace and collaborate with big data. Big data: multi-task learning, share your models Small data: transfer learning, use pre-trained models Focus on transparency and reproducibility
  • 24. Learning Resources This is a research process, not a final offering. OHBM 2019 talks on ML: https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138295 https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138032 https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138231 https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138291 https://www.pathlms.com/ohbm/courses/12238/sections/15843/video_presentations/138219 Gael Varoquaux talks: https://www.slideshare.net/GaelVaroquaux/functionalconnectome-biomarkers-to-meet-clinical-needs https://www.slideshare.net/GaelVaroquaux/machine-learning-on-non-curated-data-154905090 Machine learning in neuroimaging: Progress and challenges. Neuroimage. 2019 August 15. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6499712/pdf/nihms-1025732.pdf Learn a new Pandas trick everyday: https://www.dataschool.io/python-pandas-tips-and-tricks/
  • 25.
  • 26. Thank you! All who have supported/inspired me on my learning journey. Mike Angstadt, Chandra Sripada, Jenna Wiens, Daniel Kessler, Aman Taxali, Bennet Fauber, Marlena Duda, GirlsWhoCode Organization, Ivy Tso, Soo-Eun Chang, Steve Taylor, the entire University of Michigan community! @being_saige www.beingsaige.com

Editor's Notes

  1. How should nodes be chosen? How many nodes are needed for brain-imaging based diagnosis? How should weights of brain functional connectomes be represented? What classifiers should be used? Should linear or non-linear models be preferred? Spare or non-sparse models be used? With or without feature selection?
  2. . We study the prediction score of each pipeline relative to the mean across pipelines on each fold. This relative measure discards the variance in scores due to folds or datasets. •Regions defined functionally (with dictionary learning or ICA) give best prediction. •Prefer tangent-space parametrization of connectomes to full or partial correlation. •Non-sparse linear classifiers are best for supervised learning.
  3. Machine learning 101: a model that fits the data well doesn't necessarily generalize well. In the era of big data, generalization should be tested in separate samples, or else using split-sample approaches in which one split is kept completely hidden until the very final application of a model.
  4. Transfer learning example: knowledge gained while learning to recognize cars could apply when trying to recognize trucks.  Big Data people: think about multi-task learning, which creates more generalizable models. Also make sure you always share your saved models for people who might now have access to the big data. Small data people: think transfer learning. If you can get access to the saved models that big data people share you can use them to test on your data, even if the model wasn’t explicitly train to predict the exact phenotype you are using.
  5. Collaborative machine learning without centralized datasets.
  6. One theoretical region where there is one mode of organization (it could encode task activation, connectivity, stimulus response) running in one particular direction and in the same area there is a second mode of organization running in a perpendicular direction. Taking measurements directly would mean that we are taking the superposition of this organization and we would wrongly infer that things are organized along this diagonal. When you parcellate this data you get a completely wrong ROI atlas definition which does not at all respect the underlying data. We know this is true in motor cortex and primary visual cortex. Lots of works suggests this presence in other regions of the brain.
  7. Majority of machine learning in clinical neuroscience focused on classifying patients from healthy controls. Although this is a good starting point, its practical value is very limited, since those patients are presumably already “correctly” classified via simpler clinical examinations, hence they are used as ground truth.
  8. In the computer-science based machine learning community, the discussion of bias in predictive models is widely acknowledged and discussed. At MLHC this past summer, the ending panel spent 2 hours discussing biases and ways of overcoming them.