5 5 10

480 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
480
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

5 5 10

  1. 1. Machine-learning techniques for building a diagnostic model for very mild dementia Rong Chen & Edward H. Herskovits University of Pennsylvania NeuroImage, 2010 Journal Club - Omid Cinco de Mayo, 2010
  2. 2. Overview <ul><li>Clinical Question </li></ul><ul><li>Machine Learning Approach </li></ul><ul><li>Methods and Algorithms </li></ul><ul><li>Performance Results </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Clinical Question <ul><li>Distinguish Very Mild Dementia (VMD) from cognitively normal individuals with a diagnostic model using MRI </li></ul><ul><li>VMD is defined by the Clinical Dementia Rating (CDR) scale, which takes memory, orientation, judgment & problem solving, community affairs, home & hobbies, and personal care into account * </li></ul><ul><ul><li>0 = Normal </li></ul></ul><ul><ul><li>0.5 = Very Mild Dementia </li></ul></ul><ul><ul><li>1 = Mild Dementia </li></ul></ul><ul><ul><li>2 = Moderate Dementia </li></ul></ul><ul><ul><li>3 = Severe Dementia </li></ul></ul>* Alzheimer’s Disease Research Center
  4. 4. Approach: Machine Learning Classification <ul><li>Subject 1,Volume 1 Subject 1,Volume 2 … Subject 1,Volume p </li></ul><ul><li>Subject 2,Volume 1 Subject 2,Volume 2 … Subject 2,Volume p </li></ul><ul><li>. </li></ul><ul><li>. </li></ul><ul><li>. </li></ul><ul><li>Subject N,Volume 1 Subject N,Volume 2 … Subject N,Volume p </li></ul>Training Data / Patterns Y 1 Y 2 . . . Y N Known Labels Testing Data / Patterns Predicted Labels
  5. 5. Approach: Classification <ul><li>Previous studies implemented linear discriminant analysis and logistic regression to approach the problem of classifying VMD vs. control </li></ul>LDA LR
  6. 6. Approach: Classification <ul><li>Previous studies implemented discriminant analysis and logistic regression to approach the problem of classifying VMD vs. control </li></ul><ul><li>This paper builds on such prior work by implementing the above 2 ‘statistical’ approaches in addition to 5 machine learning algorithms, namely naïve Bayes, Bayesian network classifier with inverse tree structure, decision trees, support vector machines and multiple-layer perceptrons </li></ul>
  7. 7. Methods
  8. 8. Image Processing <ul><li>Skull stripping </li></ul><ul><li>Automated segmentation </li></ul><ul><ul><li>Gray matter (GM) </li></ul></ul><ul><ul><li>White matter (WM) </li></ul></ul><ul><ul><li>Cerebrospinal fluid (CSF) </li></ul></ul><ul><li>Spatial normalization </li></ul><ul><li>Registration </li></ul><ul><li>RAVENS analysis </li></ul><ul><ul><li>density map with voxel-wise volume measures </li></ul></ul>RAVENS = Regional Analysis of Volumes Embedded in Stereotaxic Space
  9. 9. Naïve Bayes (NB) <ul><li>Discretize the continuous voxel-wise volumes </li></ul><ul><li>Goal is to find P( C | R ), with the major underlying assumption that given C, the features are independent : </li></ul><ul><li>Bayes’s theorem would then yield: </li></ul>
  10. 10. Decision Trees <ul><li>Roots (structures), branches ( rules ) and leaves (assigned labels) </li></ul><ul><li>Rules are assigned to node/structures to yield a label for class membership (A or B) </li></ul><ul><li>Algorithm can stop at a node with no edges or continue with more rules </li></ul><ul><li>Assumption in that g( R ) is a piecewise function </li></ul><ul><li>Can “prune” tree to avoid overfitting </li></ul>Node/Structure Node/Structure rule rule A B A
  11. 11. Support Vector Machines (SVM) <ul><li>Optimization problem, where the margin of a separating hyperplane (or hypersurface) is maximized </li></ul><ul><li>Tuning of kernel parameters (e.g. radial basis function bandwidth) and cost or C (trade-off between margin width and misclassification) </li></ul>
  12. 12. Multiple Layer Perceptrons (MLP) <ul><li>Type of Artificial Neural Network (ANN) </li></ul><ul><li>Multiple layers refer to the input, output plus the additional hidden layers that are assumed in the model </li></ul><ul><li>Weights are assigned to different layers </li></ul><ul><li>Nonlinear (e.g. logistic) “activation” functions are used to go from input to output </li></ul>R C
  13. 13. Performance Comparison <ul><li>Comparison through misclassification errors (or accuracy = 1-error) </li></ul><ul><ul><li>Discrimination error (  d ) </li></ul></ul><ul><ul><ul><li>Same data for training and testing </li></ul></ul></ul><ul><ul><ul><li>Poor generalizability (ie overfits ) </li></ul></ul></ul><ul><ul><li>Cross-validation error (  cv ) </li></ul></ul><ul><ul><ul><li>k-fold or leave-one-out (where k=N) </li></ul></ul></ul><ul><ul><ul><li>Better generalizability </li></ul></ul></ul><ul><ul><li>External validation error (  ev ) </li></ul></ul><ul><ul><ul><li>Independent test set </li></ul></ul></ul><ul><ul><ul><li>Most stringent </li></ul></ul></ul>
  14. 14. Performance Comparison <ul><li>Comparison through the triangular discrimination metric </li></ul><ul><ul><li>More informative than accuracy </li></ul></ul><ul><ul><li>Provides information on sensitivity and specificity </li></ul></ul><ul><ul><li>After dividing confusion matrix by n, compute similarity by: </li></ul></ul>
  15. 15. Results
  16. 16. <ul><li>Demographics: </li></ul><ul><li>Training set: 33 VMD + 50 control patients with no significant age/sex differences </li></ul><ul><li>Testing set: 17 VMD + 13 control patients with no significant age/sex differences </li></ul>
  17. 17. Performance of all 7 algorithms on 91 atlas structures RH - right hippocampus LPG - left parahippocampal gyrus RSN - right subthalamic nucleus RNA - right nucleus accumbens LC - left cuneus LPG - left precentral gyrus RC - right cuneus LITG - left inferior temporal gyrus RAG - right angular gyrus Some algorithms (BNCIT, Decision trees, discriminant analysis and logistic regression) select subset of features for classification; others use all features (91 atlas structures)
  18. 18. BNCIT <ul><li>Used only one structure: right hippocampus (RH) </li></ul><ul><li>P ( C = VMD | RH = atrophy ) = 0 . 73 </li></ul><ul><li>P ( C = NC | RH = atrophy ) = 0 . 27 </li></ul><ul><li>P ( C = VMD | RH = normal ) = 0 . 15 </li></ul><ul><li>P ( C = NC | RH = normal ) = 0 . 85 </li></ul><ul><li>Essentially like a 2 x 2 diagnostic table! </li></ul>
  19. 19. Decision Trees RH - right hippocampus LPG - left parahippocampal gyrus RSN - right subthalamic nucleus RNA - right nucleus accumbens LC - left cuneus LPG - left precentral gyrus RC - right cuneus LITG - left inferior temporal gyrus RAG - right angular gyrus
  20. 20. Performance of all 7 algorithms on 12 atlas structures (near or in medial temporal lobe) BNCIT: same model as before (only RH is included)
  21. 21. New decision tree:
  22. 22. Receiver Operating Characteristic (ROC) curves: areas under the curve (AUC) of all 7 algorithms for both experiments mismatch between AUCs and accuracies (reported before)
  23. 23. Comparison of algorithms through triangular discriminant metrics <ul><li>Algorithms that are similar in formulation perform similarly (boxed groups with close to zero metrics) </li></ul>
  24. 24. Algorithm “Families” “Rule”-based: NB, BNCIT, DT “Margin”-based: SVM, MLP “Statistics”-based: LDA, LR
  25. 25. Conclusions <ul><li>Classification of very mild dementia from normal controls using MRI seems promising with statistical learning techniques. </li></ul><ul><li>Machine learning algorithms such as BNCIT and SVM perform better than previously reported statistical algorithms like logistic regression and discriminant analysis in distinguishing VMD from controls. </li></ul><ul><li>Cross-validation accuracies are more reliable measures and should be reported instead of or in addition to discrimination accuracies in machine learning studies (already recognized in the literature, but not so for VMD classification). </li></ul><ul><li>As one might expect, the right hippocampus is an important structure for VMD classification. </li></ul><ul><li>A model need not be complicated to perform well (e.g. BNCIT model with only right hippocampal volume). </li></ul><ul><li>Similar algorithm groups (such as decision trees + Bayes, MLP + SVM and LR + LDA) are close in their triangular performance metrics, which take false positives, false negatives, true negatives and true positives all into account. </li></ul><ul><li>Future studies should focus on prediction of longitudinal outcome and look at larger sample sizes. </li></ul>
  26. 26. A few strengths <ul><li>Answering a relatively new and perhaps more useful clinical question: classification of VMD vs. control (as opposed to the more common machine learning literature on classification of AD vs. control or MCI vs. control) </li></ul><ul><li>Applying a variety of machine learning algorithms with different error metrics, comparing performance with those of previous approaches to the same problem, and providing insight to the similarities and differences between each algorithm </li></ul>
  27. 27. Thank you!

×