Simulating Weather: Numerical Weather Prediction as Computational Simulation
Diffusion MRI, Tractography,and Connectivity: what machine learning can do?
1. DW-MRI, Tractography,
and Connectivity: what
Machine Learning can do?
Ting-Shuo Yo
Max Planck Institute
for Human Cognitive and Brain Sciences
Leipzig, Germany
Max Planck Institute for Human Cognitive and Brain Sciences
2. Where the story begins
● Diffusion Weighted MRI (DWI) is a newly
developed MR scanning protocol, which can
detect the movement/displacement of water
molecules in tissues.
● So far, the techniques used in DWI analysis are
mostly deterministic and mechanical. The
stochastic approaches (ML related) can bring
new insights to this field.
2
Max Planck Institute for Human Cognitive and Brain Sciences
3. Outline
● MPG/MPIs
● A brief introduction of DWI
● What DWI can do
● A comparison of different tractography algorithms
● What ML can do in DWI
3
Max Planck Institute for Human Cognitive and Brain Sciences
4. Outline
● MPG/MPIs
– Max Planck Society
– Objective and Organization
– MPI - CBS
● A brief introduction of DWI
● What DWI can do
● A comparison of different tractography algorithms
● What ML can do in DWI
4
Max Planck Institute for Human Cognitive and Brain Sciences
5. The Max Planck Society
● The Max Planck Society for the
Advancement of Science is an independent,
non-profit research organization.
● In particular, the Max Planck Society takes up
new and innovative and interdisciplinary
research areas that German universities are
not in a position to accommodate or deal with
adequately.
5
6. The Max Planck Institutes
● The research institutes
of the Max Planck
Society perform basic
research in the interest
of the general public in
the natural sciences,
life sciences, social
sciences, and the
humanities.
● Currently there are 81
MPIs.
6
9. Outline
● MPG/MPIs
● An Introduction of DWI tractography
– Local modelling
– Fibre tracking
● What DWI can do
● A comparison of different tractography algorithms
● What ML can do in DWI
9
Max Planck Institute for Human Cognitive and Brain Sciences
10. Diffusion Weighted MRI
● MRI can detect the
movement of water
molecules.
● The movement is
constrained by the
neural fibers.
10
Max Planck Institute for Human Cognitive and Brain Sciences
11. Diffusion Weighted MRI
● By posing a gradient magnetic field, the
displacement in the corresponding direction
can be measured.
11
Max Planck Institute for Human Cognitive and Brain Sciences
12. Tractography (1)
● Local modelling:
➢ Reconstruct the fibre
orientation within each voxel
12
Max Planck Institute for Human Cognitive and Brain Sciences
13. Tractography (2)
● Diffusion propagator
– Diffusion Tensor (DT)
– Multiple compartment models
– Persistent Angular Structure (PAS)
● Fibre Orientation Distribution Function
– Spherical Deconvolution
13
Max Planck Institute for Human Cognitive and Brain Sciences
14. Tractography (3)
● Fiber tracking:
➢ Reconstruct fibre tracts by
integrating the reconstructed
local information
14
Max Planck Institute for Human Cognitive and Brain Sciences
15. Tractography (4)
● Streamline approach
– Deterministic
– Probabilistic
● Optimization for a larger region
– Spin tracking
– Gibbs tracking
15
Max Planck Institute for Human Cognitive and Brain Sciences
16. Tractography (5)
● Deterministic tracking
– At each step, only
consider the most likely
direction
● Curvature threshold
● Step size
● Interpolation
● ......
16
Max Planck Institute for Human Cognitive and Brain Sciences
17. Tractography (6)
● Probabilistic tracking
– Perform deterministic tracking for multiple times
– Allow uncertainty at each step
17
Max Planck Institute for Human Cognitive and Brain Sciences
18. Tractography (7)
● Probabilistic tracking and tractogram
– Probability of connection
18
Max Planck Institute for Human Cognitive and Brain Sciences
19. Tractography (8)
● Optimization for a larger region
– Spin tracking
– Gibbs tracking
From Kreher et al. 2008
19
Max Planck Institute for Human Cognitive and Brain Sciences
20. Outline
● MPG/MPIs
● A brief introduction of DWI
● What DWI can do
– To reveal anatomical structure in white matter
– To construct the general brain network
– In vivo
● A comparison of different tractography algorithms
● What ML can do in DWI
20
Max Planck Institute for Human Cognitive and Brain Sciences
21. White matter structure from DWI
● Product of tractography
21
Max Planck Institute for Human Cognitive and Brain Sciences
22. Brain Network from DWI
● Hagmann 2008
22
Max Planck Institute for Human Cognitive and Brain Sciences
23. What DWI can do
● fMRI shows "where" is working.
– The "nodes" in a graph/network
● DWI shows the structure of the fiber bundles.
– The “edges" in a graph/network
– With further analysis, can also show "strength of
edges".
● The brain network:
– The amount of nodes: 10^2
– The amount of edges: 10^3
23
Max Planck Institute for Human Cognitive and Brain Sciences
24. Outline
● MPG/MPIs
● A brief introduction of DWI
● What DWI can do
● A comparison of different tractography
algorithms
– Selected algorithms
– Procedure
– Results
● What ML can do in DWI
24
Max Planck Institute for Human Cognitive and Brain Sciences
25. Selected Algorithms
25
Max Planck Institute for Human Cognitive and Brain Sciences
26. Procedure
26
Max Planck Institute for Human Cognitive and Brain Sciences
27. Results (1)
27
Max Planck Institute for Human Cognitive and Brain Sciences
28. Results (2)
28
Max Planck Institute for Human Cognitive and Brain Sciences
29. Results (3)
29
Max Planck Institute for Human Cognitive and Brain Sciences
30. Results (4)
30
Max Planck Institute for Human Cognitive and Brain Sciences
31. Results (5)
31
Max Planck Institute for Human Cognitive and Brain Sciences
32. Results (6)
32
Max Planck Institute for Human Cognitive and Brain Sciences
33. Results (7)
33
Max Planck Institute for Human Cognitive and Brain Sciences
34. Results (8)
34
Max Planck Institute for Human Cognitive and Brain Sciences
35. Quick Summary
● More connections
– Local models which allow multiple fibres
– Probabilistic tracking
● Consistent patterns across methods
– Strong connections within a lobe
– Strong connections to corpus callosum
– Weak trans-callosum connections
35
Max Planck Institute for Human Cognitive and Brain Sciences
36. Results (9)
36
Max Planck Institute for Human Cognitive and Brain Sciences
37. Outline
● MPG/MPIs
● A brief introduction of DWI
● What DWI can do
● A comparison of different tractography algorithms
● What ML can do in DWI
– Local model reconstruction
– Fiber tracking
– Further application
37
Max Planck Institute for Human Cognitive and Brain Sciences
38. ML in DWI
● Local modeling: deconvolution approach
– Assume the signals are convolution of neural
fibers and noises.
– Need to “learn" the deconvolution kernel from
data defined as "one single fiber".
– So far only GLM (2nd order polynomial) is used.
– More sophisticated kernel methods can be used.
38
Max Planck Institute for Human Cognitive and Brain Sciences
39. ML in DWI
● Fiber tracking
– Speed up the optimization process.
– Different fiber reconstruction method.
● Probabilistic modeling of fiber tracts
39
Max Planck Institute for Human Cognitive and Brain Sciences
40. MICCAI'09 Fiber Cup
● 6 datasets:
– 3 of resolution 3x3x3mm (image size: 64x64x3) and
3 b-values (650, 1500 and 2000)
– 3 of resolution 6x6x6mm (image size: 64x64x1) and
3 b-values (650, 1500, 2650)
● Participants have to return one single fiber per
spatial position selected.
40
Max Planck Institute for Human Cognitive and Brain Sciences
41. MICCAI'09 Fiber Cup
41
Max Planck Institute for Human Cognitive and Brain Sciences
42. A Very Brief Review of Tractography
● Local modeling
● Fiber tracking
42
Max Planck Institute for Human Cognitive and Brain Sciences
43. Why are we doing this?
● Streamline-based tractography:
– Each simulation (a fiber) is a possible trajectory in
the given vector field.
● What is the probability of one given fiber?
● How to select the most representative fibers?
43
Max Planck Institute for Human Cognitive and Brain Sciences
44. Probability of a Fiber Tract (1)
● Fiber tract, t = { x1, x2, ...., xl }
● P(t) = P( x1, x2, ...., xl )
44
Max Planck Institute for Human Cognitive and Brain Sciences
45. Probability of a Fiber Tract (2)
● Conditional Probability and Joint Probability
– P(A|B) = P(A,B) / P(B)
– P(A,B) = P(A|B) P(B)
● P(t) = P( x1, x2, ...., xl )
= P(xl| x1, ...., xl-1) P(x1, ...., xl-1)
= P(xl| x1, ...., xl-1) P(xl-1|x1, ...., xl-2) P(x1, ...., xl-2)
= P(xl| x1, ...., xl-1) P(xl-1|x1, ...., xl-2) ......P(x2|x1) P(x1)
45
Max Planck Institute for Human Cognitive and Brain Sciences
46. Probability of a Fiber Tract (3)
●
Assumption: fiber tracking is a 1st order Markov
process
– P(xi| x1, ...., xi-1) = P(xl|xi-1)
– P(t) = P( x1, x2, ...., xl )
= P(xl| x1, ...., xl-1) P(xl-1|x1, ...., xl-2) ......P(x2|x1) P(x1)
= P(xl|xl-1) P(xl-1|xl-2) ......P(x2|x1) P(x1)
l−1
= P x 1 ∏ P x i1∣x i
i=1
46
Max Planck Institute for Human Cognitive and Brain Sciences
47. Probability of a Fiber Tract (4)
● How do we define P(xi+1|xi) and P(xi) ?
– C: connection probability map
– P(xi) ~ C(xi)
– P(xi+1|xi) ~ C(xi+1|xi) ~ C(xi+1,xi)
l−1
P t=P x1 ∏ P x i1∣x i
i=1
47
Max Planck Institute for Human Cognitive and Brain Sciences
48. Finite State Automata (1)
● Each step of fiber tracking can lead to next
middle point or the terminal point.
48
Max Planck Institute for Human Cognitive and Brain Sciences
49. Finite State Automata (2)
t={x 1 , ... , x l }
l−1
P t=P0 x l ∏ 1−P 0 x i
i=1
49
Max Planck Institute for Human Cognitive and Brain Sciences
50. Finite State Automata (3)
● How to define P0?
– # of fibers in the neighboring voxels, NB(x)
– (1-P0(xi)) ~ C(NB(xi))
P 0 x=1−C x k
– C(NB(xi))~ C(xi) K = 20, 10, 5
l−1
P t=P0 x l ∏ 1−P 0 x i
i=1
l −1
P t≃∏ 1−1−C xi k
i=1
50
Max Planck Institute for Human Cognitive and Brain Sciences
51. Finite State Automata (4)
● Likelihood and Log-likelihood
l−1
P t=P0 x l ∏ 1−P 0 x i
i=1
l −1
P t≃∏ 1−1−C xi k
i=1
l−1 l−1
L t≃∑ ln 1−1−C x i k ≃∑ −1−C xi k
i=1 i=1
Approximation with 1st order Taylor's expansion
51
Max Planck Institute for Human Cognitive and Brain Sciences
52. Entropy of a Fiber Tract (1)
● Entropy
l
H t =∑ C x i ⋅lnC x i
i=1
● Can be seen as the log-likelihood of
l l
∑ C xi ⋅lnC xi =ln ∏ C x i C xi
i=1 i =1
52
Max Planck Institute for Human Cognitive and Brain Sciences
53. Fiber Cup Results (2)
Max. Entropy Max. Likelihood
53
Max Planck Institute for Human Cognitive and Brain Sciences
54. ML in DWI
● Connectivity based clustering
– Brain parcellation
– Brain tissue is mostly
continuous without clear
segmentation, how to
define regions on it?
– Perform clustering based
on the connectivity
matrices.
54
Max Planck Institute for Human Cognitive and Brain Sciences
55. Leipzig, Germany Saclay, Gif-sur-Yvette, France
A. Anwander M. Descoteaux
T.R. Knösche P. Fillard
T. Yo C. Poupon
55
Max Planck Institute for Human Cognitive and Brain Sciences
56. Questions
56
Max Planck Institute for Human Cognitive and Brain Sciences
57. Doing what the brain does - how
computers learn to listen
57
Max Planck Institute for Human Cognitive and Brain Sciences
58. Thank You
58
Max Planck Institute for Human Cognitive and Brain Sciences