Tracking B cells from two-photon microscopy images using Sequential Monte Carlo

1. Tracking B and T cells
from 2-photon microscopy imaging
David Olivieri, Iván Gómez and Jose Faro
(University of Vigo)
www.milegroup.net

2. Outline
 Motivation for this work
 Stochastic method for tracking: SMC
• Theoretical aspects
• Some algorithm implementation details
 Results:
• From simulations and animations
• From real microscopy data of 2D cell motility
 Conclusions (present and future work)
Iván Gómez Conde

3. Immune Response
 Understand complex details of immune response by understanding dynamics
 Several important questions related to affinity maturation process
 Cell activity and interactions
Iván Gómez Conde
Low
Affinity
Low
Affinity
High
Affinity
Activation
Produces antibodies
Motivation

4. Germinal centers
 “Germinal centers” are the
sites of affinity maturation
 Anatomic structures (in lymph
nodes) where massive
proliferation of B-cells occur
 Complex interactions between
B and Th cells; spatial zones
 Understanding dynamics in
gernminal center ; better
understand mechanisms of
immune response
Iván Gómez Conde
Motivation
Confocal microscopy image of GC:T-cells
blue, B-cells green
(photo courtesy of I. Wollenberb, IMM, Universidad de Lisboa
Portugal y J. Faro, Fac. Biología, Universidad de Vigo)

5. Dynamics
• In vivo Data:
• “2-photon Confocal microscoy” with fluorescence
excitation labelling
• Better elimination of background
Iván Gómez Conde
Motivation
 Dynamics is important!
• B and T cell motility in germinal centers give
information of function
• Useful for “Inmunologic modeling” (input &
validation)

6. Tracking in Videos
 Tracking is hard in general!
• Normally needs to be real time
• many interactions: background, camera…
• Methods: frame diff, homology, optical flow,
particle filters
 What can be learned from tracking
objects?
• Tracking cells is particularly difficult
• Cells change shape, disappear, and stick to
eachother.
• complex background,
Iván Gómez Conde
Method

7. How to Tracking cells
 Cell movement:
• Problems: Complex, overlaps, “random” component
• BUT, flourescence color is a strong feature to track
• We propose “Stochastic color based tracking”:
• SMC (stochastic monte carlo)
Iván Gómez Conde
Method

8. Software Components
Method

9. Stochastic tracking
 Sequential Monte Carlo (smc)
 Formulate tracking as an inference problem in the context
of a Hidden Markov Model (HMM)
 Observations (from
image data)
 Hidden States (object
location, scale, …)
Yt Yt+1
Xt Xt+1
SMC Method

10. Chapman Kolmogorov Eq.
 Evolution of the state (inference):
• Using the Bayesian filtering distribution:
Current Object
State
Observation
Model
Previous Object
State
Evolution
Model
SMC Method

11. Quantities of the Model
 Prior Distribution:
• Initial distribution of object states
 Evolution Model:
• How objects move between frames
 Likelihood Function:
• The probability of state x given the observation y
Iván Gómez Conde
p(x0)
p(xt | xt -1)
p(yt | xt )
SMC Method

12. Prior distribution
 User input determines the object initial position
object
Iván Gómez Conde
Initial selection
of cells by the
user
SMC Method
p(x0)

13. Evolution model
Evolution Model (second-
order, auto-regressive
dynamical model)
SMC Method
p(xt | xt -1)

14. Likelihood function
 Likelihood Model (Distance metric):
Iván Gómez Conde
SMC Method
p(yt | xt )

15. SMC algorithm (summary)
1. Determine initial regions (roi) to track.
o From roi, store reference histogram
(each node)
2. Get image samples along trajectory of cell
o Determined from the dynamics (position,
velocity, update)
o Obtain histograms of roi; compare with
reference; keep best
3. Reorder the distribution for next sampling
Iván Gómez Conde
SMC Method

16. SMC Resampling
 Resampling, we
change weights
Iván Gómez Conde
SMC Method

17. SMC Tracking pseudocode
Iván Gómez Conde
SMC Method

18. Results: simple animation
 Showing each particle  Showing tracks of max L
Iván Gómez Conde
Results

19. Tracking Accuracy
Iván Gómez Conde
Results

20. Time Performance
Iván Gómez Conde
Results

21. Cells from Simulation
Iván Gómez Conde
Results
(simulation courtesy of J. Carneiro, T. Macedo, Instituto
Gulbenkian de Ciencia, Portugal)

22. Cells Simulation: Ambiguities
Iván Gómez Conde
Results
 “unstructured” SMC leads to ambiguities
 Imagine two cells sticking to each other…
 Just based on color, particles will sample entire region
 Not sure which cell is which after contact

23. Cell Ambiguities: “Present” work
Iván Gómez Conde
Results
 Possible Solution: make constraints between particles
 Conserve area and distance; & non-overlap condition
Node
particle Constraint
Preliminary results are promising!
Modify the Weights to include constraints

24. 2-photon Microscopy videos
Iván Gómez Conde
Results
(videos courtesy of C.Allen, et.al
Science, 2006)

25. Conclusions
 SMC is a promising technique for tracking cells
o Relatively easy to implement and flexible
o Can use color histogram or shape!
o Easily extended to handle 3D image stacks
o Stochastic noise can be controlled
o Present and Future:
o Extend to Constrained SMC can solve ambiguities
o Implementation of system of “constrained particles” for each node
Iván Gómez Conde

26. Many thanks for your
attention