Marked Point Process for Neurite
           Tracing

           Sreetama Basu
              NUS-IPAL
            Supervised by
        Prof. OOI Wei Tsang
                 And
       Prof. Daniel Racoceanu
Presentation topics

•   Neurite tracing
•   Marked Point Process(MPP)
•   MPP model for neurite tracing
•   Results
Digital reconstruction
Segmentation followed by quantification
                                                                      •Structure determines functions;
                                                                      linked to higher order cognitive functions,
                                                                       neuro-degenerative diseases
                                                                      (Alzheimer’s, Parkinson’s etc).

                                                                      •Brain Atlas of common lab animals;




•Huge volume of (sparse) data, Manual reconstruction takes days and months
• C.elegans ~ 15 years; EM connectomic data ~800 terabytes

•Digital reconstruction : Easier to archive, exchange, analyze



 Figure adapted from : Ascoli, G. A., J. L. Krichmar, et al. (2001). "Generation, description and storage of dendritic morphology
 data." Philos Trans R Soc Lond B Biol Sci 356(1412): 1131-1145.
Challenges of automated reconstruction

Tile stitching artifacts                                                                   Fuzzy structures




False connections,                                                                       Halos around branches
overlaps, crossing                                                                       mistaken as parallel
                                                                                         processes



                                                                                             Intra operator
     Branch gaps                                                                             And inter operator
     And                                                                                     Variability in manual
     discontinuities                                                                          reconstruction
                                                                                             by experts


 Brown, K. M., G. Barrionuevo, et al. (2011). "The DIADEM data sets: representative light microscopy images of neuronal
 morphology to advance automation of digital reconstructions.“ Neuroinformatics 9(2-3): 143-157
Presentation topics

•   Neurite tracing
•   Marked Point Process
•   MPP model for neurite tracing
•   Results
Marked Point Process

                                                                                           Extraction of tree crowns




Extraction of arbitrarily-shaped objects using stochastic multiple birth-and-death dynamics and active contours, Maria S. Kulikova, Ian H. Jermyn,
          Xavier Descombes, Elena Zhizhina, and JosianeZerubia, IS&T/SPIE Electronic Imaging 2010 meeting, San Jose, California, USA




                                                                                                                        Extraction of
                                                                                                                        road networks




    A Point Process for Fully Automatic Road Network Detection in Satellite and Aerial Images Pierre, C. Descombes, X. Zhizhina, E. Problems of
   Information Transmission 10 3 247-256 2010
Marked Point Process
                                         Intermediate                            Image data
         0th iteration
                                         iteration                              MPP Objects
                                                                   Energy optimization through
          A           B                 A            B             Birth and death dynamics
              C                             C                      Sampling embedded in
                  D       E                              E
                                                D                  Simulated Annealing
          F                             F




          A           B                 A            B
              C                             C
                  D       E                     D        E
          F                             F
Iteration stops when all objects                    Intermediate
are detected, corresponds to minimum energy         iteration
Marked Point Process
•   Set of objects= realization of a MPP
                                         K: point process whose realization is a
                                          random number of points given by a
                                                    random variable
•   Marked point process objects defined in S= K x M

                                                      M: mark space
                                                      of the object
•   Example: M= [Rmin,Rmax]
•   And (x,y, R) defines a disc object


                                                             A                   K
                                                                         B
    centre       radius                                          C
                                                                     D       E
                                                             F
Marked point process
• image is viewed as a Gibbs energy model, where the maximum
  object density corresponds to minimum energy
                                       1
                         P ( X x=
                             = )         exp( − β E ( x))
                                       K
                                                            Energy
                                     Inverse temperature
• Optimization: Multiple Birth and Death dynamics embedded in
  Simulated Annealing
Advantages:
      Unsupervised extraction not involving any specific configuration
      initialization (unknown number of objects)
      Allows incorporation of prior – geometric and interaction constraints
      among objects
       Takes into account data at macroscopic scale , at object level instead of
      pixels (HR images)
Presentation topics

•   Neurite tracing
•   Marked Point Process
•   MPP model for neurite tracing
•   Results
MPP model for neuronal network
               extraction
 • Aim: unsupervised 3D neuronal network extraction
 • Model: object process (objects: sphere ~ U ([rmin , rmax ])
   specified by an Energy:
                      U = Ud + Ui + Up

                                                   Prior term
Data term
                                                   •Favors connectivity
• internal homogeneity
•Contrast with background Interaction term
                          •Prevents overlap of objects
                          •Prevents crowding together of objects

 • Optimization: Multiple Birth and Death dynamics
   embedded in Simulated annealing
MPP model for neuronal network
              extraction

                                                         Red circle =
                                                             cut of
                                                           spherical
                                                         object on a
          Data Energy response                            slice of the
                                                         image stack




Internal homogeneity                           Both constraints
                             No contrast       are satisfied
constraint is violated
                             with background
MPP model for neuronal network
            extraction



Pair-wise interaction constraints




Considering radiometric properties   + interaction constraints among objects
MPP model for neuronal network
               extraction




Prior Energy: based on number of neighbors,
Models multi-object interaction; favors
connectivity and elongation of network
Presentation topics

•   Neurite tracing
•   Marked Point Process
•   MPP model for neurite tracing
•   Results
Data




Olfactory Projection data stack: input to MPP   2D projection (max intensity) of the
                                                Olfactory Projection data stack
Preliminary Results




Olfactory Projection data stack with MPP objects   Volume rendering of image stacks
Preliminary Results




Approximation of the neuronal network using   Construction of Minimum Spanning Tree from
MPP with spheres as objects                   object centres on 2D projection (max intensity)
                                              of the Olfactory Projection data stack
Preliminary Results
Gold Standard Manual
Reconstruction of                 MPP +MST reconstruction
the same Axon
Thank You

Marked Point Process For Neurite Tracing

  • 1.
    Marked Point Processfor Neurite Tracing Sreetama Basu NUS-IPAL Supervised by Prof. OOI Wei Tsang And Prof. Daniel Racoceanu
  • 2.
    Presentation topics • Neurite tracing • Marked Point Process(MPP) • MPP model for neurite tracing • Results
  • 3.
    Digital reconstruction Segmentation followedby quantification •Structure determines functions; linked to higher order cognitive functions, neuro-degenerative diseases (Alzheimer’s, Parkinson’s etc). •Brain Atlas of common lab animals; •Huge volume of (sparse) data, Manual reconstruction takes days and months • C.elegans ~ 15 years; EM connectomic data ~800 terabytes •Digital reconstruction : Easier to archive, exchange, analyze Figure adapted from : Ascoli, G. A., J. L. Krichmar, et al. (2001). "Generation, description and storage of dendritic morphology data." Philos Trans R Soc Lond B Biol Sci 356(1412): 1131-1145.
  • 4.
    Challenges of automatedreconstruction Tile stitching artifacts Fuzzy structures False connections, Halos around branches overlaps, crossing mistaken as parallel processes Intra operator Branch gaps And inter operator And Variability in manual discontinuities reconstruction by experts Brown, K. M., G. Barrionuevo, et al. (2011). "The DIADEM data sets: representative light microscopy images of neuronal morphology to advance automation of digital reconstructions.“ Neuroinformatics 9(2-3): 143-157
  • 5.
    Presentation topics • Neurite tracing • Marked Point Process • MPP model for neurite tracing • Results
  • 6.
    Marked Point Process Extraction of tree crowns Extraction of arbitrarily-shaped objects using stochastic multiple birth-and-death dynamics and active contours, Maria S. Kulikova, Ian H. Jermyn, Xavier Descombes, Elena Zhizhina, and JosianeZerubia, IS&T/SPIE Electronic Imaging 2010 meeting, San Jose, California, USA Extraction of road networks A Point Process for Fully Automatic Road Network Detection in Satellite and Aerial Images Pierre, C. Descombes, X. Zhizhina, E. Problems of Information Transmission 10 3 247-256 2010
  • 7.
    Marked Point Process Intermediate Image data 0th iteration iteration MPP Objects Energy optimization through A B A B Birth and death dynamics C C Sampling embedded in D E E D Simulated Annealing F F A B A B C C D E D E F F Iteration stops when all objects Intermediate are detected, corresponds to minimum energy iteration
  • 8.
    Marked Point Process • Set of objects= realization of a MPP K: point process whose realization is a random number of points given by a random variable • Marked point process objects defined in S= K x M M: mark space of the object • Example: M= [Rmin,Rmax] • And (x,y, R) defines a disc object A K B centre radius C D E F
  • 9.
    Marked point process •image is viewed as a Gibbs energy model, where the maximum object density corresponds to minimum energy 1 P ( X x= = ) exp( − β E ( x)) K Energy Inverse temperature • Optimization: Multiple Birth and Death dynamics embedded in Simulated Annealing Advantages: Unsupervised extraction not involving any specific configuration initialization (unknown number of objects) Allows incorporation of prior – geometric and interaction constraints among objects Takes into account data at macroscopic scale , at object level instead of pixels (HR images)
  • 10.
    Presentation topics • Neurite tracing • Marked Point Process • MPP model for neurite tracing • Results
  • 11.
    MPP model forneuronal network extraction • Aim: unsupervised 3D neuronal network extraction • Model: object process (objects: sphere ~ U ([rmin , rmax ]) specified by an Energy: U = Ud + Ui + Up Prior term Data term •Favors connectivity • internal homogeneity •Contrast with background Interaction term •Prevents overlap of objects •Prevents crowding together of objects • Optimization: Multiple Birth and Death dynamics embedded in Simulated annealing
  • 12.
    MPP model forneuronal network extraction Red circle = cut of spherical object on a Data Energy response slice of the image stack Internal homogeneity Both constraints No contrast are satisfied constraint is violated with background
  • 13.
    MPP model forneuronal network extraction Pair-wise interaction constraints Considering radiometric properties + interaction constraints among objects
  • 14.
    MPP model forneuronal network extraction Prior Energy: based on number of neighbors, Models multi-object interaction; favors connectivity and elongation of network
  • 15.
    Presentation topics • Neurite tracing • Marked Point Process • MPP model for neurite tracing • Results
  • 16.
    Data Olfactory Projection datastack: input to MPP 2D projection (max intensity) of the Olfactory Projection data stack
  • 17.
    Preliminary Results Olfactory Projectiondata stack with MPP objects Volume rendering of image stacks
  • 18.
    Preliminary Results Approximation ofthe neuronal network using Construction of Minimum Spanning Tree from MPP with spheres as objects object centres on 2D projection (max intensity) of the Olfactory Projection data stack
  • 19.
    Preliminary Results Gold StandardManual Reconstruction of MPP +MST reconstruction the same Axon
  • 20.