Tracking Colliding Cells


Published on

The motion of leukocytes is significant in studying the inflammation response of the immune system. In inflammation conditions, some leukocytes slow down and eventually adhere to vessel walls. With many cells moving at a variety of speeds, collisions occur. These collisions result in abrupt changes in the motion and appearance of leukocytes. In this presentation, we propose a novel method of tracking multiple cells undergoing collision by modeling the collision states of cells and testing multiple hypotheses of their motion and appearance.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Motion model
  • Tracking Colliding Cells

    1. 1. Tracking Colliding Cells <br />Nhat ‘Rich’ Nguyen<br />Future Computing Lab<br />
    2. 2. Flu<br />Health Center<br />Blood Test<br />
    3. 3. White Count is a blood test to measure the number of white blood cells.<br />
    4. 4. In a drop of blood…<br />Number of <br />white cells<br />blood cancer<br />50,000<br />stress, viral infection, drug intake<br />25,000<br />healthy<br />5,000<br />flu, poisoning<br />0<br />
    5. 5. It isimportant to keep track of white blood cells.<br />
    6. 6. Challenges<br />Methods<br />Experiments<br />
    7. 7. Challenges<br />Methods<br />Experiments<br />
    8. 8. Video of white blood cells via a microscope<br />
    9. 9.
    10. 10. Manual<br />Automatic<br />Tedious<br />Expensive<br />Subjective<br />Little Effort<br />Economical<br />Objective<br />
    11. 11. As many cells move at a wide range of speeds…<br />Collisions<br />
    12. 12. abrupt<br />change<br />
    13. 13. Smoothness Constraints<br />Region A<br />broken tracks<br />Region A<br />robust tracks<br />
    14. 14. Challenges<br />Methods<br />Experiments<br />
    15. 15. Challenges<br />Methods<br />Experiments<br />
    16. 16. Smoothness Constraint<br />Region A<br />broken tracks<br />Region A<br />Region A<br />robust tracks<br />Our Method<br />
    17. 17. The first tracking method for colliding cells.<br />
    18. 18. Training<br />100 cell samples<br />100background samples<br />
    19. 19. Detection <br />Classify each pixel as a Cell or Background<br />
    20. 20. Tracking<br />time<br />
    21. 21. Kalman Filter<br />Popular<br />Extensively used for tracking.<br />Optimal<br />Estimate the most probable state.<br />Simple<br />Two steps: predict and correct.<br />
    22. 22. No Collision<br />Collision<br />smooth<br />smooth & abrupt<br />reliability<br />flexibility<br />?<br />Kalman filter<br />
    23. 23. Multiple Hypotheses<br />H2<br />Non- <br />Colliding<br />Colliding<br />H1<br />H3<br />H4<br />
    24. 24. No Collision<br />Non- <br />Colliding<br />Colliding<br />H1<br />
    25. 25. Collision<br />H2<br />Non- <br />Colliding<br />Colliding<br />
    26. 26. During Collision<br />Non- <br />Colliding<br />Colliding<br />H3<br />
    27. 27. After Collision<br />Non- <br />Colliding<br />Colliding<br />H4<br />
    28. 28. H2<br />Non- <br />Colliding<br />Colliding<br />H1<br />H3<br />H4<br />
    29. 29. The reliability of the Kalman filter, the flexibility of multiple hypotheses.<br />
    30. 30. Tracking Steps<br />
    31. 31. 1<br />2<br />3<br />
    32. 32. 1<br />2<br />3<br />
    33. 33. 1<br />2<br />3<br />
    34. 34. 1<br />2<br />3<br />
    35. 35. 1<br />2<br />3<br />
    36. 36. 1<br />2<br />3<br />
    37. 37. 1<br />colliding cells<br />2<br />3<br />non-colliding cell<br />
    38. 38. stay colliding<br />1<br />2<br />3<br />split away<br />keep moving<br />
    39. 39. Region B<br />Our method<br />
    40. 40. Challenges<br />Methods<br />Experiments<br />
    41. 41. Challenges<br />Methods<br />Experiments<br />
    42. 42. Data<br />8<br />300<br />~6K<br />image sequences<br />cells tracks <br />cell positions<br />
    43. 43. 112<br />188<br />colliding cells<br />non-colliding cells<br />
    44. 44. Compared Methods<br />SC <br />Smoothness Constraints <br />Single Hypothesis <br />Multiple Hypotheses <br />SH<br />MH <br />
    45. 45. Comparisons<br />MH<br />SC<br />SH<br />
    46. 46. Percentage of Tracked Positions<br />MH<br />SH<br />SC<br />
    47. 47. Colliding vs. Non-colliding<br />MH<br />SH<br />SC<br />
    48. 48. Impact of detection <br />MH <br />SH<br />SC<br />
    49. 49. Given adequate detection results, our method covers 88% of colliding cell positions.<br />
    50. 50. Challenges<br />Methods<br />Experiments<br />
    51. 51. Conclusion<br />The first tracking method for colliding cells.<br />The reliability of the Kalman filter, the flexibility of multiple hypotheses.<br />Excellent cell positions coverage.<br />Non- <br />Colliding<br />Colliding<br />88%<br />
    52. 52. Thank you.<br />
    53. 53. Questions ?<br />
    54. 54.
    55. 55. Bonus Slides<br />
    56. 56. S. J. Schmugge, S. Keller, N. Nguyen, R. Souvenir, T. H. Huynh, M. Clemens, M. C. Shin. "Segmentation of Vessels Cluttered with Cells using a Physics based Model". 11th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), New York, September 6-10 2008.<br />N. Nguyen, S. Keller, T. Huynh, M. Shin. “Tracking Colliding Cells”. IEEE Workshop on Applications of Computer Vision (WACV), Snowbird, UT December 07-09 2009.<br />[to be submitted to IEEE Transactions on Medical Imaging]<br />N. Nguyen, S. Keller, Eric Norris, T. Huynh, M. Shin. “Tracking Colliding Cells in In-Vivo Intravital Microscopy Images”. <br />Publications<br />
    57. 57. Example of Multiple cell tracking<br />
    58. 58.
    59. 59. White Blood Cells<br />Circulate in your blood<br />Defend you against bacteria<br />Protect you from disease<br />
    60. 60.
    61. 61. Previous Automatic Methods<br />Ray et al. [2002]<br />Active Contour<br />Cui et al. [2005]<br />Monte Carlo<br />Mukherjee et al. [2004]<br />Level Set Analysis<br />
    62. 62. Previous Automatic Methods<br />Eden et al. [2005]<br />SmoothnessConstraints<br />Li et al. [2005]<br />Lineage Construction<br />Smith et al. [2008]<br />Probabilistic Formalization<br />
    63. 63. Variation in cell appearances within an image<br />time<br />Varied appearance of a cell over time<br />
    64. 64. Qualitative Comparison<br />
    65. 65. Challenges<br />In a collision, cell motion and appearance<br />1. could be different<br />2. change abruptly<br />
    66. 66. Approach<br /> To improve tracking accuracy of colliding cells by:<br />having separate collision states<br />to describe cells inside and outside of collisions<br />testing multiple hypotheses<br />of cell motion and appearance as transitions between abrupt motion patterns.<br />
    67. 67. AdaBoost<br />Idea: combine many “rules of thumb” to a highly accurate prediction rule.<br />Input: visual features from training samples.<br />Schema: maintain a strategy to determine “rules of thumb” using weight distribution.<br />Output: a single strong classifier which is a linear combination of the set of weak classifiers.<br />
    68. 68. Detection Procedure <br />Scan each pixel p in the image<br />Compute image feature vector V from a window centered around p<br />Classify p as a Cell pixel if the feature score in V satisfies the learned decision rule; otherwise classify p as a Background pixel.<br />Cluster groups of Cell pixels into cell observation.<br />
    69. 69. Motion and Appearance Model<br />Collision States:<br />State Transition:<br />Hypotheses:<br />to predict the state in the next frame<br />control input vector<br />State Vector*:<br />Motion and Appearance Models: <br />(for )<br />state transition<br />matrix<br />control input matrix<br />process noise vector ~N(0,Qs)<br />Observation Vector:<br />
    70. 70. Multiple Hypotheses<br />No <br />Collision<br />Collision<br />
    71. 71. Performance RMSE<br />RMSE : Root mean squared errors of position (pixel)<br />-0.03<br />-0.17<br />+0.36<br />-0.21<br />+0.33<br />-0.20<br />SH introduces additional error in positions.<br />MH does not introduce any additional error.<br />Estimating colliding cells’ positions is more difficult.<br />
    72. 72. Collision Duration RMSE <br /> The effect of collision duration on RMSE<br />
    73. 73. Impact Detection RMSE <br /> The impact of detection on RMSE<br />-0.17<br />-0.13<br />-1.05<br />-1.09<br />Different improvement between dataset.<br />Different improvement between methods.<br />
    74. 74. Future Work<br />1. Add more features to improve detection.<br />5<br />7<br />6<br />8<br />
    75. 75. Future Work<br />2. Incorporate a probabilistic approach to transition between collision states.<br />72<br />73<br />75<br />76<br />
    76. 76. Future Work<br />3. Expand to track cells with more complex motions and behaviors.<br />49<br />50<br />51<br />52<br />
    77. 77. Detection Performance<br />Recall : TP / (TP + FN)<br />75%<br />Precision: TP / (TP + FP)<br />77%<br />
    78. 78. Collision Duration<br /> The effect of collision duration on tracking<br />6<br />112<br />Exclude SC from being considered for collision.<br />Classify colliding positions into bins based on the number of frames of the collision.<br />colliding cells<br />bins of<br />collision duration<br />
    79. 79. Detection Impact <br /> The impact of detection on tracking<br />38<br />596<br />Data with good detection results before and after collision (+/- 2 frames)<br />cell positions<br />treated<br />colliding cells<br />
    80. 80. Performance Table<br />PTP: Percentage of Tracked Positions (%)<br />+27<br />+9<br />+23<br />+3<br />+4<br />+24<br />+28<br />
    81. 81. Detection Impact Table<br />+9<br />+7<br />+16<br />+18<br />
    82. 82. More Results<br />Eden et al. [2005]<br />Our Method<br />
    83. 83. Tracking Steps<br />Predict motion<br />Predict collision<br />Get measurements<br />Get errors in position & area <br />Match with minimal error<br />
    84. 84. Collision States:<br />Hypotheses:<br />to predict the state in the next frame<br />control input vector<br />State Vector*:<br />Motion and Appearance Models: <br />(for )<br />state transition<br />matrix<br />control input matrix<br />process noise vector ~N(0,Qs)<br />Observation Vector:<br />Measurement Model:<br />measurement noise vector ~N(0,R)<br />measurement <br />transition matrix<br />
    85. 85. State Vector of cell i :<br />Predicted State Vector:<br />Zero Matrix<br />Zero Matrix<br />Zero Vector<br />Zero Vector<br />Predicted Covariance:<br />
    86. 86. Predicted State Vector:<br />Hypothesized Measurement Vector:<br />measurement <br />transition matrix<br />Error of hypothesis :<br />observation from <br />the detector<br />weight vector<br />Rule 1:<br />Rule 2:<br />error threshold of <br />Unlikely (i, k) pair<br />Stop corresponding condition:<br />
    87. 87. Remaining Observation :<br />new cell<br />leukocyte typical<br />diameter<br />area<br />Remaining Cell :<br />Not corresponded for 3 frames:<br />Updated State Vector :<br />Kalman gain<br />Updated Covariance:<br />depends on the cell <br />current state s <br /><ul><li>Eliminate the affect cause by </li></ul>abrupt change in collision<br />
    88. 88. Training<br />100 Cell Samples<br />100 Background Samples<br />Features<br />Mean Intensity<br />Standard Dev. of Intensity<br />Radial Mean<br /> Decision Rules on feature scores<br />
    89. 89. Collision Duration<br />The effect of collision duration on PTP<br />
    90. 90. H01<br />H00<br />No <br />Collision<br />(s = 0)<br />Collision<br />(s = 1)<br />H11<br />H10<br />
    91. 91. Measurements<br />Cell matches<br />Cell Detection<br />Correspondence<br />Update<br />Cell<br />image<br />Finished<br />tracks<br />Tracks<br />Predictions<br />Multiple Hypotheses<br />H00:<br />No Collision – No Collision<br />H01:<br />No Collision –Collision<br />H11:<br />Collision –Collision<br />H10:<br />Collision – <br />No Collision<br />