Object Recognition and Real-Time
                               Tracking in Microscopy Imaging



 Object Recognition and ...
MiCRoN
                                European Union IST project

  Uppsala, Lausanne, St. Ingbert, Athens, Pisa, Barcelo...
Motivation
                                   MiCRoN robot (i)



Locomotion Platform        Gripper               Rotor

...
Motivation
                                          MiCRoN robot (ii)

Power Floor, Syringe


                           ...
Motivation
                            MINIMAN vs. MiCRoN robot

      MINIMAN III-2 (5 d.o.f.)            MiCRoN (4 d.o.f...
Geometric Hashing
1988, Lamdan & Wolfson
Geometric Hashing
                         Preprocessing & Recognition


Preprocessing              Recognition




1988, ...
Geometric Hashing
                                   2 Degrees-Of-Freedom (i)


                            Solder Sphere
...
Geometric Hashing
                                        2 Degrees-Of-Freedom (ii)

                                   So...
Geometric Hashing
                                  3 Degrees-Of-Freedom (i)


                             Syringe Chip

...
Geometric Hashing
                                                          3 Degrees-Of-Freedom (ii)

                   ...
Bounded Hough Transform
2001/2004, Greenspan, Shang &
         Jasiobedzki
Bounded Hough Transform
   6 d.o.f. satellite tracking
2001/2004, Greenspan, S. & J.
Bounded Hough Transform
                2 Degrees-Of-Freedom




 Accumulate Votes
                                  t2


...
Bounded Hough Transform
                2 Degrees-Of-Freedom




 Accumulate Votes
                                       ...
Bounded Hough Transform
 3 Degrees-Of-Freedom
4 Degrees-Of-Freedom



Artificial Scene
Implementation
          Software Architecture

Application Layers
Implementation
Microstage with Custom-build Camera
Implementation
Graphical User Interface
Pushing Sugar
Results (i)



   video   reso-      time per    stack   degrees- recog- time per
           lution     frame       size  ...
Results (ii)



video   reso-      time per    stack   degrees- recog- time per
        lution     frame       size    of-...
Conclusion




• Depth estimation based on a single image is possible

• Real-time was achieved
   – Real-time recognition...
Future Work

                   Problems and Possible Solutions
Problem                           Solution
Geometric Hashi...
Homepage



MMVL official website                 MMVL MediaWiki




www.shu.ac.uk/research/meri/mmvl/   vision.eng.shu.ac.u...
Upcoming SlideShare
Loading in …5
×

Object Recognition and Real-Time Tracking in Microscope Imaging - IMVIP 2006

952 views

Published on

As the fields of micro- and nano-technology mature, there is going to be an increased need for industrial tools that enable the assembly and manipulation of micro-parts. The feedback mechanism in a future micro-factory will require computer vision.
Within the EU IST MiCRoN project, a computer vision software based on Geometric Hashing and the Bounded Hough Transform to achieve recognition of multiple micro-objects was implemented and successfully demonstrated. In this environment, the micro-objects will be of variable distance to the camera. Novel automated procedures in biology and micro-technology are thus conceivable.
This paper presents an approach to estimate the pose of multiple micro-objects with up to four degrees-of-freedom by using focus stacks as models. The paper also presents a formal definition for Geometric Hashing and the Bounded Hough Transform.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
952
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
34
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Object Recognition and Real-Time Tracking in Microscope Imaging - IMVIP 2006

  1. 1. Object Recognition and Real-Time Tracking in Microscopy Imaging Object Recognition and Real-Time Tracking in Microscopy Imaging Jan Wedekind 30th Aug – 1st Sep 2006 MMVL http://www.shu.ac.uk/research/meri/mmvl/ MiCRoN http://wwwipr.ira.uka.de/~micron/ Mimas http://sourceforge.net/projects/mimas/ MediaWiki http://vision.eng.shu.ac.uk/mediawiki/ People: J. Wedekind, M. Boissenin, B.P. Amavasai, F. Caparrelli, J. Travis
  2. 2. MiCRoN European Union IST project Uppsala, Lausanne, St. Ingbert, Athens, Pisa, Barcelona, Karlsruhe Motivation Control & GUI • prototype soldering/assembly • cell manipulation • manipulations inside vacuum chamber Project Goals • Manipulate µm-sized objects • Closed-loop control of Universit¨t Karlsruhe (Germany) a robot http://wwwipr.ira.uka.de/~micron/ • 3D object recognition http://www.cordis.lu/ist/ and tracking
  3. 3. Motivation MiCRoN robot (i) Locomotion Platform Gripper Rotor Scuola Superiore, Uppsala University Sant’Anna (Italy) (Sweden) Ecole Polytechnique F´d´rale de Lausanne e e (Switzerland)
  4. 4. Motivation MiCRoN robot (ii) Power Floor, Syringe PCB Task Planning National Technical University of Athens University of Barcelona (Greece) (Spain) Fraunhofer Institute, St. Ingbert (Germany)
  5. 5. Motivation MINIMAN vs. MiCRoN robot MINIMAN III-2 (5 d.o.f.) MiCRoN (4 d.o.f.) centre of sphere - end effector = 6.2 cm fits on a 20 cent coin
  6. 6. Geometric Hashing 1988, Lamdan & Wolfson
  7. 7. Geometric Hashing Preprocessing & Recognition Preprocessing Recognition 1988, Lamdan & Wolfson
  8. 8. Geometric Hashing 2 Degrees-Of-Freedom (i) Solder Sphere • Extract Sobel edges • Randomly select a feature location
  9. 9. Geometric Hashing 2 Degrees-Of-Freedom (ii) Solder Sphere 0 1 1   0 0 1 VM (@ A , m ) 1 t1 3 1 m   0 1 t2  = s − m 1   VM (@ A , m ) 1 0 2 1 4 0 0 m   s 1 VM (  , m) 1 0 5 5 1 m ”best match 3 with highest m VM (u, m)” u a single feature-correspondence reveals the object’s pose
  10. 10. Geometric Hashing 3 Degrees-Of-Freedom (i) Syringe Chip • Extract Sobel edges • Randomly select two feature locations
  11. 11. Geometric Hashing 3 Degrees-Of-Freedom (ii) Syringe Chip 0 1 s1 , s2 : randomly selected pair of scene features 1 VM (@ A , (m1 , m2 )) m2 1 m1 , m2 : model feature-pair with highest sum of votes 0 1 0 2 0 1 1 m1 feature tuple coord.−system 1 VM (@ A , (m1 , m2 )) 1 m2   s2 3 2 1 0 1 object coordinate system 1 VM (  , (m1 , m2 )) m1 1 s1 x2 m2 α 1 4 5 t 3 x1 4 ’’best match m1 with highest VM (u, (m1 , m2 ))” m1 m2 t, α: presumed pose of object u two feature-correspondences are revealing the object’s pose
  12. 12. Bounded Hough Transform 2001/2004, Greenspan, Shang & Jasiobedzki
  13. 13. Bounded Hough Transform 6 d.o.f. satellite tracking 2001/2004, Greenspan, S. & J.
  14. 14. Bounded Hough Transform 2 Degrees-Of-Freedom Accumulate Votes t2 t1 model features scene features local region
  15. 15. Bounded Hough Transform 2 Degrees-Of-Freedom Accumulate Votes t2 peak in Hough space t1 model features scene features local region
  16. 16. Bounded Hough Transform 3 Degrees-Of-Freedom
  17. 17. 4 Degrees-Of-Freedom Artificial Scene
  18. 18. Implementation Software Architecture Application Layers
  19. 19. Implementation Microstage with Custom-build Camera
  20. 20. Implementation Graphical User Interface
  21. 21. Pushing Sugar
  22. 22. Results (i) video reso- time per stack degrees- recog- time per lution frame size of- nition- frame (down- (recogni- freedom rate (tracking) sampled) tion) dry run (load 384×288 0.0081 s - - - - frames only) 384×288 0.20 s 7 (x, y, z) 88% 0.020 s 160×120 0.042 s 10 (x, y, z, θ) 87% 0.016 s
  23. 23. Results (ii) video reso- time per stack degrees- recog- time per lution frame size of- nition- frame (down- (recogni- freedom rate (tracking) sampled) tion) 384×288 0.27 s 16 (x, y, z, θ) 88% 0.025 s 384×288 0.072 s 14 (x, y, z, θ) 88% 0.018 s 9 (x, y, z, θ) 35% 192×144 0.32 s 0.022 s 1 (x, y, θ) 45%
  24. 24. Conclusion • Depth estimation based on a single image is possible • Real-time was achieved – Real-time recognition possible with low recognition-rate – Low recognition-rate much more tolerable than low frame-rate – Real-time tracking solves problem of low recognition-rate • Focus stack must not be self-similar • Rough surfaces are rich in features
  25. 25. Future Work Problems and Possible Solutions Problem Solution Geometric Hashing scales badly with Use RANSAC with Linear Model number of objects Hashing High memory requirements, Use local feature context, use less parametrisation for more than 2 features, use only salient features objects is difficult Sub-sampling decreases accuracy Implement non-uniform partitioning of Hough-space (adaptive accuracy) Research Topics • more than 4 degrees-of-freedom • develop micro-assembly for industrial application • develop semi-automated supporting microscope tool for biological application
  26. 26. Homepage MMVL official website MMVL MediaWiki www.shu.ac.uk/research/meri/mmvl/ vision.eng.shu.ac.uk/mediawiki/ Open source MiCRoN vision software + test data Open source Mimas real-time computer vision library

×