Object Recognition and Real-Time Tracking in Microscope Imaging - IMVIP 2006

Object Recognition and Real-Time Tracking in Microscopy Imaging Object Recognition and Real-Time Tracking in Microscopy Imaging Jan Wedekind 30th Aug – 1st Sep 2006 MMVL http://www.shu.ac.uk/research/meri/mmvl/ MiCRoN http://wwwipr.ira.uka.de/~micron/ Mimas http://sourceforge.net/projects/mimas/ MediaWiki http://vision.eng.shu.ac.uk/mediawiki/ People: J. Wedekind, M. Boissenin, B.P. Amavasai, F. Caparrelli, J. Travis

MiCRoN European Union IST project Uppsala, Lausanne, St. Ingbert, Athens, Pisa, Barcelona, Karlsruhe Motivation Control & GUI • prototype soldering/assembly • cell manipulation • manipulations inside vacuum chamber Project Goals • Manipulate µm-sized objects • Closed-loop control of Universit¨t Karlsruhe (Germany) a robot http://wwwipr.ira.uka.de/~micron/ • 3D object recognition http://www.cordis.lu/ist/ and tracking

Motivation MiCRoN robot (i) Locomotion Platform Gripper Rotor Scuola Superiore, Uppsala University Sant’Anna (Italy) (Sweden) Ecole Polytechnique F´d´rale de Lausanne e e (Switzerland)

Motivation MiCRoN robot (ii) Power Floor, Syringe PCB Task Planning National Technical University of Athens University of Barcelona (Greece) (Spain) Fraunhofer Institute, St. Ingbert (Germany)

Motivation MINIMAN vs. MiCRoN robot MINIMAN III-2 (5 d.o.f.) MiCRoN (4 d.o.f.) centre of sphere - end eﬀector = 6.2 cm ﬁts on a 20 cent coin

Geometric Hashing 1988, Lamdan & Wolfson

Geometric Hashing Preprocessing & Recognition Preprocessing Recognition 1988, Lamdan & Wolfson

Geometric Hashing 2 Degrees-Of-Freedom (i) Solder Sphere • Extract Sobel edges • Randomly select a feature location

Geometric Hashing 2 Degrees-Of-Freedom (ii) Solder Sphere 0 1 1   0 0 1 VM (@ A , m ) 1 t1 3 1 m   0 1 t2  = s − m 1   VM (@ A , m ) 1 0 2 1 4 0 0 m   s 1 VM (  , m) 1 0 5 5 1 m ”best match 3 with highest m VM (u, m)” u a single feature-correspondence reveals the object’s pose

Geometric Hashing 3 Degrees-Of-Freedom (i) Syringe Chip • Extract Sobel edges • Randomly select two feature locations

Geometric Hashing 3 Degrees-Of-Freedom (ii) Syringe Chip 0 1 s1 , s2 : randomly selected pair of scene features 1 VM (@ A , (m1 , m2 )) m2 1 m1 , m2 : model feature-pair with highest sum of votes 0 1 0 2 0 1 1 m1 feature tuple coord.−system 1 VM (@ A , (m1 , m2 )) 1 m2   s2 3 2 1 0 1 object coordinate system 1 VM (  , (m1 , m2 )) m1 1 s1 x2 m2 α 1 4 5 t 3 x1 4 ’’best match m1 with highest VM (u, (m1 , m2 ))” m1 m2 t, α: presumed pose of object u two feature-correspondences are revealing the object’s pose

Bounded Hough Transform 2001/2004, Greenspan, Shang & Jasiobedzki

Bounded Hough Transform 6 d.o.f. satellite tracking 2001/2004, Greenspan, S. & J.

Bounded Hough Transform 2 Degrees-Of-Freedom Accumulate Votes t2 t1 model features scene features local region

Bounded Hough Transform 2 Degrees-Of-Freedom Accumulate Votes t2 peak in Hough space t1 model features scene features local region

Bounded Hough Transform 3 Degrees-Of-Freedom

4 Degrees-Of-Freedom Artiﬁcial Scene

Implementation Software Architecture Application Layers

Implementation Microstage with Custom-build Camera

Implementation Graphical User Interface

Pushing Sugar

Results (i) video reso- time per stack degrees- recog- time per lution frame size of- nition- frame (down- (recogni- freedom rate (tracking) sampled) tion) dry run (load 384×288 0.0081 s - - - - frames only) 384×288 0.20 s 7 (x, y, z) 88% 0.020 s 160×120 0.042 s 10 (x, y, z, θ) 87% 0.016 s

Results (ii) video reso- time per stack degrees- recog- time per lution frame size of- nition- frame (down- (recogni- freedom rate (tracking) sampled) tion) 384×288 0.27 s 16 (x, y, z, θ) 88% 0.025 s 384×288 0.072 s 14 (x, y, z, θ) 88% 0.018 s 9 (x, y, z, θ) 35% 192×144 0.32 s 0.022 s 1 (x, y, θ) 45%

Conclusion • Depth estimation based on a single image is possible • Real-time was achieved – Real-time recognition possible with low recognition-rate – Low recognition-rate much more tolerable than low frame-rate – Real-time tracking solves problem of low recognition-rate • Focus stack must not be self-similar • Rough surfaces are rich in features

Future Work Problems and Possible Solutions Problem Solution Geometric Hashing scales badly with Use RANSAC with Linear Model number of objects Hashing High memory requirements, Use local feature context, use less parametrisation for more than 2 features, use only salient features objects is diﬃcult Sub-sampling decreases accuracy Implement non-uniform partitioning of Hough-space (adaptive accuracy) Research Topics • more than 4 degrees-of-freedom • develop micro-assembly for industrial application • develop semi-automated supporting microscope tool for biological application

Homepage MMVL oﬃcial website MMVL MediaWiki www.shu.ac.uk/research/meri/mmvl/ vision.eng.shu.ac.uk/mediawiki/ Open source MiCRoN vision software + test data Open source Mimas real-time computer vision library

Object Recognition and Real-Time Tracking in Microscope Imaging - IMVIP 2006

by Jan Wedekind

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics