Python in Computer Vision

Motivation & Background Computer Vision in Python More Information Summary Introduction to using Python in Computer Vision Kiwi PyCon, Christchurch, 2009 Brian Thorne University of Canterbury 6th November 2009 Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python More Information Summary Outline 1 Motivation & Background What is Computer Vision? Uses & Examples 2 Computer Vision in Python Tools Image Acquisition Image Filtering Background Subtraction Feature Point Detection 3 More Information Diﬀerent platforms Additional Tools Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Outline 1 Motivation & Background What is Computer Vision? Uses & Examples 2 Computer Vision in Python Tools Image Acquisition Image Filtering Background Subtraction Feature Point Detection 3 More Information Diﬀerent platforms Additional Tools Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Vision 25% of the whole brain is for vision. Around 50% of cerebral cortex is for vision, 80% of the brain is associated with vision in some manner. Brian Thorne Computer Vision in Python

Computer Vision Deﬁnition The goal of computer vision is to recognize objects and their motion What is it used for? Scene reconstruction Event detection Video tracking Object recognition Learning Indexing Motion estimation Image restoration

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Computer Vision crosses over with many domains Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary What makes it hard? What we see What the computer sees Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Vision is inferential http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Outline 1 Motivation & Background What is Computer Vision? Uses & Examples 2 Computer Vision in Python Tools Image Acquisition Image Filtering Background Subtraction Feature Point Detection 3 More Information Diﬀerent platforms Additional Tools Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Visual Object Classes Challenge 09 http://www.pascal-network.org/challenges/VOC/voc2009 Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Object Recognition and Segmentation - Texture −− − − − − −→ (Sharon, Balun, Brandt, Basri) Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Object Recognition and Segmentation - Edges http://www.robots.ox.ac.uk/~vdg/dynamics.html Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Traﬃc Monitoring Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Computer Vision More Information Uses & Examples Summary Augented Reality - Sixth Sense ’SixthSense’ is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. http://www.pranavmistry.com/projects/sixthsense/ Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Outline 1 Motivation & Background What is Computer Vision? Uses & Examples 2 Computer Vision in Python Tools Image Acquisition Image Filtering Background Subtraction Feature Point Detection 3 More Information Diﬀerent platforms Additional Tools Brian Thorne Computer Vision in Python

Python In Computer Vision: OpenCV Provides well tested, optimized and open source code for image processing and computer vision Written in C, ensuring both fast and portable. Has been compiled for many embedded platforms Has multiple language wrappers including 3 for Python Tools have been made to use graphics hardware to accelerate CV performance on the GPU Project home page and documentation is at: http://opencv.willowgarage.com

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Python In Computer Vision: Numpy & Scipy Gives strongly typed N-dimensional arrays to Python Well used and tested libraries for scientiﬁc computing Includes lots of handy tools such as optimisation and signal processing used often in computer vision. Usually used with iPython and matplotlib SciPy can be downloaded from: http://www.scipy.org Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Python In Computer Vision: Pygame Game development framework Now has basic Computer Vision support Being Python it can be used with other Python tools - integrates well with numpy/scipy pygame can be downloaded from: http://www.pygame.org Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Pycam This is the project with all the examples for this presentation. Has a bunch of simple examples like ﬁltering and background subtraction, face detection. Contains two video player classes that can work with diﬀerent backend setups, and can incorporate optional process functions. Examples of intergrating OpenCV with pygame - eg for eye and face detection. OpenCV camera class that allows an opencv camera to be used with pygame (No longer required in latest pygame) VideoCapturePlayer For the rest of this presentation, examples will use the video capture code (with error checking) from pycam. Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Acquiring & Display Of An Image Live image acquisition is such a crucial role in the majority of CV applications. Example getting and showing a frame as a most basic, but necessary test Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Python OpenCV: Image Capture Example from opencv import highgui as hg capture = hg.cvCreateCameraCapture(0) hg.cvNamedWindow("Snapshot") frame = hg.cvQueryFrame(capture) hg.cvShowImage("Snapshot", frame) hg.cvWaitKey(10000) Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Filtering - Gaussian Blur One of the simplest operations in image processing is blurring an image Reduce noise, Remove artifacts Scale an image “cleanly” Create motion blur - if done in one direction OpenCV includes a gaussian filter among many others (cvSmooth function) SciPy has a multi-dimensional Gaussian filter that acts on a NumPy array Or you could convolve an image with a filter manually Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Applying a Gaussian Blur with OpenCV OpenCV Gaussian Blur from pycam import VideoCapturePlayer as VCP from opencv import cv def gaussianBlur(im, filterSize=43): result = cv.cvCreateMat(im.rows, im.cols, im.type ) cv.cvSmooth(image,result, cv.CV_GAUSSIAN, filterSize) return result if __name__ == "__main__": VCP(gaussianBlur, "Guassian Filter").main() Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Applying a Gaussian Blur with Scipy SciPy Gaussian Blur from scipy.ndimage.filters import gaussian_filter from pycam import OpencvVideoCapturePlayer as VCP from misc import scipyFromOpenCV @scipyFromOpenCV def gaussianBlur(np_image): result = gaussian_filter(np_image, sigma=(4, 4, 0), order=0, mode=’reflect’) return result if __name__ == "__main__": VCP(gaussianBlur,"Scipy Guassian Blur").main() Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Background Subtraction In video security the camera mostly looks at the same boring background What we are usually interested in is when objects (eg people or vehicles) enter or exit a scene Aim is to isolate the interesting, and ignore the boring At the most simple level background subtraction is simply a comparison between two image frames At the more complex level many people have gotten phd’s for better background learning techniques, and better diﬀerencing algorithms Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Background Subtraction Simple Frame Differencing 1 To let the camera adjust to light levels, ignore the first few frames. 2 Store a frame as the base frame. 3 For each new frame that comes in: 1 Take the absolute intensity difference in each channel (Red/Green/Blue) with the base frame. 2 Binary threshold to ignore pixels that are only a bit different. 3 Convert difference image to a one channel mask 4 Clean up small noise areas in the mask (with median filter, erode, connected components) 5 Return the changed pixels from the original image using the created mask. Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Background Subtract Here I have placed a cellphone on my cluttered desk Can’t tell thats there is no green screen Quick demo Brian Thorne Computer Vision in Python

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Feature Point Detection Feature point detection is implemented in OpenCV you can do it in one call: cvCornerHarris or cvGoodFeatures To demonstrate the algorithm though - we will go look at it in scipy. Implementation derived from Jan Solem Brian Thorne Computer Vision in Python

Feature Detection 1 First convert to a grey scale image 2 Showing the derivative in the x and y directions 3 showing the millions of points of interest 4 ﬁltering them

Tools Motivation & Background Image Acquisition Computer Vision in Python Image Filtering More Information Background Subtraction Summary Feature Point Detection Augmented Reality Augmented reality is undergoing massive growth OpenCV provides the face detection An AR game can easily be made in Pygame using the webcam and face location as the interface Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Diﬀerent platforms More Information Additional Tools Summary Outline 1 Motivation & Background What is Computer Vision? Uses & Examples 2 Computer Vision in Python Tools Image Acquisition Image Filtering Background Subtraction Feature Point Detection 3 More Information Diﬀerent platforms Additional Tools Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Diﬀerent platforms More Information Additional Tools Summary Running on an OLPC OLPC - provide children in developing nations with access to knowledge, and opportunities to "explore, experiment and express themselves" Includes Python and a webcam - thats all you need for computer vision! Here I am running OpenCV’s facedetection on the XO laptop Lots of Computer Vision work on the XO has been done using pygame by Nirav Patel (http://eclecti.cc/olpc) Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python Diﬀerent platforms More Information Additional Tools Summary Outline 1 Motivation & Background What is Computer Vision? Uses & Examples 2 Computer Vision in Python Tools Image Acquisition Image Filtering Background Subtraction Feature Point Detection 3 More Information Diﬀerent platforms Additional Tools Brian Thorne Computer Vision in Python

IPython & MatPlotLib Using IPython, an interactive shell can be used from deep inside a nested loop in a running program. In the code add from IPython.Shell import IPShellEmbed ... IPShellEmbed()() Example In [1]: from opencv import cv In [2]: cv.cvAnd(diﬀImage,image, temp) In [3]: timeit cv.cvAnd(diﬀImage,image, temp) 1000 loops, best of 3: 229 µs per loop In [4]: from pylab import imshow, show In [5]: imshow(temp) Out[5]: <AxesImage object at 0x42489d0> In [6]: show()

Motivation & Background Computer Vision in Python Diﬀerent platforms More Information Additional Tools Summary Documentation & Support The documentation in both SciPy and OpenCV was found to be pretty good.... not entirely complete. The OpenCV book is really good. Remember Python is Free Documentation is not going to be as extensive as for a professional package like Matlab.... but you can help! Support for these open source packages is almost entirely reliant on experienced members of the community responding to requests on message boards or mailing lists. Brian Thorne Computer Vision in Python

Motivation & Background Computer Vision in Python More Information Summary Summary For the scholar of computer vision research, using Python can help in trying out new algorithms very quickly. The breadth of the additional libraries available and the ease of integrating, make new and novel solutions quickly realizable. For someone just wanting to play around with some cool stuﬀ, its easy to dive in! Limitations on using Python for CV A major limitation of using Python would be when the application is being developed for special embedded hardware or when the best possible performance is required (at YOUR expense) Brian Thorne Computer Vision in Python

References Thank You! Thank you to Raphaël Grasset - supervisor at HitLabNZ Richard Green - computer vision lecturer John Graves & Cristiano Soares for giving me detailed and helpful feedback Brian Thorne Computer Vision in Python

References For Further Reading I Library URL Pygame http://pygame.org OpenCV http://opencv.willowgarage.com Numpy/Scipy http://scipy.org Pycam http://pycam.googlecode.org G. Bradski, A. Kaehler Learning OpenCV. O’Reilly Media, September 2008. T. Oliphant Guide to NumPy. UT, Trelgol Publishing, 2006. Brian Thorne Computer Vision in Python

http://bitsofpy.blogspot.com	490
http://mj89sp3sau2k7lj1eg3k40hkeppguj6j-a-sites-opensocial.googleusercontent.com	117
http://www.slideshare.net	100
http://www.techgig.com	20
http://translate.googleusercontent.com	8
http://bitsofpy.blogspot.in	7
http://bitsofpy.blogspot.com.au	5
http://bitsofpy.blogspot.de	5
http://bitsofpy.blogspot.co.uk	5
http://bitsofpy.blogspot.fr	3
http://bitsofpy.blogspot.ca	3
http://bitsofpy.blogspot.co.nz	3
http://bitsofpy.blogspot.com.es	2
http://bitsofpy.blogspot.it	2
http://bitsofpy.blogspot.com.br	1

Python in Computer Vision

by Brian Thorne on Nov 06, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

15 Embeds 771

Statistics

Python in Computer Vision — Presentation Transcript