Open cv tutorial

  • 1,346 views
Uploaded on

An Open CV tutorial I presented to CSE/EE599 on doing circular object tracking with Hough Transforms.

An Open CV tutorial I presented to CSE/EE599 on doing circular object tracking with Hough Transforms.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,346
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
61
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Open CV TutorialUbiquitous Computing Lecture February 15th, 2012
  • 2. OpenCV installation
  • 3. Setup – Windows (1)• Download OpenCV 2.1 http://sourceforge.net/projects/opencvlibrary/files/opencv-win/2.1/
  • 4. Setup – Windows (2)• Installation Add OpenCV DLLs into system path
  • 5. Setup – Windows (3)• Include the header files and static libraries
  • 6. Setup – Linux• Install OpenCV (say yes to any dependencies): $> sudo apt-get install libcv4 libcv-dev libhighgui4 libhighgui- dev libcvaux4 libcvaux-dev• To build: $> gcc -lcv -lhighgui –lcvaux YourFile.cpp
  • 7. Links• Main wiki http://opencv.willowgarage.com/wiki/• On-line library http://opencv.willowgarage.com/documentation/c/• Tutorial http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html
  • 8. Getting Images from the WebcamTwo Methods:• Event Driven (or interrupt driven) • New Frame == interrupt • Adv: constant FPS, multi-threaded processing • Dis: overhead, must be real-time• Polling (what we will use, the norm) • Grab frame when ready • Adv: only process what we can in real time • Dis: might grab same frame, variable FPS
  • 9. Getting Images from the Webcam #include <stdio.h> #include "opencv/cv.h" #include "opencv/highgui.h" #include "opencv/cxcore.h" int main(int argc, char* argv[]) { CvCapture *capture = 0; //The camera IplImage* frame = 0; //The images you bring out of the camera capture = cvCaptureFromCAM( 0 ); //Open the camera if (!capture ){ printf("Could not connect to cameran" ); return 1; } cvNamedWindow( "demowin1", CV_WINDOW_AUTOSIZE ); //Create output window while(key != 27 /*escape key to quit*/) { //Query for the next frame frame = cvQueryFrame( capture ); if( !frame ) break; //show the raw image in the first window cvShowImage( "demowin1", frame ); //Get the last key thats been pressed for input key = cvWaitKey( 1 ); } //Shutdown cvReleaseCapture( &capture ); // WARNING: DO NOT release the "frame" variable cvDestroyWindow( "demowin1" ); return 0; }
  • 10. Saving a video file int main(int argc, char* argv[]){ CvCapture *capture = 0; //The camera IplImage* frame = 0; //The images you bring out of the camera capture = cvCaptureFromCAM( 0 ); //Open the camera if (!capture ){ printf("Could not connect to cameran" ); return 1; } CvVideoWriter *writer; int isColor = 1; int fps = 25; // or 30 writer = cvCreateVideoWriter("output.avi", CV_FOURCC_DEFAULT, fps, cvSize(640, 480)); while(key != 27 /*escape key to quit*/) { //Query for the next frame frame = cvQueryFrame( capture ); if( !frame ) break; cvWriteFrame(writer, frame); //Just like showing an image //Get the last key thats been pressed for input key = cvWaitKey( 1 ); } cvReleaseVideoWriter(&writer); //Release the video file cvReleaseCapture( &capture ); // WARNING: DO NOT release the "frame" variable return 0;}
  • 11. Accessing a pixel{ int i = frame->height/2; int j = frame->width/2; //slow way CvScalar bgr = cvGet2D(frame, i, j); printf("B:%d, G:%d, R:%dn", (int)bgr.val[0], (int)bgr.val[1], (int)bgr.val[2]); //fast way (access the array directly) printf("B:%d, G:%d, R:%dn", ((uchar *)(frame->imageData + i*frame->widthStep))[j*frame->nChannels + 0], // B ((uchar *)(frame->imageData + i*frame->widthStep))[j*frame->nChannels + 1], // G ((uchar *)(frame->imageData + i*frame->widthStep))[j*frame->nChannels + 2]);// R //set the value cvSet2D(frame, i, j, CV_RGB(0,255,0)); cvShowImage( "demowin2", frame );}
  • 12. Transforming color images• Grayscale • 0.3*R+0.59*G+0.11*B, luminance IplImage* grayscaleImg = cvCreateImage(cvSize(640,480), 8/*depth*/, 1/*channels*/); { cvCvtColor(frame, grayscaleImg, CV_BGR2GRAY); cvShowImage( "demowin2", grayscaleImg ); }• HSV • What we perceive as color, rather than “sense” as color (well… sort of) • Hue: the color value • Saturation: the richness of the color relative to brightness • Value: the gray level intensity IplImage* singleChannelImg = cvCreateImage(cvSize(640,480), 8/*depth*/, 1/*channels*/); IplImage* hsvImg = cvCreateImage(cvSize(640,480), 8/*depth*/, 3/*channels*/); { cvCvtColor(frame, hsvImg, CV_BGR2HSV); cvSplit(hsvImg, singleChannelImg, NULL, NULL, NULL); // get the "Hue" component cvShowImage( "demowin3", singleChannelImg ); }
  • 13. Reducing image size For reducing image size quickly (e.g., to increase frame rate) • Pyramid • nearest neighbor (no interpolation) • integer multiple down-sampling (throws rows and columns) • Anti-aliasing built inIplImage* smallerImg = cvCreateImage( cvSize(640/2,480/2), 8/*depth*/, 3/*channels*/);cvPyrDown( frame, smallerImg );
  • 14. Image filteringFor example low pass filtering cvSmooth(frame, frame, CV_GAUSSIAN, 15, 15); //size=15x15CV_BLUR_NO_SCALE all ones (has ringing)CV_BLUR all ones, sum to one (has ringing)CV_GAUSSIAN a gaussian smoothing kernel (no ringing if large enough)CV_MEDIAN choose the median of the block (non-linear)CV_BILATERAL “perceptual closeness” (texture/color) of the pixels
  • 15. Edge DetectionThe Canny edge detector //Requires a single-channel image: cvCvtColor(frame, grayscaleImg, CV_BGR2GRAY); //Reduce the image to the edges detected: cvCanny(grayscaleImg, edgesImg, hyst1, hyst2, sobelSize, l2needed?); 1. Image input 2. 3. 4. 5. Smoothing Gradient Local Hysteresis Maxima Thresholding Gx,y From magnitude hyst1 = image and direction of hyst2 = along lines Gx,y
  • 16. Hough Circles //Requires a single-channel image: cvCvtColor(frame, grayscaleImg, CV_BGR2GRAY); //Want to smooth it to get less noise cvSmooth(grayscaleImg, grayscaleImg, CV_GAUSSIAN, 7, 9 ); //Detect the circles in the image CvSeq* circles = cvHoughCircles(grayscaleImg, storage, CV_HOUGH_GRADIENT, downsampling, minCenterDist, CannyThresh, AccumThresh minRadius, maxRadius );• In General, Hough transform consists of 1. Edge detection 2. Accumulation in parameter space For each detected point, draw shape with different parameter Look for local maxima For a circle space is (x,y,R) R=5 R=10
  • 17. Morphology Primer• Pick a structuring element • Cross • Disk • Rectangle• Pick an operation • Dilation (Max, “OR”) • Erosion (Min, “AND”) • Opening (E+D) • Closing (D+E)
  • 18. Torso TrackingKe-Yu Chen, 2012/02/15
  • 19. Project goal • Track the human torso in real time from a single general- purpose camera 19
  • 20. Motivation • High-tech camera is not available • Only have 2D sequences 20
  • 21. Algorithm 1. BackgroundVideo ModelSource Construction 2. 3. 4. 5. Silhouette Foreground Upper body Distance extraction Object Extraction Identification Transform 21
  • 22. 1. Background Model Construction 1. BackgroundVideo ModelSource Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extraction … … 0~50 51~60 Background model 22
  • 23. 2. Foreground Object Extraction 1. BackgroundVideo ModelSource Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extraction Fragmented pixels = - Shadow 23
  • 24. 2-1. Fill the fragmented pixles 1. BackgroundVideo ModelSource Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extraction Closing by Reconstruction [5] 24
  • 25. 2-2. Shadow detection [4] 1. Background Video Model Source Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extractionR G Shadow pixel B 25
  • 26. 3. Upper body identification 1. BackgroundVideo ModelSource Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extraction Th = 50% Th * HEIGHT Th = 60% Th = 70% 26
  • 27. 4. Distance Transform 1. BackgroundVideo ModelSource Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extraction DT [1] Poisson [3] 1 1 1 1 1 1 2 2 2 1 1 2 3 2 1 1 2 2 2 1 w(x, y) = 1 + ¼ * ( w(x+1, y) + w(x, y+1) + w(x-1, y) + w(x, y -1) ) 1 1 1 1 1 27
  • 28. 5. Silhouette Extraction 1. BackgroundVideo ModelSource Construction 2. Foreground Object 3. Upper body 4. Distance 5. Silhouette Extraction Identification Transform extraction Threshold = 0.55 28
  • 29. Smooth the result• Improve shaking results t-3 t-2 t-1 t + = Final Rectangular of the torso 0.4 0.6 29
  • 30. Demohttp://youtu.be/DWy3ac9HoKY 30
  • 31. Reference1. Distance transform: http://en.wikipedia.org/wiki/Distance_transform2. Siddiqi, K.; Member, S. & Kimia, B. B. Parts of Visual Form: Computational Aspects IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17, 239-2513. Gorelick, L.; Galun, M.; Sharon, E.; Basri, R.; Society, I. C.; Society, I. C. & Br, A. Shape representation and classification using the poisson equation In In Proc. of CVPR’04, 2004, 61-674. Horprasert, T.; Harwood, D. & Davis, L. S. A statistical approach for real- time robust background subtraction and shadow detection 1999, 1-15. Closing by Reconstruction: http://artis.imag.fr/Members/Adrien.Bousseau/morphology/morphoma th.pdf 31
  • 32. Future work • Dynamic threshold for shadow detection • Track multiple objects • Program to get ground truth (4 corners of the torso) • Adaptive morphological closing operation 32
  • 33. Questions? 33