Download as PDF, PPTX
![UMat
• UMat is new type of array that wraps clmem when OpenCL is available
• Replacing Mat with UMat is often the only change needed
#include "opencv2/opencv.hpp"
using namespace cv;
int main(int argc, char** argv)
{
Mat img, gray;
img = imread(argv[1], 1);
imshow("original", img);
cvtColor(img, gray,
COLOR_BGR2GRAY);
GaussianBlur(gray, gray,
Size(7, 7), 1.5);
Canny(gray, gray, 0, 50);
imshow("edges", gray);
waitKey();
return 0;
}
#include "opencv2/opencv.hpp"
using namespace cv;
int main(int argc, char** argv)
{
UMat img, gray;
img = imread(argv[1]).getUMat();
imshow("original", img);
cvtColor(img, gray,
COLOR_BGR2GRAY);
GaussianBlur(gray, gray,
Size(7, 7), 1.5);
Canny(gray, gray, 0, 50);
imshow("edges", gray);
waitKey();
return 0;
}](https://image.slidesharecdn.com/e04opencvbradsk-150811160253-lva1-app6892/85/The-OpenCV-Open-Source-Computer-Vision-Library-Latest-Developments-a-Presentation-from-the-OpenCV-Foundation-13-320.jpg)
Uploaded byEdge AI and Vision Alliance
"The OpenCV Open Source Computer Vision Library: Latest Developments," a Presentation from the OpenCV Foundation
OpenCV 3.0, led by Gary Bradski, offers significant improvements over previous versions with a modular structure, enhanced performance, and support for various hardware accelerations. It introduces a transparent API for easier use of OpenCL, while maintaining compatibility with existing algorithms and featuring a contributions repository for new modules. The aim is to provide a stable and efficient platform for computer vision and machine learning, backed by strong community and corporate support.
More Related Content
Similar to "The OpenCV Open Source Computer Vision Library: Latest Developments," a Presentation from the OpenCV Foundation
"The OpenCV Open Source Computer Vision Library: Latest Developments," a Presentation from the OpenCV Foundation
- 1. OpenCV 3.0 Gary Bradski ChiefScientist, Perception and AI at Magic Leap CEO, OpenCV.org
- 2. OpenCV at glance •BSD license, 10M downloads, 500K+ lines of code • Huge community involvement, automated patch testing and integration process • Runs everywhere SSE, NEON, IPP, OpenCL, CUDA, OpenCV4Tegra, … core, imgproc, objdetect … OpenCV HAL OpenCV face, text, rgbd, … OpenCV Contrib Bindings: Python, Java Samples, Apps, Solutions • Find more at http://opencv.org (user) • Or http://code.opencv.org (developer)
- 3. Recent Stats NOTE: This isonly for source forge. Many more downloads come from Git and many more come on Unix distros.
- 4. OpenCV History Willow Support OpenCV Foundation IntelSupport Google Summer of Code Nvidia Support Renewed Intel Support Magic Leap
- 5. OpenCV Algorithm ModulesOverview 5 Image Processing Object recognition Machine learning Transforms Calibration Features VSLAM Fitting Optical Flow Tracking Depth, Pose Normals, Planes, 3D Features Computational Photography CORE: Data structures, Matrix math, Exceptions etc Segmentation HighGUI: I/O, Interface
- 6.
- 7.
- 8. OpenCV 3.0 atglance • Mostly compatible with OpenCV 2.x; OpenCV 1.x C API is deprecated and partially removed • Highlights: – even more modular and extendible – very stable API tailored for a long-term support – decent out-of-box performance thanks to IPP, OpenCL (T-API) and NEON instructions – lot’s of new functionality! Aug’14 Nov’14 Dec’14 May’15 3.0 alpha 3.0 beta 3.0rc 3.0 3.1 Q3’15 Goal of 3.0: make a better OpenCV 2.0, cleanup API, get better performance (with T-API, IPP, NEON), shift to modular structure and enable user contributions. – Vadim Pisarevsky
- 9. OpenCV => OpenCV+ opencv_contrib OpenCV 2.x OpenCV 3.x contributions OpenCV 3.x • OpenCV 3.0 includes mature and clean algorithms that are fully supported • A separate contribution repository is for new CV algorithms that people want to share: http://github.com/itseez/opencv_contrib • Patches to the contrib repository are tested as well by our buildbot to ensure integrity!
- 10. Using opencv_contrib The modulesin contrib have the same structure as the standard ones: $ cmake –D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules … opencv/ modules/ core/ include/, doc/, src/, test/, … CMakeLists.txt imgproc … opencv_contrib/modules text/ include/, doc/, src/, test/, … CMakeLists.txt … Path to the contrib modules can be passed to cmake to build them together with OpenCV:
- 11. New-style C++ API •All the high-level vision algorithms (face detectors, optical flow estimators, stereo matchers etc.) are declared as pure abstract classes, similarly to interfaces in Java • Smart pointers to instances are retrieved using special “factory” methods • All the implementation details are hidden • The saved space in headers is taken up instead by the verbose Doxygen comments in order to generate an always up-to-date reference manual. class StereoMatcher : public Algorithm // Algorithm is used as a base { public: // common methods for all stereo matchers virtual void compute(InputArray left, InputArray right, OutputArray disparity) = 0; … }; class StereoBM : public StereoMatcher { public: // specific methods for particular algorithm virtual void setUniquenessRatio(int ratio) = 0; … // factory method static Ptr<StereoBM> create(…); }; … Ptr<StereoBM> matcher = StereoBM::create(…); matcher->compute(left, right, disparity);
- 12. Transparent API (T-API)for HW Specializations • single API entry for each function/algorithm – no specialized cv::Canny, ocl::Canny, gpu::Canny etc. • uses dynamically loaded OpenCL runtime if available; otherwise falls back to CPU code. Dispatching is at runtime, no recompilation needed! • minimal or no changes in user code – CPU-only processing – no changes should be required • includes the following key components: – new data structure UMat – simple and robust mechanism for async processing – bonus: very convenient API for implementing custom OpenCL kernels
- 13. UMat • UMat isnew type of array that wraps clmem when OpenCL is available • Replacing Mat with UMat is often the only change needed #include "opencv2/opencv.hpp" using namespace cv; int main(int argc, char** argv) { Mat img, gray; img = imread(argv[1], 1); imshow("original", img); cvtColor(img, gray, COLOR_BGR2GRAY); GaussianBlur(gray, gray, Size(7, 7), 1.5); Canny(gray, gray, 0, 50); imshow("edges", gray); waitKey(); return 0; } #include "opencv2/opencv.hpp" using namespace cv; int main(int argc, char** argv) { UMat img, gray; img = imread(argv[1]).getUMat(); imshow("original", img); cvtColor(img, gray, COLOR_BGR2GRAY); GaussianBlur(gray, gray, Size(7, 7), 1.5); Canny(gray, gray, 0, 50); imshow("edges", gray); waitKey(); return 0; }
- 14. Transparent API: underthe hood bool _ocl_cvtColor(InputArray src, OutputArray dst, int code) { static ocl::ProgramSource oclsrc(“//cvtcolor.cl source coden …”); UMat src_ocl = src.getUMat(), dst_ocl = dst.getUMat(); if (code == COLOR_BGR2GRAY) { // get the kernel; kernel is compiled only once and cached ocl::Kernel kernel(“bgr2gray”, oclsrc, <compile_flags>); // pass 2 arrays to the kernel and run it return kernel.args(src, dst).run(0, 0, false); } else if(code == COLOR_BGR2YUV) { … } return false; // OpenCL function does not have to support all modes } void _cpu_cvtColor(const Mat& src, Mat& dst, int code) { … } // transparent API dispatcher function void cvtColor(InputArray src, OutputArray dst, int code) { dst.create(src.size(), …); if (useOpenCL(src, dst) && _ocl_cvtColor(src, dst, code)) return; // getMat() uses zero-copy if available; and with SVM it’s no op Mat src_cpu = src.getMat(); Mat dst_cpu = dst.getMat(); _cpu_cvtColor(src_cpu, dst_cpu, code); }
- 15. OpenCV+OpenCL execution model •One queue and one OpenCL device per CPU thread • Different CPU threads can share a device, but use different queues. • OpenCL kernels are executed asynchronously • cv::ocl::finish() puts the barrier in the current CPU thread. • It’s rarely needed to call cv::ocl::finish() manually. … ocl::Queue ocl::Device ocl::Queue ocl::Queue ocl::Device … … ocl::Context CPU threads
- 16. IPP + OpenCV =v. fast OpenCV • Intel gave us and our users free (as in “beer”) and royalty-free subset of IPP 8.x (IPPICV). • IPPICV is linked into OpenCV at compile stage and replaces the corresponding low-level C code. • Our buildbot ensures that all the tests pass
- 17. New Functionality and otherimprovements • Results from 20+ successful projects from GSoC 2013, 2014: – Computational photography, Text detection, Object Tracking, Matlab bindings etc. • 1000+ Pull Requests @ github (200+ PR’s between alpha & beta) • 18 new OpenCV modules! (mostly in opencv_contrib)
- 18. OpenCV QA Contribution/patch workflow: seeOpenCV wiki build.opencv.org: buildbot with 50+ builders pullrequest.opencv.org: tests each pullrequestgithub.com/itseez/opencv
- 19. OpenCV test suite •GoogleTest-based + set of Python scripts • Thousands of unit tests • Accuracy tests • Performance tests python ../modules/ts/misc/summary.py core*.xml -f "add:.*C4" -u s Geometric mean Name of Test core core core posix posix posix x64 x64 x64 6693M 6695 6695 2011-09-08--13-13-41 2011-09-08--13-30-06 2011-09-08--13-30-06 vs core posix x64 6693M 2011-09-08--13-13-41 core_arithm__add::Size_MatType::(127x61, 8UC4) 0.000 s 0.000 s 1.00 core_arithm__add::Size_MatType::(1280x720, 8UC4) 0.004 s 0.004 s 0.99 core_arithm__add::Size_MatType::(1920x1080, 8UC4) 0.009 s 0.009 s 1.02 core_arithm__add::Size_MatType::(640x480, 8UC4) 0.001 s 0.001 s 1.00
- 20. OpenVX (Khronos HAL) OpenCVwas one of the key contributors to the new Khronos accelerated vision API: OpenVX (Hardware Acceleration Library)
- 21. The HAL +Accelerators • opencv_hal - IPP-like, fastcv-like low-level API to accelerate OpenCV for different platforms. To be extended in 3.X… • As stated, opencv_ocl module (OpenCL acceleration) is now used in other modules to give transparent acceleration (T-API) • Universal Umat structure now can be used instead of cv::Mat and OclMat. • OpenVX, as it matures, may be used as one of the HW accelerators SSE, NEON, IPP, OpenCL, CUDA, OpenCV4Tegra, … core, imgproc, objdetect … OpenCV HAL OpenCV face, text, rgbd, … OpenCV Contrib Bindings: Python, Java Samples, Apps, Solutions
- 22. New from GoogleSummer of Code 2015 • Deep network optimized execution and interoperability to existing libraries • Stereo matching improvements • Projection mapping • Improved Camera Calibration • Better AR fiducial support • Improvements to text detection and tracking • Neon optimization
- 23. Other Initiatives: CVPR Stateof the Art Vision Challenge State of the Art Vision Challenge at CVPR 2015 Our aim is to make available state of the art vision in OpenCV. We thus ran a vision challenge to meet or exceed the state of the art in various areas. We will present the results. The contest details are available at: http://code.opencv.org/projects/opencv/wiki/VisionChallenge Prizes: 1. Win: $1000; Submit code: $3000 2. Win: $1000; Submit code: $3000 3. Win: $1000; Submit code: $3000 4. Win: $1000; Submit code: $3000 5. Win: $1000; Submit code: $3000
- 24. Other Initiatives: People’s Choice:Best Paper People’s Choice: Best paper We will tally the people’s vote for best paper/paper you’d most like to see implemented. We’ll present the histogram of results which is an indication of the algorithms people are interested in overall and then list the 5 top winners. Prizes will be awarded in two stages: A modest award for winning and a larger award for presenting the code w/in 5 months as a pull request to OpenCV as Detailed here: http://code.opencv.org/projects/opencv/wiki/How_to_contribute Prizes: 1. Win: $500; Submit code: $6000 2. Win: $300; Submit code: $4000 3. Win: $100; Submit code: $3000 4. Win: $50; Submit code: $3000 5. Win: $50; Submit code: $3000
- 25. Functional Language Exploration •Proliferation of new hardware makes it hard to support code. – Let the compiler port to different hardware using a no-side effects functional language “NUML.” – Can compile NUML to C++, C, Java and Python. • NUML is an array/image comprehending functional language. – https://github.com/vpisarev/numl/tree/alt_syntax/src
- 26. Learning OpenCV 3.0 •Out with 3.0/CVPR … or late summer! For Version 3.0 Other news: Intel is now supporting OpenCV Optimizations • https://software.intel.com/en-us/opencv OpenCV release candidate is now out! • http://opencv.org/opencv-3-0-rc1.html
- 27.