Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Build & Use OpenCL on OpenCV & Android NDK

7,770 views

Published on

This manual is “How to Build” manual for OpenCV with OpenCL for Android.
If you want to “Use OpenCL on OpenCV” ONLY,
Please see
http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSample

Published in: Technology

How to Build & Use OpenCL on OpenCV & Android NDK

  1. 1. ©SIProp Project, 2006-2008 1 How to Build & Use OpenCL on OpenCV & Android NDK Noritsuna Imamura noritsuna@siprop.org
  2. 2. ©SIProp Project, 2006-2008 2 !!!!!!ATTENTION!!!!!! This manual is “How to Build” manual. If you want to “Use OpenCL on OpenCV” ONLY, Please see http://github.com/noritsuna/OpenCVwithOpenCL4 AndroidNDKSample
  3. 3. How to Build OpenCL on OpenCV ©SIProp Project, 2006-2008 3 for Android NDK
  4. 4. ©SIProp Project, 2006-2008 4 Download OpenCV & NDK tool › wget https://github.com/Itseez/opencv/archive/ 2.4.10.1tar.gz › tar zxfv 2.4.10.1.tar.gz › mkdir ~/android/ › cd ~/android/ › wget http://dl.google.com/android/android-sdk_ r23.0.2-linux.tgz › wget http://dl.google.com/android/ndk/android-ndk- r10c-linux-x86_64.bin › tar zxfv android-sdk_r23.0.2-linux.tgz › ./android-ndk-r10c-linux-x86_64.bin
  5. 5. ©SIProp Project, 2006-2008 5 Edit Setup File ./opencv- 2.4.10.1/platforms/scripts/cmake_android_arm.s h Add WITH_EIGEN=off option. Currently version has some bugs. › cmake -DWITH_EIGEN=OFF - DCMAKE_BUILD_WITH_INSTALL_RPATH=ON - DCMAKE_TOOLCHAIN_FILE=../android/android.toolchain.c make $@ ../..
  6. 6. ©SIProp Project, 2006-2008 6 Set Path to SDK/NDK › export ANDROID_NDK=~/android/android-ndk- r10c › export ANDROID_SDK=~/android/android-sdk- linux
  7. 7. Add “SO loading“ func for Android 1/3 Change “SO loading“ func for Linux to this func without Android. ~/opencv- 2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp ©SIProp Project, 2006-2008 7 › #if defined(__linux__) ↓ › #if defined(__linux__) && !defined(__ANDROID__)
  8. 8. Add “SO loading“ func for Android 2/3 Add “SO loading“ func for Android. (continue to next page) ~/opencv- 2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp ©SIProp Project, 2006-2008 8 › #if defined(__ANDROID__) › #include <dlfcn.h> › #include <sys/stat.h> › #if defined(__ARM_ARCH_8A__) || defined(_X64_) › static const char *default_so_paths[] = { › "/system/lib64/libOpenCL.so", › "/system/vendor/lib64/libOpenCL.so", › "/system/vendor/lib64/egl/libGLES_mali.so" › }; › #else › static const char *default_so_paths[] = { › "/system/lib/libOpenCL.so", › "/system/vendor/lib/libOpenCL.so", › "/system/vendor/lib/egl/libGLES_mali.so" › }; › #endif ›
  9. 9. Add “SO loading“ func for Android 3/3 ©SIProp Project, 2006-2008 9 › static int access_file(const char *filename) › { › struct stat buffer; › return (stat(filename, &buffer) == 0); › } › static void* GetProcAddress (const char* name) › { › static void* h = NULL; › unsigned int i; › if (!h) › { › const char* name; › for(i=0; i<(sizeof(default_so_paths)/sizeof(char*)); i++) › { › if(access_file(default_so_paths[i])) { › name = (char *)default_so_paths[i]; › h = dlopen(name, RTLD_LAZY); › if (h) break; › } › } › if (!h) › return NULL; › } › return dlsym(h, name); › } › #define CV_CL_GET_PROC_ADDRESS(name) GetProcAddress(name) › #endif
  10. 10. › cd ~/opencv-2.4.10.1/platforms/scripts/ › ./cmake_android_arm.sh ©SIProp Project, 2006-2008 10 Execute Auto Config & make This Script make Makefile for OpenCV. In “~/opencv- 2.4.10.1/platforms/scripts/build_android_arm.sh” Make !!! › cd ~/opencv- 2.4.10.1/platforms/scripts/build_android_arm / › make
  11. 11. ©SIProp Project, 2006-2008 11 Where are Made Files? OpenCV Libs. In “~/opencv- 2.4.10.1/platforms/build_android_arm/lib” highgui core etc... 3rd Party Libs for OpenCV. In “~/opencv- 2.4.10.1/platforms/build_android_arm/3rdparty/lib” libIlmImf libjasper libjpeg libpng libtiff
  12. 12. ©SIProp Project, 2006-2008 12 Copy libs to sample program OpenCV Libs. cp ~/opencv- 2.4.10.1/platforms/build_android_arm/lib/armeabi-v7a/* ~/opencv-2.4.10.1/samples/android/native-activity/ libs_opencv/ 3rd Party Libs for OpenCV. cp ~/opencv- 2.4.10.1/platforms/build_android_arm/3rdparty/lib/ar meabi-v7a/* ~/opencv- 2.4.10.1/samples/android/native-activity/ libs_opencv/
  13. 13. How to Use OpenCL on OpenCV for Android NDK(Native App) ©SIProp Project, 2006-2008 13
  14. 14. ©SIProp Project, 2006-2008 14 Download Sample Source Code http://github.com/noritsuna/OpenCVwithOpenCL4Andr oidNDKSample Based OpenCV-2.4.10.1 & Android 4.4 Made Android Native Activity. Convert RGB Color Image to Black&White Image.
  15. 15. Call Stack Call as C/C++ APIs Call as SysCall(C/ASM) ©SIProp Project, 2006-2008 15 Native Application NDK wo/ADT Standard Android Application for C/C++ Only C/C++ on Limited Library Layer Advantage Only C/C++ DirectCall C/C++ API Dis-Advantage Use a few Android Tools A few Docs from Google Developer Site & Blogs APK File(Your Application) (C/C++) Library Layer (C/C++) Kernel/Driver Layer (C/ASM)
  16. 16. How to Make this Sample Source Code from OpenCV’s Sample Project ©SIProp Project, 2006-2008 16
  17. 17. ©SIProp Project, 2006-2008 17 Use NativeActivity of OpenCV This Sample uses Only C/C++ Code. ~/opencv-2.4.10.1/samples/android/native-activity Structure of NativeActivity AndroidManifest.xml <- Property File for Android res <- Directory that has Resource file(Image,Voice) jni <- Directory that has files for NDK Android.mk <- make file of NDK Application.mk <- property file of NDK native.cpp <- Sample App Other Files/Dir, don’t use.
  18. 18. ©SIProp Project, 2006-2008 18 Edit AndroidManifest.xml Replace “application” Section 1. <application android:label="@string/app_name" android:hasCode="false"> 2. <activity android:name="android.app.NativeActivity" 3. android:label="@string/app_name" 4. android:configChanges="orientation|keyboardHidden"> 5. <!-- Tell NativeActivity the name of or .so --> 6. <meta-data android:name="android.app.lib_name" 7. android:value="native_activity" /> 8. <intent-filter> 9. <action android:name="android.intent.action.MAIN" /> 10. <category android:name="android.intent.category.LAUNCHER" /> 11. </intent-filter> 12. </activity> 13. </application>
  19. 19. 2. <uses-sdk android:minSdkVersion=“9" /> ©SIProp Project, 2006-2008 19 About AndroidManifest.xml API Level 1. <!-- This is the platform API where NativeActivity was introduced. --> Permission 1. <uses-permission android:name="android.permission.CAMERA "/>
  20. 20. ©SIProp Project, 2006-2008 20 Create jni/Android.mk 1/9 1. Delete jni/Android.mk & Create the new jni/Android.mk 2. Set LOCAL_PATH & include 1. LOCAL_PATH := $(call my-dir) 2. include $(CLEAR_VARS)
  21. 21. ©SIProp Project, 2006-2008 21 Create jni/Android.mk 2/9 1. Set OpenCV’s Modules & Libs 1. OPENCV_MODULES:=contrib legacy ml stitching superres objdetect ts videostab video photo calib3d features2d highgui imgproc flann ocl core androidcamera 2. OPENCV_3RDPARTY_COMPONENTS:=asse tmanager tbb libjpeg libpng libtiff libjasper IlmImf 3. OPENCV_CAMERA_MODULES:= native_camera_r2.2.0 native_camera_r2.3.3 native_camera_r3.0.1 native_camera_r4.0.0 native_camera_r4.0.3 native_camera_r4.1.1 native_camera_r4.2.0 native_camera_r4.3.0 native_camera_r4.4.0
  22. 22. LOCAL_SRC_FILES:=../libs_opencv/libopenc v_$1.a 5. include $(PREBUILT_STATIC_LIBRARY) 6. endef ©SIProp Project, 2006-2008 22 Create jni/Android.mk 3/9 Set Loading OpenCV Libs Function 1. define add_opencv_module 2. include $(CLEAR_VARS) 3. LOCAL_MODULE:=opencv_$1 4. 1. $(foreach module,$(OPENCV_MODULES),$(eval $(call add_opencv_module,$(module))))
  23. 23. 1. define add_opencv_3rdparty_component 2. include $(CLEAR_VARS) 3. LOCAL_MODULE:=$1 4. LOCAL_SRC_FILES:=../libs_opencv/lib$1.a 5. include $(PREBUILT_STATIC_LIBRARY) 6. endef ©SIProp Project, 2006-2008 23 Create jni/Android.mk 4/9 Set Loading OpenCV Libs Function 1. $(foreach module,$(OPENCV_3RDPARTY_COMPONEN TS),$(eval $(call add_opencv_3rdparty_component,$(module))))
  24. 24. LOCAL_SRC_FILES:=../libs_opencv/lib$1.so 5. include $(PREBUILT_SHARED_LIBRARY) 6. endef ©SIProp Project, 2006-2008 24 Create jni/Android.mk 5/9 Set Loading AndroidCamera Libs Function 1. define add_opencv_camera_module 2. include $(CLEAR_VARS) 3. LOCAL_MODULE:=$1 4. 1. $(foreach module,$(OPENCV_CAMERA_MODULES),$(e val $(call add_opencv_camera_module,$(module))))
  25. 25. ©SIProp Project, 2006-2008 25 Create jni/Android.mk 6/9 Set loading info lib 1. include $(CLEAR_VARS) 2. LOCAL_MODULE:=opencv_info 3. LOCAL_SRC_FILES:=../libs_opencv/libopenc v_info.so 4. include $(PREBUILT_SHARED_LIBRARY)
  26. 26. ©SIProp Project, 2006-2008 26 Create jni/Android.mk 7/9 Set Include Dir Set Module name & Source Code OpenCVwithOpenCL4AndroidNDKNativeActivity.cpp 1. include $(CLEAR_VARS) 2. LOCAL_C_INCLUDES+=./jni/include 3. LOCAL_MODULE := OpenCV_CL 4. LOCAL_SRC_FILES := OpenCVwithOpenCL4AndroidNDKNativeActi vity.cpp
  27. 27. ©SIProp Project, 2006-2008 27 Create jni/Android.mk 8/9 Set Loading Lib & OpenCV Lib LOCAL_LDLIBS -lm -llog -lc -ldl -lz -landroid LOCAL_STATIC_LIBRARIES $(foreach mod, $(OPENCV_MODULES), opencv_$(mod)) $(OPENCV_3RDPARTY_COMPONENTS) 1. LOCAL_LDLIBS += -lm -llog -lc -ldl -lz - landroid 2. LOCAL_STATIC_LIBRARIES := android_native_app_glue $(foreach mod, $(OPENCV_MODULES), opencv_$(mod)) $(OPENCV_3RDPARTY_COMPONENTS)
  28. 28. ©SIProp Project, 2006-2008 28 Create jni/Android.mk 9/9 Set Library Type Make SO lib BUILD_SHARED_LIBRARY Set Loading Android module Use “native_app_glue” module $(call import-module,android/native_app_glue) 1. include $(BUILD_SHARED_LIBRARY) 2. $(call import-module, android/native_app_glue)
  29. 29. ©SIProp Project, 2006-2008 29 Setup/Make Build.xml 1/2 Making Setting File for Android using ANT Today, Development on CUI Only!!! Don’t use Eclipse! › android list target › Available Android targets: › ---------- › id: 1 or "android-9" › Name: Android 2.3.1 › Type: Platform › API level: 9 › Revision: 2 › Skins: WQVGA400, QVGA, HVGA, WVGA800 (default), WVGA854 › ABIs : armeabi › ---------- › id: 2 or "android-18" › Name: Android 4.3 › Type: Platform › API level: 18 › Revision: 2 › Skins: WXGA800, WQVGA400, QVGA, WQVGA432, WXGA800-7in, WVGA800 (default), HVGA, WXGA720, WSVGA, WVGA854 › ABIs : armeabi-v7a › ----------
  30. 30. ©SIProp Project, 2006-2008 30 Setup/Make Build.xml 2/2 Make Build.xml › android update project --path . --target 2
  31. 31. ©SIProp Project, 2006-2008 31 Build NativeActivity Build C/C++ Source Codes › ndk-build [clean all] Make APK File › ant [debug|release]
  32. 32. › adb install bin/NativeActivity-debug.apk › adb uninstall com.example.native_activity ©SIProp Project, 2006-2008 32 Install NativeActivity.apk Install APK File UnInstall APK File Show Log › adb logcat
  33. 33. ©SIProp Project, 2006-2008 33 adb(Android Debug Bridge) Functions Shell on Android Device adb shell Show Connected Android Devices adb devices Install/Uninstall APK File adb install/uninstall [App Name]/[Package Name] Send/Get File to/from Android Device adb push/pull [from File Path] [to File Path] Show Log (DDMS) adb logcat
  34. 34. ©SIProp Project, 2006-2008 34 Quick Explanation to OpenCL 1. void android_main(android_app* app) { 2. (snip) 3. cv::Mat drawing_frame; 4. (snip) 5. colorSampleImage = imread(file_path); 6. (snip) 7. // Create Mat Object for OpenCL 8. cv::ocl::oclMat oclIn(colorSampleImage), oclOut; 9. // Change Color RGB->Black&White on OpenCL 10. cv::ocl::cvtColor(oclIn, oclOut, cv::COLOR_BGR2GRAY); 11. // Download Mat Object for OpenCL to Mat for CPU 12. oclOut.download(drawing_frame); 13. 14. engine_draw_frame(&engine, drawing_frame);
  35. 35. How to Set up OpenCL Driver adreno-3xx: Qualcomm's Adreno 3xx GPUs ©SIProp Project, 2006-2008 35 mali-t6xx: ARM's Mali-T6xx GPUs
  36. 36. ©SIProp Project, 2006-2008 36 Download OpenCL Drivers OpenCL Drivers was deleted since 4.2 adreno-3xx: Qualcomm's Adreno 3xx GPUs mali-t6xx: ARM's Mali-T6xx GPUs Download from: https://github.com/madeye/opencl-android-proprietary
  37. 37. ©SIProp Project, 2006-2008 37 Set up Drivers to your Android Ex. adreno-3xx: Qualcomm's Adreno 3xx GPUs 1. $ adb push kitkat/system/vendor/lib /sdcard/ 2. $ adb shell 3. $ su 4. # mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system 5. # cp /sdcard/libOpenCL.so /system/vendor/lib/ 6. # cp /sdcard/libllvm-a3xx.so /system/vendor/lib/ 7. # chmod 644 /system/vendor/lib/libOpenCL.so 8. # chmod 644 /system/vendor/lib/libllvm-a3xx.so

×