This manual is “How to Build” manual for OpenCV with OpenCL for Android. If you want to “Use OpenCL on OpenCV” ONLY, Please see http://github.com/noritsuna/OpenCVwithOpenCL4AndroidNDKSample

1. ©SIProp Project, 2006-2008 1
How to Build & Use OpenCL
on OpenCV & Android NDK
Noritsuna Imamura
noritsuna@siprop.org

2. ©SIProp Project, 2006-2008 2
!!!!!!ATTENTION!!!!!!
This manual is “How to Build” manual.
If you want to “Use OpenCL on OpenCV” ONLY,
Please see
http://github.com/noritsuna/OpenCVwithOpenCL4
AndroidNDKSample

3. How to Build OpenCL on OpenCV
©SIProp Project, 2006-2008 3
for Android NDK

4. ©SIProp Project, 2006-2008 4
Download OpenCV & NDK tool
› wget
https://github.com/Itseez/opencv/archive/
2.4.10.1tar.gz
› tar zxfv 2.4.10.1.tar.gz
› mkdir ~/android/
› cd ~/android/
› wget http://dl.google.com/android/android-sdk_
r23.0.2-linux.tgz
› wget
http://dl.google.com/android/ndk/android-ndk-
r10c-linux-x86_64.bin
› tar zxfv android-sdk_r23.0.2-linux.tgz
› ./android-ndk-r10c-linux-x86_64.bin

5. ©SIProp Project, 2006-2008 5
Edit Setup File
./opencv-
2.4.10.1/platforms/scripts/cmake_android_arm.s
h
Add WITH_EIGEN=off option.
Currently version has some bugs.
› cmake -DWITH_EIGEN=OFF -
DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -
DCMAKE_TOOLCHAIN_FILE=../android/android.toolchain.c
make $@ ../..

6. ©SIProp Project, 2006-2008 6
Set Path to SDK/NDK
› export ANDROID_NDK=~/android/android-ndk-
r10c
› export ANDROID_SDK=~/android/android-sdk-
linux

7. Add “SO loading“ func for Android 1/3
Change “SO loading“ func for Linux to this func
without Android.
~/opencv-
2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp
©SIProp Project, 2006-2008 7
› #if defined(__linux__)
↓
› #if defined(__linux__)
&& !defined(__ANDROID__)

8. Add “SO loading“ func for Android 2/3
Add “SO loading“ func for Android. (continue to
next page)
~/opencv-
2.4.10.1/modules/ocl/src/cl_runtime/cl_runtime.cpp
©SIProp Project, 2006-2008 8
› #if defined(__ANDROID__)
› #include <dlfcn.h>
› #include <sys/stat.h>
› #if defined(__ARM_ARCH_8A__) || defined(_X64_)
› static const char *default_so_paths[] = {
› "/system/lib64/libOpenCL.so",
› "/system/vendor/lib64/libOpenCL.so",
› "/system/vendor/lib64/egl/libGLES_mali.so"
› };
› #else
› static const char *default_so_paths[] = {
› "/system/lib/libOpenCL.so",
› "/system/vendor/lib/libOpenCL.so",
› "/system/vendor/lib/egl/libGLES_mali.so"
› };
› #endif
›

9. Add “SO loading“ func for Android 3/3
©SIProp Project, 2006-2008 9
› static int access_file(const char *filename)
› {
› struct stat buffer;
› return (stat(filename, &buffer) == 0);
› }
› static void* GetProcAddress (const char* name)
› {
› static void* h = NULL;
› unsigned int i;
› if (!h)
› {
› const char* name;
› for(i=0; i<(sizeof(default_so_paths)/sizeof(char*)); i++)
› {
› if(access_file(default_so_paths[i])) {
› name = (char *)default_so_paths[i];
› h = dlopen(name, RTLD_LAZY);
› if (h) break;
› }
› }
› if (!h)
› return NULL;
› }
› return dlsym(h, name);
› }
› #define CV_CL_GET_PROC_ADDRESS(name) GetProcAddress(name)
› #endif

10. › cd ~/opencv-2.4.10.1/platforms/scripts/
› ./cmake_android_arm.sh
©SIProp Project, 2006-2008 10
Execute Auto Config & make
This Script make Makefile for OpenCV.
In “~/opencv-
2.4.10.1/platforms/scripts/build_android_arm.sh”
Make !!!
› cd ~/opencv-
2.4.10.1/platforms/scripts/build_android_arm
/
› make

11. ©SIProp Project, 2006-2008 11
Where are Made Files?
OpenCV Libs.
In “~/opencv-
2.4.10.1/platforms/build_android_arm/lib”
highgui
core
etc...
3rd Party Libs for OpenCV.
In “~/opencv-
2.4.10.1/platforms/build_android_arm/3rdparty/lib”
libIlmImf
libjasper
libjpeg
libpng
libtiff

12. ©SIProp Project, 2006-2008 12
Copy libs to sample program
OpenCV Libs.
cp ~/opencv-
2.4.10.1/platforms/build_android_arm/lib/armeabi-v7a/*
~/opencv-2.4.10.1/samples/android/native-activity/
libs_opencv/
3rd Party Libs for OpenCV.
cp ~/opencv-
2.4.10.1/platforms/build_android_arm/3rdparty/lib/ar
meabi-v7a/* ~/opencv-
2.4.10.1/samples/android/native-activity/
libs_opencv/

13. How to Use OpenCL on OpenCV
for Android NDK(Native App)
©SIProp Project, 2006-2008 13

14. ©SIProp Project, 2006-2008 14
Download Sample Source Code
http://github.com/noritsuna/OpenCVwithOpenCL4Andr
oidNDKSample
Based OpenCV-2.4.10.1 & Android 4.4
Made Android Native Activity.
Convert RGB Color Image to Black&White Image.

15. Call Stack
Call as C/C++ APIs
Call as SysCall(C/ASM)
©SIProp Project, 2006-2008 15
Native Application
NDK wo/ADT
Standard Android
Application for C/C++
Only C/C++ on Limited
Library Layer
Advantage
Only C/C++
DirectCall C/C++ API
Dis-Advantage
Use a few Android Tools
A few Docs from Google
Developer Site & Blogs
APK File(Your Application)
(C/C++)
Library Layer
(C/C++)
Kernel/Driver Layer
(C/ASM)

16. How to Make this Sample Source Code
from OpenCV’s Sample Project
©SIProp Project, 2006-2008 16

17. ©SIProp Project, 2006-2008 17
Use NativeActivity of OpenCV
This Sample uses Only C/C++ Code.
~/opencv-2.4.10.1/samples/android/native-activity
Structure of NativeActivity
AndroidManifest.xml <- Property File for Android
res <- Directory that has Resource file(Image,Voice)
jni <- Directory that has files for NDK
Android.mk <- make file of NDK
Application.mk <- property file of NDK
native.cpp <- Sample App
Other Files/Dir, don’t use.

18. ©SIProp Project, 2006-2008 18
Edit AndroidManifest.xml
Replace “application” Section
1. <application android:label="@string/app_name"
android:hasCode="false">
2. <activity android:name="android.app.NativeActivity"
3. android:label="@string/app_name"
4.
android:configChanges="orientation|keyboardHidden">
5. <!-- Tell NativeActivity the name of or .so -->
6. <meta-data android:name="android.app.lib_name"
7. android:value="native_activity" />
8. <intent-filter>
9. <action
android:name="android.intent.action.MAIN" />
10. <category
android:name="android.intent.category.LAUNCHER" />
11. </intent-filter>
12. </activity>
13. </application>

19. 2. <uses-sdk android:minSdkVersion=“9" />
©SIProp Project, 2006-2008 19
About AndroidManifest.xml
API Level
1. <!-- This is the platform API where
NativeActivity was introduced. -->
Permission
1. <uses-permission
android:name="android.permission.CAMERA
"/>

20. ©SIProp Project, 2006-2008 20
Create jni/Android.mk 1/9
1. Delete jni/Android.mk & Create the new
jni/Android.mk
2. Set LOCAL_PATH & include
1. LOCAL_PATH := $(call my-dir)
2. include $(CLEAR_VARS)

21. ©SIProp Project, 2006-2008 21
Create jni/Android.mk 2/9
1. Set OpenCV’s Modules & Libs
1. OPENCV_MODULES:=contrib legacy ml
stitching superres objdetect ts videostab
video photo calib3d features2d highgui
imgproc flann ocl core androidcamera
2. OPENCV_3RDPARTY_COMPONENTS:=asse
tmanager tbb libjpeg libpng libtiff libjasper
IlmImf
3. OPENCV_CAMERA_MODULES:=
native_camera_r2.2.0 native_camera_r2.3.3
native_camera_r3.0.1 native_camera_r4.0.0
native_camera_r4.0.3 native_camera_r4.1.1
native_camera_r4.2.0 native_camera_r4.3.0
native_camera_r4.4.0

22. LOCAL_SRC_FILES:=../libs_opencv/libopenc
v_$1.a
5. include $(PREBUILT_STATIC_LIBRARY)
6. endef
©SIProp Project, 2006-2008 22
Create jni/Android.mk 3/9
Set Loading OpenCV Libs Function
1. define add_opencv_module
2. include $(CLEAR_VARS)
3. LOCAL_MODULE:=opencv_$1
4.
1. $(foreach
module,$(OPENCV_MODULES),$(eval $(call
add_opencv_module,$(module))))

23. 1. define add_opencv_3rdparty_component
2. include $(CLEAR_VARS)
3. LOCAL_MODULE:=$1
4.
LOCAL_SRC_FILES:=../libs_opencv/lib$1.a
5. include $(PREBUILT_STATIC_LIBRARY)
6. endef
©SIProp Project, 2006-2008 23
Create jni/Android.mk 4/9
Set Loading OpenCV Libs Function
1. $(foreach
module,$(OPENCV_3RDPARTY_COMPONEN
TS),$(eval $(call
add_opencv_3rdparty_component,$(module))))

24. LOCAL_SRC_FILES:=../libs_opencv/lib$1.so
5. include $(PREBUILT_SHARED_LIBRARY)
6. endef
©SIProp Project, 2006-2008 24
Create jni/Android.mk 5/9
Set Loading AndroidCamera Libs Function
1. define add_opencv_camera_module
2. include $(CLEAR_VARS)
3. LOCAL_MODULE:=$1
4.
1. $(foreach
module,$(OPENCV_CAMERA_MODULES),$(e
val $(call
add_opencv_camera_module,$(module))))

25. ©SIProp Project, 2006-2008 25
Create jni/Android.mk 6/9
Set loading info lib
1. include $(CLEAR_VARS)
2. LOCAL_MODULE:=opencv_info
3. LOCAL_SRC_FILES:=../libs_opencv/libopenc
v_info.so
4. include $(PREBUILT_SHARED_LIBRARY)

26. ©SIProp Project, 2006-2008 26
Create jni/Android.mk 7/9
Set Include Dir
Set Module name & Source Code
OpenCVwithOpenCL4AndroidNDKNativeActivity.cpp
1. include $(CLEAR_VARS)
2. LOCAL_C_INCLUDES+=./jni/include
3. LOCAL_MODULE := OpenCV_CL
4. LOCAL_SRC_FILES :=
OpenCVwithOpenCL4AndroidNDKNativeActi
vity.cpp

27. ©SIProp Project, 2006-2008 27
Create jni/Android.mk 8/9
Set Loading Lib & OpenCV Lib
LOCAL_LDLIBS
-lm -llog -lc -ldl -lz -landroid
LOCAL_STATIC_LIBRARIES
$(foreach mod, $(OPENCV_MODULES), opencv_$(mod))
$(OPENCV_3RDPARTY_COMPONENTS)
1. LOCAL_LDLIBS += -lm -llog -lc -ldl -lz -
landroid
2. LOCAL_STATIC_LIBRARIES :=
android_native_app_glue $(foreach mod,
$(OPENCV_MODULES), opencv_$(mod))
$(OPENCV_3RDPARTY_COMPONENTS)

28. ©SIProp Project, 2006-2008 28
Create jni/Android.mk 9/9
Set Library Type
Make SO lib
BUILD_SHARED_LIBRARY
Set Loading Android module
Use “native_app_glue” module
$(call import-module,android/native_app_glue)
1. include $(BUILD_SHARED_LIBRARY)
2. $(call import-module,
android/native_app_glue)

29. ©SIProp Project, 2006-2008 29
Setup/Make Build.xml 1/2
Making Setting File for Android using ANT
Today, Development on CUI Only!!!
Don’t use Eclipse!
› android list target
› Available Android targets:
› ----------
› id: 1 or "android-9"
› Name: Android 2.3.1
› Type: Platform
› API level: 9
› Revision: 2
› Skins: WQVGA400, QVGA, HVGA, WVGA800 (default), WVGA854
› ABIs : armeabi
› ----------
› id: 2 or "android-18"
› Name: Android 4.3
› Type: Platform
› API level: 18
› Revision: 2
› Skins: WXGA800, WQVGA400, QVGA, WQVGA432, WXGA800-7in, WVGA800
(default), HVGA, WXGA720, WSVGA, WVGA854
› ABIs : armeabi-v7a
› ----------

30. ©SIProp Project, 2006-2008 30
Setup/Make Build.xml 2/2
Make Build.xml
› android update project --path . --target 2

31. ©SIProp Project, 2006-2008 31
Build NativeActivity
Build C/C++ Source Codes
› ndk-build [clean all]
Make APK File
› ant [debug|release]

32. › adb install bin/NativeActivity-debug.apk
› adb uninstall com.example.native_activity
©SIProp Project, 2006-2008 32
Install NativeActivity.apk
Install APK File
UnInstall APK File
Show Log
› adb logcat

33. ©SIProp Project, 2006-2008 33
adb(Android Debug Bridge)
Functions
Shell on Android Device
adb shell
Show Connected Android Devices
adb devices
Install/Uninstall APK File
adb install/uninstall [App Name]/[Package Name]
Send/Get File to/from Android Device
adb push/pull [from File Path] [to File Path]
Show Log (DDMS)
adb logcat

34. ©SIProp Project, 2006-2008 34
Quick Explanation to OpenCL
1. void android_main(android_app* app) {
2. （snip）
3. cv::Mat drawing_frame;
4. (snip)
5. colorSampleImage = imread(file_path);
6. (snip)
7. // Create Mat Object for OpenCL
8. cv::ocl::oclMat oclIn(colorSampleImage), oclOut;
9. // Change Color RGB->Black&White on OpenCL
10. cv::ocl::cvtColor(oclIn, oclOut, cv::COLOR_BGR2GRAY);
11. // Download Mat Object for OpenCL to Mat for CPU
12. oclOut.download(drawing_frame);
13.
14. engine_draw_frame(&engine, drawing_frame);

35. How to Set up OpenCL Driver
adreno-3xx: Qualcomm's Adreno 3xx GPUs
©SIProp Project, 2006-2008 35
mali-t6xx: ARM's Mali-T6xx GPUs

36. ©SIProp Project, 2006-2008 36
Download OpenCL Drivers
OpenCL Drivers was deleted since 4.2
adreno-3xx: Qualcomm's Adreno 3xx GPUs
mali-t6xx: ARM's Mali-T6xx GPUs
Download from:
https://github.com/madeye/opencl-android-proprietary

37. ©SIProp Project, 2006-2008 37
Set up Drivers to your Android
Ex. adreno-3xx: Qualcomm's Adreno 3xx GPUs
1. $ adb push kitkat/system/vendor/lib /sdcard/
2. $ adb shell
3. $ su
4. # mount -o rw,remount -t yaffs2 /dev/block/mtdblock3
/system
5. # cp /sdcard/libOpenCL.so /system/vendor/lib/
6. # cp /sdcard/libllvm-a3xx.so /system/vendor/lib/
7. # chmod 644 /system/vendor/lib/libOpenCL.so
8. # chmod 644 /system/vendor/lib/libllvm-a3xx.so