Technical notes‎ > ‎

Tutorial - part 2: Use OpenCV Nonfree module (SIFT, SURF) in Android Applications via JNI

In the previous tutorial, we have demonstrated how to compile nonfree module in OpenCV into Android NDK libraries. 

In this article, I will show how to use JNI interface to call the NDK functions in the JAVA code of an Android applications. I will reuse some of the project files from the previous tutorial.

The following is a step-by-step tutorial, in which we will build a simple Android application with only one "run demo" button, as is shown in the figure below. We still use SIFT as an example. The main program performing SIFT processing is almost the same as the one in the previous tutorial. I only made some necessary changes for the JNI interface. 
The demo application reads input image from "/sdcard/nonfree/img1.jpg", and write the output image to "sdcard/nonfree/img1_results.jpg". 

Environment setup:

- Windows 7 Pro X64
- cygwin
- adt-bundle-windows-x86-20130522
- android-ndk-r9 (install path: D:\Android_dev\android-ndk-r9)
- OpenCV-2.4.8-android-sdk (install path: D:\CV_dev\OpenCV-2.4.8-android-sdk)

1. Create new Android application project. 
Start Eclipse. Create an Android project: nonfree-jni-demo. The package name is com.example.nonfreejnidemo.

2. Create native JNI code files and makefiles
In the Android application folder, create a folder "jni", which contains:

jni folder

- Android.mk
- Application.mk
- libnonfree.so
- libopencv-java.so
- nonfree_jni.cpp

Application.mk is the same as in the previous tutorial. 

Application.mk

APP_ABI := armeabi 
#APP_ABI += armeabi-v7a
APP_STL := gnustl_static
APP_CPPFLAGS := -frtti -fexceptions
APP_PLATFORM := android-15

We modified Android.mk, since after compiling the JNI code, we want a shared library.

Android.mk

LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE    := nonfree_prebuilt
LOCAL_SRC_FILES := libnonfree.so
include $(PREBUILT_SHARED_LIBRARY)

include $(CLEAR_VARS)
LOCAL_MODULE    := opencv_java_prebuilt
LOCAL_SRC_FILES := libopencv_java.so
include $(PREBUILT_SHARED_LIBRARY)

include $(CLEAR_VARS)
# Modify LOCAL_C_INCLUDES with your path to OpenCV for Android.
LOCAL_C_INCLUDES:= /cygdrive/d/CV_dev/OpenCV-2.4.8-android-sdk/sdk/native/jni/include
LOCAL_MODULE    := nonfree_jni
LOCAL_CFLAGS    := -Werror -O3 -ffast-math
LOCAL_LDLIBS    += -llog -ldl 
LOCAL_SHARED_LIBRARIES := nonfree_prebuilt opencv_java_prebuilt
LOCAL_SRC_FILES := nonfree_jni.cpp
include $(BUILD_SHARED_LIBRARY)

nonfree_jni.cpp file defines JNI interface. We also implement major SIFT processing here, in which we called OpenCV SIFT functions.

nonfree_jni.cpp

// The major functions performing the SIFT processing
int run_demo()
{
    const char * imgInFile = "/sdcard/nonfree/img1.jpg";
const char * imgOutFile = "/sdcard/nonfree/img1_result.jpg";

Mat image;
image = imread(imgInFile, CV_LOAD_IMAGE_COLOR);

vector<KeyPoint> keypoints;
Mat descriptors;

SiftFeatureDetector detector;
detector.detect(image, keypoints);
detector.compute(image,keypoints, descriptors);
       /* Some other processing, please check the download package for details. */

return 0;
}

// JNI interface functions, be careful about the naming.
extern "C" 
{
    JNIEXPORT void JNICALL Java_com_example_nonfreejnidemo_NonfreeJNILib_runDemo(JNIEnv * env, jobject obj);
};

JNIEXPORT void JNICALL Java_com_example_nonfreejnidemo_NonfreeJNILib_runDemo(JNIEnv * env, jobject obj)
{
    run_demo();
}

3. Compile libraries
In the project root folder, type "ndk-build" to compile the libraries. After this step, you should be able to see three library files under \nonfree-jni-demo\libs\armeabi:
- libnonfree.so
- libnonfree_jni.so
- libopencv_java.so

4. Create JNI interface class
Add a new class NonfreeJNILib.

NonfreeJNILib.java

package com.example.nonfreejnidemo;
public class NonfreeJNILib {
    static 
    {
    try
    { 
    // Load necessary libraries.
    System.loadLibrary("opencv_java");
    System.loadLibrary("nonfree");
    System.loadLibrary("nonfree_jni");
    }
    catch( UnsatisfiedLinkError e )
{
           System.err.println("Native code library failed to load.\n" + e);
}
    }
    public static native void runDemo();
}

5. Implement the Android application.
In the layout editor, add a button with text "Rum Demo".
In MainActivity.java, we add event function for the click button to start the SIFT processing.

MainActivity.java

runBtn = (Button)findViewById(R.id.button_run_demo);
runBtn.setOnClickListener(new Button.OnClickListener()
{
@Override
public void onClick(View v) 
{
// Call the JNI interface
NonfreeJNILib.runDemo();
}
});

Finally, add permission to write the external storage to the Android applications. Add the following code in AndroidManifest.xml. (without this, we cannot write the results image into sdcard. If you don't write card, you can skip this step)
 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

6. Run the application.
Create /sdcard/nonfree folder on your test device. adb push nonfree-jni-demo/img/img1.jpg into this folder. Then start the application. Click the "run demo" button, the SIFT processing will start. Since this is a simple demo, I didn't implement any multi-threading scheme. During the SIFT processing, the GUI of the application will freeze. You may want to use multi-threading and message passing in a real application to enhance the GUI experience. 
Once the processing is done, you will see the results on sdcard. You can also check some intermediate information using logcat.

Resources:
2. If you only need SIFT in your project and don't need other functions from OpenCV, you can also consider EZSIFT. Find more information here: http://sourceforge.net/projects/ezsift/


Disqus comments