This document discusses creating novel augmented reality applications for Android. It provides a brief history of mobile AR, describes some popular mobile AR SDKs and APIs including the Qualcomm AR SDK (QCAR). It discusses how to develop for QCAR using both Android Studio and Unity3D. Examples are given of games and applications created using QCAR and Unity3D. The document concludes that the future of mobile AR is markerless tracking using computer vision techniques.

1. Creating Novel Augmented Reality
Applications for Android
www.diotasoft.com
Sebastian Knödel Ludovic Perrier Yohan Hadjedi
skn@diotasoft.com lpe@diotasoft.com yha@diotasoft.com
sekond ludovic_perrier nah0y
PAUG 1

2. Content
Brief History of Mobile AR
Mobile AR SDKs & APIs
Qualcomm AR SDK (QCAR)
QCAR & Unity3D
Mobile AR Games
Future of mobile AR
PAUG 2

3. Brief History of Mobile AR
Sensor based tracking
GPS position
Compass orientation
Courtesy Wikitude Wikitude 2009
Vision based tracking
Courtesy TU Graz
Special Marker 1999 ARToolkitPlus 2007 NyARToolkit 2009
PAUG 3

4. Brief History of Mobile AR
Vision based tracking
Courtesy Qualcomm
Qualcomm AR
Courtesy Metaio
Natural Features
Diotasoft - Werber 2010 Metaio/Juneio
PAUG 4

5. Mobile AR SDKs & APIs
Proprietary Public
PAUG 5

6. Qualcomm AR SDK (QCAR)
• Library for mobile AR development Different target types
• Android only
• Supported devices Qualcomm CPU
(Nexus One/S, HTC Desire)
• SDK with Eclipse integration
PAUG 6

7. Workflow QCAR
Mobile application
Target Images (jpg/png)
App Logic & Rendering
Engine
Target Position
QDevNet
QCAR library
Target Management
Application
Target.dat Config.xml Target Resources
PAUG 7

8. QCAR API
Virtual Multi Image Image
Button Target Target
Marker Developer Application
Tracker
Camera Rendering
Android
PAUG 8

9. Developing with QCAR
Initialize
Java
SDK myApp extends Activity{ C/CPP
void onCreate(){ myNativeRenderFrame(JNIEnv *, jobject){
glClear(…);
Initialize_App();
Initialize_QCAR(); // Render video background:
Load QCAR::Renderer::getInstance().begin();
Tracker mLoadTrackerTask = new LoadTrackerTask(); for(i … state.getNumActiveTrackables()) {
QCAR::Trackable* trackable =
state.getActiveTrackable(i);
mRenderer = new myRenderer();
mGlView.setRenderer(mRenderer); modelViewMatrix =
Update convertPose2GLMatrix(trackable->getPose());
Camera myRenderer implements glMatrixMode(GL_MODELVIEW);
Pose GLSurfaceView.Renderer{ glLoadMatrixf(modelViewMatrix.data);
void onDrawFrame(GL10 gl){ …
myNativeRenderFrame(); …
} Add Your OpenGLES 1.1/2.0 Render Code Here
} …
}
Update GL }
Rendering // finish GL Rendering
void onPause(){ QCAR::Renderer::getInstance().end();
QCAR.onPause(); }
}
void onDestroy(){
QCAR.deinit();
Close SDK }
};
PAUG 9

10. QCAR and Unity3D
PAUG 10

11. QCAR and Unity3D
Workflow
• Create Target QDevNet
• Add target resources to project
• Add ARCamera Prefab
• Add myTarget Prefab
• Add 3D objects, physics, shaders,
Game Logic, etc.
• Deploy on Android device
PAUG 11

12. Soundplugin for Unity3D
public class JavaClass{ JavaClass.java SoundManager.java
private Activity mActivity;
public static void initSounds(Context theContext)
public JavaClass(Activity currentActivity) { {
mActivity = currentActivity; …
SoundManager.initSounds(mActivity);
SDK }
mAudioManager =
(AudioManager) mContext.getSystemService
public int ActivitySoundPlay(int id, (Context.AUDIO_SERVICE);
float volume) { }
return SoundManager.playSound(id, volume,…);
}
jint JNI_OnLoad(JavaVM* vm, void* reserved) {
NativeJavaBridge.CPP (JNI)
jclass cls_Activity = jni_env->FindClass("com/unity3d/player/UnityPlayer");
jclass cls_JavaClass = jni_env->FindClass("com/diotasoft/pluginunity/JavaClass");
activitySoundPlay = jni_env->GetMethodID(cls_JavaClass, "ActivitySoundPlay", "(IFFIF)I");
…
}
NDK int playSound(int index, float volume, int repeat, float pitch) {
jint currentStream = (jint)jni_env->CallObjectMethod(JavaClass, activitySoundPlay, index, volume,
repeat, pitch);
…
}
public class AndroidSoundPoolPlugin : MonoBehaviour {
AndroidSoundPlugin.CS (C#)
Unity [DllImport("javabridge")]
private static extern int playSound(int index, float volume, int repeat, float pitch);
[DllImport("javabridge")]
private static extern void stopSound(int currentStream);
};
PAUG 12

13. Diotasoft Games
The Circus ARLabyrinth
PAUG 13

14. Conclusions
QCAR Android SDK/NDK QCAR Unity
• Open system (hands on) • Closed system
• Existant code integration • Fast content creation
• Features you implement • Plugins & profiling
• Developer community • Rich feature sets provided
PAUG 14

15. Future is Markerless
PAUG 15

16. Merci beaucoup
PAUG 16