Presented by Mr Prasanna Veerapandi (Bala), Associate Lecturer & Consultant, Software Systems Practice ,NUS-ISS, at NUS-ISS Learning Day 2019

1. Deploying AI Apps using
TensorFlow Lite in Mobile
Devices
Colab - Keras - h5 - tflite - Android
#ISSLearningDay
Mr. Prasanna Veerapandi (Bala), NUS-ISS
2 Aug 2018

2. Agenda
• Intro to Deep Learning
• Build & Train CNN Models - colab
• Export Model to tflite
• Deploy tflite in android app
• Live Demo
• From Train Model ….. to Deploy in device
#ISSLearningDay

3. About me
• My name is Bala (Prasanna Veerapandi)
• I teach Python for Data , Ops & Things
#ISSLearningDay
Google : NUS ISS PYTHON

4. Intro to Deep Learning
SGD, NN, Back Prop, Arch, CNN, Keras, tflite
#ISSLearningDay

5. ML vs DL

6. Let’s see how with a simple
classifier
CAT vs DOG

7. Get Some Data- Binary Classifier (2 folders)

8. Choose a function y = f(x) *Simple

9. Choose a function y = f(x) *Intermediate

10. Choose a function y = f(x) *Deep

11. How it learns ?
TF Playground

12. SGD - Optimization

13. Once Trained : You get predictions

14. Build & Train CNN Models - colab
Data Preprocessing, Transform, Scale, Bachify, Hyper Param, Arch,
Training & Evaluate
#ISSLearningDay

15. Lets see some Keras Code

16. Import
import tensorflow as tf
from tensorflow.keras.datasets import mnist
from tensorflow.keras.callbacks import LearningRateScheduler
from tensorflow.keras.initializers import Constant
from tensorflow.keras.layers import Input, Conv2D, BatchNormalization
from tensorflow.keras.layers Activation, Dropout, Flatten, Dense
from tensorflow.keras.models import Model
from tensorflow.keras.utils import to_categorical

17. Load Data
NUM_CLASSES = 10
(x_train, y_train), (x_test, y_test) = mnist.load_data(path='mnist.npz')
x_train = x_train.reshape(*x_train.shape, 1)
x_test = x_test.reshape(*x_test.shape, 1)
x_train = x_train / 255.0
x_test = x_test / 255.0
y_train = to_categorical(y_train, NUM_CLASSES)
y_test = to_categorical(y_test, NUM_CLASSES)
print("x_train.shape = {}, y_train.shape = {}".format(x_train.shape,
y_train.shape))

18. Build a Model (Arch)
INPUT_SHAPE = (28, 28, 1)
NUM_CLASSES = 10
model = keras.Sequential()
model.add( Input(shape=INPUT_SHAPE, name='input') )
model.add( Conv2D(24, kernel_size=(6, 6), strides=1) )
model.add( MaxPooling2D( pool_size=(2, 2) ) )
model.add( BatchNormalization(scale=False, beta_initializer=Constant(0.01)) )
model.add( Activation('relu') )
model.add( tf.keras.layers.Flatten() )
model.add( Dense(200) )
model.add( BatchNormalization(scale=False, beta_initializer=Constant(0.01)) )
model.add( Activation('relu'))
model.add( Dropout(rate=0.25))
model.add( Dense(NUM_CLASSES, activation='softmax', name='output') )

19. Model Summary

20. Remember the model Arch... *Intermediate

21. Build a Model (Arch) - Add more layers...
INPUT_SHAPE = (28, 28, 1)
NUM_CLASSES = 10
model = keras.Sequential()
model.add( Input(shape=INPUT_SHAPE, name='input') )
model.add( Conv2D(24, kernel_size=(6, 6), strides=1) )
model.add( BatchNormalization(scale=False, beta_initializer=Constant(0.01)) )
model.add( Activation('relu') )
model.add( Conv2D(48, kernel_size=(5, 5), strides=2) )
model.add( MaxPooling2D( pool_size=(2, 2) ) )
model.add( BatchNormalization(scale=False, beta_initializer=Constant(0.01)) )
model.add( Activation('relu') )
model.add( Conv2D(64, kernel_size=(4, 4), strides=2) )
model.add( MaxPooling2D( pool_size=(2, 2) ) )
model.add( BatchNormalization(scale=False, beta_initializer=Constant(0.01)) )
model.add( Activation('relu') )
model.add( tf.keras.layers.Flatten() )
model.add( Dense(200) )
model.add( BatchNormalization(scale=False, beta_initializer=Constant(0.01)) )
model.add( Activation('relu'))
model.add( Dropout(rate=0.25))

22. Model Summary

23. Compile the model
# Get an Optimizer
adam = keras.optimizers.Adam(lr=0.001)
model.compile(loss='categorical_crossentropy',
optimizer=adam,
metrics=['accuracy'])

24. Lets Train the model
history = model.fit(x_train,
y_train, batch_size=128,
epochs=2, verbose=1,
validation_data=(x_test, y_test))

25. Export Model to tflite
Keras, .h5, tflite converter
#ISSLearningDay

27. Export h5 to tflite
model.save('mnist.h5')
converter = tf.lite.TFLiteConverter.from_keras_model_file('mnist.h5')
tflite_model = converter.convert()
with open('mnist.tflite', 'wb') as f:
f.write(tflite_model)

28. Demo Time
http://bit.ly/train-mnist

29. Deploy tflite in android app
org.tensorflow:tensorflow-lite:1.13.1 , Interpreter
classifier , labels.txt,
Interpreter.run( data , result )
#ISSLearningDay

30. Android APP

31. build.gradle
aaptOptions {
noCompress "tflite"
noCompress "lite"
}
dependencies {
implementation 'org.tensorflow:tensorflow-lite:1.13.1'
}

32. Copy model into assets folder
mnist.tflite
labels.txt || labels.json
File Size in 1..5 Mbs

33. Model file size
quick-draw-tflite = 442 Kb
File Size in 1..5 Mbs

34. Configure input Shapes
public class MnistDigitClassifier {
private static final String MODEL_NAME = "mnist.tflite";
private static final int BATCH_SIZE = 1;
public static final int IMG_HEIGHT = 28;
public static final int IMG_WIDTH = 28;
private static final int NUM_CHANNEL = 1;
private static final int NUM_CLASSES = 10;
private final Interpreter.Options options = new Interpreter.Options();
private final Interpreter mInterpreter;
private final ByteBuffer mImageData;
private final int[] mImagePixels = new int[IMG_HEIGHT * IMG_WIDTH];
private final float[][] mResult = new float[1][NUM_CLASSES];
}

35. new Interpreter()
public class MnistDigitClassifier {
public MnistDigitClassifier(Activity activity) throws IOException {
mInterpreter = new Interpreter(loadModelFile(activity), options);
mImageData = ByteBuffer.allocateDirect(
4 * BATCH_SIZE * IMG_HEIGHT * IMG_WIDTH * NUM_CHANNEL);
mImageData.order(ByteOrder.nativeOrder());
}
}

36. Predict : Classify
public class MnistDigitClassifier {
public Result classify(Bitmap bitmap) {
convertBitmapToByteBuffer(bitmap);
mInterpreter.run(mImageData, mResult);
Result r = new Result(mResult[0], timeCost);
r.setLabel(String.valueOf(r.getNumber()));
return r;
}

37. Demo Time
http://bit.ly/quickdraw-tflite

38. Appendix
// How to get the bitmap
Bitmap image = mFpvPaint.exportToBitmap(
QuickDrawClassifier.IMG_WIDTH,
QuickDrawClassifier.IMG_HEIGHT);
private MappedByteBuffer loadModelFile(Activity activity) throws IOException {
AssetFileDescriptor fileDescriptor = activity.getAssets().openFd(MODEL_NAME);
FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
FileChannel fileChannel = inputStream.getChannel();
long startOffset = fileDescriptor.getStartOffset();
long declaredLength = fileDescriptor.getDeclaredLength();
return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}

39. https://github.com/balaprasanna/quick-
draw-keras-tflite-android
Code

40. Q/A
Thank You!
bala@nus.edu.sg
#ISSLearningDay