Nowadays, mobile devices have enough computing power to run pre-trained models on them. This results in an optimal use of the hardware and an increase in speed, because the data are not sent over the Internet, which also means more privacy. In my presentation I will show different ways to integrate machine learning models into an iOS and Android application. The participants will learn how to integrate a pre-training model into an iOS and Android application with CoreML and Tensorflow Lite, and how to re-train a model for own pictures and use it instead of the pre-trained model. All examples are shown in a small demo. https://www.mcubed.london/sessions/machine-learning-models-mobile-devices/

1. Machine Learning Models
on Mobile Devices
Lars Gregori

2. Why Machine Learning
Models on Mobile Devices?
•Privacy
•Availability
•Speed

3. What Models?
•Image
•Text
•…

4. What Models (CoreML)?
https://developer.apple.com/documentation/coreml/converting_trained_models_to_core_ml

5. What Models (CoreML)?
convolutional
Caffe
Keras
random forests scikit-learn
XGBoost
multiclass classification
logic regression
https://developer.apple.com/documentation/coreml/converting_trained_models_to_core_ml

6. How?
•CoreML
•TensorFlow Lite
•Other e.g. Caffe2Go

7. CoreML
•Apple
•WWDC 2017
•iOS, macOS, watchOS, tvOS

8. Application
Vision NaturalLanguage GameplayKit
CoreML
Accelerate und BNNS MetalPerformanceShaders
HardwareCPU GPU
CoreML
BNNS - Basic neural network subroutines
https://developer.apple.com/documentation/accelerate/bnns

9. TensorFlow Lite
•Google
•Based on TensorFlow Mobile
•Android, iOS and Raspberry Pi

10. TensorFlow Lite
.tflite fileAndroid App
Java API
C++ API
Interpreter
Android Neural Network API
Operators
.tflite fileiOS App
C++ API
Interpreter
Operators

11. DEMO
Object Detection

12. 95.1% African elephant, Loxodonta africana
4.2% tusker
0.4% Indian elephant, Elephas maximus
0.0% moped
0.0% warthog

13. Inception V3
•Object detection (96.5% top-5)
•Google Brain, arXiv:1512.00567
•1000 classes; ILSVRC2012*
ILSVRC2012 - ImageNet Large Scale Visual Recognition Challenge 2012

14. Inception V3 and CoreML
https://developer.apple.com/machine-learning/build-run-models/

15. Inception V3 and CoreML

16. Inception V3 and CoreML
override func viewDidLoad() {
super.viewDidLoad()
let model = try! VNCoreMLModel(for: Inceptionv3().model)
let request = VNCoreMLRequest(model: model,
completionHandler: resHandler)
self.requests.append(request)
}
Inceptionv3().model
resHandler
self.requests

17. Inception V3 and CoreML
func resHandler(request: VNRequest, _: Error?) {
let results = request.results
as! [VNClassificationObservation]
let percent = Int(results[0].confidence * 100)
let identifier = results[0].identifier
resultLabel.text = "(percent)% (identifier)"
}
resHandler
results
results
results

18. Inception V3 and CoreML

19. Inception V3 and CoreML
@IBAction func predict(_ sender: UIButton) {
selectedImage.image = sender.currentImage
let image = sender.currentImage!.cgImage!
let handler = VNImageRequestHandler(cgImage: image,
options: [:])
try? handler.perform(self.requests)
}
@IBAction
sender.currentImage!.cgImage!
handler.preform
handler
image =
image
self.requests

20. Inception V3 and TF Lite
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/g3doc/models.md

21. Inception V3 and TF Lite
String tffile = “inception_v3.tflite”;
protected void onCreate(Bundle savedInstanceState) {
tflite = new Interpreter(loadModelFile(tffile));
button1.setOnClickListener(new View.OnClickListener() {
@Override public void onClick(View v) {
predict(context, R.drawable.image1);
Interpreter
setOnClickListener
predict

22. Inception V3 and TF Lite
private void predict(Context context, int imgId) {
Bitmap bitmap = BitmapFactory.decodeResource(...,
imgId, ...);
image.setImageBitmap(bitmap);
ByteBuffer imgBuf = convertBitmapToByteBuffer(bitmap);
float[][] labelProb = new float[1][labels.size()];
tflite.run(imgBuf, labelProb);
>>>
predict
imgBuf
imgBuf
labelProb
labelProbtflite.run

23. Inception V3 and TF Lite
String label = "";
Float percent = -1.0f;
for(int index = 0; index < labels.size(); index++) {
if (labelProb[0][index] > percent) {
percent = labelProb[0][index];
label = labels.get(index);
}
labelProb
labelProb
index
index
index
index
labels
labels

24. Image Retraining
37.7% strawberry
11.9% ice cream, icecream
7.9% pineapple, ananas
2.8% ear, spike, capitulum
2.6% trifle

25. Image Retraining
•Transfer learning
•Already trained model
•New classification layer on top
https://www.tensorflow.org/hub/tutorials/image_retraining

26. Example
XOR

27. XOR
•0 xor 0 = 0
•0 xor 1 = 1
•1 xor 0 = 1
•1 xor 1 = 0

28. XOR Keras Model
training_data = np.array([[0,0], [0,1], [1,0], [1,1]])
target_data = np.array([ [0], [1], [1], [0]])
model = Sequential()
model.add(Dense(8, input_dim=2, activation=sigmoid))
model.add(Dense(1, activation=sigmoid))
model.compile(loss=mean_squared_error, optimizer=SGD(lr=1.0))
model.fit(training_data, target_data, epochs=2000)
>> Jupyter Notebook
Jupyter Notebook: https://github.com/choas/Machine_Learning_Models_In_Apps/blob/master/XOR/models/XOR.ipynb
SGD - stochastic gradient descent - https://en.wikipedia.org/wiki/Stochastic_gradient_descent
Sigmoid - https://en.wikipedia.org/wiki/Sigmoid_function

29. XOR Keras Model
print model.predict(training_data)
[[0.03998361]
[0.9534271 ]
[0.9549079 ]
[0.05220655]]

30. Save Model
•CoreML Tools (Keras, …)
•tfcoreml (Tensorflow)
•TensorFlow Lite Optimizing Converter
(TOCO)

31. DEMO
XOR

32. XOR Example

33. XOR and CoreML
override func viewDidLoad() {
super.viewDidLoad()
out00.text = "(predict(0, 0))"
out01.text = "(predict(0, 1))"
out10.text = "(predict(1, 0))"
out11.text = "(predict(1, 1))"
}
predict(0, 0)
predict(0, 1)
predict(1, 0)
predict(1, 1)

34. XOR and CoreML
func predict(_ a: NSNumber, _ b: NSNumber) -> NSNumber {
let input_data = try! MLMultiArray(shape:[2], …)
input_data[0] = a
input_data[1] = b
let xor_input = xorInput(input: input_data)
let prediction = try! xor().prediction(input: xor_input)
return prediction.result[0]
}
predict
input_data
input_data
input_data
input_data
xor().predictionprediction
prediction.result[0]
xorInput

35. XOR and TF Lite
protected void onCreate(Bundle savedInstanceState) {
// …
tflite = new Interpreter(loadModelFile("xor.tflite"));
result00.setText(prediction(0, 0));
result01.setText(prediction(0, 1));
result10.setText(prediction(1, 0));
result11.setText(prediction(1, 1));
}
tflite
prediction(0, 0)
prediction(0, 1)
prediction(1, 0)
prediction(1, 1)

36. XOR and TF Lite
private String prediction(int a, int b) {
float[][] in = new float[][]{{a, b}};
float[][] out = new float[][]{{0}};
tflite.run(in, out);
return String.valueOf(out[0][0]);
}
prediction
in
in
out
out
out[0][0]
tflite.run(in, out)

37. Quantization

38. Quantization
•A discrete set of values.
•TensorFlow Lite: 8-bits
•CoreML 2: Weight Quantization

39. Quantization - Inception V3
https://ai.googleblog.com/2016/08/improving-inception-and-image.html

40. Quantization - Inception V3
Convolution
https://ai.googleblog.com/2016/08/improving-inception-and-image.html

41. DEMO
Quantization

42. 97% original Inception V3
97% 16-bit linear
98% 8-bit linear
2% 4-bit linear (pillow is wrong)

43. Summary

44. Summary
•Why
•How
•What will you build?

45. Thank you
@choas