Intro talk for non data scientists, on what is ML and

1. Who’s afraid of
Machine Learning?
Britt Barak

2. Britt Barak
Google Developer Expert - Android
Women Techmakers Israel
Britt Barak @brittBarak

12. In a machine...

14. Strawberry
Not
Strawberry

15. Input
Red
Seeds
pattern
Top
leaves

16. 0.64
0.75
0.4
Input
Red
Seeds
pattern
Top
leaves

17. 0.64
0.75
0.4
Input
Red
Seeds
pattern
Top
leaves

18. 0.64
0.75
0.4
Input
Red
Seeds
pattern
Top
leaves

19. 0.64
0.75
0.4
Input
Red
Seeds
pattern
Top
leaves
0.5
0.8
0.3

20. 0.64
0.75
0.4
Input
Red
Seeds
pattern
Top
leaves
0.5
0.8
0.3
0.5 * 0.64
+ 0.8 * 0.75
+ 0.3 * 0.4

21. 0.64
0.75
0.4
Input
Red
Seeds
pattern
Top
leaves
0.5
0.8
0.3
0.5 * 0.64
+ 0.8 * 0.75
+ 0.3 * 0.4
___________
1.04

22. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
0.5
0.8
0.3
0.5 * 0.64
+ 0.8 * 0.75
+ 0.3 * 0.4
___________
1.04
+ 0.7
___________
Input

23. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
1.74
0.5
0.8
0.3
0.5 * 0.64
+ 0.8 * 0.75
+ 0.3 * 0.4
___________
1.04
+ 0.7
___________
1.74
Input

24. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
1.74
Input

25. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
1.02
1.74
0.97
Input

26. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
1.02
1.74
0.97
Output
Strawberry
Not
Strawberry
Input

27. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
1.02
1.74
0.97
0.87
0.13
Strawberry
Not
Strawberry
OutputInput

29. 0.7
0.03
0.01
Red
Seeds
pattern
Top
leaves
Strawberry
Not
Strawberry
OutputInput

30. 0.7
0.03
0.01
Red
Seeds
pattern
Top
leaves
3.72
0.89
1.92
Strawberry
Not
Strawberry
OutputInput

31. 0.7
0.03
0.01
Red
Seeds
pattern
Top
leaves
3.72
0.89
1.92
0.2
0.8
Strawberry
Not
Strawberry
OutputInput

32. Strawberry Not Not
StrawberryNot Not
Strawberry NotNot
0.5 * 0.64
+ 0.8 * 0.75
+ 0.3 * 0.4
___________
1.04
+ 0.7
___________
1.74

33. Training

34. 0.64
0.75
0.4
Red
Seeds
pattern
Top
leaves
1.02
1.74
0.97
0.89
0.11
Strawberry
Not
Strawberry
OutputInput

35. Red
Seeds
pattern
Top
leaves
Strawberry
Not
Strawberry
OutputInput Hidden

37. Data science

38. TensorFlow
- Open source
- Widely used
- Flexible for scale:
- 1 or more CPUs / GPUs
- desktop, server, mobile device

39. Strawberry

40. Strawberry

41. TensorFlow
Mobile
- Speech Recognition
- Image Recognition
- Object Localization
- Gesture Recognition
- Translation
- Text Classification
- Voice Synthesis

42. Lightweight Fast Cross platform

43. MobileNet Inception-V3 SmartReply
Models

44. Example implementation

45. Image
Classifier
classifier
.classify(bitmap)
label

46. 1. Add assets

48. labels.txt
strawberry
orange
lemon
fig
pineapple
banana
jackfruit
custard apple
pomegranate
hay
carbonara
chocolate sauce
dough
meat loaf
pizza

49. 2. Add TensorFlow Lite

50. repositories {
maven {
url 'https://google.bintray.com/tensorflow'
}
}
dependencies {
// ...
implementation 'org.tensorflow:tensorflow-lite:+'
}
build.gradle

51. android {
aaptOptions {
noCompress "tflite"
noCompress "lite"
}
}
build.gradle

52. 3. Create
ImageClassifier.java

53. Image
Classifier

54. ImageClassifier.java
tflite = new Interpreter();

55. ImageClassifier.java
model = loadModelFile();
tflite = new Interpreter(model);

56. MappedByteBuffer loadModelFile() {
AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
}

57. MappedByteBuffer loadModelFile() {
AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
FileInputStream inputStream = new
FileInputStream(descriptor.getFileDescriptor());
FileChannel channel = inputStream.getChannel();
}

58. MappedByteBuffer loadModelFile() {
AssetFileDescriptor descriptor= getAssets().openFd(MODEL_PATH);
FileInputStream inputStream = new
FileInputStream(descriptor.getFileDescriptor());
FileChannel channel = inputStream.getChannel();
long start = descriptor.getStartOffset();
long length = descriptor.getDeclaredLength();
return
channel.map(FileChannel.MapMode.READ_ONLY, start, length);
}

59. Image
Classifier
[strawberry, apple, ...]
labels.txt

60. ImageClassifier.java
model = loadModelFile();
tflite = new Interpreter(model);
labelList = loadLabelList();

61. labels.txt
strawberry
orange
lemon
fig
pineapple
banana
jackfruit
custard apple
pomegranate
hay
carbonara
chocolate sauce
dough
meat loaf
pizza

62. List<String> loadLabelList() throws IOException {
` InputStreamReader inputStream =
new InputStreamReader(getAssets().open(LABEL_PATH));
}

63. List<String> loadLabelList() throws IOException {
` InputStreamReader inputStream =
new InputStreamReader(getAssets().open(LABEL_PATH));
BufferedReader reader = new BufferedReader(inputStream);
}

64. List<String> loadLabelList() throws IOException {
` InputStreamReader inputStream =
new InputStreamReader(getAssets().open(LABEL_PATH));
BufferedReader reader = new BufferedReader(inputStream);
List<String> labelList = new ArrayList<String>();
String line;
while ((line = reader.readLine()) != null) {
labelList.add(line);
}
}

65. List<String> loadLabelList() throws IOException {
` InputStreamReader inputStream =
new InputStreamReader(getAssets().open(LABEL_PATH));
BufferedReader reader = new BufferedReader(inputStream);
List<String> labelList = new ArrayList<String>();
String line;
while ((line = reader.readLine()) != null) {
labelList.add(line);
}
reader.close();
return labelList;
}

66. Image
Classifier
[ [0..6] , [ 0.1 ] , ...]
[strawberry, apple, ...]
probArray
labels.txt

67. probArray =
{
[0.7],
[0.3],
[0],
[0],
}
labelList =
{
strawberry,
apple,
pineapple,
banana
}
0.3

68. ImageClassifier.java
model = loadModelFile();
tflite = new Interpreter(model);
labelList = loadLabelList();
probArray = new float[1][labelList.size()];

69. Image
Classifier
[......] [ [0..6] , [ 0.1 ] , ...]
[strawberry, apple, ...]
ByteBuffer probArray
labels.txt

70. ImageClassifier.java
model = loadModelFile();
tflite = new Interpreter(model);
labelList = loadLabelList();
probArray = new float[1][labelList.size()];
imgData =
ByteBuffer.allocateDirect(
DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE);
imgData.order(ByteOrder.nativeOrder());

71. 4. Run the model / classify

72. classifier
.classify(bitmap)
Image
Classifier
[......] [ [0..6] , [ 0.1 ] , ...]
[strawberry, apple, ...]
ByteBuffer probArray
labels.txt

73. ImageClassifier.java
String classify(Bitmap bitmap) {
convertBitmapToByteBuffer(bitmap);
}

74. void convertBitmapToByteBuffer(Bitmap bitmap) {
//...
bitmap.getPixels(intValues, 0, bitmap.getWidth(),
0, 0,bitmap.getWidth(), bitmap.getHeight());
}
}

75. void convertBitmapToByteBuffer(Bitmap bitmap) {
//...
bitmap.getPixels(intValues, 0, bitmap.getWidth(),
0, 0,bitmap.getWidth(), bitmap.getHeight());
int pixel = 0;
for (int i = 0; i < DIM_IMG_SIZE_X; ++i) {
for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) {
final int val = intValues[pixel++];
imgData.put((byte) ((val >> 16) & 0xFF));
imgData.put((byte) ((val >> 8) & 0xFF));
imgData.put((byte) (val & 0xFF));
}
}
}

76. ImageClassifier.java
String classify(Bitmap bitmap) {
convertBitmapToByteBuffer(bitmap);
tflite.run(imgData, probArray);

77. ImageClassifier.java
String classify(Bitmap bitmap) {
convertBitmapToByteBuffer(bitmap);
tflite.run(imgData, probArray);
String textToShow = getTopLabels();
return textToShow;
}

78. private String getTopKLabels() {
for (int i = 0; i < labelList.size(); ++i) {
sortedLabels.add(
new AbstractMap.SimpleEntry<>(labelList.get(i), (labelProbArray[0][i] &
0xff) / 255.0f));
if (sortedLabels.size() > RESULTS_TO_SHOW) {
sortedLabels.poll();
}
}
String textToShow = "";
final int size = sortedLabels.size();
for (int i = 0; i < size; ++i) {
Map.Entry<String, Float> label = sortedLabels.poll();
textToShow = String.format("n%s: %4.2f", label.getKey(), label.getValue())
+ textToShow;
}
return textToShow;
}

79. Strawberry - 0.87
Apple - 0.13
Tomato - 0.01

80. Machine
Learning
is a
new world

81. Links
- Tensorflow
- https://www.tensorflow.org/
- Tensorflow lite
- https://www.tensorflow.org/mobile/tflite/
- Codes labs
- codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/
- Google’s Machine Learning Crash Course
- developers.google.com/machine-learning/crash-course/
- [Dr. Joe Dispenza]

82. Thank you!
Keep in touch!
Britt Barak
@brittBarak

83. BETA

84. Key capabilities
● Production-ready for common use cases
● On-device or in the cloud
● Deploy custom models

86. How to implement?

87. 1. Add the sdk
Quickly include the SDK using Gradle or CocoaPods.

88. 2. Prepare input data
For example, if you're using a vision feature, capture an image from the camera
and generate the necessary metadata such as image rotation, or prompt the user
to select a photo from their gallery.

89. 3. Apply the ML model to your data
By applying the ML model to your data, you generate insights such as the
emotional state of detected faces or the objects and concepts that were
recognized in the image, depending on the feature you used. Use these insights to
power features in your app like photo embellishment, automatic metadata
generation, or whatever else you can imagine.

90. Ready to use models

91. Ready to use models: 1. Text Recognition

92. Ready to use models: 2. Face Detection

93. Ready to use models: 3. Barcode Scanning

94. Corners
(87,87) (612,87) (612,612) (87,612)
Raw value
WIFI:S:SB1Guest;P:12345;T:WEP;;
WiFi information

95. Ready to use models: 4. Image Labeling