1. Tensorflow.js
Develop ML with JavaScript
1

2. What?
2

3. 3

4. History
4

5. 5

6. What we can do?
6

7. What we can do?
92% it’s cat
7
52% it’s cat

8. How it works
8

9. Single neuron
f(∑)
x1
x2
x3
w1
w2
w3
out
9

10. Single neuron
f(∑)
x1
x2
x3
w1
w2
w3
out
f(x[1]*w[1] + x[2]*w[2] + x[3]*w[3])
f(∑) = (0...1)
10

11. Single neuron
f(∑)
x1
x2
x3
w1
w2
w3
out
f(x[1]*w[1] + x[2]*w[2] + x[3]*w[3])
f = sigmoid / ReLU / tanH ...
11

12. Single neuron
12
1
-1
-1 1
Sigmoid
f(x) = 1 / (1 + exp(-x))
1
-1
-1 1
ReLU
f(x) = max(0, x)
1
-1
-1 1
TanH
f(x) = (exp(x) - exp(-x)) /
(exp(x) + exp(-x))

13. 13

14. Single neuron
Activation Function

15. Single neuron
f(∑)
x1
x2
x3
w1
w2
w3
out
W1
W2
W3
X1 X2 X3 *
15

16. More neurons…
f(∑)
x1
x2
x3
W11 W12
W21 W22
W31 W32
X1 X2 X3 *
f(∑)
out2
out1
Matrix DOT
16
w11
w1
w21
w22
w31
w32

17. One Layer!
f(∑)
x1
x2
x3
W11 W12
W21 W22
W31 W32
X1 X2 X3
*
f(∑)
out2
out1
O1 O2=
17

18. More layers...
f(∑)
x1
x2
x3
W11 W12
W21 W22
W31 W32
X1 X2 X3
*
f(∑)
*
f(∑)
f(∑)
f(∑)
W11 W12 W13
W21 W22 W13
18

19. Deep Network!
f(∑)
x1
x2
x3
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
O1 O2 O3 O4
19

20. Deep Network
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
X1 …. Xn O1 …. On
20

21. Deep Network
W11 W12
W21 W22
W31 W32
W11 W12 W13
W21 W22 W13
W11 W12
W21 W22
W31 W32
21

22. Recognize Cat
22

23. Recognize Cat
23
134 50 233
12 189 199
... ... ...
213 181 71

24. Recognize Cat
24
0.52 0.19 0.91
0.098 0.74 0.78
... ... ...
0.83 0.71 0.28
x’ = x / 255

25. Recognize Cat
25
0.52 0.19 0.91 0.098 0.74 0.78 ….

26. Recognize Cat
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)0.25 0.81 ... 0.25 0.023 0.977
yes, it’s cat!
26

27. Wrong result :(
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)0.25 0.81 ... 0.25 0.65 0.35
may be cat…
or not…
i don’t know!
27

28. Learning
28

29. Learning flow
1. Get Labeled data
2. Get results for labeled data (Forward
propagation)
3. Compare with expected results (Fe - Fa)
4. If error is big, update weights (Back propagation),
repeat 2-4
5. If error is low (E < 0.001), try on unlabeled data
not cat
cat
29

30. Learning flow
not cat
cat
Input
[ 0.24, 0.56, … 0.567 ],
[ 0.87, 0.122, … 0.78 ],
[ 0.911, 0.796, … 0.24 ],
...
Expected Output
[ 1, 0 ],
[ 0, 1 ],
[ 0, 1 ],
...
30

31. Learning flow
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
Forward Propagation
31

32. Learning flow
Error = expected - actual
= [ 1, 0 ] - [ 0.23, 0.77 ]
= [0.77, -0.77,]
32
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)

33. Learning flow
Back Propagation
Back prop (impl)
33
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)
f(∑)

34. Learning flow
Gradient descent

35. Tensorflow
35

36. Why in browser?
36

37. Tensorflow.js
In browser:
WebGL
On server
CUDA or Native C++
37

38. Install
● npm install @tensorflow/tfjs
● npm install @tensorflow/tfjs-node
38

39. Wait! Who is “Tensor”?
39

40. Wait! Who is “Tensor”?
[1.0, 2.0, 3.0],
[10.0, 20.0, 30.0]
tensor =
40

41. Wait! Who is “Tensor”?
const b = tf.tensor([[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]]);
const c = tf.tensor2d([[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]]);
const e = tf.tensor2d([[1.0, 2.0], [3.0, 4.0]]);
const f = tf.tensor2d([[5.0, 6.0], [7.0, 8.0]]);
41

42. Ops API
const e = tf.tensor([[1.0, 2.0], [20.0, 30.0]]);
const f = tf.tensor2d([[2.0, 3.0], [10.0, 20.0]]);
const e_plus_f = e.add(f);
const f_plus_e = tf.add(e, f);
const e_sub_f = e.sub(f);
const e_mul_f = e.mul(f);
const square_f = tf.square(f);
42

43. Memory
Dispose
const e = tf.tensor([[1.0, 2.0], [20.0, 30.0]]);
const x_squared = x.square();
x.dispose();
x_squared.dispose();
43

44. Memory
Tidy
const average = tf.tidy(() => {
const y = tf.tensor1d([1.0, 2.0, 3.0, 4.0]);
const z = tf.ones([4]);
return y.sub(z).square().mean();
});
average.print()
44

45. 45

46. TensorFlow.js Layers API
46

47. Layers API
import * as tf from '@tensorlowjs/tfjs';
// Build and compile model.
const model = tf.sequential();
model.add(tf.layers.dense({units: 1, inputShape: [1]}));
model.compile({optimizer: 'sgd', loss: 'meanSquaredError'});
// Generate data for training.
const xs = tf.tensor2d([[1], [2], [3], [4]], [4, 1]);
const ys = tf.tensor2d([[1], [3], [5], [7]], [4, 1]);
// Train model
await model.fit(xs, ys, {epochs: 1000}); 47

48. Load pretrained model
import * as tf from '@tensorlowjs/tfjs';
// Load model.
const model = await tf.loadModel('https://foo.bar/model.json');
48

49. Live example
49

50. Capture from camera
public capture(): Tensor {
return tf.tidy(() => {
const croppedImage = this.cropImage(tf.fromPixels(this.video));
const batchedImage = croppedImage.expandDims(0);
return batchedImage.toFloat().div(tf.scalar(127)).sub(tf.scalar(1));
});
}
50

51. Load pre-trained model
private loadTruncatedMobileNet$(): Observable<Model> {
return Observable.create(observer => {
tf.loadModel(
'https://storage.googleapis.com/tfjs-models/tfjs/mobilenet_v1_0.25_224/model.json'
).then(
(mobilenet: Model) => {
const layer = mobilenet.getLayer('conv_pw_13_relu');
observer.next(tf.model({ inputs: mobilenet.inputs, outputs: layer.output }));
observer.complete();
}
);
});
}
51

52. Create own model
tf.sequential({
layers: [
tf.layers.flatten({
inputShape: this.truncatedMobileNet.outputs[0].shape.slice(1)
}),
tf.layers.dense({
units: Config.NUM_HIDDEN_LAYERS,
activation: 'relu',
kernelInitializer: 'varianceScaling',
useBias: true
}),
tf.layers.dense({
units: Config.NUM_CLASSES,
kernelInitializer: 'varianceScaling',
useBias: false,
activation: 'softmax'
})
]
});
52

53. Train
public train$(): Observable<number> {
return Observable.create((observer: Observer<number>) => {
this.model = this.getModel();
const optimizer = tf.train.adam(Config.LEARNING_RATE);
this.model.compile({
optimizer: optimizer,
loss: 'categoricalCrossentropy'
});
this.actualModelTrain(observer);
});
}
53
// ...
this.model.fit(this.datasetController.savedX,
this.datasetController.savedY, {
batchSize,
epochs: Config.NUM_EPOCHS,
callbacks: {
onBatchEnd: async (batch, logs) => observer.next(logs.loss),
}
})
.then(() => observer.complete())

54. Predict image class
public predictFromCameraCapture$(): Observable<number> {
return Observable.create(observer => {
const predict = () => {
const predictedClass = tf.tidy(() => {
const img = this.webcam.capture();
const embeddings = this.truncatedMobileNet.predict(img);
const predictions: Tensor<Rank> | Tensor<Rank>[] = this.model.predict(embeddings);
return (predictions as Tensor<Rank>).as1D().argMax();
});
predictedClass.data()
.then((data: Int32Array) => observer.next(data[0]))
};
predict();
})
}
54

55. Playing Mortal Kombat with TensorFlow.js. Transfer learning
and data augmentation
55

56. 56

57. 57

58. Essence of calculus
Linear algebra
Coursera: Deep learning
Awesome Deep Learning Project Ideas
Live example
58
GALLERY

59. Roadmap
59
watch
presentation
again
multiplication,
forward prop,
back prop
build your own
network
implementation
try to
recognize
digits
face id
lock for your
app
convolutional
network
transfer
learning

60. Lazy Roadmap
60
instal ml5.js done!
const classifier = ml5.imageClassifier('MobileNet', onModelReady);
// Make a prediction
let prediction = classifier.predict(img, (err, results) => {
console.log(results);
});

61. THANKS FOR WATCHING!!!
YOU ARE AWESOME
61

62. QUESTIONS? 62