Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Functional dnn in elixir

369 views

Published on

Slides used for speech about implementation of Convolutional Neural Network in Elixir.
You can also find the speech on https://youtube.com/watch?v=RYLPiUjKcQ8 .

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Functional dnn in elixir

  1. 1. Functional CNN in Elixir Masayoshi Tamura Retrieva, Inc. © 2020, Retrieva, Inc. All rights reserved.
  2. 2. Agenda • Self-introduction • What is Elixir • What is Convolutional Newral Network(CNN) • CNN in Elixir • Functional CNN • Acceleration • Tips • Future work © 2020, Retrieva, Inc. All rights reserved. 2
  3. 3. Who am I? Self-introduction © 2020, Retrieva, Inc. All rights reserved. 3 Java C C# C++ JavaScript Python Elixir Haskell prolog R matlab Lua php SAS SVF Objective-C cocos2d-x AngularJS jQuery D3.js TwitterAPI Docker CircleCI Apache Tomcat OpenGL GLSL WebGL OpenCL Oracle MySQL PostgreSQL SQLServer memcached Spring Seasar2 Servlet Jenkins Nginx WebSphere RPM Hadoop maven ProtocolBuffer gRPC Tensorflow Chainer scikit-learn k-means Naïve Bayes Classifier Random Forest Ray Tracer Photon Mapper Dynamic Programing Greedy Algorithm Binary Indexed Tree Segement Tree
  4. 4. Masayoshi Tamura • Educational Background |> Waseda University Senior High School |> Waseda University(BAPsy) |> Oregon State University(MEng) • Engineering Career |> System Development Company |> Mobile Game Development Company |> Freelance Smartphone Game Developer |> Retrieva © 2020, Retrieva, Inc. All rights reserved. 4
  5. 5. Masayoshi Tamura • Community |> fukuoka.ex |> Japan Android Group • Hobby |> Boxing |> Bouldering |> Footsal |> Rock Festival |> Sake © 2020, Retrieva, Inc. All rights reserved. 5
  6. 6. My Events in 2019 • Events |> ElixirConf JP 2019 Lightning Talk |> Forkwell press interview |> Elixir Advent Calendar 2019 Day 18 |> fukuoka.ex Advent Calendar 2019 Day 17 © 2020, Retrieva, Inc. All rights reserved. 6
  7. 7. © 2020, Retrieva, Inc. All rights reserved. 7 First of all, why Deep Learning in Elixir?
  8. 8. Motivation • Fast training on the machine without CUDA • To learn how deep learning is working • Favorite Language • No practical library in Elixir so far • Ecosystem is growing © 2020, Retrieva, Inc. All rights reserved. 8
  9. 9. Resource • My implementation is based on “DeepLeening from scratch” written by Koki Saito[1] https://github.com/oreilly-japan/deep-learning-from-scratch © 2020, Retrieva, Inc. All rights reserved. 9
  10. 10. What is Elixir © 2020, Retrieva, Inc. All rights reserved. 10
  11. 11. Elixir • Functional Programming Language • Pipe |> Operator • BEAM(Bogdan's Erlang Abstract Machine) = Erlang VM • Immutable • Concurrency • Failure Resistance • Not assignment but binding • Availability • Let it crash © 2020, Retrieva, Inc. All rights reserved. 11
  12. 12. Convolutional Neural Network © 2020, Retrieva, Inc. All rights reserved. 12
  13. 13. Neural Network © 2020, Retrieva, Inc. All rights reserved. 13 Input Layer Hidden Layer Output Layer
  14. 14. Deep Neural Network © 2020, Retrieva, Inc. All rights reserved. 14 Input Layer Hidden Layers Output Layer
  15. 15. Convolutional Neural Network © 2020, Retrieva, Inc. All rights reserved. 15 Input Layer Convolution Pooling Output Layer
  16. 16. © 2020, Retrieva, Inc. All rights reserved. 16 how to use the network
  17. 17. Means to use the model • Predict • Infer the label(s) for the data • Train • Fix the parameters to obtain beter predictions • Partial differentiation • Backward propagation © 2020, Retrieva, Inc. All rights reserved. 17
  18. 18. Predict(forward) • Make data pass through layers © 2020, Retrieva, Inc. All rights reserved. 18
  19. 19. Train(forward->backward) • Make data pass through layers © 2020, Retrieva, Inc. All rights reserved. 19
  20. 20. Train(forward->backward) • go back and adjust parameters as they should be © 2020, Retrieva, Inc. All rights reserved. 20 t
  21. 21. Update params • Partial differentiation • ex). Affine • 𝑊 = 𝑤11 𝑤12 𝑤13 𝑤21 𝑤22 𝑤23 • 𝜕𝐿 𝜕𝑊 = 𝜕𝐿 𝜕𝑤11 𝜕𝐿 𝜕𝑤12 𝜕𝐿 𝜕𝑤13 𝜕𝐿 𝜕𝑤21 𝜕𝐿 𝜕𝑤22 𝜕𝐿 𝜕𝑤23 © 2020, Retrieva, Inc. All rights reserved. 21
  22. 22. Update params • Partial differentiation • ex). Affine • 𝑊 = 𝑤11 𝑤12 𝑤13 𝑤21 𝑤22 𝑤23 • 𝜕𝐿 𝜕𝑊 = 𝜕𝐿 𝜕𝑤11 𝜕𝐿 𝜕𝑤12 𝜕𝐿 𝜕𝑤13 𝜕𝐿 𝜕𝑤21 𝜕𝐿 𝜕𝑤22 𝜕𝐿 𝜕𝑤23 © 2020, Retrieva, Inc. All rights reserved. 22
  23. 23. Update params • Backward Propagation -> chain rule • ex). Affine © 2020, Retrieva, Inc. All rights reserved. 23 dot + X(N, 2) W(2, 3) X・W(N, 3) Y(N, 3) B(3,)
  24. 24. Update params • Backward Propagation • ex). Affine © 2020, Retrieva, Inc. All rights reserved. 24 dot + X(N, 2) W(2, 3) X・W(N, 3) Y(N, 3) B(3,) 𝜕𝐿 𝜕𝑌 (N, 3) 𝜕𝐿 𝜕𝑌 (N, 3) 𝜕𝐿 𝜕𝑌 (3,) sum of cols(axis=0) 𝐿11 𝐿12 𝐿13 𝐿21 𝐿22 𝐿23 𝜕𝐿 𝜕𝑋 = 𝜕𝐿 𝜕𝑌 ∙ 𝑊 𝑇 𝜕𝐿 𝜕𝑊 = 𝑋 𝑇 ∙ 𝜕𝐿 𝜕𝑌
  25. 25. CNN in Elixir © 2020, Retrieva, Inc. All rights reserved. 25
  26. 26. © 2020, Retrieva, Inc. All rights reserved. 26 Simple Two Layer Net
  27. 27. Simple Two Layer Net • 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 27 no type config Activation parameters 1 Affine (784, 100) ReLU 78,500 2 Affine (100, 10) Softmax 1,010
  28. 28. Simple Two Layer Net • 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 28 no type config Activation parameters 1 Affine (784, 100) ReLU 78,500 2 Affine (100, 10) Softmax 1,010
  29. 29. Affine Layer • Full-Connected Layer © 2020, Retrieva, Inc. All rights reserved. 29 biasweight x1 x2 x3
  30. 30. Affine Layer • Forward • Y = WX + B © 2020, Retrieva, Inc. All rights reserved. 30
  31. 31. Simple Two Layer Net • 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 31 no type config Activation parameters 1 Affine (784, 100) ReLU 78,500 2 Affine (100, 10) Softmax 1,010
  32. 32. ReLU • Forward • ℎ 𝑥 = 𝑥, 𝑥 > 0 0, 𝑥 ≤ 0 © 2020, Retrieva, Inc. All rights reserved. 32
  33. 33. Simple Two Layer Net • 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 33 no type config Activation parameters 1 Affine (784, 100) ReLU 78,500 2 Affine (100, 10) Softmax 1,010
  34. 34. Softmax • Forward • 𝑦 𝑘 = 𝑒𝑥𝑝 𝑎 𝑘 𝑖=0 𝑛 𝑒𝑥𝑝 𝑎 𝑖 © 2020, Retrieva, Inc. All rights reserved. 34
  35. 35. Softmax • Forward • 𝑦 𝑘 = 𝑒𝑥𝑝 𝑎 𝑘 𝑖=0 𝑛 𝑒𝑥𝑝 𝑎 𝑖 = 𝑒𝑥𝑝 𝑎 𝑘+𝐶′ 𝑖=0 𝑛 𝑒𝑥𝑝 𝑎 𝑖+𝐶′ Due to overflow, it is better to subtruct maximum value in the list © 2020, Retrieva, Inc. All rights reserved. 35
  36. 36. CrossEntropyError • 𝐸 = − 𝑘 𝑡 𝑘log 𝑦 𝑘 © 2020, Retrieva, Inc. All rights reserved. 36
  37. 37. Softmax with CrossEntropyError • Backward • 𝑦 𝑘 − 𝑡 𝑘 © 2020, Retrieva, Inc. All rights reserved. 37
  38. 38. ReLU • Backward • ℎ 𝑑𝑜𝑢𝑡 = 𝑑𝑜𝑢𝑡, 𝑥 > 0 𝑤ℎ𝑒𝑟𝑒 𝑥 𝑖𝑠 𝑡ℎ𝑒 𝑖𝑛𝑝𝑢𝑡 𝑜𝑓 𝑓𝑜𝑟𝑤𝑎𝑟𝑑 0, 𝑥 ≤ 0 © 2020, Retrieva, Inc. All rights reserved. 38
  39. 39. Affine Layer • Backward • W = W – (XT・dout) • B = sum(dout, axis=0) • dx = dout ・WT © 2020, Retrieva, Inc. All rights reserved. 39
  40. 40. Forward • protocol © 2020, Retrieva, Inc. All rights reserved. 40
  41. 41. © 2020, Retrieva, Inc. All rights reserved. 41
  42. 42. © 2020, Retrieva, Inc. All rights reserved. 42
  43. 43. © 2020, Retrieva, Inc. All rights reserved. 43 Simple ConvNet
  44. 44. Simple ConvNet • 1 Convolution + 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 44 no type config Activation parameters 1 Convolution2D 30, (5, 5), 1, 0 ReLU 780 2 MaxPooling (2, 2, 0) - 0 3 Affine (4320, 100) ReLU 432,100 4 Affine (100, 10) Softmax 1,010
  45. 45. im2col • pick the values in the kernel © 2020, Retrieva, Inc. All rights reserved. 45
  46. 46. im2col • pick the values in the kernel © 2020, Retrieva, Inc. All rights reserved. 46
  47. 47. im2col • pick the values in the kernel © 2020, Retrieva, Inc. All rights reserved. 47 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 0 1 5 6
  48. 48. im2col • pick the values in the kernel © 2020, Retrieva, Inc. All rights reserved. 48 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 0 1 5 6 25 26 30 31
  49. 49. im2col • pick the values in the kernel © 2020, Retrieva, Inc. All rights reserved. 49 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32
  50. 50. im2col • pick the values in the kernel © 2020, Retrieva, Inc. All rights reserved. 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 out_w
  51. 51. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 51 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  52. 52. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 52 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  53. 53. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 53 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
  54. 54. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 54 5 6 7 8 9 10 11 12 13 14
  55. 55. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 55 5 6 7 8 9 10 11 12 13 14
  56. 56. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 56 6 7 8 9 11 12 13 14
  57. 57. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 57 6 7 11 12
  58. 58. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 58 6 7 11 12
  59. 59. im2col • pick the values in the kernel • ex). y=1, x=1 © 2020, Retrieva, Inc. All rights reserved. 59 6 7 11 12 31 32 36 37
  60. 60. Simple ConvNet • 1 Convolution + 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 60 no type config Activation parameters 1 Convolution2D 30, (5, 5), 1, 0 ReLU 780 2 MaxPooling2D (2, 2, 0) - 0 3 Affine (4320, 100) ReLU 432,100 4 Affine (100, 10) Softmax 1,010
  61. 61. MaxPooling2D • Remain maximum value in the kernel © 2020, Retrieva, Inc. All rights reserved. 61 0 3 1 6 10 4 5 2 1 5 7 1 8 4 2 3 3 2 9 4 3 6 2 8
  62. 62. MaxPooling2D • Remain maximum value in the kernel © 2020, Retrieva, Inc. All rights reserved. 62 0 3 1 6 10 4 5 2 1 5 7 1 8 4 2 3 3 2 9 4 3 6 2 8
  63. 63. MaxPooling2D • Remain maximum value in the kernel © 2020, Retrieva, Inc. All rights reserved. 63 10 4 5 2 1 5 7 1 8 4 2 3 3 2 9 4 3 6 2 8 6
  64. 64. MaxPooling2D • Remain maximum value in the kernel © 2020, Retrieva, Inc. All rights reserved. 64 6 10 7 8 9 8
  65. 65. MaxPooling2D • Remain maximum value in the kernel © 2020, Retrieva, Inc. All rights reserved. 65 6 10 7 8 9 8 8 6 5 9 7 8 9 7 10 7
  66. 66. MaxPooling2D • Forward © 2020, Retrieva, Inc. All rights reserved. 66
  67. 67. Simple ConvNet • 1 Convolution + 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 67 no type config Activation parameters 1 Convolution2D 30, (5, 5), 1, 0 ReLU 780 2 MaxPooling2D (2, 2, 0) - 0 3 Affine (4320, 100) ReLU 432,100 4 Affine (100, 10) Softmax 1,010
  68. 68. Convolution2D • Forward • im2col(2x2 kernel) © 2020, Retrieva, Inc. All rights reserved. 68 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34
  69. 69. Convolution2D • Forward • im2col ・ WT + B © 2020, Retrieva, Inc. All rights reserved. 69 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34 (N, 8) (8, out_channel)
  70. 70. Convolution2D • Forward • im2col ・ WT + B © 2020, Retrieva, Inc. All rights reserved. 70 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34 (N, 8) (8, out_channel) o1
  71. 71. o1 o2 Convolution2D • Forward • im2col ・ WT + B © 2020, Retrieva, Inc. All rights reserved. 71 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34 (N, 8) (8, out_channel)
  72. 72. o1 o2 o3 Convolution2D • Forward • im2col ・ WT + B © 2020, Retrieva, Inc. All rights reserved. 72 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34 (N, 8) (8, out_channel)
  73. 73. • Forward • im2col ・ WT + B o1 o2 o3 Convolution2D © 2020, Retrieva, Inc. All rights reserved. 73 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34 (N, 8) (8, out_channel) b1 b2 b3 +
  74. 74. • Forward • im2col ・ WT + B o1 o2 o3 Convolution2D © 2020, Retrieva, Inc. All rights reserved. 74 0 1 5 6 25 26 30 31 1 2 6 7 26 27 31 32 2 3 7 8 27 28 32 33 3 4 8 9 28 29 33 34 (N, 8) (8, out_channel) b1 b2 b3 + (N, out_channel)
  75. 75. Convolution2D • Forward • reshape+transpose © 2020, Retrieva, Inc. All rights reserved. 75 o11 o12 o13 o21 o22 o23 o31 o32 o33 o41 o42 o43 o11 o12 o13 o21 o22 o23 o31 o32 o33 o41 o42 o43 reshape (batch, height, width, channel)
  76. 76. Convolution2D • Forward • reshape+transpose © 2020, Retrieva, Inc. All rights reserved. 76 o11 o12 o13 o21 o22 o23 o31 o32 o33 o41 o42 o43 o11 o12 o13 o21 o22 o23 o31 o32 o33 o41 o42 o43 reshape (batch, height, width, channel) o13 o23 o33 o41 o12 o22 o32 o41 o11 o21 o31 o41 transpose(0, 3, 1, 2)
  77. 77. Convolution2D • Forward © 2020, Retrieva, Inc. All rights reserved. 77
  78. 78. MaxPooling2D • Backward © 2020, Retrieva, Inc. All rights reserved. 78 d1 d2 d3 d4 d5 d6 d7 d8 d9 d d d d d d d 0 3 1 6 10 4 5 2 1 5 7 1 8 4 2 3 3 2 9 4 3 6 2 8 im2col
  79. 79. MaxPooling2D • Backward © 2020, Retrieva, Inc. All rights reserved. 79 d1 d2 d3 d4 d5 d6 d7 d8 d9 d d d d d d d 0 3 1 6 10 4 5 2 1 5 7 1 8 4 2 3 3 2 9 4 3 6 2 8 0 3 10 4 1 5 8 4 1 6 5 2 7 1 2 3 3 2 3 6 7 2 12 5 9 4 2 8 8 1 3 8 im2colinput
  80. 80. MaxPooling2D • Backward © 2020, Retrieva, Inc. All rights reserved. 80
  81. 81. Convolution2D • Backward • dw: colT・dout’ © 2020, Retrieva, Inc. All rights reserved. 81 d1 d2 d3 d4 d5 d6 d7 d8 d9 d d d d d d d d11 d12 d13 d14 d21 d22 d23 d24 d31 d32 d33 d34 d41 d42 d43 d44 d51 d52 d53 d54 d61 d62 d63 d64 reshape+transpose(0, 2, 3, 1) dout’
  82. 82. Convolution2D • Backward • dw: colT・dout’ • db: sum of columns © 2020, Retrieva, Inc. All rights reserved. 82 d1 d2 d3 d4 d5 d6 d7 d8 d9 d d d d d d d d11 d12 d13 d14 d21 d22 d23 d24 d31 d32 d33 d34 d41 d42 d43 d44 d51 d52 d53 d54 d61 d62 d63 d64 dout’
  83. 83. Convolution2D • Backward • dw: colT・dout’ • db: sum of columns © 2020, Retrieva, Inc. All rights reserved. 83 d1 d2 d3 d4 d5 d6 d7 d8 d9 d d d d d d d d11 d12 d13 d14 d21 d22 d23 d24 d31 d32 d33 d34 d41 d42 d43 d44 d51 d52 d53 d54 d61 d62 d63 d64
  84. 84. Convolution2D • Backward • dw: colT・dout’ • db: sum of columns • dx: same as MaxPooing2D © 2020, Retrieva, Inc. All rights reserved. 84 d1 d2 d3 d4 d5 d6 d7 d8 d9 d d d d d d d 0 3 1 6 10 4 5 2 1 5 7 1 8 4 2 3 3 2 9 4 3 6 2 8 0 3 10 4 1 5 8 4 1 6 5 2 7 1 2 3 3 2 3 6 7 2 12 5 9 4 2 8 8 1 3 8 w im2colinput
  85. 85. Convolution2D • Backward © 2020, Retrieva, Inc. All rights reserved. 85
  86. 86. Convolution2D • Backward © 2020, Retrieva, Inc. All rights reserved. 86
  87. 87. Result • MNIST(N, 1, 28, 28) © 2020, Retrieva, Inc. All rights reserved. 87
  88. 88. © 2020, Retrieva, Inc. All rights reserved. 88 Is this like Functional programming?
  89. 89. Make it functional • Remove struct! © 2020, Retrieva, Inc. All rights reserved. 89
  90. 90. Make it functional • Forward © 2020, Retrieva, Inc. All rights reserved. 90
  91. 91. Pros and Cons • Pros • Simple(pass only as the function needs) • No protocol needed • Less memory usage • Cons • Pattern match increases • Doctest does not work with pattern match? • Huge result lists are displayed © 2020, Retrieva, Inc. All rights reserved. 91
  92. 92. Pros and Cons • Pros • Simple(pass only as the function needs) • No protocol needed • Less memory usage • Cons • Pattern match increases • Doctest does not work with pattern match? • Huge result lists are displayed © 2020, Retrieva, Inc. All rights reserved. 92
  93. 93. CIFAR10 • 4 Convolution + 2 Affine Network © 2020, Retrieva, Inc. All rights reserved. 93 no type config Activation parameters 1 Convolution2D 32, (3, 3), 1, 1 ReLU 896 2 Convolution2D 32, (3, 3), 1, 1 ReLU 9,248 3 MaxPooling (2, 2, 0) - 0 4 Dropout 0.25 - 0 5 Convolution2D 64, (3, 3), 1, 1 ReLU 18,496 6 Convolution2D 64, (3, 3), 1, 1 ReLU 36,928 7 MaxPooling (2, 2, 0) - 0 8 Dropout 0.25 - 0 9 Affine (4096, 512) ReLU 2,097,664 10 Dropout 0.5 - 0 4 Affine (512, 10) Softmax 5,130
  94. 94. © 2020, Retrieva, Inc. All rights reserved. 94 Acceleration
  95. 95. Matrex © 2020, Retrieva, Inc. All rights reserved. 95
  96. 96. Matrex • Fast matrix manipulation library with CBLAS © 2020, Retrieva, Inc. All rights reserved. 96
  97. 97. Matrex • Fast matrix manipulation library with CBLAS • Problem • Only for matrix • Huge cost to create Matrex struct © 2020, Retrieva, Inc. All rights reserved. 97
  98. 98. NIF © 2020, Retrieva, Inc. All rights reserved. 98
  99. 99. NIF • Native Implemented Functions • Enable to call C/C++ functions © 2020, Retrieva, Inc. All rights reserved. 99
  100. 100. NIF • Native Implemented Functions • Enable to call C/C++ functions © 2020, Retrieva, Inc. All rights reserved. 100
  101. 101. Bench • Checked by Benchfella © 2020, Retrieva, Inc. All rights reserved. 101
  102. 102. Tips © 2020, Retrieva, Inc. All rights reserved. 102
  103. 103. Use Float for tests • Numbers are automatically converted to chars © 2020, Retrieva, Inc. All rights reserved. 103
  104. 104. Memory usage • Explicitly use processes • Avoid defstruct • Do NOT rush data • Enum.reduce consumes bigger amount of memory © 2020, Retrieva, Inc. All rights reserved. 104
  105. 105. Future work © 2020, Retrieva, Inc. All rights reserved. 105
  106. 106. Future work • Use OpenCL(NIF) • Implement more layers • Apply Pelemay • Apply Cockatrice © 2020, Retrieva, Inc. All rights reserved. 106
  107. 107. Github • Release in a few days https://github.com/muramasa8191/broca © 2020, Retrieva, Inc. All rights reserved. 107
  108. 108. Reference [1] K. Saito, “ゼロから作るDeep Learning”, O’REILLY Japan, 2016 © 2020, Retrieva, Inc. All rights reserved. 108
  109. 109. © 2020, Retrieva, Inc. All rights reserved.

×