Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Stock prediction
Next
Download to read offline and view in fullscreen.

Share

Deep dive into deeplearn.js

Download to read offline

deeplearn.js is a deep learning library running on Web Browser accelerated by WebGL

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Deep dive into deeplearn.js

  1. 1. Deep Dive into deeplearn.js Kai Sasaki Treasure Data Inc
  2. 2. About me Kai Sasaki - 佐々木 海 (@Lewuathe) Software Engineer at Treasure Data Apache Hivemall Committer Hadoop, Spark, deeplearnjs Contributor
  3. 3. What is deeplearn.js? deeplearn.js is a deep learning library running on your browser accelerated by WebGL. Mainly developed by Google PAIR project. https://research.googleblog.com/2017/08/harness-power-of-machine-learning-in.html
  4. 4. Demo https://deeplearnjs.org/ - SqueezeNet - Teachable Machine
  5. 5. Similar Products WebDNN - https://mil-tokyo.github.io/webdnn/ Keras.js - https://github.com/transcranial/keras-js
  6. 6. Technology - Written in TypeScript - Available in TypeScript and ES6 project - Accelerated by WebGL - Tested on Chrome and Firefox - Can run both training and inference https://www.npmjs.com/ types
  7. 7. Architecture Graph: Representing computation data flow
 much like graph in TensorFlow etc Op: A graph node that represents a
 computation on tensor NDArray: Real data copied from/to CPU 
 and GPU backed by Typed Array
  8. 8. Computation Graph
  9. 9. Graph Node Node Node Tensor Tensor Tensor
  10. 10. Graph Node Node Node Node Node Tensor Tensor Tensor
  11. 11. Graph Node Node Node Node Node Node Tensor Tensor Tensor Tensor Tensor add multiply conv2d relu etc
  12. 12. Operation Op Op Op Op Op Op NDArray NDArray NDArray NDArray NDArray
  13. 13. Kernel Implementation
  14. 14. math Web Browser WebGL
  15. 15. math NDArrayMathCPU NDArrayMathGPU Web Browser WebGL
  16. 16. math math NDArrayMathCPU NDArrayMathGPU Web Browser WebGL kernelNDArray
  17. 17. math math NDArrayMathCPU NDArrayMathGPU Web Browser WebGL kernelNDArray Conv2d softmax etc… Graph matmul
  18. 18. Kernel Example import {Array1D, NDArrayMathGPU, Scalar} from 'deeplearn'; const math = new NDArrayMathGPU(); const a = Array1D.new([1, 2, 3]); const b = Scalar.new(2); const result = math.add(a, b); // Option 1: With async/await. // Caveat: in non-Chrome browsers you need to put this in an async function. console.log(await result.data()); // Float32Array([3, 4, 5]) // Option 2: With a Promise. result.data().then(data => console.log(data)); // Option 3: Synchronous download of data. // This is simpler, but blocks the UI until the GPU is done. console.log(result.dataSync());
  19. 19. TensorFlow like Example const graph = new Graph(); // Make a new input in the graph, called 'x', with shape [] (a Scalar). const x: Tensor = graph.placeholder('x', []); // Make new variables in the graph, 'a', 'b', 'c' with shape [] and random // initial values. const a: Tensor = graph.variable('a', Scalar.new(Math.random())); const b: Tensor = graph.variable('b', Scalar.new(Math.random())); const c: Tensor = graph.variable('c', Scalar.new(Math.random())); //… // At this point the graph is set up, but has not yet been evaluated. // **deeplearn.js** needs a Session object to evaluate a graph. const math = new NDArrayMathGPU(); const session = new Session(graph, math);
  20. 20. TensorFlow Integration
  21. 21. Port TensorFlow model deeplearn.js can import TensorFlow model from checkpoint. Need to port the weights from checkpoint files. https://deeplearnjs.org/demos/mnist/mnist.html
  22. 22. Porting weights $ python scripts/dump_checkpoints/dump_checkpoint_vars.py 
 —model_type=tensorflow 
 —output_dir=demos/mnist/ 
 —checkpoint_file=/tmp/tensorflow/mnist/logs/ fully_connected_feed/model.ckpt-1999 # Will save a lot of files # (one file per variable and manifest.json)
  23. 23. Porting weights import {CheckpointLoader} from 'deeplearn'; const varLoader = new CheckpointLoader('.'); varLoader.getAllVariables().then(async vars => { const math = new NDArrayMathGPU(); const hidden1W = vars[‘hidden1/weights’] as Array2D; const hidden1b = vars[‘hidden1/biases’] as Array1D; //… }
  24. 24. GPU Acceleration
  25. 25. WebGL WebGL (Web Graphics Library) is a JavaScript API for rendering interactive 3D and 2D graphics within any compatible web browser without the use of plug-ins. - Mozilla Developer Network
  26. 26. WebGL - Provides the standard API - Implemented by major web browsers such 
 as Chrome, Firefox, Safari and Opera - Hardware Agnostic
  27. 27. WebGL Graphic Pipeline https://www.tutorialspoint.com/webgl/webgl_graphics_pipeline.htm
  28. 28. WebGL Graphic Pipeline http://math.hws.edu/graphicsbook/c6/s4.html
  29. 29. WebGL 1. deeplearnjs compiles GLSL shader 2. Send input NDArray as texture into GPU 3. Run a fragment shader 4. Fetch result NDArray from GPU
  30. 30. 1. Compile Shader float unaryOperation(float x) { return (x < 0.0) ? 0.0 : x; } void main() { float x = getAAtCoords(); // Sample a value from a texture float y = unaryOperation(x); setOutput(y); }
  31. 31. 2. Upload NDArray to GPU gl.bindTexture(gl.TEXTURE_2D, texture); gl.texSubImage2D( gl.TEXTURE_2D, 0, 0, 0, width, height, textureFormat, getTextureType(gl), data); // Texture format is RGBA // Texture type is gl.UNSIGNED_BYTE or gl.FLOAT gl.bindTexture(gl.TEXTURE_2D, null);
  32. 32. 3. Run shader program gl.bindBuffer(gl.ARRAY_BUFFER, buffer); // Map buffer data to texture coordination gl.vertexAttribPointer( loc, arrayEntriesPerItem, gl.FLOAT, false, 
 itemStrideInBytes, itemOffsetInBytes); // Draw something gl.drawElements(gl.TRIANGLES, 6, gl.UNSIGNED_SHORT, 0);
  33. 33. 4. Download the result getBufferSubDataAsyncExtension.getBufferSubDataAsync( gl2.PIXEL_PACK_BUFFER, 0, downloadTarget); // Download as Typed Array -> NDArray // Typed Array is backed by raw buffer memory (ArrayBuffer) 
 // accessed through views (e.g. Float32Array) https://www.khronos.org/registry/webgl/extensions/WEBGL_get_buffer_sub_data_async/
  34. 34. Future Works
  35. 35. Automatic TensorFlow to deeplearn.js Direct importing from TensorFlow GraphDef format makes model importing easier. https://deeplearnjs.org/docs/roadmap.html
  36. 36. Eager Mode Eager execution mode enables us to run computation without Graph definition. Similar to TensorFlow eager mode.
  37. 37. Decoupling NDArray from storage mechanism NDArray is now tightly coupled with storage layer (GPU texture) Tracking NDArray to avoid memory leak often becomes hard work.
  38. 38. Great Demos One of the reason we use deeplearn.js is that it enables us to create interesting application by using deep learning easily. deeplearn.js is now enhancing demos to show the potential of deep learning on Web Browser.
  39. 39. Patches Welcome!
  • ManrajSingh4

    Aug. 18, 2018
  • roadlabs

    Feb. 20, 2018
  • atsushineki

    Nov. 25, 2017
  • hunter9x

    Nov. 22, 2017
  • RyousukeNakazato

    Nov. 17, 2017

deeplearn.js is a deep learning library running on Web Browser accelerated by WebGL

Views

Total views

2,769

On Slideshare

0

From embeds

0

Number of embeds

839

Actions

Downloads

38

Shares

0

Comments

0

Likes

5

×