Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rajat Monga at AI Frontiers: Deep Learning with TensorFlow

3,762 views

Published on

In this talk at AI Frontiers Conference, Rajat Monga shares about TensorFlow that has enabled cutting-edge machine learning research at the top AI labs in the world. At the same time it has made the technology accessible to a large audience leading to some amazing uses. TensorFlow is used for classification, recommendation, text parsing, sentiment analysis and more. This talk goes over the design that makes it fast, flexible, and easy to use, and describe how we continue to make it better.

Published in: Technology
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Rajat Monga at AI Frontiers: Deep Learning with TensorFlow

  1. 1. Deep Learning with TensorFlow Rajat Monga Engineering Director, TensorFlow
  2. 2. How complex are Deep Learning Systems?
  3. 3. Neural networks
  4. 4. Inception (2015)
  5. 5. That’s not all... Input Parameters Prediction
  6. 6. That’s not all... Input Parameters Labels Prediction Loss Update
  7. 7. Parameters Labels Prediction Loss Update Input batch ... That’s not all...
  8. 8. That’s not all... Input batch ... Parameters Labels Prediction Loss Update Parameter Server
  9. 9. That’s not all... Input batch ... Parameters Update Model Worker
  10. 10. UpdateUpdateUpdate Model That’s not all... Input batch ... Parameters Update Model Model Model ...
  11. 11. Raspberry Pi Android iOS Portability is a requirement TPU GPU CPU
  12. 12. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  13. 13. repository for “machine learning” category on GitHub #1
  14. 14. 12,000+ commits since Nov, 2015 570+ contributors 1M+ binary downloads 5000+ TensorFlow related repositories on GitHub #15 most popular repository on GitHub by stars - across all categories Used in ML classes at many universities: Toronto, Berkeley, Stanford, ... Some Stats
  15. 15. TensorFlow powered Cucumber Sorter From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/
  16. 16. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  17. 17. Platforms
  18. 18. Languages
  19. 19. TensorFlow Architecture CPU GPU Android iOS ... C++ front end Python front end ... Kernels Bindings + Compound Ops add mul print reshape ... Core TensorFlow Execution System Ops
  20. 20. Ecosystem
  21. 21. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  22. 22. Parallelism in Op implementations MatMul ... ...
  23. 23. ... ... ... ... ...MatMul MatMul Task Parallelism in DataFlow graph
  24. 24. Data Parallelism MatMul Input MatMulParam ... ...MatMul MatMul ... ...
  25. 25. Model Parallelism MatMul Matrix Matrix MatMul Split Concat
  26. 26. GPU 0 CPU Add Mul biases learning rate −=... ... Distribution across Devices
  27. 27. GPU 0 CPU Add Mul biases learning rate −=... ... Distribution ● TensorFlow inserts Send/Recv Ops to transport tensors across devices ● Recv ops pull data from Send ops Send Recv
  28. 28. GPU 0 CPU Add Mul biases learning rate −=... ... Distribution ● TensorFlow inserts Send/Recv Ops to transport tensors across devices ● Recv ops pull data from Send ops Send Recv Send Recv Send Recv Send Recv
  29. 29. Just-In-Time Compilation 0x00000000 movq (%rdx), %rax 0x00000003 vmovaps (%rax), %xmm0 0x00000007 vmulps %xmm0, %xmm0, %xmm0 0x0000000b vmovaps %xmm0, (%rdi) ... TF graphs go in, Optimized & specialized assembly comes out. XLA: “Accelerated Linear Algebra” Compiler
  30. 30. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  31. 31. MatMul Add Relu biases weights input input = ... biases = tf.get_variable(’biases’, ...) weights = tf.get_variable(’weights’, ...) out = tf.matmul(input, weights) out = tf.add(out, biases) out = tf.nn.relu(out)
  32. 32. MatMul Add Relu biases weights input input = ... output = tf.contrib.layers.fully_connected(input, ...)
  33. 33. MatMul Add Relu biases weights input input = ... output = tf.contrib.layers.fully_connected(input, ...)
  34. 34. Align cognitive model with programming model Each box is one line of code!
  35. 35. TensorFlow contains complete algorithms Linear{Classifier,Regressor} DNN{Classifier,Regressor} DNNLinearCombined{Classifier,Regressor} SVM KMeansClustering GMM ...
  36. 36. Simple machine learning classifier = learn.LinearClassifier(feature_columns=feature_columns, n_classes=10) classifier.fit(data, labels, batch_size=100, steps=1000) classifier.evaluate(eval_data, eval_labels) Tooling provided for distributed training and evaluation, graphical debugging, and export to production server (tensorflow/serving).
  37. 37. UpdateUpdateUpdate Model Remember this? Input batch ... Parameters Update Model Model Model ...
  38. 38. TensorFlow handles complexity for you... ...so you can focus on your ideas
  39. 39. Rajat Monga @rajatmonga Thank You!

×