Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016

1,433 views

Published on

Machine Learning with TensorFlow: TensorFlow has enabled cutting-edge machine learning research at the top AI labs in the world. At the same time it has made the technology accessible to a large audience leading to some amazing uses. TensorFlow is used for classification, recommendation, text parsing, sentiment analysis and more. This talk will go over the design that makes it fast, flexible, and easy to use, and describe how we continue to make it better.

Published in: Technology
  • Be the first to comment

Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016

  1. 1. Deep Learning with TensorFlow Rajat Monga Engineering Director, TensorFlow
  2. 2. How complex are Machine Learning Systems?
  3. 3. Neural networks
  4. 4. Inception (2015)
  5. 5. That’s not all... Input Parameters Labels Prediction Loss Update
  6. 6. Parameters Labels Prediction Loss Update Input batch ... That’s not all...
  7. 7. That’s not all... Input batch ... Parameters Labels Prediction Loss Update Parameter Server
  8. 8. That’s not all... Input batch ... Parameters Update Model Worker
  9. 9. UpdateUpdateUpdate Model That’s not all... Input batch ... Parameters Update Model Model Model ... Parallelize this! Distribute this!
  10. 10. Raspberry Pi Android iOS Portability is a requirement TPU GPU CPU
  11. 11. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  12. 12. repository for “machine learning” category on GitHub #1 Released in Nov. 2015
  13. 13. 10,000+ commits since Nov, 2015 490+ contributors 1M+ binary downloads #20 most popular repository on GitHub by stars Used in ML classes at many universities: Toronto, Berkeley, Stanford, ... Some Stats
  14. 14. TensorFlow powered Cucumber Sorter From: http://workpiles.com/2016/02/tensorflow-cnn-cucumber/
  15. 15. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  16. 16. Platforms
  17. 17. Languages
  18. 18. TensorFlow Architecture CPU GPU Android iOS ... C++ front end Python front end ... Kernels Bindings + Compound Ops add mul Print reshape ... Core TensorFlow Execution System Ops
  19. 19. Ecosystem
  20. 20. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  21. 21. Parallelism in Op implementations MatMul ... ...
  22. 22. ... ... ... ... ...MatMul MatMul Task Parallelism in DataFlow graph
  23. 23. Data Parallelism MatMul Input MatMulParam ... ...MatMul MatMul ... ...
  24. 24. Model Parallelism MatMul Matrix Matrix MatMul Split Concat
  25. 25. GPU 0 CPU Add Mul biases learning rate −=... ... Distribution across Devices
  26. 26. GPU 0 CPU Add Mul biases learning rate −=... ... Distribution ● TensorFlow inserts Send/Recv Ops to transport tensors across devices ● Recv ops pull data from Send ops Send Recv
  27. 27. GPU 0 CPU Add Mul biases learning rate −=... ... Distribution ● TensorFlow inserts Send/Recv Ops to transport tensors across devices ● Recv ops pull data from Send ops Send Recv Send Recv Send Recv Send Recv
  28. 28. Scale across machines
  29. 29. machine learning gets complex quickly Modeling complexityHeterogenous System Distributed System
  30. 30. MatMul Add Relu biases weights input input = ... biases = tf.get_variable(’biases’, ...) weights = tf.get_variable(’weights’, ...) out = tf.matmul(input, weights) out = tf.add(out, biases) out = tf.nn.relu(out)
  31. 31. MatMul Add Relu biases weights input input = ... output = tf.contrib.layers.fully_connected(input, ...)
  32. 32. Align cognitive model with programming model Each box is one line of code!
  33. 33. TensorFlow contains complete algorithms Linear{Classifier,Regressor} DNN{Classifier,Regressor} DNNLinearCombined{Classifier,Regressor} SVM KMeansClustering GMM ...
  34. 34. Simple machine learning classifier = learn.LinearClassifier(feature_columns=feature_columns, n_classes=10) classifier.fit(data, labels, batch_size=100, steps=1000) classifier.evaluate(eval_data, eval_labels) Tooling provided for distributed training and evaluation, graphical debugging, and export to production server (tensorflow/serving).
  35. 35. UpdateUpdateUpdate Model Remember this? Input batch ... Parameters Update Model Model Model ... Parallelize this! Distribute this!
  36. 36. TensorFlow handles complexity for you... ...so you can focus on your ideas
  37. 37. 41 Ready to use Machine Learning models Use your own data to train models Cloud Vision API Cloud Translate API Cloud Machine Learning Develop - Model - Test Google BigQuery Cloud Storage Cloud Datalab Beta GA BetaGA Beta GA GA Cloud Natural Language API Beta Cloud Speech API TensorFlow
  38. 38. Cats
  39. 39. AutoTrash http://myplace.frontier.com/~r.bond/cats/cats.htm
  40. 40. AutoTrash http://myplace.frontier.com/~r.bond/cats/cats.htm
  41. 41. Rajat Monga @rajatmonga Thank You!

×