Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning with Spark

989 views

Published on

How to get the best of both worlds : Big Data and Data Science?
Run Deep Learning on Spark easily with BigDL library!
Slides of my short conference, introduction to BigDL, for Christmas JUG event in Montpellier

Published in: Technology

Deep Learning with Spark

  1. 1. Deep Learning with Spark Anastasia Lieva Fuzzy Humanist, Data-Scientist @lievAnastazia
  2. 2. Spark is a new Hero
  3. 3. Deep Learning is a new Hero
  4. 4. BigDL is a new epic story
  5. 5. BigDL High-level deep learning library
  6. 6. BigDL High-level deep learning library
  7. 7. BigDL High-level deep learning library Intel MKL Scale-out w/ Spark
  8. 8. BigDL Intel MKL
  9. 9. BigDL : Deep Learning on Spark
  10. 10. BigDL : Deep Learning on Spark
  11. 11. API: Scala and Python
  12. 12. API: Scala and Python BUT
  13. 13. API: Scala and Python BUT the disadvantage of all Python APIs is
  14. 14. API: Scala and Python BUT the disadvantage of all Python APIs is that they are written in Python
  15. 15. API: Scala a̶̶̶n̶̶̶d̶̶̶ ̶̶̶P̶̶̶y̶̶̶t̶̶̶h̶̶̶o̶̶̶n̶̶̶
  16. 16. val conf = Engine.createSparkConf() .setAppName("DeepLearningOnSpark") .setMaster("local[3]") val sparkSession = SparkSession.builder() .config(conf).getOrCreate() val sqlContext = sparkSession.sqlContext val sparkContext = sparkSession.sparkContext Engine.init The same configs as Spark
  17. 17. val conf = Engine.createSparkConf() .setAppName("DeepLearningOnSpark") .setMaster("local[3]") val sparkSession = SparkSession.builder() .config(conf).getOrCreate() val sqlContext = sparkSession.sqlContext val sparkContext = sparkSession.sparkContext Engine.init The same configs as Spark
  18. 18. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Model Architecture
  19. 19. Tensor https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/linear_algebra.html DATA
  20. 20. Tensor Sparse TensorTable Sample DATA
  21. 21. Tensor Sparse TensorTable Sample Lua / Torch Tables (Tensor of Features, Tensor of Targets) Tensor(indices, values, shape) DATA
  22. 22. Tensor Sparse TensorTable Sample Mini-batch Batch of Samples DATA
  23. 23. DATA Tensor Sparse TensorTable Sample Mini-batch DataSet For advanced applications only
  24. 24. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Model Architecture
  25. 25. More than 100 layers ! Embedding Pooling Convolution Normalization Reccurent DropOut Sparse … and others Layers
  26. 26. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Expected Learning by Backpropagation
  27. 27. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Prediction Learning by Backpropagation
  28. 28. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Prediction Ground truth Error Learning by Backpropagation
  29. 29. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Prediction Ground truth Error Update weights in every layer w/ an optimization algorithm Learning by Backpropagation
  30. 30. L A Y E R 1 L A Y E R 2 L A Y E R 3 L A Y E R 4 L A Y E R 5 Input Data Prediction Ground truth Error Update weights in every layer w/ an optimization algorithm Retry prediction with updated weights Learning by Backpropagation
  31. 31. Losses More than 30 criterions : mean squared error, binary cross entropy, negative log likelihood criterion, KL-divergence of the Gaussian distribution...
  32. 32. Losses More than 30 criterions : mean squared error, binary cross entropy, negative log likelihood criterion, KL-divergence of the Gaussian distribution... Optimization algorithms Most popular gradient descent algorithms : SGD, Adam, Adagrad, Adadelta, AdaMax
  33. 33. Let’s predict something!
  34. 34. Let’s predict something! X X
  35. 35. Let’s predict something! X X Good BadMore Or Less
  36. 36. RegexTokenizer() Word2Vec() SpakMLlib Preprocess unstructured data
  37. 37. RegexTokenizer() Word2Vec() Tensor[Vector] Sample(featureTensor, label) SpakMLlib BigDL Preprocess unstructured data
  38. 38. http://intellabs.github.io/RiverTrail/tutorial/ Convolutional Neural Network
  39. 39. Convolutional Neural Network
  40. 40. Convolutional Neural Network Bonjour, on recrute à Montpellier (#systeme, reseau , #Devops, #Linux ). n'hésitez pas à postuler et à diffuser, Merci beaucoup . PS nous ne sommes pas une SSII
  41. 41. Convolutional Neural Network Bonjour, on recrute à Montpellier (#systeme, reseau , #Devops, #Linux ). n'hésitez pas à postuler et à diffuser, Merci beaucoup . PS nous ne sommes pas une SSII Montpellier #systeme, reseau , #Devops, #Linux pas une SSII
  42. 42. Convolutional Neural Network Bonjour, on recrute à Montpellier (#systeme, reseau , #Devops, #Linux ). n'hésitez pas à postuler et à diffuser, Merci beaucoup . PS nous ne sommes pas une SSII Montpellier #systeme, reseau , #Devops, #Linux pas une SSII $$$$$ ?
  43. 43. Convolutional Neural Network Bonjour, on recrute à Montpellier (#systeme, reseau , #Devops, #Linux ). n'hésitez pas à postuler et à diffuser, Merci beaucoup . PS nous ne sommes pas une SSII Montpellier #systeme, reseau , #Devops, #Linux pas une SSII $$$$$ ? Bad
  44. 44. T E M P O R A L Conv R E L U T E M P O R A L MaxP ool L I N E A R D R O P O U T R E L U L I N E A R L O G S O F T M A X Model Architecture
  45. 45. val model = Sequential[Double]() .add(TemporalConvolution(inputSize, outputSizeTempConv, kernelSize)) .add(ReLU()) .add(TemporalMaxPooling(outputSizeMaxPooling) .add(Linear(inputSizeLinearLayer, outputSizeLinearLayer)) .add(Dropout(0.1)) .add(ReLU()) .add(Linear(inputSizeLinearLayer2, outputSizeLinearLayer2)) .add(LogSoftMax()) Model Architecture In BigDL
  46. 46. val criterion = new ClassNLLCriterion[Double] val optimizer = Optimizer(model, trainData, criterion, batchSize) optimizer .setOptimMethod( new Adagrad(learningRate, learningRateDecay)) .optimize() Training model In BigDL
  47. 47. val criterion = new ClassNLLCriterion[Double] val optimizer = Optimizer(model, trainData, criterion, batchSize) optimizer .setOptimMethod( new Adagrad(learningRate, learningRateDecay)) .optimize() Training model In BigDL
  48. 48. val optimizer = Optimizer.apply(model, trainData, criterion, 6) val logdir = "mylogdir" val appName = "job-offers-filter" val trainSummary = TrainSummary(logdir, appName) val validationSummary = ValidationSummary(logdir, appName) optimizer.setTrainSummary(trainSummary) optimizer.setValidationSummary(validationSummary) optimizer .setOptimMethod( new Adagrad(learningRate = 0.01, learningRateDecay = 0.0002)) .optimize() Config for tensorboard
  49. 49. BigDL & Tensorboard
  50. 50. BigDL & Tensorboard
  51. 51. Spark Pipelines Integration
  52. 52. Preprocess unstructured data RegexTokenizer() Word2Vec() Dataframe .select(“features”, “label”) SpakMLlib
  53. 53. val model = Sequential[Double]() .add(TemporalConvolution(100, 20, 5)) .add(ReLU()) .add(TemporalMaxPooling(96)) .add(Linear(20, 100)) .add(Dropout(0.1)) .add(ReLU()) .add(Linear(100, 3)) .add(LogSoftMax()) val criterion = new ClassNLLCriterion[Double] Spark Integration
  54. 54. Spark Integration val estimator = new DLEstimator(model, criterion, featureSize, labelSize) .setLearningRate(0.01) .setBatchSize(6) val trainedModel = estimator.fit(trainDataframe) val predictions = trainedModel.transform(testDataframe)
  55. 55. Spark Integration val estimator = new DLEstimator(model, criterion, featureSize, labelSize) .setLearningRate(0.01) .setBatchSize(6) val trainedModel = estimator.fit(trainDataframe) val predictions = trainedModel.transform(testDataframe) Estimator Transformer
  56. 56. Interoperability Your model BigDL Torch Tensor flowCaffe Keras
  57. 57. Post more job offers on comm-montpellier.slack ! https://bit.ly/comm-mtp offres qualifiées correctement tant sur le domaine, les technos que la fourchette salariale. Ou à minima avec un pitch marrant ;)

×