JD
Uploaded byJason Dai
PDF, PPTX509 views

Build Deep Learning Applications for Big Data Platforms (CVPR 2018 tutorial)

This document outlines an agenda for a talk on building deep learning applications on big data platforms using Analytics Zoo. The agenda covers motivations around trends in big data, deep learning frameworks on Apache Spark like BigDL and TensorFlowOnSpark, an introduction to Analytics Zoo and its high-level pipeline APIs, built-in models, and reference use cases. It also covers distributed training in BigDL, advanced applications, and real-world use cases of deep learning on big data at companies like JD.com and World Bank. The talk concludes with a question and answer session.

Embed presentation

Download as PDF, PPTX
CVPR 2018 Building Deep Learning Applications on Big Data Platform An Introduction to Analytics Zoo for Apache Spark and BigDL Jason Dai
CVPR 2018 Agenda • Motivation (10 minutes) • Trends, real-world scenarios • DL frameworks on Apache Spark (20 minutes) • BigDL, TensorFlowOnSpark, DL Pipelines, SparkNet • Analytics Zoo for Spark and BigDL (15 minutes) • High level pipeline APIs, feature engineering, built-in models, reference use cases • Analytics Zoo Examples (15 minutes) • Dogs vs. cats, object detection, TFNet • Break (30 minutes)
CVPR 2018 Agenda • Distributed training in BigDL (30 minutes) • Data parallel training, parameter synchronization, scaling & convergence, task scheduling, etc. • Advanced applications (15 minutes) • Variational autoencoder, movie recommendations • Real-world applications (30 minutes) • Object detection and image feature extraction at JD.com • Image similarity based house recommendation for MLSlistings • Transfer learning based image classifications for World Bank • Fraud detection for payment transactions for UnionPay • Conclusion and Q&A (15 minutes)
CVPR 2018 Motivations Technology and Industry Trends Real World Scenarios
CVPR 2018 Trend #1: Data Scale Driving Deep Learning Process “Machine Learning Yearning”, Andrew Ng, 2016
CVPR 2018 Trend #2: Hadoop Becoming the Center of Data Gravity Phillip Radley, BT Group Strata + Hadoop World 2016 San Jose Matthew Glickman, Goldman Sachs Spark Summit East 2015
CVPR 2018 Trend #3: Real-World ML/DL Systems Are Complex Big Data Analytics Pipelines “Hidden Technical Debt in Machine Learning Systems”, Sculley et al., Google, NIPS 2015 Paper
CVPR 2018 Trend #4: Unified Big Data Platform Driving Analytics & Data Science Ion Stoica, UC Berkeley, Spark Summit 2013 Keynote
CVPR 2018 Unified Big Data Analytics Platform Data Input Flume Kafka Storage HBaseHDFS Resource Mgmt & Co-ordination ZooKeeperYARN Data Processing & Analysis MR Storm Apache Hadoop & Spark Platform Parquet Avro Spark Core SQL Streaming MLlib GraphX DataFrame ML Pipelines SparkR Flink Giraph Batch Streaming Interactive Machine Leaning Graph Analytics SQL R PythonJava Notebook Spreadsheet
CVPR 2018 Chasm b/w Deep Learning and Big Data Communities Average users (big data users, data scientists, analysts, etc.)Deep learning experts The Chasm
CVPR 2018 Large-Scale Image Recognition at JD.com
CVPR 2018 Bridging the Chasm Make deep learning more accessible to big data and data science communities • Continue the use of familiar SW tools and HW infrastructure to build deep learning applications • Analyze “big data” using deep learning on the same Hadoop/Spark cluster where the data are stored • Add deep learning functionalities to large-scale big data programs and/or workflow • Leverage existing Hadoop/Spark clusters to run deep learning applications • Shared, monitored and managed with other workloads (e.g., ETL, data warehouse, feature engineering, traditional ML, graph analytics, etc.) in a dynamic and elastic fashion
CVPR 2018 DL Frameworks on Apache Spark BigDL, DL Pipelines for Spark, TensorflowOnSpark, SparkNet, etc.
CVPR 2018 Apache Spark Spark Job Driver Spark Task Worker Spark Task Spark Task Spark Task Worker Spark Task Spark Task • A Spark cluster consists of a single driver node and multiple worker nodes • A Spark job contains many Spark tasks, each working on a data partition • Driver is responsible for scheduling and dispatching the tasks to workers, which runs the actual Spark tasks Low Latency, Distributed Data Processing Framework https://spark.apache.org
CVPR 2018 Apache Spark Spark Program • Spark runs as a library in your program (1 instance per app) • Runs tasks locally or on cluster • K8s, YARN, Mesos or standalone mode • Accesses storage systems via Hadoop InputFormat API • Can use HBase, HDFS, S3, … Your application SparkContext Local threads Cluster manager Worker Spark executor Worker Spark executor HDFS or other storageSource: “Parallel programming with Spark”, Matei Zaharia, AMPCamp 3
CVPR 2018 Apache Spark join union groupBy map Stage 3 Stage 1 Stage 2 A: B: C: D: E: F: G: Task = Cached Partition= RDD • General task graphs • Automatically pipelines functions • Data locality aware • Partitioning aware to avoid shuffles Source: “Parallel programming with Spark”, Matei Zaharia, AMPCamp 3 Distributed Task Execution
CVPR 2018 BigDL Bringing Deep Learning To Big Data Platform https://github.com/intel-analytics/BigDL Spark Core SQL SparkR Streaming MLlib GraphX ML Pipeline DataFrame https://bigdl-project.github.io/ • Distributed deep learning framework for Apache Spark* • Make deep learning more accessible to big data users and data scientists • Write deep learning applications as standard Spark programs • Run on existing Spark/Hadoop clusters (no changes needed) • Feature parity with popular deep learning frameworks • E.g., Caffe, Torch, Tensorflow, etc. • High performance (on CPU) • Powered by Intel MKL and multi-threaded programming • Efficient scale-out • Leveraging Spark for distributed training & inference
CVPR 2018 BigDL Run as Standard Spark Programs Spark Program DL App on Driver Spark Executor (JVM) Spark Task BigDL lib Worker Intel MKL Standard Spark jobs Worker Worker Worker Worker Spark Executor (JVM) Spark Task BigDL lib Worker Intel MKL BigDL library Spark jobs BigDL Program Standard Spark jobs • No changes to the Spark or Hadoop clusters needed Iterative • Each iteration of the training runs as a Spark job Data parallel • Each Spark task runs the same model on a subset of the data (batch)
CVPR 2018 Distributed Training in BigDL … Training Set Parameter Server Architecture directly inside Spark (using Block Manager) Partition 1 Partition 2 Partition n Worker Gradient 1 2 Weight 3 4 5 Worker Gradient 1 2 Weight 3 4 5 Worker Gradient 1 2 Weight 3 4 5 … … … … … … … Peer-2-Peer All-Reduce synchronization
CVPR 2018 TensoflowOnSpark Spark Task Python Queue TF Worker Func Worker Spark Task Worker TF PS Func Spark Task Python Queue TF Worker Func Worker Spark Task TF Worker Func Worker Spark Task Worker TF PS Func Spark Task TF Worker Func Worker HDFS feed_dict: TF worker func runs as independent process in background, reading data from Python queue queue_runner: direct HDFS access from TF work func Standalone TF jobs on Spark cluster • Use Spark as the orchestration layer to allocate resources • Launch distributed TensorFlow job on the allocated resources • Coarse-grained integration of two independent frameworks • Memory overheads, no gang scheduling, limited interactions with data pipelines, etc. https://github.com/yahoo/TensorFlowOnSpark
CVPR 2018 DL Pipelines for Spark Spark ML Pipelines Dataframe Dataframe Dataframe Transformer Transformer Transformer Estimator Source: “Deep Dive into Deep Learning Pipelines”, Sue Ann Hong and Tim Hunter, Spark Summit EU 2017 Load existing TF or Keras models in Spark ML pipelines • Load into transformer: inference only • Load into estimator: single node training/tuning only https://github.com/databricks/spark-deep-learning
CVPR 2018 SparkNet Source: “SparkNet: Training Deep Networks in Spark”, Philipp Moritz, et al., ICLR 2016 Distributed DL training by running Caffe in each worker • Asynchronous parameter synchronization through master (driver) mode • Very inefficient (~20 seconds with just 5 workers)
CVPR 2018 Analytics Zoo Analytics + AI Platform for Spark and BigDL https://github.com/intel-analytics/analytics-zoo
CVPR 2018 Analytics Zoo Build and Productionize Deep Learning Apps for Big Data at Scale Reference Use Cases • Anomaly detection • Sentiment analysis • Fraud detection • Chatbot, sequence prediction, etc. Built-In Deep Learning Models • Image classification • Object detection • Text classification • Recommendations • Sequence-to-sequence, GAN, etc. Feature Engineering Feature transformations for • Image, text, 3D imaging, time series, speech, etc. High-Level Pipeline APIs • Native deep learning support in Spark DataFrames and ML Pipelines • Autograd, Keras and transfer learning APIs for model definition • Model serving API for model serving/inference pipelines Backbends Spark, BigDL, TensorFlow, etc. https://github.com/intel-analytics/analytics-zoo/ https://analytics-zoo.github.io/
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications for big data at scale • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
CVPR 2018 nnframes 1. Initialize NNContext and load images into DataFrames using NNImageReader Native DL support in Spark DataFrames and ML Pipelines from zoo.common.nncontext import * from zoo.pipeline.nnframes import * sc = init_nncontext() imageDF = NNImageReader.readImages(image_path, sc) 2. Process loaded data using DataFrame transformations getName = udf(lambda row: ...) df = imageDF.withColumn("name", getName(col("image"))) 3. Processing image using built-in feature engineering operations from zoo.feature.image import * transformer = ChainedPreprocessing( [RowToImageFeature(), ImageChannelNormalize(123.0, 117.0, 104.0), ImageMatToTensor(), ImageFeatureToTensor()])
CVPR 2018 nnframes 4. Define model using Keras-style API Native DL support in Spark DataFrames and ML Pipelines from zoo.pipeline.api.keras.layers import * from zoo.pipeline.api.keras.models import * model = Sequential() .add(Convolution2D(32, 3, 3, activation='relu', input_shape=(1, 28, 28))) .add(MaxPooling2D(pool_size=(2, 2))) .add(Flatten()).add(Dense(10, activation='softmax'))) 5. Train model using Spark ML Pipelines Estimater = NNEstimater(model, CrossEntropyCriterion(), transformer) .setLearningRate(0.003).setBatchSize(40).setMaxEpoch(1) .setFeaturesCol("image").setCachingSample(False) nnModel = estimater.fit(df)
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
CVPR 2018 Autograd, Keras &Transfer Learning APIs from zoo.pipeline.api.net import * full_model = Net.load_caffe(def_path, model_path) # Remove layers after pool5 model = full_model.new_graph(outputs=["pool5"]).to_keras()) # freeze layers from input to res4f inclusive model.freeze_up_to(["res4f"]) # append a few layers image = Input(name="input", shape=(3, 224, 224)) resnet = model.to_keras()(image) resnet50 = Flatten()(resnet) 1. Use transfer learning APIs to • Load an existing Caffe model • Remove last few layers • Freeze first few layers • Append a few layers Build Siamese Network Using Transfer Learning
CVPR 2018 Autograd, Keras &Transfer Learning APIs Build Siamese Network Using Transfer Learning import zoo.pipeline.api.autograd as A from zoo.pipeline.api.keras.layers import * from zoo.pipeline.api.keras.models import * input = Input(shape=[2, 3, 226, 226]) features = TimeDistributed(layer=resnet50)(input) f1 = features.index_select(1, 0) #image1 f2 = features.index_select(1, 1) #image2 diff = A.abs(f1 - f2) fc = Dense(1)(diff) output = Activation("sigmoid")(fc) model = Model(input, output) 2. Use autograd and Keras-style APIs to build the Siamese Network
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
CVPR 2018 Feature Engineering from zoo.common.nncontext import * from zoo.feature.image import * spark = init_nncontext() local_image_set = ImageSet.read(image_path) distributed_image_set = ImageSet.read(image_path, spark, 2) 1. Read images into local or distributed ImageSet 2. Image augmentations using built-in ImageProcessing operations transformer = ChainedPreprocessing([ImageBytesToMat(), ImageColorJitter(), ImageExpand(max_expand_ratio=2.0), ImageResize(300, 300, -1), ImageHFlip()]) new_local_image_set = transformer(local_image_set) new_distributed_image_set = transformer(distributed_image_set) Image Augmentations Using Built-in Image Transformations (w/ OpenCV on Spark)
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
CVPR 2018 Built-in Deep Learning Models • Object detection API • High-level API and pretrained models (e.g., SSD, Faster-RCNN, etc.) for object detection • Image classification API • High-level API and pretrained models (e.g., VGG, Inception, ResNet, MobileNet, etc.) for image classification • Text classification API • High-level API and pre-defined models (using CNN, LSTM, etc.) for text classification • Recommendation API • High-level API and pre-defined models (e.g., Neural Collaborative Filtering, Wide and Deep Learning, etc.) for recommendation
CVPR 2018 Object Detection API from zoo.common.nncontext import * from zoo.models.image.objectdetection import * spark = init_nncontext() model = ObjectDetector.load_model(model_path) 1. Load pretrained model in Detection Model Zoo 2. Off-the-shell inference using the loaded model image_set = ImageSet.read(img_path, spark) output = model.predict_image_set(image_set) Off-the-shell Inference Using Analytics Zoo Object Detection API 3. Visualize the results using utility methods config = model.get_config() visualizer = Visualizer(config.label_map(), encoding="jpg") visualized = visualizer(output).get_image(to_chw=False).collect() https://github.com/intel-analytics/analytics-zoo/tree/master/pyzoo/zoo/examples/objectdetection
CVPR 2018 Reference Use Cases • Anomaly Detection • Using LSTM network to detect anomalies in time series data • Fraud Detection • Using feed-forward neural network to detect frauds in credit card transaction data • Recommendation • Use Analytics Zoo Recommendation API (i.e., Neural Collaborative Filtering, Wide and Deep Learning) for recommendations on data with explicit feedback. • Sentiment Analysis • Sentiment analysis using neural network models (e.g. CNN, LSTM, GRU, Bi-LSTM) • Variational Autoencoder (VAE) • Use VAE to generate faces and digital numbers https://github.com/intel-analytics/analytics-zoo/tree/master/apps
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
CVPR 2018 POJO Model Serving API import com.intel.analytics.zoo.pipeline.inference.AbstractInferenceModel; public class TextClassification extends AbstractInferenceModel { public RankerInferenceModel(int concurrentNum) { super(concurrentNum); } ... } public class ServingExample { public static void main(String[] args) throws IOException { TextClassification model = new TextClassification(); model.load(modelPath, weightPath); texts = … List<JTensor> inputs = preprocess(texts); for (JTensor input : inputs) { List<Float> result = model.predict(input.getData(), input.getShape()); ... } }
CVPR 2018 Model Serving & Inference HDFS/S3 Kafka Flume Kinesis Twitter Spout Analytics Zoo Model Spout Bolt Bolt Bolt Bolt Bolt Analytics Zoo Model Seamless integration in Web Services, Storm, Flink, Kafka, etc. (using POJO local Java APIs)
CVPR 2018 Model Serving in Spark DataFrames Seamless support of DL functionalities in Spark SQL queries, Dataframe operation and stream processing ImageNet dataset (http://www.image-net.org) Kafka File Data Frame (Batch/Stream) Analytics Zoo UDF Filtered Data Frame (Batch/Stream) df.select($’image’) .withColumn( “image_type”, ImgClassifyInAnalyticsZoo(“image”)) .filter($’image_type’ == ‘dog’)
CVPR 2018 Analytics Zoo Build end-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference & fine tuning using TFNet
CVPR 2018 Distributed TensorFlow Model Inference import tensorflow as tf batch_size_tensor = tf.placeholder_with_default(128, shape=[]) x_batch, y_batch = tf.train.shuffle_batch (..., batch_size=batch_size_tensor, ...) cnn = cnn_model_fn(x_batch, y_batch) sess = tf.Session() init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()) sess.run(init_op) for step in range(600): _, loss = sess.run([cnn.train_op, cnn.loss], ...) from zoo.utils.tf import * export_tf(sess, folder_path, [x_batch], [cnn.prediction]) 1. Export TensorFlow models (in your TensorFlow program)
CVPR 2018 Distributed TensorFlow Model Inference 2. Load exported TensorFlow model into Analytics Zoo from zoo.pipeline.api.net import * model = TFNet.from_export_folder(folder_path) # Alternatively, you may directly load a frozen TensorFlow model as follows # model = TFNet(model_path, [“image_tensor:0”], [“output_tensor:0”]) 3. Add a few layers and run distributed model inference import zoo.pipeline.api.autograd as A from zoo.pipeline.api.keras.layers import * from zoo.pipeline.api.keras.models import * input = Input(shape=[2, 3, 226, 226]) features = TimeDistributed(layer=model)(input) f1 = features.index_select(1, 0) f2 = features.index_select(1, 1) diff = A.abs(f1 - f2) result = Model(input, diff) result.predict_image_set(...)
CVPR 2018 Analytics Zoo Examples Dogs vs. cats, object detections, TFNet
CVPR 2018 Dogs vs. Cats https://github.com/intel-analytics/analytics-zoo/blob/master/apps/dogs-vs-cats/transfer-learning.ipynb Notebook:
CVPR 2018 Object Detection API https://github.com/intel-analytics/analytics-zoo/blob/master/apps/object-detection/object-detection.ipynb Notebook:
CVPR 2018 Image Classification Using TFNet https://github.com/intel-analytics/analytics-zoo/blob/master/apps/tfnet/image_classification_inference.ipynb Notebook:
CVPR 2018 Break
CVPR 2018 Distributed Training In BigDL Data parallel training Parameter synchronization Scaling and Convergence Task scheduling “BigDL: A Distributed Deep Learning Framework for Big Data”, https://arxiv.org/abs/1804.05839
CVPR 2018 Apache Spark Spark Job Driver Spark Task Worker Spark Task Spark Task Spark Task Worker Spark Task Spark Task Single master (driver), multiple workers
CVPR 2018 Apache Spark join union groupBy map Stage 3 Stage 1 Stage 2 A: B: C: D: E: F: G: Task = Cached Partition= RDD Source: “Parallel programming with Spark”, Matei Zaharia, AMPCamp 3 Spark compute model • Data parallel • Functional, coarse-grained operators • Immutable RDDs • Applying the same operation (e.g., map, filter, etc.) to all data items
CVPR 2018 Distributed Training in BigDL Data Parallel, Synchronous Mini-Batch SGD Prepare training data as an RDD of Samples Construct an RDD of models (each being a replica of the original model) for (i <- 1 to N) { //”model forward-backward” job for each task in the Spark job: read the latest weights get a random batch of data from local Sample partition compute errors (forward on local model replica) compute gradients (backward on local model replica) //”parameter synchronization” job aggregate (sum) all the gradients update the weights per specified optimization method }
CVPR 2018 Data Parallel Training Worker 2 Worker nWorker 1 Partition 1 Partition 2 Partition n Sample RDD Partition 1 Partition 2 Partition nModel RDD Task n: zip Sample and model RDDs, and compute gradient on co-located Sample and model partitions Task 2Task 1 “Model Forward-Backward” Job
CVPR 2018 Parameter Synchronization Task n 1 2 n 1 2 n 1 2 n 1 2 n 1 2 n Task 1 Task 2 local gradient local gradient local gradient ∑ ∑ ∑ gradient 1 gradient n weight 1 weight 2 weight n “Parameter Synchronization” Job update update update gradient 2
CVPR 2018 Parameter Synchronization For each task n in the ”parameter synchronization” job { shuffle the nth partition of all gradients to this task aggregate (sum) the gradients updates the nth partition of the weights broadcast the nth partition of the updated weights } “Parameter Synchronization” Job (managing nth partition of the parameters - similar to a parameter server) “Parameter Server” style architecture (directly on top of primitives in Spark) • Gradient aggregation: shuffle • Weight sync: task-side broadcast • In-memory persistence
CVPR 2018 Training Scalability Throughput of ImageNet Inception v1 training (w/ BigDL 0.3.0 and dual-socket Intel Broadwell 2.1 GHz); the throughput scales almost linear up to 128 nodes (and continue to scale reasonably up to 256 nodes). Source: Scalable Deep Learning with BigDL on the Urika-XC Software Suite (https://www.cray.com/blog/scalable-deep-learning-bigdl-urika-xc-software-suite/)
CVPR 2018 Increased Mini-Batch Size • Distributed synchronous mini-batch SGD • Increased mini-batch size total_batch_size = batch_size_per_worker * num_of_workers • Can lead to loss in test accuracy • State-of-art method for scaling mini-batch size* • Linear scaling rule • Warm-up strategy • Layer-wise adaptive rate scaling • Adding batch normalization *Source: “Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour”, Priya Goyal, et at. https://arxiv.org/abs/1706.02677 *Source: “ImageNet Training in Minutes”, Yang You, et at. https://arxiv.org/abs/1709.05011
CVPR 2018 Training Convergence: Inception v1 Strategies • Warm-up • Linear scaling • Gradient clipping • TODO: adding batch normalization Source: Very large-scale distributed deep learning with BigDL, Jason Dai and Ding Ding. O’Reilly AI Conference 2017
CVPR 2018 Training Convergence: SSD Strategies • Warm-up • Linear scaling • Gradient clipping Source: Very large-scale distributed deep learning with BigDL, Jason Dai and Ding Ding. O’Reilly AI Conference 2017
CVPR 2018 Difference vs. Classical PS Architecture Classical PS architecture • Multiple long-running, potentially stateful tasks • Interact with each other (in a blocking fashion for synchronization) • Require fine-grained data access and in- place data mutation • Not directly supported by existing big data systems BigDL implementations • Run a series of short-lived Spark jobs (e.g., two jobs per mini-batch) • Each task in the job is stateless and non-blocking • Automatically adapt to the dynamic resource changes (e.g., preemption, failures, resource sharing, etc.) • Built on top of existing primitives in Spark (e.g., shuffle, broadcast, and in- memory data persistence)
CVPR 2018 Task Scheduling Overheads Spark overheads (task scheduling & task dispatch ) as a fraction of average compute time for Inception v1 training BigDL implementations • Run a single, multi- threaded task on each worker • Achieve high scalability on large clusters (e.g., up to 256 servers Source: Accelerating Deep Learning Training with BigDL and Drizzle on Apache Spark, Shivaram Venkataraman, Ding Ding, and Sergey Ermolin. (https://rise.cs.berkeley.edu/blog/accelerating-deep-learning-training-with-bigdl-and-drizzle-on-apache-spark/)
CVPR 2018 Reducing Scheduling Overheads Using Drizzle Scaling to even larger (>500) workers • Iterative model training • Same operations run repeatedly • Drizzle • A low latency execution engine for Spark • Group scheduling for multiple iterations of computations at once Source: Accelerating Deep Learning Training with BigDL and Drizzle on Apache Spark, Shivaram Venkataraman, Ding Ding, and Sergey Ermolin. (https://rise.cs.berkeley.edu/blog/accelerating-deep-learning-training-with-bigdl-and-drizzle-on-apache-spark/)
CVPR 2018 Advanced Analytics Zoo Applications Variational autoencoder, movie recommendations
CVPR 2018 Variational AutoEncoder https://github.com/intel-analytics/analytics-zoo/blob/master/apps/variational- autoencoder/using_variational_autoencoder_to_generate_digital_numbers.ipynb https://github.com/intel-analytics/analytics-zoo/blob/master/apps/variational- autoencoder/using_variational_autoencoder_to_generate_faces.ipynb Notebook:
CVPR 2018 Movie Recommendations https://github.com/intel-analytics/analytics-zoo/blob/master/apps/recommendation/ncf-explicit- feedback.ipynb Notebook:
CVPR 2018 Real-World Applications Object detection and image feature extraction at JD.com Image similarity based house recommendation for MLSlisting Transfer learning based image classifications for World Bank Fraud detection for payment transactions for UnionPay
CVPR 2018 Object Detection and Image Feature Extraction at JD.com
CVPR 2018 Applications Large-scale image feature extraction • Object detect (remove background, optional) • Feature extraction Application • Similar image search • Image Deduplication • Competitive price monitoring • IP (image copyright) protection system Source: “Bringing deep learning into big data analytics using BigDL”, Xianyan Jia and Zhenhua Wang, Strata Data Conference Singapore 2017
CVPR 2018 Similar Image Search Query Search Result Query Search Result ------ Source: “Bringing deep learning into big data analytics using BigDL”, Xianyan Jia and Zhenhua Wang, Strata Data Conference Singapore 2017
CVPR 2018 Challenges of Productionizing Large-Scale Deep Learning Solutions Productionizing large-scale seep learning solutions is challenging • Very complex and error-prone in managing large-scale distributed systems • E.g., resource management and allocation, data partitioning, task balance, fault tolerance, model deployment, etc. • Low end-to-end performance in GPU solutions • E.g., reading images out from HBase takes about half of the total time • Very inefficient to develop the end-to-end processing pipeline • E.g., image pre-processing on HBase can be very complex
CVPR 2018 Production Deployment with Analytics Zoo for Spark and BigDL http://mp.weixin.qq.com/s/xUCkzbHK4K06-v5qUsaNQQ https://software.intel.com/en-us/articles/building-large-scale-image-feature-extraction-with-bigdl-at-jdcom • Reuse existing Hadoop/Spark clusters for deep learning with no changes (image search, IP protection, etc.) • Efficiently scale out on Spark with superior performance (3.83x speed-up vs. GPU severs) as benchmarked by JD
CVPR 2018 Image Similarity Based House Recommendation for MLSlistings https://software.intel.com/en-us/articles/using-bigdl-to-build-image-similarity-based-house-recommendations MLSlistings built image-similarity based house recommendations using BigDL on Microsoft Azure
CVPR 2018 Image Similarity Based House Recommendation for MLSlistings RDD of house photos Is house exterior? {0, 1} House Style {0, 1, 2, 3} Image Processing Is house exterior? {0, 1} House Style {0, 1, 2, 3} Image Processing Is house exterior? {0, 1} House Style {0, 1, 2, 3} Image Processing Is house exterior? {0, 1} Tags (is_exterior, style, floors) of images RDD of house photos Image features Store image tags and feature in table storage Image pre- processing Image pre- processing Pre-trained VGG16 model (to extract features) Three pre-trained Inception v1 models (fine- tuned as classifiers)
CVPR 2018 Image Similarity Based House Recommendation for MLSlistings https://github.com/intel-analytics/analytics-zoo/blob/master/apps/image- similarity/Image%20similarity.ipynb Notebook:
CVPR 2018 Transfer Learning Based Image Classifications for World Bank Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018 Classifying Real Food Images is not a Cat vs. Dog Problem
CVPR 2018 Project Layout Phase 1: • Image preprocessing (eliminate poor quality images and invalid images) • Classify images (by food type) to validate existing labels Phase 2: • Identify texts in the image and make bounding box around them • Text recognition (words/sentences in the image text) • Determine whether text contains PII (personal identifiable information) • Blur areas with PII text Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
CVPR 2018 Code – Phase 1 Fine-tuning Training Prediction and Evaluation Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
CVPR 2018 Result – Phase 1 • Fine tune with Inception v1 on a full dataset • Dataset: 994325 images, 69 categories Nodes Cores Batch Size Epochs Training Time (sec) Throughput (images/sec) Accuracy (%) 20 30 1200 12 61125 170 81.7 * This model training was performed using multinode cluster on AWS R4.8xlarge instance with 20 nodes Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
CVPR 2018 Next Steps – Phase 2 • Image Quality Preprocessing • Filter with print text only • Rescaling, Binarisation, Noise Removal, Rotation / Deskewing (OpenCV, Python, etc.) • Detect text and bounding box circle text • Recognize text • Determine whether text contains PII (personal identifiable information) • Recognize PII with leading words • Blur areas with PII text • Image tools Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
CVPR 2018 Fraud Detection for Payment Transactions for UnionPay Train one model all features selected features model candidate model sampled partition Training Data … normal fraud Train one model Train one model sampled partition sampled partition Post Processing Pre- Processing model model Spark Pipeline Test Data Predictions Test Spark DataFrameHive Table Pre-processing Feature Engineering Feature Engineering Feature Selection Model Ensemble SparkPipeline Neural Network Model Using BigDL Feature Selection* Model Training Model Evaluation & Fine Tune https://mp.weixin.qq.com/s?__biz=MzI3NDAwNDUwNg==&mid=2648307335&idx=1&sn=8eb9f63eaf2e40e24a90601b9cc03d1f
CVPR 2018 Fraud Detection for Payment Transactions for UnionPay https://github.com/intel-analytics/analytics-zoo/blob/master/apps/fraud-detection/fraud-detection.ipynb Notebook:
CVPR 2018 Summary Make deep learning more accessible to big data and data science communities • Analyze “big data” using deep learning on the same Hadoop/Spark cluster where the data are stored • Add deep learning functionalities to large-scale big data programs and/or workflow • Leverage existing Hadoop/Spark clusters to run deep learning applications • Shared, managed and monitored with other workloads (ETL, data warehouse, traditional ML, etc.) Analytics Zoo: https://github.com/intel-analytics/analytics-zoo • End-to-end Analytics + AI platform for Apache Spark and BigDL • Build and productionize deep learning application for Big Data at Scale
CVPR 2018
CVPR 2018 Legal Disclaimers • Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. • No computer system can be absolutely secure. • Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance. Intel, the Intel logo, Xeon, Xeon phi, Lake Crest, etc. are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © 2018 Intel Corporation

More Related Content

PDF
Building AI to play the FIFA video game using distributed TensorFlow on Analy...
byJason Dai
 
PDF
Using BigDL on Apache Spark to Improve the MLS Real Estate Search Experience ...
byDatabricks
 
PPTX
AI from your data lake: Using Solr for analytics
byDataWorks Summit
 
PDF
Distributed Deep Learning At Scale On Apache Spark With BigDL
byYulia Tell
 
PDF
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
byDatabricks
 
PDF
Scalable Machine Learning in R and Python with H2O
bySri Ambati
 
PDF
Bringing an AI Ecosystem to the Domain Expert and Enterprise AI Developer wit...
byDatabricks
 
PDF
Applications of Deep Learning in Telematics
byDatabricks
 
Building AI to play the FIFA video game using distributed TensorFlow on Analy...
byJason Dai
 
Using BigDL on Apache Spark to Improve the MLS Real Estate Search Experience ...
byDatabricks
 
AI from your data lake: Using Solr for analytics
byDataWorks Summit
 
Distributed Deep Learning At Scale On Apache Spark With BigDL
byYulia Tell
 
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
byDatabricks
 
Scalable Machine Learning in R and Python with H2O
bySri Ambati
 
Bringing an AI Ecosystem to the Domain Expert and Enterprise AI Developer wit...
byDatabricks
 
Applications of Deep Learning in Telematics
byDatabricks
 

What's hot

PDF
Accelerate Your AI Today
byDESMOND YUEN
 
PDF
Enabling Python to be a Better Big Data Citizen
byWes McKinney
 
PDF
AI on Spark for Malware Analysis and Anomalous Threat Detection
byDatabricks
 
PDF
Distributed Deep Learning with Hadoop and TensorFlow
byJan Wiegelmann
 
PDF
Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...
byWes McKinney
 
PPTX
END-TO-END MACHINE LEARNING STACK
byJan Wiegelmann
 
PDF
H2O PySparkling Water
bySri Ambati
 
PDF
Creating an end-to-end Recommender System with Apache Spark and Elasticsearch...
bysparktc
 
PPTX
Democratizing AI with Apache Spark
bySpark Summit
 
PPTX
Flux - Open Machine Learning Stack / Pipeline
byJan Wiegelmann
 
PDF
Data Analysis and Statistics in Python using pandas and statsmodels
byWes McKinney
 
PDF
Tiny Batches, in the wine: Shiny New Bits in Spark Streaming
byPaco Nathan
 
PDF
A Spark-Based Intelligent Assistant: Making Data Exploration in Natural Langu...
byDatabricks
 
PPTX
Spark ML Pipeline serving
byStepan Pushkarev
 
PDF
Taking Jupyter Notebooks and Apache Spark to the Next Level PixieDust with Da...
byDatabricks
 
PDF
Building Better Analytics Workflows (Strata-Hadoop World 2013)
byWes McKinney
 
PDF
PyCon Singapore 2013 Keynote
byWes McKinney
 
PDF
The Bitter Lesson of ML Pipelines
byJim Dowling
 
PDF
Introduction to Analytics with Azure Notebooks and Python
byJen Stirrup
 
PDF
Data Warehousing with Spark Streaming at Zalando
byDatabricks
 
Accelerate Your AI Today
byDESMOND YUEN
 
Enabling Python to be a Better Big Data Citizen
byWes McKinney
 
AI on Spark for Malware Analysis and Anomalous Threat Detection
byDatabricks
 
Distributed Deep Learning with Hadoop and TensorFlow
byJan Wiegelmann
 
Practical Medium Data Analytics with Python (10 Things I Hate About pandas, P...
byWes McKinney
 
END-TO-END MACHINE LEARNING STACK
byJan Wiegelmann
 
H2O PySparkling Water
bySri Ambati
 
Creating an end-to-end Recommender System with Apache Spark and Elasticsearch...
bysparktc
 
Democratizing AI with Apache Spark
bySpark Summit
 
Flux - Open Machine Learning Stack / Pipeline
byJan Wiegelmann
 
Data Analysis and Statistics in Python using pandas and statsmodels
byWes McKinney
 
Tiny Batches, in the wine: Shiny New Bits in Spark Streaming
byPaco Nathan
 
A Spark-Based Intelligent Assistant: Making Data Exploration in Natural Langu...
byDatabricks
 
Spark ML Pipeline serving
byStepan Pushkarev
 
Taking Jupyter Notebooks and Apache Spark to the Next Level PixieDust with Da...
byDatabricks
 
Building Better Analytics Workflows (Strata-Hadoop World 2013)
byWes McKinney
 
PyCon Singapore 2013 Keynote
byWes McKinney
 
The Bitter Lesson of ML Pipelines
byJim Dowling
 
Introduction to Analytics with Azure Notebooks and Python
byJen Stirrup
 
Data Warehousing with Spark Streaming at Zalando
byDatabricks
 

Similar to Build Deep Learning Applications for Big Data Platforms (CVPR 2018 tutorial)

PDF
BigDL: Bringing Ease of Use of Deep Learning for Apache Spark with Jason Dai ...
byDatabricks
 
PDF
Integrating Deep Learning Libraries with Apache Spark
byDatabricks
 
PDF
Deep Learning on Apache® Spark™: Workflows and Best Practices
byDatabricks
 
PDF
Deep Learning on Apache® Spark™ : Workflows and Best Practices
byJen Aman
 
PDF
Deep Learning on Apache® Spark™: Workflows and Best Practices
byJen Aman
 
PDF
Build a deep learning pipeline on apache spark for ads optimization
byCraig Chao
 
PDF
Very large scale distributed deep learning on BigDL
byDESMOND YUEN
 
PDF
BigDL webinar - Deep Learning Library for Spark
byDESMOND YUEN
 
PDF
Build, Scale, and Deploy Deep Learning Pipelines Using Apache Spark
byDatabricks
 
PDF
Build, Scale, and Deploy Deep Learning Pipelines with Ease Using Apache Spark
byDatabricks
 
PPTX
Emiliano Martinez | Deep learning in Spark Slides | Codemotion Madrid 2018
byCodemotion
 
PPTX
Combining Machine Learning frameworks with Apache Spark
byDataWorks Summit/Hadoop Summit
 
PDF
Automated ML Workflow for Distributed Big Data Using Analytics Zoo (CVPR2020 ...
byJason Dai
 
PPTX
AI and Spark - IBM Community AI Day
byNick Pentreath
 
PPTX
Combining Machine Learning Frameworks with Apache Spark
byDatabricks
 
PDF
Build, Scale, and Deploy Deep Learning Pipelines with Ease
byDatabricks
 
PDF
Spark Based Distributed Deep Learning Framework For Big Data Applications
byHumoyun Ahmedov
 
PPTX
Deep-Dive into Deep Learning Pipelines with Sue Ann Hong and Tim Hunter
byDatabricks
 
PDF
Index conf sparkai-feb20-n-pentreath
byChester Chen
 
PDF
Scalable Data Science in Python and R on Apache Spark
byfelixcss
 
BigDL: Bringing Ease of Use of Deep Learning for Apache Spark with Jason Dai ...
byDatabricks
 
Integrating Deep Learning Libraries with Apache Spark
byDatabricks
 
Deep Learning on Apache® Spark™: Workflows and Best Practices
byDatabricks
 
Deep Learning on Apache® Spark™ : Workflows and Best Practices
byJen Aman
 
Deep Learning on Apache® Spark™: Workflows and Best Practices
byJen Aman
 
Build a deep learning pipeline on apache spark for ads optimization
byCraig Chao
 
Very large scale distributed deep learning on BigDL
byDESMOND YUEN
 
BigDL webinar - Deep Learning Library for Spark
byDESMOND YUEN
 
Build, Scale, and Deploy Deep Learning Pipelines Using Apache Spark
byDatabricks
 
Build, Scale, and Deploy Deep Learning Pipelines with Ease Using Apache Spark
byDatabricks
 
Emiliano Martinez | Deep learning in Spark Slides | Codemotion Madrid 2018
byCodemotion
 
Combining Machine Learning frameworks with Apache Spark
byDataWorks Summit/Hadoop Summit
 
Automated ML Workflow for Distributed Big Data Using Analytics Zoo (CVPR2020 ...
byJason Dai
 
AI and Spark - IBM Community AI Day
byNick Pentreath
 
Combining Machine Learning Frameworks with Apache Spark
byDatabricks
 
Build, Scale, and Deploy Deep Learning Pipelines with Ease
byDatabricks
 
Spark Based Distributed Deep Learning Framework For Big Data Applications
byHumoyun Ahmedov
 
Deep-Dive into Deep Learning Pipelines with Sue Ann Hong and Tim Hunter
byDatabricks
 
Index conf sparkai-feb20-n-pentreath
byChester Chen
 
Scalable Data Science in Python and R on Apache Spark
byfelixcss
 

Recently uploaded

PDF
DSD-INT 2025 DevOps - Automated testing and delivery of Delft3D FM - van West...
byDeltares
 
PDF
Bring AI and build AI agents into your Jakarta EE Apps with LangChain4J-CDI
byBuhake Sindi
 
PDF
SCORM Cloud: The 5 categories of content distribution
byRustici Software
 
PPTX
The Sync Strikes Back: Tales from the MOPs Trenches
bybbedford2
 
PDF
DSD-INT 2025 Century-Scale Impacts of Sediment-Based Coastal Restoration unde...
byDeltares
 
PDF
Internship Project Training Report-3.pdf
byPawank36
 
PPTX
Future of Software Testing: AI-Powered Open Source Testing Tools
bySophie Lane
 
PDF
DSD-INT 2025 Understanding the Paraguay River Response to Hard Bottom Dredgin...
byDeltares
 
PDF
BCA 1st Semester Fundamentals Solved Question Paper 44121
byKuvempu University
 
PDF
DSD-INT 2025 Delft3D FM Suite 2026.01 2D3D - New features + Improvements - Sp...
byDeltares
 
PDF
DSD-INT 2025 Timor-Leste Flood exposure and Climate Change assessment - Ogunwumi
byDeltares
 
PDF
DSD-INT 2025 Validation of SFINCS on Historical River Floods at the Global Sc...
byDeltares
 
PPTX
Building AI agents in Java - Devoxx Belgium 2025
byJulien Dubois
 
PPTX
CRM Development Company | Custom CRM Development Services
byyugababu033
 
PDF
Everything You Ever Wanted to Know About Quarkus and LangChain4j
byMarkus Eisele
 
PDF
Fundamental of Information Systems introduction.pdf
byYeabsraLakew
 
PPTX
Presentation on AWS Cloud RDS Deep dive
byBimal Jain
 
PDF
DSD-INT 2025 UK Coastal Flooding Incident Guide - Dam
byDeltares
 
PPT
Presentation on TCL/TK scripting language
byBimal Jain
 
PDF
DSD-INT 2025 Modelling Non-Newtonian Slurry Beaching with Delft3D-Slurry - Bi
byDeltares
 
DSD-INT 2025 DevOps - Automated testing and delivery of Delft3D FM - van West...
byDeltares
 
Bring AI and build AI agents into your Jakarta EE Apps with LangChain4J-CDI
byBuhake Sindi
 
SCORM Cloud: The 5 categories of content distribution
byRustici Software
 
The Sync Strikes Back: Tales from the MOPs Trenches
bybbedford2
 
DSD-INT 2025 Century-Scale Impacts of Sediment-Based Coastal Restoration unde...
byDeltares
 
Internship Project Training Report-3.pdf
byPawank36
 
Future of Software Testing: AI-Powered Open Source Testing Tools
bySophie Lane
 
DSD-INT 2025 Understanding the Paraguay River Response to Hard Bottom Dredgin...
byDeltares
 
BCA 1st Semester Fundamentals Solved Question Paper 44121
byKuvempu University
 
DSD-INT 2025 Delft3D FM Suite 2026.01 2D3D - New features + Improvements - Sp...
byDeltares
 
DSD-INT 2025 Timor-Leste Flood exposure and Climate Change assessment - Ogunwumi
byDeltares
 
DSD-INT 2025 Validation of SFINCS on Historical River Floods at the Global Sc...
byDeltares
 
Building AI agents in Java - Devoxx Belgium 2025
byJulien Dubois
 
CRM Development Company | Custom CRM Development Services
byyugababu033
 
Everything You Ever Wanted to Know About Quarkus and LangChain4j
byMarkus Eisele
 
Fundamental of Information Systems introduction.pdf
byYeabsraLakew
 
Presentation on AWS Cloud RDS Deep dive
byBimal Jain
 
DSD-INT 2025 UK Coastal Flooding Incident Guide - Dam
byDeltares
 
Presentation on TCL/TK scripting language
byBimal Jain
 
DSD-INT 2025 Modelling Non-Newtonian Slurry Beaching with Delft3D-Slurry - Bi
byDeltares
 

Build Deep Learning Applications for Big Data Platforms (CVPR 2018 tutorial)

  • 1.
    CVPR 2018 Building DeepLearning Applications on Big Data Platform An Introduction to Analytics Zoo for Apache Spark and BigDL Jason Dai
  • 2.
    CVPR 2018 Agenda • Motivation(10 minutes) • Trends, real-world scenarios • DL frameworks on Apache Spark (20 minutes) • BigDL, TensorFlowOnSpark, DL Pipelines, SparkNet • Analytics Zoo for Spark and BigDL (15 minutes) • High level pipeline APIs, feature engineering, built-in models, reference use cases • Analytics Zoo Examples (15 minutes) • Dogs vs. cats, object detection, TFNet • Break (30 minutes)
  • 3.
    CVPR 2018 Agenda • Distributedtraining in BigDL (30 minutes) • Data parallel training, parameter synchronization, scaling & convergence, task scheduling, etc. • Advanced applications (15 minutes) • Variational autoencoder, movie recommendations • Real-world applications (30 minutes) • Object detection and image feature extraction at JD.com • Image similarity based house recommendation for MLSlistings • Transfer learning based image classifications for World Bank • Fraud detection for payment transactions for UnionPay • Conclusion and Q&A (15 minutes)
  • 4.
    CVPR 2018 Motivations Technology andIndustry Trends Real World Scenarios
  • 5.
    CVPR 2018 Trend #1:Data Scale Driving Deep Learning Process “Machine Learning Yearning”, Andrew Ng, 2016
  • 6.
    CVPR 2018 Trend #2:Hadoop Becoming the Center of Data Gravity Phillip Radley, BT Group Strata + Hadoop World 2016 San Jose Matthew Glickman, Goldman Sachs Spark Summit East 2015
  • 7.
    CVPR 2018 Trend #3:Real-World ML/DL Systems Are Complex Big Data Analytics Pipelines “Hidden Technical Debt in Machine Learning Systems”, Sculley et al., Google, NIPS 2015 Paper
  • 8.
    CVPR 2018 Trend #4:Unified Big Data Platform Driving Analytics & Data Science Ion Stoica, UC Berkeley, Spark Summit 2013 Keynote
  • 9.
    CVPR 2018 Unified BigData Analytics Platform Data Input Flume Kafka Storage HBaseHDFS Resource Mgmt & Co-ordination ZooKeeperYARN Data Processing & Analysis MR Storm Apache Hadoop & Spark Platform Parquet Avro Spark Core SQL Streaming MLlib GraphX DataFrame ML Pipelines SparkR Flink Giraph Batch Streaming Interactive Machine Leaning Graph Analytics SQL R PythonJava Notebook Spreadsheet
  • 10.
    CVPR 2018 Chasm b/wDeep Learning and Big Data Communities Average users (big data users, data scientists, analysts, etc.)Deep learning experts The Chasm
  • 11.
    CVPR 2018 Large-Scale ImageRecognition at JD.com
  • 12.
    CVPR 2018 Bridging theChasm Make deep learning more accessible to big data and data science communities • Continue the use of familiar SW tools and HW infrastructure to build deep learning applications • Analyze “big data” using deep learning on the same Hadoop/Spark cluster where the data are stored • Add deep learning functionalities to large-scale big data programs and/or workflow • Leverage existing Hadoop/Spark clusters to run deep learning applications • Shared, monitored and managed with other workloads (e.g., ETL, data warehouse, feature engineering, traditional ML, graph analytics, etc.) in a dynamic and elastic fashion
  • 13.
    CVPR 2018 DL Frameworkson Apache Spark BigDL, DL Pipelines for Spark, TensorflowOnSpark, SparkNet, etc.
  • 14.
    CVPR 2018 Apache Spark Spark Job Driver SparkTask Worker Spark Task Spark Task Spark Task Worker Spark Task Spark Task • A Spark cluster consists of a single driver node and multiple worker nodes • A Spark job contains many Spark tasks, each working on a data partition • Driver is responsible for scheduling and dispatching the tasks to workers, which runs the actual Spark tasks Low Latency, Distributed Data Processing Framework https://spark.apache.org
  • 15.
    CVPR 2018 Apache Spark SparkProgram • Spark runs as a library in your program (1 instance per app) • Runs tasks locally or on cluster • K8s, YARN, Mesos or standalone mode • Accesses storage systems via Hadoop InputFormat API • Can use HBase, HDFS, S3, … Your application SparkContext Local threads Cluster manager Worker Spark executor Worker Spark executor HDFS or other storageSource: “Parallel programming with Spark”, Matei Zaharia, AMPCamp 3
  • 16.
    CVPR 2018 Apache Spark join union groupBy map Stage3 Stage 1 Stage 2 A: B: C: D: E: F: G: Task = Cached Partition= RDD • General task graphs • Automatically pipelines functions • Data locality aware • Partitioning aware to avoid shuffles Source: “Parallel programming with Spark”, Matei Zaharia, AMPCamp 3 Distributed Task Execution
  • 17.
    CVPR 2018 BigDL Bringing DeepLearning To Big Data Platform https://github.com/intel-analytics/BigDL Spark Core SQL SparkR Streaming MLlib GraphX ML Pipeline DataFrame https://bigdl-project.github.io/ • Distributed deep learning framework for Apache Spark* • Make deep learning more accessible to big data users and data scientists • Write deep learning applications as standard Spark programs • Run on existing Spark/Hadoop clusters (no changes needed) • Feature parity with popular deep learning frameworks • E.g., Caffe, Torch, Tensorflow, etc. • High performance (on CPU) • Powered by Intel MKL and multi-threaded programming • Efficient scale-out • Leveraging Spark for distributed training & inference
  • 18.
    CVPR 2018 BigDL Runas Standard Spark Programs Spark Program DL App on Driver Spark Executor (JVM) Spark Task BigDL lib Worker Intel MKL Standard Spark jobs Worker Worker Worker Worker Spark Executor (JVM) Spark Task BigDL lib Worker Intel MKL BigDL library Spark jobs BigDL Program Standard Spark jobs • No changes to the Spark or Hadoop clusters needed Iterative • Each iteration of the training runs as a Spark job Data parallel • Each Spark task runs the same model on a subset of the data (batch)
  • 19.
    CVPR 2018 Distributed Trainingin BigDL … Training Set Parameter Server Architecture directly inside Spark (using Block Manager) Partition 1 Partition 2 Partition n Worker Gradient 1 2 Weight 3 4 5 Worker Gradient 1 2 Weight 3 4 5 Worker Gradient 1 2 Weight 3 4 5 … … … … … … … Peer-2-Peer All-Reduce synchronization
  • 20.
    CVPR 2018 TensoflowOnSpark Spark Task Python Queue TF Worker Func Worker Spark Task Worker TF PS Func Spark Task Python Queue TF Worker Func Worker Spark Task TF Worker Func Worker Spark Task Worker TF PS Func Spark Task TF Worker Func Worker HDFS feed_dict: TFworker func runs as independent process in background, reading data from Python queue queue_runner: direct HDFS access from TF work func Standalone TF jobs on Spark cluster • Use Spark as the orchestration layer to allocate resources • Launch distributed TensorFlow job on the allocated resources • Coarse-grained integration of two independent frameworks • Memory overheads, no gang scheduling, limited interactions with data pipelines, etc. https://github.com/yahoo/TensorFlowOnSpark
  • 21.
    CVPR 2018 DL Pipelinesfor Spark Spark ML Pipelines Dataframe Dataframe Dataframe Transformer Transformer Transformer Estimator Source: “Deep Dive into Deep Learning Pipelines”, Sue Ann Hong and Tim Hunter, Spark Summit EU 2017 Load existing TF or Keras models in Spark ML pipelines • Load into transformer: inference only • Load into estimator: single node training/tuning only https://github.com/databricks/spark-deep-learning
  • 22.
    CVPR 2018 SparkNet Source: “SparkNet:Training Deep Networks in Spark”, Philipp Moritz, et al., ICLR 2016 Distributed DL training by running Caffe in each worker • Asynchronous parameter synchronization through master (driver) mode • Very inefficient (~20 seconds with just 5 workers)
  • 23.
    CVPR 2018 Analytics Zoo Analytics+ AI Platform for Spark and BigDL https://github.com/intel-analytics/analytics-zoo
  • 24.
    CVPR 2018 Analytics Zoo Buildand Productionize Deep Learning Apps for Big Data at Scale Reference Use Cases • Anomaly detection • Sentiment analysis • Fraud detection • Chatbot, sequence prediction, etc. Built-In Deep Learning Models • Image classification • Object detection • Text classification • Recommendations • Sequence-to-sequence, GAN, etc. Feature Engineering Feature transformations for • Image, text, 3D imaging, time series, speech, etc. High-Level Pipeline APIs • Native deep learning support in Spark DataFrames and ML Pipelines • Autograd, Keras and transfer learning APIs for model definition • Model serving API for model serving/inference pipelines Backbends Spark, BigDL, TensorFlow, etc. https://github.com/intel-analytics/analytics-zoo/ https://analytics-zoo.github.io/
  • 25.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications for big data at scale • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
  • 26.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
  • 27.
    CVPR 2018 nnframes 1. InitializeNNContext and load images into DataFrames using NNImageReader Native DL support in Spark DataFrames and ML Pipelines from zoo.common.nncontext import * from zoo.pipeline.nnframes import * sc = init_nncontext() imageDF = NNImageReader.readImages(image_path, sc) 2. Process loaded data using DataFrame transformations getName = udf(lambda row: ...) df = imageDF.withColumn("name", getName(col("image"))) 3. Processing image using built-in feature engineering operations from zoo.feature.image import * transformer = ChainedPreprocessing( [RowToImageFeature(), ImageChannelNormalize(123.0, 117.0, 104.0), ImageMatToTensor(), ImageFeatureToTensor()])
  • 28.
    CVPR 2018 nnframes 4. Definemodel using Keras-style API Native DL support in Spark DataFrames and ML Pipelines from zoo.pipeline.api.keras.layers import * from zoo.pipeline.api.keras.models import * model = Sequential() .add(Convolution2D(32, 3, 3, activation='relu', input_shape=(1, 28, 28))) .add(MaxPooling2D(pool_size=(2, 2))) .add(Flatten()).add(Dense(10, activation='softmax'))) 5. Train model using Spark ML Pipelines Estimater = NNEstimater(model, CrossEntropyCriterion(), transformer) .setLearningRate(0.003).setBatchSize(40).setMaxEpoch(1) .setFeaturesCol("image").setCachingSample(False) nnModel = estimater.fit(df)
  • 29.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
  • 30.
    CVPR 2018 Autograd, Keras&Transfer Learning APIs from zoo.pipeline.api.net import * full_model = Net.load_caffe(def_path, model_path) # Remove layers after pool5 model = full_model.new_graph(outputs=["pool5"]).to_keras()) # freeze layers from input to res4f inclusive model.freeze_up_to(["res4f"]) # append a few layers image = Input(name="input", shape=(3, 224, 224)) resnet = model.to_keras()(image) resnet50 = Flatten()(resnet) 1. Use transfer learning APIs to • Load an existing Caffe model • Remove last few layers • Freeze first few layers • Append a few layers Build Siamese Network Using Transfer Learning
  • 31.
    CVPR 2018 Autograd, Keras&Transfer Learning APIs Build Siamese Network Using Transfer Learning import zoo.pipeline.api.autograd as A from zoo.pipeline.api.keras.layers import * from zoo.pipeline.api.keras.models import * input = Input(shape=[2, 3, 226, 226]) features = TimeDistributed(layer=resnet50)(input) f1 = features.index_select(1, 0) #image1 f2 = features.index_select(1, 1) #image2 diff = A.abs(f1 - f2) fc = Dense(1)(diff) output = Activation("sigmoid")(fc) model = Model(input, output) 2. Use autograd and Keras-style APIs to build the Siamese Network
  • 32.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
  • 33.
    CVPR 2018 Feature Engineering fromzoo.common.nncontext import * from zoo.feature.image import * spark = init_nncontext() local_image_set = ImageSet.read(image_path) distributed_image_set = ImageSet.read(image_path, spark, 2) 1. Read images into local or distributed ImageSet 2. Image augmentations using built-in ImageProcessing operations transformer = ChainedPreprocessing([ImageBytesToMat(), ImageColorJitter(), ImageExpand(max_expand_ratio=2.0), ImageResize(300, 300, -1), ImageHFlip()]) new_local_image_set = transformer(local_image_set) new_distributed_image_set = transformer(distributed_image_set) Image Augmentations Using Built-in Image Transformations (w/ OpenCV on Spark)
  • 34.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
  • 35.
    CVPR 2018 Built-in DeepLearning Models • Object detection API • High-level API and pretrained models (e.g., SSD, Faster-RCNN, etc.) for object detection • Image classification API • High-level API and pretrained models (e.g., VGG, Inception, ResNet, MobileNet, etc.) for image classification • Text classification API • High-level API and pre-defined models (using CNN, LSTM, etc.) for text classification • Recommendation API • High-level API and pre-defined models (e.g., Neural Collaborative Filtering, Wide and Deep Learning, etc.) for recommendation
  • 36.
    CVPR 2018 Object DetectionAPI from zoo.common.nncontext import * from zoo.models.image.objectdetection import * spark = init_nncontext() model = ObjectDetector.load_model(model_path) 1. Load pretrained model in Detection Model Zoo 2. Off-the-shell inference using the loaded model image_set = ImageSet.read(img_path, spark) output = model.predict_image_set(image_set) Off-the-shell Inference Using Analytics Zoo Object Detection API 3. Visualize the results using utility methods config = model.get_config() visualizer = Visualizer(config.label_map(), encoding="jpg") visualized = visualizer(output).get_image(to_chw=False).collect() https://github.com/intel-analytics/analytics-zoo/tree/master/pyzoo/zoo/examples/objectdetection
  • 37.
    CVPR 2018 Reference UseCases • Anomaly Detection • Using LSTM network to detect anomalies in time series data • Fraud Detection • Using feed-forward neural network to detect frauds in credit card transaction data • Recommendation • Use Analytics Zoo Recommendation API (i.e., Neural Collaborative Filtering, Wide and Deep Learning) for recommendations on data with explicit feedback. • Sentiment Analysis • Sentiment analysis using neural network models (e.g. CNN, LSTM, GRU, Bi-LSTM) • Variational Autoencoder (VAE) • Use VAE to generate faces and digital numbers https://github.com/intel-analytics/analytics-zoo/tree/master/apps
  • 38.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference using TFNet
  • 39.
    CVPR 2018 POJO ModelServing API import com.intel.analytics.zoo.pipeline.inference.AbstractInferenceModel; public class TextClassification extends AbstractInferenceModel { public RankerInferenceModel(int concurrentNum) { super(concurrentNum); } ... } public class ServingExample { public static void main(String[] args) throws IOException { TextClassification model = new TextClassification(); model.load(modelPath, weightPath); texts = … List<JTensor> inputs = preprocess(texts); for (JTensor input : inputs) { List<Float> result = model.predict(input.getData(), input.getShape()); ... } }
  • 40.
    CVPR 2018 Model Serving& Inference HDFS/S3 Kafka Flume Kinesis Twitter Spout Analytics Zoo Model Spout Bolt Bolt Bolt Bolt Bolt Analytics Zoo Model Seamless integration in Web Services, Storm, Flink, Kafka, etc. (using POJO local Java APIs)
  • 41.
    CVPR 2018 Model Servingin Spark DataFrames Seamless support of DL functionalities in Spark SQL queries, Dataframe operation and stream processing ImageNet dataset (http://www.image-net.org) Kafka File Data Frame (Batch/Stream) Analytics Zoo UDF Filtered Data Frame (Batch/Stream) df.select($’image’) .withColumn( “image_type”, ImgClassifyInAnalyticsZoo(“image”)) .filter($’image_type’ == ‘dog’)
  • 42.
    CVPR 2018 Analytics Zoo Buildend-to-end deep learning applications for big data • E2E analytics + AI pipelines (natively in Spark DataFrames and ML Pipelines) using nnframes • Flexible model definition using autograd, Keras-style & transfer learning APIs • Data preprocessing using built-in feature engineering operations • Out-of-the-box solutions for a variety of problem types using built-in deep learning models and reference use cases Productionize deep learning applications at scale for big data • Serving models in web services and big data frameworks (e.g., Storm or Kafka) using POJO model serving APIs • Large-scale distributed TensorFlow model inference & fine tuning using TFNet
  • 43.
    CVPR 2018 Distributed TensorFlowModel Inference import tensorflow as tf batch_size_tensor = tf.placeholder_with_default(128, shape=[]) x_batch, y_batch = tf.train.shuffle_batch (..., batch_size=batch_size_tensor, ...) cnn = cnn_model_fn(x_batch, y_batch) sess = tf.Session() init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()) sess.run(init_op) for step in range(600): _, loss = sess.run([cnn.train_op, cnn.loss], ...) from zoo.utils.tf import * export_tf(sess, folder_path, [x_batch], [cnn.prediction]) 1. Export TensorFlow models (in your TensorFlow program)
  • 44.
    CVPR 2018 Distributed TensorFlowModel Inference 2. Load exported TensorFlow model into Analytics Zoo from zoo.pipeline.api.net import * model = TFNet.from_export_folder(folder_path) # Alternatively, you may directly load a frozen TensorFlow model as follows # model = TFNet(model_path, [“image_tensor:0”], [“output_tensor:0”]) 3. Add a few layers and run distributed model inference import zoo.pipeline.api.autograd as A from zoo.pipeline.api.keras.layers import * from zoo.pipeline.api.keras.models import * input = Input(shape=[2, 3, 226, 226]) features = TimeDistributed(layer=model)(input) f1 = features.index_select(1, 0) f2 = features.index_select(1, 1) diff = A.abs(f1 - f2) result = Model(input, diff) result.predict_image_set(...)
  • 45.
    CVPR 2018 Analytics ZooExamples Dogs vs. cats, object detections, TFNet
  • 46.
    CVPR 2018 Dogs vs.Cats https://github.com/intel-analytics/analytics-zoo/blob/master/apps/dogs-vs-cats/transfer-learning.ipynb Notebook:
  • 47.
    CVPR 2018 Object DetectionAPI https://github.com/intel-analytics/analytics-zoo/blob/master/apps/object-detection/object-detection.ipynb Notebook:
  • 48.
    CVPR 2018 Image ClassificationUsing TFNet https://github.com/intel-analytics/analytics-zoo/blob/master/apps/tfnet/image_classification_inference.ipynb Notebook:
  • 49.
  • 50.
    CVPR 2018 Distributed TrainingIn BigDL Data parallel training Parameter synchronization Scaling and Convergence Task scheduling “BigDL: A Distributed Deep Learning Framework for Big Data”, https://arxiv.org/abs/1804.05839
  • 51.
    CVPR 2018 Apache Spark Spark Job Driver SparkTask Worker Spark Task Spark Task Spark Task Worker Spark Task Spark Task Single master (driver), multiple workers
  • 52.
    CVPR 2018 Apache Spark join union groupBy map Stage3 Stage 1 Stage 2 A: B: C: D: E: F: G: Task = Cached Partition= RDD Source: “Parallel programming with Spark”, Matei Zaharia, AMPCamp 3 Spark compute model • Data parallel • Functional, coarse-grained operators • Immutable RDDs • Applying the same operation (e.g., map, filter, etc.) to all data items
  • 53.
    CVPR 2018 Distributed Trainingin BigDL Data Parallel, Synchronous Mini-Batch SGD Prepare training data as an RDD of Samples Construct an RDD of models (each being a replica of the original model) for (i <- 1 to N) { //”model forward-backward” job for each task in the Spark job: read the latest weights get a random batch of data from local Sample partition compute errors (forward on local model replica) compute gradients (backward on local model replica) //”parameter synchronization” job aggregate (sum) all the gradients update the weights per specified optimization method }
  • 54.
    CVPR 2018 Data ParallelTraining Worker 2 Worker nWorker 1 Partition 1 Partition 2 Partition n Sample RDD Partition 1 Partition 2 Partition nModel RDD Task n: zip Sample and model RDDs, and compute gradient on co-located Sample and model partitions Task 2Task 1 “Model Forward-Backward” Job
  • 55.
    CVPR 2018 Parameter Synchronization Taskn 1 2 n 1 2 n 1 2 n 1 2 n 1 2 n Task 1 Task 2 local gradient local gradient local gradient ∑ ∑ ∑ gradient 1 gradient n weight 1 weight 2 weight n “Parameter Synchronization” Job update update update gradient 2
  • 56.
    CVPR 2018 Parameter Synchronization Foreach task n in the ”parameter synchronization” job { shuffle the nth partition of all gradients to this task aggregate (sum) the gradients updates the nth partition of the weights broadcast the nth partition of the updated weights } “Parameter Synchronization” Job (managing nth partition of the parameters - similar to a parameter server) “Parameter Server” style architecture (directly on top of primitives in Spark) • Gradient aggregation: shuffle • Weight sync: task-side broadcast • In-memory persistence
  • 57.
    CVPR 2018 Training Scalability Throughputof ImageNet Inception v1 training (w/ BigDL 0.3.0 and dual-socket Intel Broadwell 2.1 GHz); the throughput scales almost linear up to 128 nodes (and continue to scale reasonably up to 256 nodes). Source: Scalable Deep Learning with BigDL on the Urika-XC Software Suite (https://www.cray.com/blog/scalable-deep-learning-bigdl-urika-xc-software-suite/)
  • 58.
    CVPR 2018 Increased Mini-BatchSize • Distributed synchronous mini-batch SGD • Increased mini-batch size total_batch_size = batch_size_per_worker * num_of_workers • Can lead to loss in test accuracy • State-of-art method for scaling mini-batch size* • Linear scaling rule • Warm-up strategy • Layer-wise adaptive rate scaling • Adding batch normalization *Source: “Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour”, Priya Goyal, et at. https://arxiv.org/abs/1706.02677 *Source: “ImageNet Training in Minutes”, Yang You, et at. https://arxiv.org/abs/1709.05011
  • 59.
    CVPR 2018 Training Convergence:Inception v1 Strategies • Warm-up • Linear scaling • Gradient clipping • TODO: adding batch normalization Source: Very large-scale distributed deep learning with BigDL, Jason Dai and Ding Ding. O’Reilly AI Conference 2017
  • 60.
    CVPR 2018 Training Convergence:SSD Strategies • Warm-up • Linear scaling • Gradient clipping Source: Very large-scale distributed deep learning with BigDL, Jason Dai and Ding Ding. O’Reilly AI Conference 2017
  • 61.
    CVPR 2018 Difference vs.Classical PS Architecture Classical PS architecture • Multiple long-running, potentially stateful tasks • Interact with each other (in a blocking fashion for synchronization) • Require fine-grained data access and in- place data mutation • Not directly supported by existing big data systems BigDL implementations • Run a series of short-lived Spark jobs (e.g., two jobs per mini-batch) • Each task in the job is stateless and non-blocking • Automatically adapt to the dynamic resource changes (e.g., preemption, failures, resource sharing, etc.) • Built on top of existing primitives in Spark (e.g., shuffle, broadcast, and in- memory data persistence)
  • 62.
    CVPR 2018 Task SchedulingOverheads Spark overheads (task scheduling & task dispatch ) as a fraction of average compute time for Inception v1 training BigDL implementations • Run a single, multi- threaded task on each worker • Achieve high scalability on large clusters (e.g., up to 256 servers Source: Accelerating Deep Learning Training with BigDL and Drizzle on Apache Spark, Shivaram Venkataraman, Ding Ding, and Sergey Ermolin. (https://rise.cs.berkeley.edu/blog/accelerating-deep-learning-training-with-bigdl-and-drizzle-on-apache-spark/)
  • 63.
    CVPR 2018 Reducing SchedulingOverheads Using Drizzle Scaling to even larger (>500) workers • Iterative model training • Same operations run repeatedly • Drizzle • A low latency execution engine for Spark • Group scheduling for multiple iterations of computations at once Source: Accelerating Deep Learning Training with BigDL and Drizzle on Apache Spark, Shivaram Venkataraman, Ding Ding, and Sergey Ermolin. (https://rise.cs.berkeley.edu/blog/accelerating-deep-learning-training-with-bigdl-and-drizzle-on-apache-spark/)
  • 64.
    CVPR 2018 Advanced AnalyticsZoo Applications Variational autoencoder, movie recommendations
  • 65.
  • 66.
  • 67.
    CVPR 2018 Real-World Applications Objectdetection and image feature extraction at JD.com Image similarity based house recommendation for MLSlisting Transfer learning based image classifications for World Bank Fraud detection for payment transactions for UnionPay
  • 68.
    CVPR 2018 Object Detectionand Image Feature Extraction at JD.com
  • 69.
    CVPR 2018 Applications Large-scale imagefeature extraction • Object detect (remove background, optional) • Feature extraction Application • Similar image search • Image Deduplication • Competitive price monitoring • IP (image copyright) protection system Source: “Bringing deep learning into big data analytics using BigDL”, Xianyan Jia and Zhenhua Wang, Strata Data Conference Singapore 2017
  • 70.
    CVPR 2018 Similar ImageSearch Query Search Result Query Search Result ------ Source: “Bringing deep learning into big data analytics using BigDL”, Xianyan Jia and Zhenhua Wang, Strata Data Conference Singapore 2017
  • 71.
    CVPR 2018 Challenges ofProductionizing Large-Scale Deep Learning Solutions Productionizing large-scale seep learning solutions is challenging • Very complex and error-prone in managing large-scale distributed systems • E.g., resource management and allocation, data partitioning, task balance, fault tolerance, model deployment, etc. • Low end-to-end performance in GPU solutions • E.g., reading images out from HBase takes about half of the total time • Very inefficient to develop the end-to-end processing pipeline • E.g., image pre-processing on HBase can be very complex
  • 72.
    CVPR 2018 Production Deploymentwith Analytics Zoo for Spark and BigDL http://mp.weixin.qq.com/s/xUCkzbHK4K06-v5qUsaNQQ https://software.intel.com/en-us/articles/building-large-scale-image-feature-extraction-with-bigdl-at-jdcom • Reuse existing Hadoop/Spark clusters for deep learning with no changes (image search, IP protection, etc.) • Efficiently scale out on Spark with superior performance (3.83x speed-up vs. GPU severs) as benchmarked by JD
  • 73.
    CVPR 2018 Image SimilarityBased House Recommendation for MLSlistings https://software.intel.com/en-us/articles/using-bigdl-to-build-image-similarity-based-house-recommendations MLSlistings built image-similarity based house recommendations using BigDL on Microsoft Azure
  • 74.
    CVPR 2018 Image SimilarityBased House Recommendation for MLSlistings RDD of house photos Is house exterior? {0, 1} House Style {0, 1, 2, 3} Image Processing Is house exterior? {0, 1} House Style {0, 1, 2, 3} Image Processing Is house exterior? {0, 1} House Style {0, 1, 2, 3} Image Processing Is house exterior? {0, 1} Tags (is_exterior, style, floors) of images RDD of house photos Image features Store image tags and feature in table storage Image pre- processing Image pre- processing Pre-trained VGG16 model (to extract features) Three pre-trained Inception v1 models (fine- tuned as classifiers)
  • 75.
    CVPR 2018 Image SimilarityBased House Recommendation for MLSlistings https://github.com/intel-analytics/analytics-zoo/blob/master/apps/image- similarity/Image%20similarity.ipynb Notebook:
  • 76.
    CVPR 2018 Transfer LearningBased Image Classifications for World Bank Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018 Classifying Real Food Images is not a Cat vs. Dog Problem
  • 77.
    CVPR 2018 Project Layout Phase1: • Image preprocessing (eliminate poor quality images and invalid images) • Classify images (by food type) to validate existing labels Phase 2: • Identify texts in the image and make bounding box around them • Text recognition (words/sentences in the image text) • Determine whether text contains PII (personal identifiable information) • Blur areas with PII text Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
  • 78.
    CVPR 2018 Code –Phase 1 Fine-tuning Training Prediction and Evaluation Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
  • 79.
    CVPR 2018 Result –Phase 1 • Fine tune with Inception v1 on a full dataset • Dataset: 994325 images, 69 categories Nodes Cores Batch Size Epochs Training Time (sec) Throughput (images/sec) Accuracy (%) 20 30 1200 12 61125 170 81.7 * This model training was performed using multinode cluster on AWS R4.8xlarge instance with 20 nodes Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
  • 80.
    CVPR 2018 Next Steps– Phase 2 • Image Quality Preprocessing • Filter with print text only • Rescaling, Binarisation, Noise Removal, Rotation / Deskewing (OpenCV, Python, etc.) • Detect text and bounding box circle text • Recognize text • Determine whether text contains PII (personal identifiable information) • Recognize PII with leading words • Blur areas with PII text • Image tools Source: Using Crowdsourced Images to Create Image Recognition Models with Analytics Zoo using BigDL, Maurice Nsabimana and Jiao Wang, Spark Summit 2018
  • 81.
    CVPR 2018 Fraud Detectionfor Payment Transactions for UnionPay Train one model all features selected features model candidate model sampled partition Training Data … normal fraud Train one model Train one model sampled partition sampled partition Post Processing Pre- Processing model model Spark Pipeline Test Data Predictions Test Spark DataFrameHive Table Pre-processing Feature Engineering Feature Engineering Feature Selection Model Ensemble SparkPipeline Neural Network Model Using BigDL Feature Selection* Model Training Model Evaluation & Fine Tune https://mp.weixin.qq.com/s?__biz=MzI3NDAwNDUwNg==&mid=2648307335&idx=1&sn=8eb9f63eaf2e40e24a90601b9cc03d1f
  • 82.
    CVPR 2018 Fraud Detectionfor Payment Transactions for UnionPay https://github.com/intel-analytics/analytics-zoo/blob/master/apps/fraud-detection/fraud-detection.ipynb Notebook:
  • 83.
    CVPR 2018 Summary Make deeplearning more accessible to big data and data science communities • Analyze “big data” using deep learning on the same Hadoop/Spark cluster where the data are stored • Add deep learning functionalities to large-scale big data programs and/or workflow • Leverage existing Hadoop/Spark clusters to run deep learning applications • Shared, managed and monitored with other workloads (ETL, data warehouse, traditional ML, etc.) Analytics Zoo: https://github.com/intel-analytics/analytics-zoo • End-to-end Analytics + AI platform for Apache Spark and BigDL • Build and productionize deep learning application for Big Data at Scale
  • 84.
  • 85.
    CVPR 2018 Legal Disclaimers •Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. • No computer system can be absolutely secure. • Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance. Intel, the Intel logo, Xeon, Xeon phi, Lake Crest, etc. are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © 2018 Intel Corporation