Tensorflow on Apache Hadoop YARN
•Download as PPTX, PDF•
14 likes•4,834 views
Tensorflow™ is one of the most popular open source projects for machine learning and deep learning, which can handle enterprise use cases like image recognition, video analytics, audio translation, etc. However, training deep learning model was very expensive which requires lots of GPU resources. Also, a real-life distributed Tensorflow application needs a bunch of services such as workers, parameter servers, TensorBoard, etc. work together. Those services need to be carefully configured to make them can talk to each other. To make distributed TF applications can be easily launched, managed, monitored by YARN, we introduced YARN service assembly along with other improvements such as GPU support, container-DNS support, scheduling improvements, etc. These improvements make distributed TF applications can run on YARN as simple as run it locally, which can let TF developers focus on deep learning algorithms instead of worrying about underlying infrastructure. Also, YARN can better manage a shared cluster which runs TF and other services/batch jobs with these improvements. During this session, we will take a closer look at these improvements, and we will do a demo of running a distributed TF assembly which consists of workers, parameter servers, TensorBoard and prediction servers on YARN. Speaker: Sunil Govindan, Senior Software Engineer, Hortonworks
Recommended
Recommended
More Related Content
More from DataWorks Summit
More from DataWorks Summit (20)
Recently uploaded
Recently uploaded (20)
Tensorflow on Apache Hadoop YARN
- 1. TensorFlow on Apache Hadoop YARN Sunil Govindan Apache Hadoop PMC member Hortonworks
- 2. 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Agenda Machine Learning on Big Data Platform TensorFlow on Apache Hadoop YARN Example walkthrough with demo
- 3. 3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache YARN - Past, Present and Future Sunil G Hortonworks https://dataworkssummit.com/sydney-2017/sessions/yarn-past-present-future/ Wednesday September 20th Room C4.5 5.10 PM -- Related Session -- Rohith Sharma K S Hortonworks
- 4. 4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine Learning on Big Data Platform
- 5. 5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine Learning on Big Data Platform Apache Zeppelin
- 6. 6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine Learning on Big Data Platform Data Scientists Software engineers Explore data Create pipeline Find best params Save model Load model Deploy in production Scoring on batch/streaming data Apache Zeppelin
- 7. 7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine learning workflow Feature Selection Data Feature Transform Feature Encoding Feature Evaluation Model Training Feature Model Evaluation Model Validation Model Staging Experiment Online Feature Model Database Exper- iment Model as Service Real-time Feature Calibration Data Preprocessing Feature Engineering Model Training Online Service
- 8. 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine Learning in a Unified Platform “Hidden Technical Debt in Machine Learning Systems”, Google
- 9. 9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine learning – Data Preprocessing Feature Selection Data Feature Transform Feature Encoding Feature Evaluation Feature Engineering Import data – HDFS – AWS – RDBMS Join data Data exploration Data sample Training/Test random split
- 10. 10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine learning – Feature Engineering Feature Selection Data Feature Transform Feature Encoding Feature Evaluation Feature Engineering Feature transform/selection Feature embedding
- 11. 11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine learning – Model Training Model Training Feature Model Evaluation Model Validation Model Staging Model Training Traditional machine learning models – Logistic Regression – Gradient boosting tree – Recommendation/ALS – LDA Libraries – Apache Spark MLlib – XGBoost Deep learning models – DNN – CNN – RNN – LSTM Libraries – TensorFlow – Apache MXNet – BigDL
- 12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Model Training - Deep learning can’t fit all Natural language processing Computer vision Speech/Video Anti-fraud Recommendation CTR estimation Topic model PageRank
- 13. 13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine learning – Model Serving Experiment Online Feature Model Database Exper- iment Model as Service Real-time Feature Calibration Online Service Model deploy Model serving – Batch – Streaming Experiment – offline – online (A/B test)
- 14. 14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved TensorFlow on Apache Hadoop YARN
- 15. 15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Hadoop YARN Overview YARN is Apache Hadoop’s resource management framework YARN is made up of the following components: – ResourceManager (RM) allocates resources to applications, is the master service for YARN – Per-node NodeManager (NM) launches application containers and monitors their resource usage – Per-application ApplicationMaster (AM) requests containers for the application At its core, YARN is responsible for managing “containers” across a collection of servers
- 16. 16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Machine learning platform on YARN CPU GPU SSD YARN: Data Operating System (Cluster Resource Management) Spark MLlib XGBoost Hive/LLAP Spark SQLTensorFlow Zeppelin HDFS AWS S3 RDBMS
- 17. 17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Why all under YARN • CPU / Memory, (WIP) GPU, FPGA, Network • Queues / Users quota, user access control • Time line services, Grafana, etc • Capacity Planning, Preemption, Reservation System Better SLA Monitoring IsolationGovernance
- 18. 18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved All running on the same YARN platform LLAP 128 G 128 G 128 G 128 G 128 G LLAP LLAP 128 G 128 G GPUs
- 19. 19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Recent works in YARN to support ML workloads like Tensorflow GPU Support – Boost workload performance from GPU Resource Profile and support for Custom Resource Types - Support for new resources Native Service - Easy to define and run any custom service YARN Assembly - Easy to wire different apps
- 20. 20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved GPU support on YARN Effort led by Wangda Tan from Hortonworks Why? GPU: Many cores to handle massive (but simple) computation tasks simultaneously: GPU CPU GPU Computation Intensive Other Without GPU support, researchers/engineers has to wait longer to finish apps.
- 21. 21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Challenges of GPU support Different levels of support – Take me to a machine where GPUs are available with Partitions / Node Labels. (Current status) – Take me to a machine where GPUs are available • give me a full device only to me for the lifetime of my container • give me multiple full devices only to me for the lifetime of my container • give me full device(s) only to me for a portion of the lifetime of my container • give me a slice of device(s) to me for a full / portion of the lifetime of my container More dimensions: – Bandwidths and on-GPU memory – Topology of multiple GPUs Slide credit to: Vinod Kumar Vavilapalli
- 22. 22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Resource profiles and custom resource types Past – Supports only Memory and CPU Now – A generalized vector – Admins can create custom Resource Types! – Ease of resource requesting model using profiles NodeManager Memory CPU GPU NodeManager Memory CPU GPU ResourceManager Small Medium Large Profile Memory CPU GPU Small 2 GB 4 Cores 1 Cores Medium 4 GB 8 Cores 1 Cores Large 16 GB 16 Cores 4 Cores Application Master Small
- 23. 23 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN Native Services • Long Running – Simplify the deployment and management of long running applications on YARN. • Easily Add New Applications – Remove tedious process of bringing new applications to YARN. • Easy to Use – REST API and Command Line tools. • Declarative Configuration – Provide configuration to the applications, declare resource needs, specify placement policies. • Service Discovery – Simplified service discovery on YARN
- 24. 24 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN Service assembly • Collection of services/apps logically wired together for a common purpose • Easily deploy services (zookeeper + Hbase) as a combination through REST API. • User defined service dependencies and readiness • Container localities - affinity and anti-affinity scheduling
- 25. 25 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN assembly: Makes everything easier! Forget about writing an application master, this is how you can run app on YARN .. Write assembly spec in JSON (we call it Yarnfile) Post the JSON as REST request to YARN server. YARN to figure out rest of it. An example:
- 26. 26 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN assembly: Run multi-stages job
- 27. 27 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN assembly: Run Distributed Tensorflow Training (with PS)
- 28. 28 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN Assembly: Parallel Parameter Tuning
- 29. 29 © Hortonworks Inc. 2011 – 2016. All Rights Reserved YARN Assembly: Model Serving and Update Application & Services upgrades – ”Do an upgrade of my Tensorflow serving model with minimal impact to end- users” - Use serving.tensorflow-mode-serving.wtan.domain:1234 to access the service. - YARN could do load balancing for launched instances.
- 30. 30 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Example walkthrough: How to do Click- Through-Rate on a big data platform
- 31. 31 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Click-Through Rate (CTR) Prediction Given a user and context, predict probability of a click for an ad. Probably the most “profitable” machine learning problem in industry Basic setting quite well-studied; scale make it challenging – Google, Facebook, Yahoo, Bing Challenges – Simple binary problem; but want probabilities, not just the label – Very skewed label distribution: clicks << skips – Tons of data (every impression generates a training example) – Limitations at serving: need to predict quickly
- 32. 32 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Labeled events Impression0 click Impression1 non-click Impression2 non-click … … Impression10 non-click Impression11 click Impression12 non-click … … Impression100 non-click Impression101 non-click Impression102 non-click … … Labeled events Labeled events
- 33. 33 © Hortonworks Inc. 2011 – 2016. All Rights Reserved CTR model Logistic regression (LR) – LR on SGD/LBFGS - batch – Follow the regularized leader (FTRL) - online Factorization Machines (FM/FFM) Gradient boosting tree (GBT) Deep neural networks (DNN)
- 34. 34 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Questions?
- 35. 35 © Hortonworks Inc. 2011 – 2016. All Rights Reserved BoF’s: Apache Hadoop – YARN, HDFS Sanjay Radia Hortonworks Sunil G Hortonworks https://dataworkssummit.com/sydney-2017/birds-of-a-feather/apache-hadoop-yarn-hdfs Thursday September 21st Room C2.1 6.00PM -- Related Session -- Rohith Sharma K S Hortonworks
- 36. 36 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Thank you!
Editor's Notes
- Data is flooding into every business. In many applications, we need more training data and bigger models means better result. We use Hadoop to store large amount of data, use Spark on YARN for simple data processing, can also can try some machine learning frameworks such as TensorFlow or XGBoost on the hadoop-based big data platform for machine learning or deep learning.
- Another important change is the roles in machine learning. As the increasing dataset and more and more complex problem, one person can’t do all of the work, we need data scientists work together with software engineers. Data scientist usually explore data, find the best machine learning pipeline. After that, software engineer will deploy the model and make prediction based on new input. The input data could be batch data or streaming data.
- This is a typical machine learning, which involves three steps: feature engineering, model training and online service. Not surprisingly, the most important thing is to have the right features: those capturing historical information dominate other types of features. Once we have the right features and the right model, other factors play small roles. We first get feature representation from raw data, and then feed these features into machine learning model, and then evaluate the model and choose the best one to push into online service. The machine learning workflow is complicated, usually involves several steps under the help of several infrastructure components.
- Just like the workflow shows, only a tiny fraction of the code is actually devoted to model learning. The machine learning workflow usually need lots of supports from the big data platform, such as data collection from different data sources, feature extraction, feature transform, and so on. Let’s find out how big data infrastructure could help machine learning step by step.
- Machine learning workflow starts with loading data from different data sources, like HDFS, AWS S3 or database system. After that, we usually join data from different source to generate a wide table. Apache Hive or Apache Spark is the most appropriate tools to handle this workload. And then, data scientists starts data exploration via Zeppelin. The most common issue is unbalanced label for the dataset, for example, the number of positive label is far more than the negative label. To get more accurate model, we need to subsample data from the group which has more instances to make it balanced. After that, we random split the dataset for training and test under the help of Spark. Once we get training data, we can start feature engineering.
- Feature engineering technology has made great progress over the past decade, from hand-designed features to automating feature discovery by deep learning. In many cases, hand-designed features can leverage the understanding of the domain knowledge which will lead to optimal results, Spark MLlib provides lots of feature transform/selection operators to make it simple and easily. But it will involves heavy physical work and need hire experienced engineers. DNNs has been successful applied in computer vision, speech recognition and natural language processing during recent years. More and more scientists and engineers applied deep neural network in computer vision, speech recognition and natural language and it has achieved good results. DNN can learn features automatically via embedding, the most famous embedding trick is word2vec which can produce a vector space, with each unique word in the corpus being assigned a corresponding vector in the space.
- Model training is the most important step of the whole pipeline.
- Deep learning is becoming more and more powerful, but it can’t solve all of humanity‘s problems. In natural language processing, computer vision, speech or video recognition areas, deep learning may behavior better than traditional model. But for problems like recommendation or CTR estimation, very scalable linear models still play a major roles. And some graphic related model like topic model or PageRank, we still need graph calculation engine. Further more, hybrid model is becoming more and more useful. For example, Facebook presents a hybrid model structure: the concatenation of boosted decision trees and of a probabilistic sparse linear classier, illustrated in the figure. Their experience tells us, this hybrid structure significantly increases the prediction accuracy. Google also developed a hybrid model - wide and deep learning model, which jointly train a wide linear model (for memorization) alongside a deep neural network (for generalization), one can combine the strengths of both to bring us one step closer. From the above cases, we can learn that a machine learning platform should support traditional machine learning model and deep learning model, both of them are very useful.
- Deploy the model distributed for parallel model serving on batch mode or streaming mode. Evaluate the model offline or online by different metrics.
- Predicting Ad CTR is a massive-scale machine learning problem that is central to the multi-billion dollar online advertising industry. A typical CTR prediction problem shares some similarities with many other industry machine learning problems, which makes it very representative.
- Usually there are billions of Ad impressions daily. Each impression has an unique id. We need join impressions with click stream every x minutes as the dataset for machine learning.
- Each model has advantage and disadvantage: Non-linear models, on the other hand, are able to utilize different feature combinations and thus could potentially improve estimation performance, but can’t scale to a large number of parameter. Deep neural networks (DNNs) are able to extract the hidden structures and intrinsic patterns at different levels of abstractions from training data. But training deep neural networks on a large input feature space requires tuning a huge number of parameters, which is computationally expensive. And the input raw features are high dimensional and sparse binary features converted from the raw categorical features, which makes it hard to train traditional DNNs in large scale.