Chug dl presentation
•Download as PPTX, PDF•
1 like•401 views
Slides from The Deep Learning Meetup
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Chug dl presentation
Similar to Chug dl presentation (20)
More from Chicago Hadoop Users Group
More from Chicago Hadoop Users Group (18)
Recently uploaded
Recently uploaded (20)
Chug dl presentation
- 1. Introducing a new exascale era for innovation DEEP LEARNING ON GPUS by Alex Volkov avolkov@nvidia.com
- 2. 2 AGENDA OUTLINE • Introduction: Brief discussion of what is motivating popularity of GPUs. • Why GPUs? Moore’s law and breakdown of Dennard Scaling is resulting in emergence of domain specific architectures (DSAs). • GPUs match application to the processor architecture (graphics, virtual reality, neural networks and Deep Learning, massive matrix computations) • What is deep learning and how it differs from traditional data analytics? • Traditionally experts manually extract relevant features. These features drive some process or model for information discovery or decision making. Deep learning models are driven by raw data directly. No manual feature extraction. • Example: Image processing. In the past would use image processing such as edge detection, optical flow, etc. With deep learning feed the image directly to the network and use CNN layers (convolutional neural networks), RNN/LSTM (recurrent NN and long-short-term-memory networks). These layers more or less achieve automatically what used to be done manually, but better and automated. • Discussion on different types of Deep Learning networks. Attached image reference. • Discussion how Analytics/Big Data can be enhanced by DL and technology examples. • Yahoo projects: Caffe on Spark, Tensorflow on Spark. • DL4J on Spark • Tentatively do a quick demo or overview of a demo done ahead of time with Tensorflow on Spark accelerated with GPUs. • Conclusion.
- 3. 3 DEEP LEARNING ON GPUS Abstract: Modern day computational challenges are going beyond capabilities of traditional multiprocessors. Graphical Processing Units (GPUs) are filling the performance gap by taking advantage of its massively parallel architecture. GPUs enable practical applications of Artificial Intelligence and Deep Learning (DL), Machine Learning, and state of the art analytics methodologies. The presentation will give a general overview of DL and the areas where GPUs can help accelerate the analytics workflows using DL. DL applications will illustrate the challenges that data scientists are faced with and how DL is meeting these challenges. A GPU enabled Spark ecosystem using Tensorflow DL framework will be presented to demonstrate advantages that GPUs bring to the datacenters and data scientists. Data Analytics and Deep Learning on GPUs
- 4. 4 INNOVATION: A HISTORICAL VIEW h
- 5. 5 WHY GPUS GPUs are cost effective computing engines for demands of Exascale Era End of Dennard Scaling places a cap on single threaded performance Increasing application performance will require fine grain parallel code with significant computational intensity AI and Data Science emerging as important new components of scientific discovery Dramatic improvements in accuracy, completeness and response time yield increased insight from huge volumes of data Cloud based usage models, in- situ execution and visualization emerging as new workflows critical to the science process and productivity Tight coupling of interactive simulation, visualization, data analysis/AI
- 6. 6 RISE OF GPU COMPUTING ARCHITECTURE Moore’s Law is NOT Dead as transistor count keeps growing 1980 1990 2000 2010 2020 GPU-Computing perf 1.5X per year 1000X by 2025 Original data up to the year 2010 collected and plotted by M. Horowitz, F. Labonte, O. Shacham, K. Olukotun, L. Hammond, and C. Batten New plot and data collected for 2010-2015 by K. Rupp 102 103 104 105 106 107 Single-threaded perf 1.5X per year 1.1X per year APPLICATIONS SYSTEMS ALGORITHMS CUDA ARCHITECTURE
- 7. 7 SO WHAT IS DEEP LEARNING (DL)? And what is a Neural Network? Fundamentally it is just a geometric transformation. “A geometric transformation is a function whose domain and range are sets of points.” DL is a sophisticated Neural Network
- 8. 8 MANY TYPES OF NEURAL NETWORKS DL does not require manual feature extraction Automation is the name of the game https://www.asimovinstitute.org/author/fjodorvanveen/ • Popular Neural Nets in DL • CNN – convolution neural nets • RNN – recurrent neural nets • LSTM – long-short term memory • GAN – Generative Adversarial Network • GRU – Gated Recurrent Unit “Success of deep learning so far has been the ability to map space X to space Y using a continuous geometric transform, given large amounts of human-annotated data. Doing this well is a game- changer for essentially every industry.” https://blog.keras.io/the-limitations-of-deep-learning.html
- 9. 9 CONVERGENCE OF DATA ANALYTICS AND DL GPU acceleration at the heart of the envisioned converged analytics (Lambda) architecture Data Sources Ingest Storage Stream Processing Batch Processing Serving Layer Notebook Visualization Syslog Netflow Graph Visualization Interactivity QuerySpeed cuSTINGER
- 10. 10 GOAI: GPU OPEN ANALYTICS INITIATIVE GDF (GPU Data Frame) Data Remains Resident on GPU for efficient to avoid io bottleneck https://github.com/gpuopenanalytics/pygdf In the past GPU Data had to be copied unnecessarily between host and device memory resulting in io bottlenecks
- 11. 11 GPU ANALYTICS SOFTWARE STACK Achieve unprecedented speedup in your day to day workflows GBM training benchmark comparing a dual-Xeon CPU- only system to a system with multiple Tesla P100 GPUs. https://devblogs.nvidia.com/parallelforall/goai-open-gpu-accelerated-data-analytics/
- 12. 12 BATCH PROCESSING WITH SPARK AND DL Spark is not efficient as a computation layer for DL calculations, but can be used for fast ETL. Typically orchestrate jobs to GPUs. Popular frameworks for Spark and DL: DL4J - https://deeplearning4j.org/spark, https://github.com/deeplearning4j/scalnet Java and Scala based Yahoo: https://github.com/yahoo/CaffeOnSpark, https://mapr.com/blog/distributed-deep-learning-caffe-using-mapr-cluster/ https://github.com/yahoo/TensorFlowOnSpark Databricks: https://github.com/databricks/tensorframes CERNDB: https://github.com/cerndb/dist-keras https://github.com/adatao/tensorspark List of frameworks to run DL on Spark clusters
- 13. 13 TENSORFLOW ON SPARK Sparks-tensorflow-connector - library for loading and storing TensorFlow records with Apache Spark. https://github.com/tensorflow/ecosystem/tree/master/spark/spark-tensorflow-connector Demo Yahoo’s TensorflowOnSpark implementation. https://github.com/yahoo/TensorFlowOnSpark/tree/master/examples https://github.com/yahoo/TensorFlowOnSpark/wiki https://github.com/yahoo/TensorFlowOnSpark/wiki/GetStarted_YARN https://github.com/yahoo/TensorFlowOnSpark/wiki/GetStarted_Standalone https://github.com/yahoo/TensorFlowOnSpark/wiki/Conversion-Guide DEMO TF on SPARK
Editor's Notes
- Brief discussion of the convergence of high performance computing and artificial intelligence. Generally there is a pattern for the introduction of new technology into the market. This pattern takes the shape of an S-curve. Penetration of technology is typically slow during the initial phase when technology if first introduced. As the applications using the new technology mature penetration and adoption picks up dramatically. Eventually as new technology appears older technology tends to taper off. This trend has been typical in diverse industries from textile to steam engines to the automotive era. We are currently entering the distributed intelligence era in which artificial intelligence is becoming commonplace and widely adopted.
- Look at the three major factors driving innovation. The end of Dennard scaling; this tracks the tapering off of the era of traditional CPU based computing which fundamentally means that single threaded performance is hitting a wall. If your application cannot be executed in parallel, you’re running into a fundamental limitation in the performance that you can achieve with that application. GPUs offer the opportunity to run those applications in parallel in an economically scalable manner. While some applications are challenged in their ability to go in parallel, artificial intelligence is being introduced to market and is ramping up at a very high speed in the commercial sector. The beauty is that the training for artificial intelligence runs extraordinarily well on GPUs. The throughput capability that a GPU offers can be fully realized with AI. Once an application has been trained, the inference can run at or near real time. Cloud computing is also evolving. The cloud adoption democratizes large scale computing. Visualization and AI applications that require speed can be offloaded to the cloud and processed by GPUs.
- Dennard (1974) (famous electrical engineer and inventor) observed that voltage and current should be proportional to the linear dimensions of a transistor ♦ Thus, as transistors shrank, so did necessary voltage and current; power is proportional to the area of the transistor. As transistors get smaller, power density increases because these don’t scale with size. These created a “Power Wall” that has limited practical processor frequency to around 4 GHz since 2006
- https://en.wikipedia.org/wiki/Geometric_transformation The idea of DL is actually pretty simple. All you need is sufficiently large parametric models trained with gradient descent on sufficiently many examples. As the quote attributed to Richard Feynman goes, "It's not complicated, it's just a lot of it".
- Taken from: https://blog.keras.io/the-limitations-of-deep-learning.html In deep learning, everything is a vector, i.e. everything is a point in a geometric space. Model inputs (it could be text, images, etc) and targets are first "vectorized", i.e. turned into some initial input vector space and target vector space. Each layer in a deep learning model operates one simple geometric transformation on the data that goes through it. Together, the chain of layers of the model forms one very complex geometric transformation, broken down into a series of simple ones. This complex transformation attempts to maps the input space to the target space, one point at a time. This transformation is parametrized by the weights of the layers, which are iteratively updated based on how well the model is currently performing. (A key characteristic of this geometric transformation is that it must be differentiable, which is required in order for us to be able to learn its parameters via gradient descent. Intuitively, this means that the geometric morphing from inputs to outputs must be smooth and continuous—a significant constraint.) That's the magic of deep learning: turning meaning into vectors, into geometric spaces, then incrementally learning complex geometric transformations that map one space to another. All you need are spaces of sufficiently high dimensionality in order to capture the full scope of the relationships found in the original data. The limitations of deep learning In general, anything that requires reasoning—like programming, or applying the scientific method—long-term planning, and algorithmic-like data manipulation, is out of reach for deep learning models, no matter how much data you throw at them. Even learning a sorting algorithm with a deep neural network is tremendously difficult. The risk of anthropomorphizing machine learning models Taken from: https://blog.keras.io/the-future-of-deep-learning.html The future of deep learning In summary: the long-term vision In short, here is my long-term vision for machine learning: Models will be more like programs, and will have capabilities that go far beyond the continuous geometric transformations of the input data that we currently work with. These programs will arguably be much closer to the abstract mental models that humans maintain about their surroundings and themselves, and they will be capable of stronger generalization due to their rich algorithmic nature. In particular, models will blend algorithmic modules providing formal reasoning, search, and abstraction capabilities, with geometric modules providing informal intuition and pattern recognition capabilities. AlphaGo (a system that required a lot of manual software engineering and human-made design decisions) provides an early example of what such a blend between symbolic and geometric AI could look like. They will be grown automatically rather than handcrafted by human engineers, using modular parts stored in a global library of reusable subroutines—a library evolved by learning high-performing models on thousands of previous tasks and datasets. As common problem-solving patterns are identified by the meta-learning system, they would be turned into a reusable subroutine—much like functions and classes in contemporary software engineering—and added to the global library. This achieves the capability for abstraction. This global library and associated model-growing system will be able to achieve some form of human-like "extreme generalization": given a new task, a new situation, the system would be able to assemble a new working model appropriate for the task using very little data, thanks to 1) rich program-like primitives that generalize well and 2) extensive experience with similar tasks. In the same way that humans can learn to play a complex new video game using very little play time because they have experience with many previous games, and because the models derived from this previous experience are abstract and program-like, rather than a basic mapping between stimuli and action. As such, this perpetually-learning model-growing system could be interpreted as an AGI—an Artificial General Intelligence. But don't expect any singularitarian robot apocalypse to ensue: that's a pure fantasy, coming from a long series of profound misunderstandings of both intelligence and technology. This critique, however, does not belong here.
- Lambda Architecture has two processing layers in stream and batch so there’s two code bases to manage. When building analytics you often times want different analytics run on historical vs real-time data anyway so you end up with two code bases either way. Lambda Architecture Uses batch and stream processing to try to balance latency, throughput, and fault-tolerance. Uses batch processing to provide comprehensive views of historical data. Uses stream processing to provide an online view of real-time data. GPU Accelerated architecture has the same layers as the big data architecture, but there are dramatically less technologies, where certain GPU-accelerated technologies such as MapD and Kinetica end up eliminating multiple technologies by themselves.
- As the software stack has matured, GOAI appeared in order to efficiently manage the data pipeline through the converged data analytics architecture. The heart of the pipeline is the GPU Data Frame which keeps the GPU data resident on GPU memory to workaround i/o bottlenecks.
- Checkout the blog that will guide you through several demos. Use these demos as starting points to augment your workflows and see what acceleration you can achieve.