Slides from my talk at the useR Group Münster 04/17/18 on how to start with GPU enabled deep learning in R. First I'm showing how to create a NVIDIA docker based image with RStudio, TensorFlow and Keras for R and then comes an introduction to deep learning (classic MNIST classification with MLP and CNN).
Slides for JavaOne 2015 talk by Brendan Gregg, Netflix (video/audio, of some sort, hopefully pending: follow @brendangregg on twitter for updates). Description: "At Netflix we dreamed of one visualization to show all CPU consumers: Java methods, GC, JVM internals, system libraries, and the kernel. With the help of Oracle this is now possible on x86 systems using system profilers (eg, Linux perf_events) and the new JDK option -XX:+PreserveFramePointer. This lets us create Java mixed-mode CPU flame graphs, exposing all CPU consumers. We can also use system profilers to analyze memory page faults, TCP events, storage I/O, and scheduler events, also with Java method context. This talk describes the background for this work, instructions generating Java mixed-mode flame graphs, and examples from our use at Netflix where Java on x86 is the primary platform for the Netflix cloud."
AdamCloud: A Cloud infrastructure for a Genomic project. The AdamCloud project aims to simplify the installation of the AmpLab genomic pipeline (Snap, Adam, Avocado).
The results of the first iteration (part II) were presented here:
http://www.slideshare.net/davidonlaptop/bdm32-adam-cloud-part-2-43514904
Slides for JavaOne 2015 talk by Brendan Gregg, Netflix (video/audio, of some sort, hopefully pending: follow @brendangregg on twitter for updates). Description: "At Netflix we dreamed of one visualization to show all CPU consumers: Java methods, GC, JVM internals, system libraries, and the kernel. With the help of Oracle this is now possible on x86 systems using system profilers (eg, Linux perf_events) and the new JDK option -XX:+PreserveFramePointer. This lets us create Java mixed-mode CPU flame graphs, exposing all CPU consumers. We can also use system profilers to analyze memory page faults, TCP events, storage I/O, and scheduler events, also with Java method context. This talk describes the background for this work, instructions generating Java mixed-mode flame graphs, and examples from our use at Netflix where Java on x86 is the primary platform for the Netflix cloud."
AdamCloud: A Cloud infrastructure for a Genomic project. The AdamCloud project aims to simplify the installation of the AmpLab genomic pipeline (Snap, Adam, Avocado).
The results of the first iteration (part II) were presented here:
http://www.slideshare.net/davidonlaptop/bdm32-adam-cloud-part-2-43514904
Agenda:
The Linux kernel has multiple "tracers" built-in, with various degrees of support for aggregation, dynamic probes, parameter processing, filtering, histograms, and other features. Starting from the venerable ftrace, introduced in kernel 2.6, all the way through eBPF, which is still under development, there are many options to choose from when you need to statically instrument your software with probes, or diagnose issues in the field using the system's dynamic probes. Modern tools include SystemTap, Sysdig, ktap, perf, bcc, and others. In this talk, we will begin by reviewing the modern tracing landscape -- ftrace, perf_events, kprobes, uprobes, eBPF -- and what insight into system activity these tools can offer. Then, we will look at specific examples of using tracing tools for diagnostics: tracing a memory leak using low-overhead kmalloc/kfree instrumentation, diagnosing a CPU caching issue using perf stat, probing network and block I/O latency distributions under load, or merely snooping user activities by capturing terminal input and output.
Speaker:
Sasha is the CTO of Sela Group, a training and consulting company based in Israel that employs over 400 developers world-wide. Most of Sasha's work revolves around performance optimization, production debugging, and low-level system diagnostics, but he also dabbles in mobile application development on iOS and Android. Sasha is the author of two books and three Pluralsight courses, and a contributor to multiple open-source projects. He blogs at http://blog.sashag.net.
Cgroups, namespaces and beyond: what are containers made from?Docker, Inc.
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
Make Your Containers Faster: Linux Container Performance ToolsKernel TLV
If you look under the hood, Linux containers are just processes with some isolation features and resource quotas sprinkled on top. In this talk, we will apply modern Linux performance tools to container analysis: get high-level resource utilization on running containers with docker stats, htop, and nsenter; dig into high-CPU issues with perf; detect slow filesystem latency with BPF-based tools; and generate flame graphs of interesting event call stacks.
Sasha Goldshtein is the CTO of Sela Group, a Microsoft MVP and Regional Director, Pluralsight and O'Reilly author, and international consultant and trainer. Sasha is the author of two books and multiple online courses, and a prolific blogger. He is also an active open source contributor to projects focused on system diagnostics, performance monitoring, and tracing -- across multiple operating systems and runtimes. Sasha authored and delivered training courses on Linux performance optimization, event tracing, production debugging, mobile application development, and modern C++. Between his consulting engagements, Sasha speaks at international conferences world-wide.
You can find more details on the meetup page - https://www.meetup.com/Tel-Aviv-Yafo-Linux-Kernel-Meetup/events/245319189/
Video: https://www.facebook.com/atscaleevents/videos/1693888610884236/ . Talk by Brendan Gregg from Facebook's Performance @Scale: "Linux performance analysis has been the domain of ancient tools and metrics, but that's now changing in the Linux 4.x series. A new tracer is available in the mainline kernel, built from dynamic tracing (kprobes, uprobes) and enhanced BPF (Berkeley Packet Filter), aka, eBPF. It allows us to measure latency distributions for file system I/O and run queue latency, print details of storage device I/O and TCP retransmits, investigate blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. This talk will summarize this new technology and some long-standing issues that it can solve, and how we intend to use it at Netflix."
When working with big data or complex algorithms, we often look to parallelize our code to optimize runtime. By taking advantage of a GPUs 1000+ cores, a data scientist can quickly scale out solutions inexpensively and sometime more quickly than using traditional CPU cluster computing. In this webinar, we will present ways to incorporate GPU computing to complete computationally intensive tasks in both Python and R.
See the full presentation here: 👉 https://vimeo.com/153290051
Learn more about the Domino data science platform: https://www.dominodatalab.com
From USENIX LISA 2010, San Jose.
Visualizations that include heat maps can be an effective way to present performance data: I/O latency, resource utilization, and more. Patterns can emerge that would be difficult to notice from columns of numbers or line graphs, which are revealing previously unknown behavior. These visualizations are used in a product as a replacement for traditional metrics such as %CPU and are allowing end users to identify more issues much more easily (and some issues are becoming nearly impossible to identify with tools such as vmstat(1)). This talk covers what has been learned, crazy heat map discoveries, and thoughts for future applications beyond performance analysis.
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
With the following report I show how to host and execute a deep learning project on a cloud. The cloud is hosted by google Colab and enables working and testing in teams. Lazarus and FreePascal is also being built in colab and the deep learning network is compiled and trained too in a Jupyter notebook with Python scripts.
MeetBSDCA 2014 Performance Analysis for BSD, by Brendan Gregg. A tour of five relevant topics: observability tools, methodologies, benchmarking, profiling, and tracing. Tools summarized include pmcstat and DTrace.
Настройка окружения для кросскомпиляции проектов на основе docker'acorehard_by
Как быстро и легко настраивать/обновлять окружения для кросскомпиляции проектов под различные платформы(на основе docker), как быстро переключаться между ними, как используя эти кирпичики организовать CI и тестирование(на основе GitLab и Docker).
Agenda:
The Linux kernel has multiple "tracers" built-in, with various degrees of support for aggregation, dynamic probes, parameter processing, filtering, histograms, and other features. Starting from the venerable ftrace, introduced in kernel 2.6, all the way through eBPF, which is still under development, there are many options to choose from when you need to statically instrument your software with probes, or diagnose issues in the field using the system's dynamic probes. Modern tools include SystemTap, Sysdig, ktap, perf, bcc, and others. In this talk, we will begin by reviewing the modern tracing landscape -- ftrace, perf_events, kprobes, uprobes, eBPF -- and what insight into system activity these tools can offer. Then, we will look at specific examples of using tracing tools for diagnostics: tracing a memory leak using low-overhead kmalloc/kfree instrumentation, diagnosing a CPU caching issue using perf stat, probing network and block I/O latency distributions under load, or merely snooping user activities by capturing terminal input and output.
Speaker:
Sasha is the CTO of Sela Group, a training and consulting company based in Israel that employs over 400 developers world-wide. Most of Sasha's work revolves around performance optimization, production debugging, and low-level system diagnostics, but he also dabbles in mobile application development on iOS and Android. Sasha is the author of two books and three Pluralsight courses, and a contributor to multiple open-source projects. He blogs at http://blog.sashag.net.
Cgroups, namespaces and beyond: what are containers made from?Docker, Inc.
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
Make Your Containers Faster: Linux Container Performance ToolsKernel TLV
If you look under the hood, Linux containers are just processes with some isolation features and resource quotas sprinkled on top. In this talk, we will apply modern Linux performance tools to container analysis: get high-level resource utilization on running containers with docker stats, htop, and nsenter; dig into high-CPU issues with perf; detect slow filesystem latency with BPF-based tools; and generate flame graphs of interesting event call stacks.
Sasha Goldshtein is the CTO of Sela Group, a Microsoft MVP and Regional Director, Pluralsight and O'Reilly author, and international consultant and trainer. Sasha is the author of two books and multiple online courses, and a prolific blogger. He is also an active open source contributor to projects focused on system diagnostics, performance monitoring, and tracing -- across multiple operating systems and runtimes. Sasha authored and delivered training courses on Linux performance optimization, event tracing, production debugging, mobile application development, and modern C++. Between his consulting engagements, Sasha speaks at international conferences world-wide.
You can find more details on the meetup page - https://www.meetup.com/Tel-Aviv-Yafo-Linux-Kernel-Meetup/events/245319189/
Video: https://www.facebook.com/atscaleevents/videos/1693888610884236/ . Talk by Brendan Gregg from Facebook's Performance @Scale: "Linux performance analysis has been the domain of ancient tools and metrics, but that's now changing in the Linux 4.x series. A new tracer is available in the mainline kernel, built from dynamic tracing (kprobes, uprobes) and enhanced BPF (Berkeley Packet Filter), aka, eBPF. It allows us to measure latency distributions for file system I/O and run queue latency, print details of storage device I/O and TCP retransmits, investigate blocked stack traces and memory leaks, and a whole lot more. These lead to performance wins large and small, especially when instrumenting areas that previously had zero visibility. This talk will summarize this new technology and some long-standing issues that it can solve, and how we intend to use it at Netflix."
When working with big data or complex algorithms, we often look to parallelize our code to optimize runtime. By taking advantage of a GPUs 1000+ cores, a data scientist can quickly scale out solutions inexpensively and sometime more quickly than using traditional CPU cluster computing. In this webinar, we will present ways to incorporate GPU computing to complete computationally intensive tasks in both Python and R.
See the full presentation here: 👉 https://vimeo.com/153290051
Learn more about the Domino data science platform: https://www.dominodatalab.com
From USENIX LISA 2010, San Jose.
Visualizations that include heat maps can be an effective way to present performance data: I/O latency, resource utilization, and more. Patterns can emerge that would be difficult to notice from columns of numbers or line graphs, which are revealing previously unknown behavior. These visualizations are used in a product as a replacement for traditional metrics such as %CPU and are allowing end users to identify more issues much more easily (and some issues are becoming nearly impossible to identify with tools such as vmstat(1)). This talk covers what has been learned, crazy heat map discoveries, and thoughts for future applications beyond performance analysis.
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
Talk for USENIX/LISA2014 by Brendan Gregg, Netflix. At Netflix performance is crucial, and we use many high to low level tools to analyze our stack in different ways. In this talk, I will introduce new system observability tools we are using at Netflix, which I've ported from my DTraceToolkit, and are intended for our Linux 3.2 cloud instances. These show that Linux can do more than you may think, by using creative hacks and workarounds with existing kernel features (ftrace, perf_events). While these are solving issues on current versions of Linux, I'll also briefly summarize the future in this space: eBPF, ktap, SystemTap, sysdig, etc.
With the following report I show how to host and execute a deep learning project on a cloud. The cloud is hosted by google Colab and enables working and testing in teams. Lazarus and FreePascal is also being built in colab and the deep learning network is compiled and trained too in a Jupyter notebook with Python scripts.
MeetBSDCA 2014 Performance Analysis for BSD, by Brendan Gregg. A tour of five relevant topics: observability tools, methodologies, benchmarking, profiling, and tracing. Tools summarized include pmcstat and DTrace.
Настройка окружения для кросскомпиляции проектов на основе docker'acorehard_by
Как быстро и легко настраивать/обновлять окружения для кросскомпиляции проектов под различные платформы(на основе docker), как быстро переключаться между ними, как используя эти кирпичики организовать CI и тестирование(на основе GitLab и Docker).
SenchaCon 2016: Develop, Test & Deploy with Docker - Jonas Schwabe Sencha
Have you ever heard the phrase: "Everything works fine on my machine?" Docker is here to rescue you. Running your toolchain, Ext JS application, back-end server, and even your database - all in a standardized container format that can be transported and reused, throughout your process. In this session, you will learn how to automate a typical workflow, including developing, testing, and deploying, by using Docker containers and common continuous integration solutions.
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...Jérôme Petazzoni
If you're not familiar yet with Docker, here is your chance to catch up. This presentation includes a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. Recent features are listed, as well as a glimpse at what's next in the Docker world.
This presentation was given during OSCON, at a meet-up hosted by New Relic, with co-presentations from CoreOS and Rackspace OnMetal.
Introduction to Docker at SF Peninsula Software Development Meetup @GuidewiredotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Introduction to Docker at the Azure Meet-up in New YorkJérôme Petazzoni
This is the presentation given at the Azure New York Meet-Up group, September 3rd.
It includes a quick overview of the Open Source Docker Engine and its associated services delivered through the Docker Hub. It also covers the new features of Docker 1.0, and briefly explains how to get started with Docker on Azure.
Tracing MariaDB server with bpftrace - MariaDB Server Fest 2021Valeriy Kravchuk
Bpftrace is a relatively new eBPF-based open source tracer for modern Linux versions (kernels 5.x.y) that is useful for analyzing production performance problems and troubleshooting software. Basic usage of the tool, as well as bpftrace one liners and advanced scripts useful for MariaDB DBAs are presented. Problems of MariaDB Server dynamic tracing with bpftrace and some possible solutions and alternative tracing tools are discussed.
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...Eric Smalling
Slides from my 2.5 hour hands-on workshop covering Docker basics, the Docker MTA program and how it applies to legacy Java applications and some tips on running those apps in containers in production.
Most developers start adopting Docker by integrating it with their development environment. Unfortunately development environments are nuanced. Using Docker to automate and isolate development environments is rewarding, but you'll need to keep a few things in mind when designing that integration.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Machine learning and optimization techniques for electrical drives.pptx
From Zero to Hero - All you need to do serious deep learning stuff in R
1. From Zero to Hero
ALL YOU NEED TO DO SERIOUS DEEP LEARNING STUFF IN R
2. Agenda
• About Me
• How I use R/RStudio/(Nvidia)Docker
• Image recognition with R and Keras
3. Kai Lichtenberg, 33
Machine Learning Deep Learning
Sales Engineer
Mechanical
Engineer
Data Scientist
MSc
Sales Engineering & Product
Management
BSc
Sales Engineering & Product
Management
PhD @ Bosch
MSc
Mechanical Engineering
BSc
Mechanical Engineering
Working student
Gearbox
Development
Advanced Engineering
Transmission Units
Advanced Engineering
Connectivity Solutions
Founder, Data Scientist,
Accountant, Office
Manager, Janitor, …
Classic Stuff
Language Journey
TODAY
Quantum Computing
Q#
4. Some Projects
• Modeling reliability of equipment with high dimensional data (PhD)
• Non-linear exogenous autoregression (neural nets) to predict temperature
• Various production datasets, e.g. high pressure diesel pump and injector
• Who’s driving the machine? Driver classification and profiling
• Condition monitoring via acceleration and acoustic signals
• Customer churn prediction
• Image recognition with deep neural nets
• Blogging @Bosch
6. How my Dev Environment looked like
• Installed on the same system I use for office, browsing, etc.
• Pretty big stack with numerous packages, IDE’s, dependen-
cies, drivers, venvs, environment variables, …
• Especially the Keras->TensorFlow->cuDNN toolchain is tricky
• Takes hours to set up and just seconds to mess up
• I tend to fiddle around and break things :-)
• Needs to be installed on my laptop and my workstation (and
behave the same)
7. How I want my Dev Environment
• Very easy to install
• Portable to any Linux
• GPU support!
• No big overhead, use of native system resources
• No fiddling, tweaking an tuning
• All my beloved packages and tools
No cloud services! Only a container registry
8. Sneak Peek
#Docker CE
apt-get update
apt-get install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update
apt-get install docker-ce
usermod -a -G docker $USER
# Add the package repositories for nvidia docker
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/amd64/nvidia-docker.list |
tee /etc/apt/sources.list.d/nvidia-docker.list
apt-get update
# Install nvidia-docker2 and reload the Docker daemon
apt-get install -y nvidia-docker2
pkill -SIGHUP dockerd
#Log into the nvidia repo and get tensorflow python 3
docker login nvcr.io
docker pull nvcr.io/nvidia/tensorflow:18.02-py3
#Clone Repo and build images
git clone https://github.com/KaiLicht/DataScience_Toolbox
cd DataScience_Toolbox/dockerfiles/Rbase_keras_tf
docker build -t kailicht/Rbase_keras_tf .
cd DataScience_Toolbox/dockerfiles/RStudio_deeplearning
docker build -t kailicht/rstudio_deeplearning .
cd DataScience_Toolbox/dockerfiles/My_Rstudio
docker build -t kailicht/myrstudio:1.0 .
No need to copy! I’m going to publish a
blogpost on how to create this stuff from
scratch and link it in the meetup group.
9. What is Docker?
Server
Host OS
Hypervisor
Guest OS
Bins/Libs
App A
Guest OS
Bins/Libs
App B
Server
Host OS
Docker Engine
Bins/Libs
App A
Bins/Libs
App B
Virtualization (VM’s) Isolation (Container)
Server
Host OS
Docker Engine
Bins/Libs
App A
Bins/Libs
App B
Container with GPU
GPU GPU GPU
CUDA Driver
CUDA toolkit CUDA toolkit
• No shared resources
• Complete OS is emulated
• Big overhead
• Not really suited for Data
Science environments
• Shared resources with host
• Only isolated file system
• Very little overhead
• Perfect for reproducible
Data Science
• Containers with GPU
passthrough
• Nvidia offers preconfigured
and optimized images for
different use cases (e.g. a
python-TensorFlow image)
See for yourself!
10. How to make a docker image
FROM ubuntu:16.04
MAINTAINER Kai Lichtenberg <kai@sentin.ai>
# Set a default user.
RUN useradd docker
&& mkdir /home/docker
&& chown docker:docker /home/docker
&& addgroup docker staff
RUN apt-get update
&& apt-get install -y --no-install-recommends
ed
locales
vim-tiny
fonts-texgyre
gnupg2
libcurl4-openssl-dev
libssl-dev
libssh2-1-dev
sudo
&& rm -rf /var/lib/apt/lists/*
dockerfile:
kai@XPS15:~$ docker build -t my_ubuntu:1.0 .
Building the image:
What is happening?
• By starting the build process docker first downloads the ubuntu:16.04
base image from dockerhub
• Then it starts the base image and basically parses the commands in the
dockerfile (here: adding a user and installing some stuff with apt-get)
• After everything is done the image is saved with the tag my_ubuntu:1.0
kai@XPS15:~$ docker run –it my_ubuntu:1.0
root@52d676fa5956:/workspace# uname –a
Linux 9b6e2b8f6e53 4.13.0-38-generic #43~16.04.1-Ubuntu SMP
Wed Mar 14 17:48:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Starting the image:
11. My Rstudio Docker Containers
kailicht/my_rstudio:1.0
Building it in 4 stages • The foundation for this build is the
tensorflow:18.02-py3 image from NVIDIA
• It comes batteries included:
• Ubuntu 16.04 with python 3.5
• NVIDIA CUDA
• cuBLAS (Linear Algebra)
• NCCL (For Multi-GPU usage)
• Horovod Distributed DL Framework
• OpenMPI
• Tensorboard
• TensorRT to optimize deployment
kailicht/rstudio_deeplearning
kailicht/rbase_keras_tf
Base: tensorflow:18.02-py3
Let’s have a look
into the dockerfiles!
0. Get the base image
1. Install R, Keras &TensorFlow for R
2. Install Rstudio and dependencies
3. Customize (Settings, Packages)
kailicht/my_rstudio:1.x 4. Install new packages
Images for different use cases:
• Do Data Science REPL-Work: kailicht/my_rstudio:1.x
• Deploy a shiny app with image recognition: kailicht/rbase_keras_tf
• Deploy a flask microservice with a TensorFlow model: tensorflow:18.02-py3
• …
13. But wait, isn’t python better for Deep Learning?
• No.
• In python and R you only define the neural network!
• All the magic happens in the toolchain.
• It creates the computational graph and performs all the matrix operations needed on
your CUDA enabled GPU.
• Sorry but using R or python for Deep Learning is pretty much like using a GUI
• If you want to deploy a trained network (e.g. as an API) then python is the better choice
• It’s important to differentiate between creating a model and deploying it!
14. Before we start: The MNIST Data Set
In R:
• The Hello World of Machine Learning
• 70k handwritten digits as 28x28 monochrome pictures
• And off course the class labels (0-9)
btw: this is a tensor! (like
any other matrix or vector)
15. Before we start: Piping
#Select 3 columns in a data frame
MyDataFrame2 <- select(MyDataFrame1, MyCol1, MyCol2, MyCol3)
#Is the same like:
MyDataFrame2 <- MyDataFrame1 %>%
select(MyCol1, MyCol2, MyCol3)
#Chain it!
MyDataFrame2 <- MyDataFrame1 %>%
select(MyCol1, MyCol2, MyCol3) %>%
filter(MyCol1 <= "SomeValue") %>%
mutate(MyCol4 = MyCol1 / MyCol2) %>%
summarize_all(mean)
Also very neat for operating with tensors flowing through a neural net!
16. Before we start: The MLP Architecture
Inputs Weights
Sum
Activation
Function
Bias
Symbol:
Input Layer Hidden Layer Output Layer
• The weights and biases are optimized to best fit the data in a process called backpropagation
• There are a lot of (simple) tricks to prevent overfitting (dropout, learning rate decay, …)
• For a more detailed introduction I recommend the 2 hour talk from Martin Görner @ Google
Cloud Next ‘17
17. But how does it work with a 28x28 picture?
It’s tidy! :)
Let’s code that!
https://cntk.ai/pythondocs/CNTK_103C_MNIST_MultiLayerPerceptron.html
19. What are we missing?
• The spatial correlation of the pixel in their 2D space is lost!
• This information is important! Convolutional Neural Network!
https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#4
