SlideShare a Scribd company logo

Introduction to Chainer 11 may,2018

Published on 11 may, 2018 Chainer is a deep learning framework which is flexible, intuitive, and powerful. This slide introduces some unique features of Chainer and its additional packages such as ChainerMN (distributed learning), ChainerCV (computer vision), ChainerRL (reinforcement learning), Chainer Chemistry (biology and chemistry), and ChainerUI (visualization).

1 of 67
Download to read offline
Last update: 11 May, 2018
Chainer – a deep learning framework
Chainer is a Python framework that lets researchers quickly
implement, train, and evaluate deep learning models.
Designing a network Training, evaluation
Data
set
Written in pure Python and well-documented.
No need to learn a new tensor API since Chainer uses Numpy and CuPy (Numpy-like API)
User-friendly error messages. Easy to debug using pure Python debuggers.
Easy and intuitive to write a network. Supports dynamic graphs.
Chainer features
Fast
☑ CUDA
☑ cuDNN
☑ NCCL
Full
featured
☑ Convolutional Networks
☑ Recurrent Networks
☑ Backprop of backprop
Intuitive
☑ Define-by-Run
☑ High debuggability
Supports GPU acceleration using CUDA with CuPy
High-speed training/inference with cuDNN’s optimized deep learning functions with CuPy
Supports a fast, multi-GPU learning using NCCL with CuPy
N-dimensional Convolution, Deconvolution, Pooling, BN, etc.
RNN components such as LSTM, Bi-directional LSTM, GRU and Bi-directional GRU
Higher order derivatives (a.k.a. gradient of gradient) is supported
Well-abstracted common tools for various NN learning, easy to write a set of learning flows☑ Easy to use APIs
☑ Low learning curve
☑ Maintainable codebase
Basic concept of Define-by-Run approach
Neural network = Computational graph
Define-by-Run: The history of computation applied to the input variable does
represent the network architecture
Basic concept of Define-by-Run approach
Neural network = Computational graph
Define-by-Run: The history of computation applied to the input variable does
represent the network architecture
Basic concept of Define-by-Run approach
Neural network = Computational graph
Define-by-Run: The history of computation applied to the input variable does
represent the network architecture
Basic concept of Define-by-Run approach
Neural network = Computational graph
Define-by-Run: The history of computation applied to the input variable does
represent the network architecture
*
if a > 0
Basic concept of Define-by-Run approach
Neural network = Computational graph
Define-by-Run: The history of computation applied to the input variable does
represent the network architecture
*
if a > 0
Basic concept of Define-by-Run approach
Neural network = Computational graph
Define-by-Run: The history of computation applied to the input variable does
represent the network architecture
*
if a > 0
Static vs Dynamic
“Define-and-run”
Same computational
graph for every
iteration.
Static
graph
framework
The graph is allowed
to change each
iteration.
“Define-by-run”
Dynamic
graph
framework
Define-and-Run and Define-by-Run
# Define the (static) graph
x = Variable(‘x’)
y = Variable(‘y’)
z = x + 2 * y
# Evaluation (z represents a graph)
for xi, yi in data:
eval(z, (xi, yi))
# Build, evaluate at the same time
for xi, yi in data:
x = Variable(xi)
y = Variable(yi)
if y > 0:
z = x + 2 * y
else:
z = x - y
You can conditionally
perform different
forward computations
depending on the data
Define-and-Run Define-by-Run
Easy to optimize, but difficult to change
behavior depending on the data
How to write a convolutional network in Chainer
import chainer
import chainer.links as L
import chainer.functions as F
class LeNet5(chainer.Chain):
def __init__(self):
super(LeNet5, self).__init__()
with self.init_scope():
self.conv1 = L.Convolution2D(1, 6, 5, 1)
self.conv2 = L.Convolution2D(6, 16, 5, 1)
self.conv3 = L.Convolution2D(16, 120, 4, 1)
self.fc4 = L.Linear(None, 84)
self.fc5 = L.Linear(84, 10)
• Start writing a model by inheriting Chain class
• Register parametric layers inside the
init_scope
• Write forward computation in
__call__ method (no need to
write backward computation)
def __call__(self, x):
h = F.sigmoid(self.conv1(x))
h = F.max_pooling_2d(h, 2, 2)
h = F.sigmoid(self.conv2(h))
h = F.max_pooling_2d(h, 2, 2)
h = F.sigmoid(self.conv3(h))
h = F.sigmoid(self.fc4(h))
return self.fc5(h)
Training models
model = LeNet5()
model = L.Classifier(model)
# Dataset is a list! ([] to access, having __len__)
dataset = [(x1, t1), (x2, t2), ...]
# iterator to return a mini-batch retrieved from dataset
it = iterators.SerialIterator(dataset, batchsize=32)
# Optimization methods (you can easily try various methods by changing SGD to
# MomentumSGD, Adam, RMSprop, AdaGrad, etc.)
opt = optimizers.SGD(lr=0.01)
opt.setup(model)
updater = training.StandardUpdater(it, opt, device=0) # device=-1 if you use CPU
trainer = training.Trainer(updater, stop_trigger=(100, 'epoch'))
trainer.run()
For more details, refer to official examples: https://github.com/pfnet/chainer/tree/master/examples
Define-by-Run brings flexibility and intuitiveness
“Forward computation” becomes a definition of network
• Depending on data, it is easy to change a network structure (e.g. conditionally, stochastically,
etc.)
• You can define a network itself by Python code
=The network structure can be treated as a program instead of data.
For Chainer, the “forward computation” can be written in Python
• Enables you to write a network structure freely using the syntax of Python
• Define-by-Run makes it easy to insert any process like putting a print statement between network
computations (In case of define-and-run which compiles a network, this kind of debugging is
difficult)
• Easy to reuse code of the same network for other purposes with few changes (e.g. by just adding
a conditional branch partially)
• Easy to check intermediate values and the design of the network itself using external debugging
tools etc.
CuPy - Drop-in replacement of NumPy for GPU acceleration
Independent library to handle all GPU calculations in Chainer
Lower cost to migrate CPU code to GPU with NumPy-compatible API
GPU-execute linear algebra algorithms such as a singular value decomposition
Rich in examples such as KMeans, Gaussian Mixture Model
import numpy as np
x = np.random.rand(10)
W = np.random.rand(10, 5)
y = np.dot(x, W)
import cupy as cp
x = cp.random.rand(10)
W = cp.random.rand(10, 5)
y = cp.dot(x, W)
GPU
https://github.com/cupy/cupy
● How to write custom kernels: https://docs-cupy.chainer.org/en/latest/tutorial/kernel.html
● Custom kernel example: https://github.com/cupy/cupy/tree/master/examples/gemm
● Example: element-wise kernel to compute: f(x, y) = (x − y)^2
Write you own CUDA kernel with CuPy
Usage (note that broadcasting is supported):
>>> x = cp.arange(10, dtype=np.float32).reshape(2, 5)
>>> squared_diff(x, 5)
array([[ 25., 16., 9., 4., 1.],
[ 0., 1., 4., 9., 16.]], dtype=float32)
Code:
>>> squared_diff = cp.ElementwiseKernel(
... 'float32 x, float32 y',
... 'float32 z',
... 'z = (x - y) * (x - y)',
... 'squared_diff')
Training speed comparison
ilkarman/DeepLearningFrameworks
https://github.com/ilkarman/DeepLearningFrameworks
Chainer is 5th place
at an external benchmark on CIFAR-10 dataset
to compare the training time of VGG-style
network
● Faster than PyTorch, TensorFlow, Keras
(with all backends), CNTK
Note
● Chainer + CuPy has auto-tune function to
select the best convolution algorithm in
cuDNN
● This benchmark runs on NVIDIA K80 GPU
Popularity Growth of Chainer
https://trends.google.com/trends/explore?cat=5&q=chainer,tensorflow,pytorch,mxnet,cntk
Chainer comes with several examples of models such as
● Natural language processing: RNNs, Recursive nets,
Word2Vec, Seq2Seq, etc.
● Image classification: various convnets for MNIST,
CIFAR10, ImageNet
● Generative models: VAE, DCGAN
For details, see
https://github.com/chainer/chainer/tree/master/examples
Additional models such as chainer-gan-lib can be found
on pfnet-research:
https://github.com/pfnet-research/chainer-gan-lib
A large number of external examples are also available:
https://github.com/chainer/chainer/wiki/External-examples
Chainer examples
pomegranate (probabilistic models):
https://github.com/jmschrei/pomegranate
PyINN (fused PyTorch ops in CuPy):
https://github.com/szagoruyko/pyinn
Vanilla LSTM with CuPy (no Chainer, easily and quickly ported from NumPy):
https://github.com/mitmul/chainer-notebooks/blob/master/9_vanilla-LSTM-with-cupy.ipynb
QRNN for PyTorch (uses CuPy for custom kernel)
https://github.com/salesforce/pytorch-qrnn
External projects/examples using CuPy
Add-on packages for Chainer
Distributed deep learning, deep reinforcement learning, computer vision
ChainerMN (Multi-Node): additional package for distributed deep learning
  High scalability (100 times faster with 128GPU)
ChainerRL: deep reinforcement learning library
  DQN, DDPG, A3C, ACER, NSQ, PCL, etc. OpenAI Gym support
ChainerCV: provides image recognition algorithms, dataset wrappers
  Faster R-CNN, Single Shot Multibox Detector (SSD), SegNet, etc.
ChainerMN: Multi-Node training
Keeping the easy-to-use characteristics of Chainer as is,
ChainerMN enables to use multiple nodes which have multiple
GPUs easily to make training faster
GPU
GPU
InfiniBand
GPU
GPU
InfiniBand
MPI
NVIDIA NCCL
Distributed deep learning with ChainerMN
100x speed up with 128 Maxwell GPUs
Comparison with other frameworks
ChainerMN is the fastest at the comparison of elapsed time to train ResNet-50 on
ImageNet dataset for 100 epochs (May 2017)
Recently we achieved
15 mins to train ResNet50 on
ImageNet dataset with 8 times
larger cluster (1024 GPUs
over 128 nodes)
See the details in this paper:
“Extremely Large Minibatch SGD:
Training ResNet-50 on ImageNet in 15
Minutes”
https://arxiv.org/abs/1711.04325
We confirmed that if we increase the number of nodes,
the almost same accuracy can be achieved
Speedup without dropping the accuracy
Scale-out test on Microsoft Azure
Easy-to-use API of ChainerMN
You can start using ChainerMN just by wrapping one line!
optimizer = chainer.optimizers.MomentumSGD()
optimizer = chainermn.DistributedOptimizer(
chainer.optimizers.MomentumSGD())
Add-on packages for Chainer
Distribute deep learning, deep reinforcement learning, computer vision
ChainerMN (Multi-Node): additional package for distributed deep learning
  High scalability (100 times faster with 128GPU)
ChainerRL: deep reinforcement learning library
  DQN, DDPG, A3C, ACER, NSQ, PCL, etc. OpenAI Gym support
ChainerCV: provides image recognition algorithms, dataset wrappers
  Faster R-CNN, Single Shot Multibox Detector (SSD), SegNet, etc.
Reinforcement Learning:
ChainerRL: Deep Reinforcement Learning Library
Train an agent which interacts with the environment to maximize
the rewards
Action
Env
Observation, Reward
Reinforcement Learning with ChainerRL
1. Create an environment
Action
Env
Observation, Reward
Distribution: Softmax, Mellowmax, Gaussian,…
Policy: Observation → Distribution of actions
2. Define an agent model
Reinforcement Learning with ChainerRL
2. Define an agent model (contd.)
Q-Function: Observation → Value of each action (expectation of the sum of future rewards)
ActionValue: Discrete, Quadratic
Reinforcement Learning with ChainerRL
Action
Env
Observation, Reward
3. Create an agent
Reinforcement Learning with ChainerRL
4. Interact with the environment!
Reinforcement Learning with ChainerRL
Algorithms provided by ChainerRL
• Deep Q-Network (Mnih et al., 2015)
• Double DQN (Hasselt et al., 2016)
• Normalized Advantage Function (Gu et al., 2016)
• (Persistent) Advantage Learning (Bellemare et al., 2016)
• Deep Deterministic Policy Gradient (Lillicrap et al., 2016)
• SVG(0) (Heese et al., 2015)
• Asynchronous Advantage Actor-Critic (Mnih et al., 2016)
• Asynchronous N-step Q-learning (Mnih et al., 2016)
• Actor-Critic with Experience Replay (Wang et al., 2017) <- NEW!
• Path Consistency Learning (Nachum et al., 2017) <- NEW!
• etc.
ChainerRL Quickstart Guide
• Define a Q-function in a Jupyter notebook and learn the Cart Pole
Balancing problem with DQN
https://github.com/chainer/chainerrl/blob/master/examples/quickstart/quickstart.ipynb
Add-on packages for Chainer
Distribute deep learning, deep reinforcement learning, computer vision
ChainerMN (Multi-Node): additional package for distributed deep learning
  High scalability (100 times faster with 128GPU)
ChainerRL: deep reinforcement learning library
  DQN, DDPG, A3C, ACER, NSQ, PCL, etc. OpenAI Gym support
ChainerCV: provides image recognition algorithms, dataset wrappers
  Faster R-CNN, Single Shot Multibox Detector (SSD), SegNet, etc.
Evaluate your
model on
popular
datasets
Running and training deep-learning models easier for Computer Vision tasks
ChainerCV https://github.com/chainer/chainercv
Datasets
Pascal VOC,
Caltech-UCSD
Birds-200-2011,
Stanford Online
Products, CamVid, etc.
Models
Faster R-CNN, SSD,
SegNet (will add more
models!)
Training
tools
Evaluation
tools
Dataset
Abstraction
Train popular
models with
your data
Start computer vision research using deep learning much easier
ChainerCV
Latest algorithms with your data
Provide complete model code, training code, inference code for
segmentation algorithms (SegNet, etc.) and object detection algorithms
(Faster R-CNN, SSD, etc.), and so on
All code is confirmed to reproduce the results
All training code and model code reproduced the experimental results
shown in the original paper
• If you want to see some
examples of ChainerCV
and the reproducing code
for some papers, please
check the official Github
repository
(chainer/chainercv)
• The right figure shows the
result of the inference code
of Faster RCNN example
• The pre-trained weights
are automatically
downloaded!
https://github.com/chainer/chainercv
Install: $ pip install chainercv
•
•
→
←
•
•
•
ChainerUI
ChainerUI is a visualization and
experiment management tool for
Chainer
Install:
$ pip install chainerui
● You can compare the effect of
different hyperparameters visually
by loss curves and the scores in the
table
● You can change the learning rate
dynamically during training
See the details in:
https://github.com/chainer/chainerui
Chainer Chemistry
Chainer Chemisry is a collection of tools for
training neural networks on biology and
chemistry tasks using Chainer
Install:
$ pip install chainer-chemistry
● It currently provides widely used graph
convolution implementation:
○ NFP: Neural fingerprint
○ GGNN: Gated Graph Neural
Network
○ Weave
○ SchNet
See the details in:
https://github.com/pfnet-research/chainer-chemistry
Summary
• Chainer is a deep learning framework focusing on flexibility
• There are 5 additional packages for specific task/domain:
• ChainerMN - Distributed learning
• ChainerRL - Reinforcement learning
• ChainerCV - Computer vision
• ChainerUI - Visualization and experiment management
• Chainer Chemistry - Graph convolutions for biology/chemistry tasks
Intel Chainer
Intel Chainer with MKL-DNN Backend
CPU
CuPy
NVIDIA GPU
CUDA
cuDNN
BLAS
NumPy
Chainer
MKL-DNN
Intel Xeon/Xeon Phi
MKL
Intel Chainer with MKL-DNN Backend
MKL-DNN
• Neural Network library optimized for Intel architectures
• Supported CPUs:
✓ Intel Atom(R) processor with Intel(R) SSE4.1 support
✓ 4th, 5th, 6th and 7th generation Intel(R) Core processor
✓ Intel(R) Xeon(R) processor E5 v3 family (code named Haswell)
✓ Intel(R) Xeon(R) processor E5 v4 family (code named Broadwell)
✓ Intel(R) Xeon(R) Platinum processor family (code name Skylake)
✓ Intel(R) Xeon Phi(TM) product family x200 (code named Knights Landing)
✓ Future Intel(R) Xeon Phi(TM) processor (code named Knights Mill)
• MKL-DNN accelerates the computation of NN on the above CPUs
Intel Chainer with MKL-DNN Backend
convnet-benchmarks* result:
Intel Chainer Chainer with NumPy (MKL-Build)
Alexnet Forward 429.16 ms 5041.91 ms
Alexnet Backward 841.73 ms 5569.49 ms
Alexnet Total 1270.89 ms 10611.40 ms
~8.35x faster than NumPy backend!
Intel Chainer with MKL-DNN Backend
Intel is developing Intel Chainer as a fork of Chainer v2
https://github.com/intel/chainer
Applications using Chainer
Object Detection
https://www.youtube.com/watch?v=yNc5N1MOOt4
Semantic Segmentation
https://www.youtube.com/watch?v=lGOjchGdVQs
Ponanza Chainer
● Won the 2nd
place at The 27th
World Computer Shogi Championship
● Based on Ponanza which was the champion for two years in a row (2015, 2016)
● “Ponanza Chainer” applied Deep Learning for ordering the possible next moves for which
“Ponanza” should think ahead deeply
● “Ponanza Chainer” wins “Ponanza” with a probability of 80%
Team
PFN
Issei
Yamamoto
Akira
Shimoyama
Team
Ponanza
Paints Chainer
● Auto Sketch Colorization
● Train a neural network with
a large dataset of paintings
● It takes a line drawings as
input, and output a
colorized image!
● You can also give color hits
which indicates preferable
colors
https://paintschainer.preferred.tech
Installation of Chainer
1. Install CUDA Toolkit 8.0
https://developer.nvidia.com/cuda-downloads
2. Install cuDNN v6.0 Library
https://developer.nvidia.com/rdp/cudnn-download
3. Install NCCL for Multi-GPUs
https://github.com/NVIDIA/nccl
4. Install CuPy and Chainer
% pip install cupy
% pip install chainer
Chainer on Ubuntu
For more details, see the official installation guide:
http://docs.chainer.org/en/stable/install.html
Chainer on Windows with NVIDIA GPU
1. Install Visual C++ 2015 Build Tools
http://landinghub.visualstudio.com/visual-cpp-build-tools
2. Install CUDA Toolkit 8.0
https://developer.nvidia.com/cuda-downloads
3. Install cuDNN v6.0 Library for Windows 10
https://developer.nvidia.com/rdp/cudnn-download
Put all files under C:Program FilesNVIDIA GPU Computing ToolkitCUDAv8.0
4. Install Anaconda 4.3.1 Python 3.6 or 2.7
https://www.continuum.io/downloads
5. Add environmental variables
- Add “C:Program Files (x86)Microsoft Visual Studio 14.0VCbin” to PATH variable
- Add “C:Program Files (x86)Windows Kits10Include10.0.10240.0ucrt” to INCLUDE variable
6. Install Chainer on Anaconda Prompt
> pip install cupy
> pip install chainer
Note: If Visual C++ 2017 was previously installed, some additional steps might be required. In that case, please
visit our support channels for help.
Chainer on Azure
Use Data Science Virtual Machine for Linux (Ubuntu)
• Ready for CUDA & cuDNN
• Chainer, CuPy, ChainerCV, and ChainerRL are pre-installed!
1
2
3
• Easy model export to ONNX format
ONNX-Chainer
https://github.com/chainer/onnx-chainer
Chainer Model Export
tfchain: TensorFlow export (experimental)
Caffe-export
• https://github.com/mitmul/tfchain
• Supports Linear, Convolution2D, MaxPooling2D, ReLU
• Just add @totf decorator right before the forward method of the model
import chainer
import chainer.functions as F
import chainer.links as L
import numpy as np
from chainer.exporters import caffe
model = chainer.Sequential(
L.Linear(None, 10),
F.relu,
L.Linear(10, 10)
F.relu,
L.Linear(10, 10)
)
x = chainer.Variable(
x.random.rand(1, 10).astype('f'))
caffe.export(model, [x], './, True)
External Projects for Model Portability
DLPack
• https://mil-tokyo.github.io/webdnn/
• The model conversion to run it on a web browser supports Chainer
WebDNN
• https://github.com/dmlc/dlpa
ck
• MXNet, Torch, Caffe2 have
joined to discuss the
guideline of memory layout
of tensor and the common
operator interfaces
Companies supporting Chainer
The Chainer project is now supported by
these Leading computing companies
Contributing to Chainer
Chainer is an open-source project.
• You can send a PR from here: https://github.com/chainer/chainer
• The development speed of Deep Learning research is super fast, therefore,
to provide the state-of-the-art technologies through Chainer, we
continuously update the development plans:
• Chainer v3.0.0 will be released on 26th
September!
• Will support gradient of gradient (higher order differentiation)
• Will add the official Windows support ensured by Microsoft
The release schedule after
v2.0.1 (4th
July)→
● Started as a fork of Chainer. API is
similar to Chainer.
● Codebase is written in a combination of
Python, C, C++ -> More complicated
codebase than Chainer but can also be
faster in some cases.
● Uses familiar Torch tensor API. Easy
for torch users but requires extra
learning curve for NumPy users.
Conversion functions sometimes
needed.
Comparison of Chainer with PyTorch
● Codebase is written entirely in Python
and therefore accessible to
researchers who are not familiar with
C/C++ -> Keep codebase in pure
Python if possible. Value an easily
understandable and maintainable
codebase over extreme performance in
all cases.
● Uses familiar Numpy API: Numpy
(CPU) or CuPy (GPU).
● Easy to write custom GPU kernels in
Chainer using CuPy.
● CuPy makes it easy for researchers to
write custom CUDA kernels.
Chainer PyTorch
So, the differences are mostly in the array libraries and
different philosophies on how to achieve the best balance
between performance optimizations and maintainability
and flexibility of the core codebase.

Recommended

“Introduction to the TVM Open Source Deep Learning Compiler Stack,” a Present...
“Introduction to the TVM Open Source Deep Learning Compiler Stack,” a Present...“Introduction to the TVM Open Source Deep Learning Compiler Stack,” a Present...
“Introduction to the TVM Open Source Deep Learning Compiler Stack,” a Present...Edge AI and Vision Alliance
 
[MIRU2018] Global Average Poolingの特性を用いたAttention Branch Network
[MIRU2018] Global Average Poolingの特性を用いたAttention Branch Network[MIRU2018] Global Average Poolingの特性を用いたAttention Branch Network
[MIRU2018] Global Average Poolingの特性を用いたAttention Branch NetworkHiroshi Fukui
 
PFP:材料探索のための汎用Neural Network Potential_中郷_20220422POLセミナー
PFP:材料探索のための汎用Neural Network Potential_中郷_20220422POLセミナーPFP:材料探索のための汎用Neural Network Potential_中郷_20220422POLセミナー
PFP:材料探索のための汎用Neural Network Potential_中郷_20220422POLセミナーMatlantis
 
Introduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningIntroduction of Deep Reinforcement Learning
Introduction of Deep Reinforcement LearningNAVER Engineering
 
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習台灣資料科學年會
 
Kaggle RSNA Pneumonia Detection Challenge 解法紹介
Kaggle RSNA Pneumonia Detection Challenge 解法紹介Kaggle RSNA Pneumonia Detection Challenge 解法紹介
Kaggle RSNA Pneumonia Detection Challenge 解法紹介理 秋山
 
論文紹介: Fast R-CNN&Faster R-CNN
論文紹介: Fast R-CNN&Faster R-CNN論文紹介: Fast R-CNN&Faster R-CNN
論文紹介: Fast R-CNN&Faster R-CNNTakashi Abe
 

More Related Content

What's hot

Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkKnoldus Inc.
 
You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話
You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話
You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話Yusuke Uchida
 
Computer Vision for Beginners
Computer Vision for BeginnersComputer Vision for Beginners
Computer Vision for BeginnersSanghamitra Deb
 
How to use Apache TVM to optimize your ML models
How to use Apache TVM to optimize your ML modelsHow to use Apache TVM to optimize your ML models
How to use Apache TVM to optimize your ML modelsDatabricks
 
Forward-Forward Algorithm
Forward-Forward AlgorithmForward-Forward Algorithm
Forward-Forward AlgorithmDong Heon Cho
 
論文紹介 "DARTS: Differentiable Architecture Search"
論文紹介 "DARTS: Differentiable Architecture Search"論文紹介 "DARTS: Differentiable Architecture Search"
論文紹介 "DARTS: Differentiable Architecture Search"Yuta Koreeda
 
Decision Transformer: Reinforcement Learning via Sequence Modeling
Decision Transformer: Reinforcement Learning via Sequence ModelingDecision Transformer: Reinforcement Learning via Sequence Modeling
Decision Transformer: Reinforcement Learning via Sequence ModelingYasunori Ozaki
 
adversarial training.pptx
adversarial training.pptxadversarial training.pptx
adversarial training.pptxssuserc45ddf
 
introduction to Dueling network
introduction to Dueling networkintroduction to Dueling network
introduction to Dueling networkWEBFARMER. ltd.
 
ディープラーニングの2値化(Binarized Neural Network)
ディープラーニングの2値化(Binarized Neural Network)ディープラーニングの2値化(Binarized Neural Network)
ディープラーニングの2値化(Binarized Neural Network)Hideo Terada
 
効率的学習 / Efficient Training(メタサーベイ)
効率的学習 / Efficient Training(メタサーベイ)効率的学習 / Efficient Training(メタサーベイ)
効率的学習 / Efficient Training(メタサーベイ)cvpaper. challenge
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network Yan Xu
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsBill Liu
 
Action Recognitionの歴史と最新動向
Action Recognitionの歴史と最新動向Action Recognitionの歴史と最新動向
Action Recognitionの歴史と最新動向Ohnishi Katsunori
 
[DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima...
 [DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima... [DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima...
[DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima...Deep Learning JP
 
論文紹介:Dueling network architectures for deep reinforcement learning
論文紹介:Dueling network architectures for deep reinforcement learning論文紹介:Dueling network architectures for deep reinforcement learning
論文紹介:Dueling network architectures for deep reinforcement learningKazuki Adachi
 
2020 08 05_dl_DETR
2020 08 05_dl_DETR2020 08 05_dl_DETR
2020 08 05_dl_DETRharmonylab
 
Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkYan Xu
 

What's hot (20)

Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural Network
 
You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話
You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話
You Only Look One-level Featureの解説と見せかけた物体検出のよもやま話
 
Computer Vision for Beginners
Computer Vision for BeginnersComputer Vision for Beginners
Computer Vision for Beginners
 
How to use Apache TVM to optimize your ML models
How to use Apache TVM to optimize your ML modelsHow to use Apache TVM to optimize your ML models
How to use Apache TVM to optimize your ML models
 
Forward-Forward Algorithm
Forward-Forward AlgorithmForward-Forward Algorithm
Forward-Forward Algorithm
 
論文紹介 "DARTS: Differentiable Architecture Search"
論文紹介 "DARTS: Differentiable Architecture Search"論文紹介 "DARTS: Differentiable Architecture Search"
論文紹介 "DARTS: Differentiable Architecture Search"
 
Decision Transformer: Reinforcement Learning via Sequence Modeling
Decision Transformer: Reinforcement Learning via Sequence ModelingDecision Transformer: Reinforcement Learning via Sequence Modeling
Decision Transformer: Reinforcement Learning via Sequence Modeling
 
adversarial training.pptx
adversarial training.pptxadversarial training.pptx
adversarial training.pptx
 
introduction to Dueling network
introduction to Dueling networkintroduction to Dueling network
introduction to Dueling network
 
ディープラーニングの2値化(Binarized Neural Network)
ディープラーニングの2値化(Binarized Neural Network)ディープラーニングの2値化(Binarized Neural Network)
ディープラーニングの2値化(Binarized Neural Network)
 
NumPy闇入門
NumPy闇入門NumPy闇入門
NumPy闇入門
 
効率的学習 / Efficient Training(メタサーベイ)
効率的学習 / Efficient Training(メタサーベイ)効率的学習 / Efficient Training(メタサーベイ)
効率的学習 / Efficient Training(メタサーベイ)
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
 
CuPy v4 and v5 roadmap
CuPy v4 and v5 roadmapCuPy v4 and v5 roadmap
CuPy v4 and v5 roadmap
 
Deep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its ApplicationsDeep Reinforcement Learning and Its Applications
Deep Reinforcement Learning and Its Applications
 
Action Recognitionの歴史と最新動向
Action Recognitionの歴史と最新動向Action Recognitionの歴史と最新動向
Action Recognitionの歴史と最新動向
 
[DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima...
 [DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima... [DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima...
[DL輪読会]StarGAN: Unified Generative Adversarial Networks for Multi-Domain Ima...
 
論文紹介:Dueling network architectures for deep reinforcement learning
論文紹介:Dueling network architectures for deep reinforcement learning論文紹介:Dueling network architectures for deep reinforcement learning
論文紹介:Dueling network architectures for deep reinforcement learning
 
2020 08 05_dl_DETR
2020 08 05_dl_DETR2020 08 05_dl_DETR
2020 08 05_dl_DETR
 
Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural Network
 

Similar to Introduction to Chainer 11 may,2018

Overview of Chainer and Its Features
Overview of Chainer and Its FeaturesOverview of Chainer and Its Features
Overview of Chainer and Its FeaturesSeiya Tokui
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to ChainerShunta Saito
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to ChainerSeiya Tokui
 
CuPy: A NumPy-compatible Library for GPU
CuPy: A NumPy-compatible Library for GPUCuPy: A NumPy-compatible Library for GPU
CuPy: A NumPy-compatible Library for GPUShohei Hido
 
Distributed Multi-GPU Computing with Dask, CuPy and RAPIDS
Distributed Multi-GPU Computing with Dask, CuPy and RAPIDSDistributed Multi-GPU Computing with Dask, CuPy and RAPIDS
Distributed Multi-GPU Computing with Dask, CuPy and RAPIDSPeterAndreasEntschev
 
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...Universitat Politècnica de Catalunya
 
Viktor Tsykunov: Azure Machine Learning Service
Viktor Tsykunov: Azure Machine Learning ServiceViktor Tsykunov: Azure Machine Learning Service
Viktor Tsykunov: Azure Machine Learning ServiceLviv Startup Club
 
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdfS51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdfDLow6
 
Profiling deep learning network using NVIDIA nsight systems
Profiling deep learning network using NVIDIA nsight systemsProfiling deep learning network using NVIDIA nsight systems
Profiling deep learning network using NVIDIA nsight systemsJack (Jaegeun) Han
 
GPU and Deep learning best practices
GPU and Deep learning best practicesGPU and Deep learning best practices
GPU and Deep learning best practicesLior Sidi
 
Tutorial: Image Generation and Image-to-Image Translation using GAN
Tutorial: Image Generation and Image-to-Image Translation using GANTutorial: Image Generation and Image-to-Image Translation using GAN
Tutorial: Image Generation and Image-to-Image Translation using GANWuhyun Rico Shin
 
Azure machine learning service
Azure machine learning serviceAzure machine learning service
Azure machine learning serviceRuth Yakubu
 
Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117Ganesan Narayanasamy
 
NVIDIA HPC ソフトウエア斜め読み
NVIDIA HPC ソフトウエア斜め読みNVIDIA HPC ソフトウエア斜め読み
NVIDIA HPC ソフトウエア斜め読みNVIDIA Japan
 
Euro python2011 High Performance Python
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance PythonIan Ozsvald
 
Introduction to GPUs for Machine Learning
Introduction to GPUs for Machine LearningIntroduction to GPUs for Machine Learning
Introduction to GPUs for Machine LearningSri Ambati
 
Machine-Learning-based Performance Heuristics for Runtime CPU/GPU Selection
Machine-Learning-based Performance Heuristics for Runtime CPU/GPU SelectionMachine-Learning-based Performance Heuristics for Runtime CPU/GPU Selection
Machine-Learning-based Performance Heuristics for Runtime CPU/GPU SelectionAkihiro Hayashi
 
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...Masashi Shibata
 

Similar to Introduction to Chainer 11 may,2018 (20)

Overview of Chainer and Its Features
Overview of Chainer and Its FeaturesOverview of Chainer and Its Features
Overview of Chainer and Its Features
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
CuPy: A NumPy-compatible Library for GPU
CuPy: A NumPy-compatible Library for GPUCuPy: A NumPy-compatible Library for GPU
CuPy: A NumPy-compatible Library for GPU
 
Deep Learning for Computer Vision: Software Frameworks (UPC 2016)
Deep Learning for Computer Vision: Software Frameworks (UPC 2016)Deep Learning for Computer Vision: Software Frameworks (UPC 2016)
Deep Learning for Computer Vision: Software Frameworks (UPC 2016)
 
Distributed Multi-GPU Computing with Dask, CuPy and RAPIDS
Distributed Multi-GPU Computing with Dask, CuPy and RAPIDSDistributed Multi-GPU Computing with Dask, CuPy and RAPIDS
Distributed Multi-GPU Computing with Dask, CuPy and RAPIDS
 
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
 
Viktor Tsykunov: Azure Machine Learning Service
Viktor Tsykunov: Azure Machine Learning ServiceViktor Tsykunov: Azure Machine Learning Service
Viktor Tsykunov: Azure Machine Learning Service
 
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdfS51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
S51281 - Accelerate Data Science in Python with RAPIDS_1679330128290001YmT7.pdf
 
Profiling deep learning network using NVIDIA nsight systems
Profiling deep learning network using NVIDIA nsight systemsProfiling deep learning network using NVIDIA nsight systems
Profiling deep learning network using NVIDIA nsight systems
 
GPU and Deep learning best practices
GPU and Deep learning best practicesGPU and Deep learning best practices
GPU and Deep learning best practices
 
Tutorial: Image Generation and Image-to-Image Translation using GAN
Tutorial: Image Generation and Image-to-Image Translation using GANTutorial: Image Generation and Image-to-Image Translation using GAN
Tutorial: Image Generation and Image-to-Image Translation using GAN
 
Azure machine learning service
Azure machine learning serviceAzure machine learning service
Azure machine learning service
 
Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117
 
NVIDIA HPC ソフトウエア斜め読み
NVIDIA HPC ソフトウエア斜め読みNVIDIA HPC ソフトウエア斜め読み
NVIDIA HPC ソフトウエア斜め読み
 
Euro python2011 High Performance Python
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance Python
 
Introduction to GPUs for Machine Learning
Introduction to GPUs for Machine LearningIntroduction to GPUs for Machine Learning
Introduction to GPUs for Machine Learning
 
Machine-Learning-based Performance Heuristics for Runtime CPU/GPU Selection
Machine-Learning-based Performance Heuristics for Runtime CPU/GPU SelectionMachine-Learning-based Performance Heuristics for Runtime CPU/GPU Selection
Machine-Learning-based Performance Heuristics for Runtime CPU/GPU Selection
 
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
MLOps Case Studies: Building fast, scalable, and high-accuracy ML systems at ...
 

More from Preferred Networks

Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3
Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3
Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3Preferred Networks
 
深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...
深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...
深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...Preferred Networks
 
Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55
Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55
Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55Preferred Networks
 
Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2
Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2
Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2Preferred Networks
 
最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2
最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2
最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2Preferred Networks
 
Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2
Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2
Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2Preferred Networks
 
スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演
スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演
スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演Preferred Networks
 
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)Preferred Networks
 
PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)
PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)
PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)Preferred Networks
 
自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)
自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)
自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)Preferred Networks
 
Kubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語る
Kubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語るKubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語る
Kubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語るPreferred Networks
 
Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張
Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張
Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張Preferred Networks
 
続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2
続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2
続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2Preferred Networks
 
Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...
Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...
Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...Preferred Networks
 
KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...
KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...
KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...Preferred Networks
 
KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...
KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...
KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...Preferred Networks
 
Topology Managerについて / Kubernetes Meetup Tokyo 50
Topology Managerについて / Kubernetes Meetup Tokyo 50Topology Managerについて / Kubernetes Meetup Tokyo 50
Topology Managerについて / Kubernetes Meetup Tokyo 50Preferred Networks
 
PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...
PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...
PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...Preferred Networks
 
PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜
PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜
PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜Preferred Networks
 
わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48
わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48
わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48Preferred Networks
 

More from Preferred Networks (20)

Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3
Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3
Optunaを使ったHuman-in-the-loop最適化の紹介 - 2023/04/27 W&B 東京ミートアップ #3
 
深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...
深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...
深層学習の新しい応用と、 それを支える計算機の進化 - Preferred Networks CEO 西川徹 (SEMICON Japan 2022 Ke...
 
Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55
Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55
Kubernetes ControllerをScale-Outさせる方法 / Kubernetes Meetup Tokyo #55
 
Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2
Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2
Kaggle Happywhaleコンペ優勝解法でのOptuna使用事例 - 2022/12/10 Optuna Meetup #2
 
最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2
最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2
最新リリース:Optuna V3の全て - 2022/12/10 Optuna Meetup #2
 
Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2
Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2
Optuna Dashboardの紹介と設計解説 - 2022/12/10 Optuna Meetup #2
 
スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演
スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演
スタートアップが提案する2030年の材料開発 - 2022/11/11 QPARC講演
 
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用(2022/10/19東大大学院「 融合情報学特別講義Ⅲ」)
 
PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)
PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)
PFNにおける研究開発(2022/10/19 東大大学院「融合情報学特別講義Ⅲ」)
 
自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)
自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)
自然言語処理を 役立てるのはなぜ難しいのか(2022/10/25東大大学院「自然言語処理応用」)
 
Kubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語る
Kubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語るKubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語る
Kubernetes にこれから入るかもしれない注目機能!(2022年11月版) / TechFeed Experts Night #7 〜 コンテナ技術を語る
 
Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張
Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張
Matlantis™のニューラルネットワークポテンシャルPFPの適用範囲拡張
 
続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2
続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2
続・PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜 #2
 
Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...
Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...
Kubernetes Service Account As Multi-Cloud Identity / Cloud Native Security Co...
 
KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...
KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...
KubeCon + CloudNativeCon Europe 2022 Recap / Kubernetes Meetup Tokyo #51 / #k...
 
KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...
KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...
KubeCon + CloudNativeCon Europe 2022 Recap - Batch/HPCの潮流とScheduler拡張事例 / Kub...
 
Topology Managerについて / Kubernetes Meetup Tokyo 50
Topology Managerについて / Kubernetes Meetup Tokyo 50Topology Managerについて / Kubernetes Meetup Tokyo 50
Topology Managerについて / Kubernetes Meetup Tokyo 50
 
PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...
PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...
PFN Summer Internship 2021 / Kohei Shinohara: Charge Transfer Modeling in Neu...
 
PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜
PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜
PFN のオンプレML基盤の取り組み / オンプレML基盤 on Kubernetes 〜PFN、ヤフー〜
 
わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48
わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48
わかる!metadata.managedFields / Kubernetes Meetup Tokyo 48
 

Recently uploaded

How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHubHow We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHubShapeBlue
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueShapeBlue
 
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsLow Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsScyllaDB
 
Achieving Excellence IESVE for HVAC Simulation.pdf
Achieving Excellence IESVE for HVAC Simulation.pdfAchieving Excellence IESVE for HVAC Simulation.pdf
Achieving Excellence IESVE for HVAC Simulation.pdfIES VE
 
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...Cprime
 
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider LectureMuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider LectureManik S Magar
 
Trading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdfTrading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdfLucas Lagone
 
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...ShapeBlue
 
AI-Plugins-Planners-Persona-SemanticKernel.pptx
AI-Plugins-Planners-Persona-SemanticKernel.pptxAI-Plugins-Planners-Persona-SemanticKernel.pptx
AI-Plugins-Planners-Persona-SemanticKernel.pptxUdaiappa Ramachandran
 
Artificial Intelligence - AI For Everyone
Artificial Intelligence - AI For EveryoneArtificial Intelligence - AI For Everyone
Artificial Intelligence - AI For EveryoneSridhar Seshadri
 
Key projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AIKey projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AIVijayananda Mohire
 
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...2toLead Limited
 
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...Chris Bingham
 
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...James Anderson
 
CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...
CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...
CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...ShapeBlue
 
software-quality-assurance question paper 2023
software-quality-assurance question paper 2023software-quality-assurance question paper 2023
software-quality-assurance question paper 2023RohanMistry15
 
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUGBoosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUGRick Ossendrijver
 
eXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductionseXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductionsElanthirayan Madhavan
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerSaiLinnThu2
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationAppsthentic Technology
 

Recently uploaded (20)

How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHubHow We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
 
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsLow Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & Pitfalls
 
Achieving Excellence IESVE for HVAC Simulation.pdf
Achieving Excellence IESVE for HVAC Simulation.pdfAchieving Excellence IESVE for HVAC Simulation.pdf
Achieving Excellence IESVE for HVAC Simulation.pdf
 
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
 
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider LectureMuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
MuleSoft Online Meetup Group - B2B Crash Course: PM Insider Lecture
 
Trading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdfTrading Software Development_ Trends to Watch in 2024.pdf
Trading Software Development_ Trends to Watch in 2024.pdf
 
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
 
AI-Plugins-Planners-Persona-SemanticKernel.pptx
AI-Plugins-Planners-Persona-SemanticKernel.pptxAI-Plugins-Planners-Persona-SemanticKernel.pptx
AI-Plugins-Planners-Persona-SemanticKernel.pptx
 
Artificial Intelligence - AI For Everyone
Artificial Intelligence - AI For EveryoneArtificial Intelligence - AI For Everyone
Artificial Intelligence - AI For Everyone
 
Key projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AIKey projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AI
 
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
 
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
Learning About GenAI Engineering with AWS PartyRock [AWS User Group Basel - F...
 
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
GDG Cloud Southlake 30 Brian Demers Breeding 10x Developers with Developer Pr...
 
CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...
CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...
CloudStack 101: The Best Way to Build Your Private Cloud – Rohit Yadav, VP Ap...
 
software-quality-assurance question paper 2023
software-quality-assurance question paper 2023software-quality-assurance question paper 2023
software-quality-assurance question paper 2023
 
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUGBoosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
Boosting Developer Effectiveness with a Java platform team 1.4 - ArnhemJUG
 
eXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductionseXtended Reality(XR) Basic introductions
eXtended Reality(XR) Basic introductions
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and Optimization
 

Introduction to Chainer 11 may,2018

  • 1. Last update: 11 May, 2018
  • 2. Chainer – a deep learning framework Chainer is a Python framework that lets researchers quickly implement, train, and evaluate deep learning models. Designing a network Training, evaluation Data set
  • 3. Written in pure Python and well-documented. No need to learn a new tensor API since Chainer uses Numpy and CuPy (Numpy-like API) User-friendly error messages. Easy to debug using pure Python debuggers. Easy and intuitive to write a network. Supports dynamic graphs. Chainer features Fast ☑ CUDA ☑ cuDNN ☑ NCCL Full featured ☑ Convolutional Networks ☑ Recurrent Networks ☑ Backprop of backprop Intuitive ☑ Define-by-Run ☑ High debuggability Supports GPU acceleration using CUDA with CuPy High-speed training/inference with cuDNN’s optimized deep learning functions with CuPy Supports a fast, multi-GPU learning using NCCL with CuPy N-dimensional Convolution, Deconvolution, Pooling, BN, etc. RNN components such as LSTM, Bi-directional LSTM, GRU and Bi-directional GRU Higher order derivatives (a.k.a. gradient of gradient) is supported Well-abstracted common tools for various NN learning, easy to write a set of learning flows☑ Easy to use APIs ☑ Low learning curve ☑ Maintainable codebase
  • 4. Basic concept of Define-by-Run approach Neural network = Computational graph Define-by-Run: The history of computation applied to the input variable does represent the network architecture
  • 5. Basic concept of Define-by-Run approach Neural network = Computational graph Define-by-Run: The history of computation applied to the input variable does represent the network architecture
  • 6. Basic concept of Define-by-Run approach Neural network = Computational graph Define-by-Run: The history of computation applied to the input variable does represent the network architecture
  • 7. Basic concept of Define-by-Run approach Neural network = Computational graph Define-by-Run: The history of computation applied to the input variable does represent the network architecture * if a > 0
  • 8. Basic concept of Define-by-Run approach Neural network = Computational graph Define-by-Run: The history of computation applied to the input variable does represent the network architecture * if a > 0
  • 9. Basic concept of Define-by-Run approach Neural network = Computational graph Define-by-Run: The history of computation applied to the input variable does represent the network architecture * if a > 0
  • 10. Static vs Dynamic “Define-and-run” Same computational graph for every iteration. Static graph framework The graph is allowed to change each iteration. “Define-by-run” Dynamic graph framework
  • 11. Define-and-Run and Define-by-Run # Define the (static) graph x = Variable(‘x’) y = Variable(‘y’) z = x + 2 * y # Evaluation (z represents a graph) for xi, yi in data: eval(z, (xi, yi)) # Build, evaluate at the same time for xi, yi in data: x = Variable(xi) y = Variable(yi) if y > 0: z = x + 2 * y else: z = x - y You can conditionally perform different forward computations depending on the data Define-and-Run Define-by-Run Easy to optimize, but difficult to change behavior depending on the data
  • 12. How to write a convolutional network in Chainer import chainer import chainer.links as L import chainer.functions as F class LeNet5(chainer.Chain): def __init__(self): super(LeNet5, self).__init__() with self.init_scope(): self.conv1 = L.Convolution2D(1, 6, 5, 1) self.conv2 = L.Convolution2D(6, 16, 5, 1) self.conv3 = L.Convolution2D(16, 120, 4, 1) self.fc4 = L.Linear(None, 84) self.fc5 = L.Linear(84, 10) • Start writing a model by inheriting Chain class • Register parametric layers inside the init_scope • Write forward computation in __call__ method (no need to write backward computation) def __call__(self, x): h = F.sigmoid(self.conv1(x)) h = F.max_pooling_2d(h, 2, 2) h = F.sigmoid(self.conv2(h)) h = F.max_pooling_2d(h, 2, 2) h = F.sigmoid(self.conv3(h)) h = F.sigmoid(self.fc4(h)) return self.fc5(h)
  • 13. Training models model = LeNet5() model = L.Classifier(model) # Dataset is a list! ([] to access, having __len__) dataset = [(x1, t1), (x2, t2), ...] # iterator to return a mini-batch retrieved from dataset it = iterators.SerialIterator(dataset, batchsize=32) # Optimization methods (you can easily try various methods by changing SGD to # MomentumSGD, Adam, RMSprop, AdaGrad, etc.) opt = optimizers.SGD(lr=0.01) opt.setup(model) updater = training.StandardUpdater(it, opt, device=0) # device=-1 if you use CPU trainer = training.Trainer(updater, stop_trigger=(100, 'epoch')) trainer.run() For more details, refer to official examples: https://github.com/pfnet/chainer/tree/master/examples
  • 14. Define-by-Run brings flexibility and intuitiveness “Forward computation” becomes a definition of network • Depending on data, it is easy to change a network structure (e.g. conditionally, stochastically, etc.) • You can define a network itself by Python code =The network structure can be treated as a program instead of data. For Chainer, the “forward computation” can be written in Python • Enables you to write a network structure freely using the syntax of Python • Define-by-Run makes it easy to insert any process like putting a print statement between network computations (In case of define-and-run which compiles a network, this kind of debugging is difficult) • Easy to reuse code of the same network for other purposes with few changes (e.g. by just adding a conditional branch partially) • Easy to check intermediate values and the design of the network itself using external debugging tools etc.
  • 15. CuPy - Drop-in replacement of NumPy for GPU acceleration Independent library to handle all GPU calculations in Chainer Lower cost to migrate CPU code to GPU with NumPy-compatible API GPU-execute linear algebra algorithms such as a singular value decomposition Rich in examples such as KMeans, Gaussian Mixture Model import numpy as np x = np.random.rand(10) W = np.random.rand(10, 5) y = np.dot(x, W) import cupy as cp x = cp.random.rand(10) W = cp.random.rand(10, 5) y = cp.dot(x, W) GPU https://github.com/cupy/cupy
  • 16. ● How to write custom kernels: https://docs-cupy.chainer.org/en/latest/tutorial/kernel.html ● Custom kernel example: https://github.com/cupy/cupy/tree/master/examples/gemm ● Example: element-wise kernel to compute: f(x, y) = (x − y)^2 Write you own CUDA kernel with CuPy Usage (note that broadcasting is supported): >>> x = cp.arange(10, dtype=np.float32).reshape(2, 5) >>> squared_diff(x, 5) array([[ 25., 16., 9., 4., 1.], [ 0., 1., 4., 9., 16.]], dtype=float32) Code: >>> squared_diff = cp.ElementwiseKernel( ... 'float32 x, float32 y', ... 'float32 z', ... 'z = (x - y) * (x - y)', ... 'squared_diff')
  • 17. Training speed comparison ilkarman/DeepLearningFrameworks https://github.com/ilkarman/DeepLearningFrameworks Chainer is 5th place at an external benchmark on CIFAR-10 dataset to compare the training time of VGG-style network ● Faster than PyTorch, TensorFlow, Keras (with all backends), CNTK Note ● Chainer + CuPy has auto-tune function to select the best convolution algorithm in cuDNN ● This benchmark runs on NVIDIA K80 GPU
  • 18. Popularity Growth of Chainer https://trends.google.com/trends/explore?cat=5&q=chainer,tensorflow,pytorch,mxnet,cntk
  • 19. Chainer comes with several examples of models such as ● Natural language processing: RNNs, Recursive nets, Word2Vec, Seq2Seq, etc. ● Image classification: various convnets for MNIST, CIFAR10, ImageNet ● Generative models: VAE, DCGAN For details, see https://github.com/chainer/chainer/tree/master/examples Additional models such as chainer-gan-lib can be found on pfnet-research: https://github.com/pfnet-research/chainer-gan-lib A large number of external examples are also available: https://github.com/chainer/chainer/wiki/External-examples Chainer examples
  • 20. pomegranate (probabilistic models): https://github.com/jmschrei/pomegranate PyINN (fused PyTorch ops in CuPy): https://github.com/szagoruyko/pyinn Vanilla LSTM with CuPy (no Chainer, easily and quickly ported from NumPy): https://github.com/mitmul/chainer-notebooks/blob/master/9_vanilla-LSTM-with-cupy.ipynb QRNN for PyTorch (uses CuPy for custom kernel) https://github.com/salesforce/pytorch-qrnn External projects/examples using CuPy
  • 21. Add-on packages for Chainer Distributed deep learning, deep reinforcement learning, computer vision ChainerMN (Multi-Node): additional package for distributed deep learning   High scalability (100 times faster with 128GPU) ChainerRL: deep reinforcement learning library   DQN, DDPG, A3C, ACER, NSQ, PCL, etc. OpenAI Gym support ChainerCV: provides image recognition algorithms, dataset wrappers   Faster R-CNN, Single Shot Multibox Detector (SSD), SegNet, etc.
  • 22. ChainerMN: Multi-Node training Keeping the easy-to-use characteristics of Chainer as is, ChainerMN enables to use multiple nodes which have multiple GPUs easily to make training faster GPU GPU InfiniBand GPU GPU InfiniBand MPI NVIDIA NCCL
  • 23. Distributed deep learning with ChainerMN 100x speed up with 128 Maxwell GPUs
  • 24. Comparison with other frameworks ChainerMN is the fastest at the comparison of elapsed time to train ResNet-50 on ImageNet dataset for 100 epochs (May 2017) Recently we achieved 15 mins to train ResNet50 on ImageNet dataset with 8 times larger cluster (1024 GPUs over 128 nodes) See the details in this paper: “Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes” https://arxiv.org/abs/1711.04325
  • 25. We confirmed that if we increase the number of nodes, the almost same accuracy can be achieved Speedup without dropping the accuracy
  • 26. Scale-out test on Microsoft Azure
  • 27. Easy-to-use API of ChainerMN You can start using ChainerMN just by wrapping one line! optimizer = chainer.optimizers.MomentumSGD() optimizer = chainermn.DistributedOptimizer( chainer.optimizers.MomentumSGD())
  • 28. Add-on packages for Chainer Distribute deep learning, deep reinforcement learning, computer vision ChainerMN (Multi-Node): additional package for distributed deep learning   High scalability (100 times faster with 128GPU) ChainerRL: deep reinforcement learning library   DQN, DDPG, A3C, ACER, NSQ, PCL, etc. OpenAI Gym support ChainerCV: provides image recognition algorithms, dataset wrappers   Faster R-CNN, Single Shot Multibox Detector (SSD), SegNet, etc.
  • 29. Reinforcement Learning: ChainerRL: Deep Reinforcement Learning Library Train an agent which interacts with the environment to maximize the rewards Action Env Observation, Reward
  • 30. Reinforcement Learning with ChainerRL 1. Create an environment Action Env Observation, Reward
  • 31. Distribution: Softmax, Mellowmax, Gaussian,… Policy: Observation → Distribution of actions 2. Define an agent model Reinforcement Learning with ChainerRL
  • 32. 2. Define an agent model (contd.) Q-Function: Observation → Value of each action (expectation of the sum of future rewards) ActionValue: Discrete, Quadratic Reinforcement Learning with ChainerRL
  • 33. Action Env Observation, Reward 3. Create an agent Reinforcement Learning with ChainerRL
  • 34. 4. Interact with the environment! Reinforcement Learning with ChainerRL
  • 35. Algorithms provided by ChainerRL • Deep Q-Network (Mnih et al., 2015) • Double DQN (Hasselt et al., 2016) • Normalized Advantage Function (Gu et al., 2016) • (Persistent) Advantage Learning (Bellemare et al., 2016) • Deep Deterministic Policy Gradient (Lillicrap et al., 2016) • SVG(0) (Heese et al., 2015) • Asynchronous Advantage Actor-Critic (Mnih et al., 2016) • Asynchronous N-step Q-learning (Mnih et al., 2016) • Actor-Critic with Experience Replay (Wang et al., 2017) <- NEW! • Path Consistency Learning (Nachum et al., 2017) <- NEW! • etc.
  • 36. ChainerRL Quickstart Guide • Define a Q-function in a Jupyter notebook and learn the Cart Pole Balancing problem with DQN https://github.com/chainer/chainerrl/blob/master/examples/quickstart/quickstart.ipynb
  • 37. Add-on packages for Chainer Distribute deep learning, deep reinforcement learning, computer vision ChainerMN (Multi-Node): additional package for distributed deep learning   High scalability (100 times faster with 128GPU) ChainerRL: deep reinforcement learning library   DQN, DDPG, A3C, ACER, NSQ, PCL, etc. OpenAI Gym support ChainerCV: provides image recognition algorithms, dataset wrappers   Faster R-CNN, Single Shot Multibox Detector (SSD), SegNet, etc.
  • 38. Evaluate your model on popular datasets Running and training deep-learning models easier for Computer Vision tasks ChainerCV https://github.com/chainer/chainercv Datasets Pascal VOC, Caltech-UCSD Birds-200-2011, Stanford Online Products, CamVid, etc. Models Faster R-CNN, SSD, SegNet (will add more models!) Training tools Evaluation tools Dataset Abstraction Train popular models with your data
  • 39. Start computer vision research using deep learning much easier ChainerCV Latest algorithms with your data Provide complete model code, training code, inference code for segmentation algorithms (SegNet, etc.) and object detection algorithms (Faster R-CNN, SSD, etc.), and so on All code is confirmed to reproduce the results All training code and model code reproduced the experimental results shown in the original paper
  • 40. • If you want to see some examples of ChainerCV and the reproducing code for some papers, please check the official Github repository (chainer/chainercv) • The right figure shows the result of the inference code of Faster RCNN example • The pre-trained weights are automatically downloaded! https://github.com/chainer/chainercv Install: $ pip install chainercv
  • 43. ChainerUI ChainerUI is a visualization and experiment management tool for Chainer Install: $ pip install chainerui ● You can compare the effect of different hyperparameters visually by loss curves and the scores in the table ● You can change the learning rate dynamically during training See the details in: https://github.com/chainer/chainerui
  • 44. Chainer Chemistry Chainer Chemisry is a collection of tools for training neural networks on biology and chemistry tasks using Chainer Install: $ pip install chainer-chemistry ● It currently provides widely used graph convolution implementation: ○ NFP: Neural fingerprint ○ GGNN: Gated Graph Neural Network ○ Weave ○ SchNet See the details in: https://github.com/pfnet-research/chainer-chemistry
  • 45. Summary • Chainer is a deep learning framework focusing on flexibility • There are 5 additional packages for specific task/domain: • ChainerMN - Distributed learning • ChainerRL - Reinforcement learning • ChainerCV - Computer vision • ChainerUI - Visualization and experiment management • Chainer Chemistry - Graph convolutions for biology/chemistry tasks
  • 47. Intel Chainer with MKL-DNN Backend CPU CuPy NVIDIA GPU CUDA cuDNN BLAS NumPy Chainer MKL-DNN Intel Xeon/Xeon Phi MKL
  • 48. Intel Chainer with MKL-DNN Backend MKL-DNN • Neural Network library optimized for Intel architectures • Supported CPUs: ✓ Intel Atom(R) processor with Intel(R) SSE4.1 support ✓ 4th, 5th, 6th and 7th generation Intel(R) Core processor ✓ Intel(R) Xeon(R) processor E5 v3 family (code named Haswell) ✓ Intel(R) Xeon(R) processor E5 v4 family (code named Broadwell) ✓ Intel(R) Xeon(R) Platinum processor family (code name Skylake) ✓ Intel(R) Xeon Phi(TM) product family x200 (code named Knights Landing) ✓ Future Intel(R) Xeon Phi(TM) processor (code named Knights Mill) • MKL-DNN accelerates the computation of NN on the above CPUs
  • 49. Intel Chainer with MKL-DNN Backend convnet-benchmarks* result: Intel Chainer Chainer with NumPy (MKL-Build) Alexnet Forward 429.16 ms 5041.91 ms Alexnet Backward 841.73 ms 5569.49 ms Alexnet Total 1270.89 ms 10611.40 ms ~8.35x faster than NumPy backend!
  • 50. Intel Chainer with MKL-DNN Backend Intel is developing Intel Chainer as a fork of Chainer v2 https://github.com/intel/chainer
  • 54. Ponanza Chainer ● Won the 2nd place at The 27th World Computer Shogi Championship ● Based on Ponanza which was the champion for two years in a row (2015, 2016) ● “Ponanza Chainer” applied Deep Learning for ordering the possible next moves for which “Ponanza” should think ahead deeply ● “Ponanza Chainer” wins “Ponanza” with a probability of 80% Team PFN Issei Yamamoto Akira Shimoyama Team Ponanza
  • 55. Paints Chainer ● Auto Sketch Colorization ● Train a neural network with a large dataset of paintings ● It takes a line drawings as input, and output a colorized image! ● You can also give color hits which indicates preferable colors https://paintschainer.preferred.tech
  • 57. 1. Install CUDA Toolkit 8.0 https://developer.nvidia.com/cuda-downloads 2. Install cuDNN v6.0 Library https://developer.nvidia.com/rdp/cudnn-download 3. Install NCCL for Multi-GPUs https://github.com/NVIDIA/nccl 4. Install CuPy and Chainer % pip install cupy % pip install chainer Chainer on Ubuntu For more details, see the official installation guide: http://docs.chainer.org/en/stable/install.html
  • 58. Chainer on Windows with NVIDIA GPU 1. Install Visual C++ 2015 Build Tools http://landinghub.visualstudio.com/visual-cpp-build-tools 2. Install CUDA Toolkit 8.0 https://developer.nvidia.com/cuda-downloads 3. Install cuDNN v6.0 Library for Windows 10 https://developer.nvidia.com/rdp/cudnn-download Put all files under C:Program FilesNVIDIA GPU Computing ToolkitCUDAv8.0 4. Install Anaconda 4.3.1 Python 3.6 or 2.7 https://www.continuum.io/downloads 5. Add environmental variables - Add “C:Program Files (x86)Microsoft Visual Studio 14.0VCbin” to PATH variable - Add “C:Program Files (x86)Windows Kits10Include10.0.10240.0ucrt” to INCLUDE variable 6. Install Chainer on Anaconda Prompt > pip install cupy > pip install chainer Note: If Visual C++ 2017 was previously installed, some additional steps might be required. In that case, please visit our support channels for help.
  • 59. Chainer on Azure Use Data Science Virtual Machine for Linux (Ubuntu) • Ready for CUDA & cuDNN • Chainer, CuPy, ChainerCV, and ChainerRL are pre-installed! 1 2 3
  • 60. • Easy model export to ONNX format ONNX-Chainer https://github.com/chainer/onnx-chainer
  • 61. Chainer Model Export tfchain: TensorFlow export (experimental) Caffe-export • https://github.com/mitmul/tfchain • Supports Linear, Convolution2D, MaxPooling2D, ReLU • Just add @totf decorator right before the forward method of the model import chainer import chainer.functions as F import chainer.links as L import numpy as np from chainer.exporters import caffe model = chainer.Sequential( L.Linear(None, 10), F.relu, L.Linear(10, 10) F.relu, L.Linear(10, 10) ) x = chainer.Variable( x.random.rand(1, 10).astype('f')) caffe.export(model, [x], './, True)
  • 62. External Projects for Model Portability DLPack • https://mil-tokyo.github.io/webdnn/ • The model conversion to run it on a web browser supports Chainer WebDNN • https://github.com/dmlc/dlpa ck • MXNet, Torch, Caffe2 have joined to discuss the guideline of memory layout of tensor and the common operator interfaces
  • 64. The Chainer project is now supported by these Leading computing companies
  • 66. Chainer is an open-source project. • You can send a PR from here: https://github.com/chainer/chainer • The development speed of Deep Learning research is super fast, therefore, to provide the state-of-the-art technologies through Chainer, we continuously update the development plans: • Chainer v3.0.0 will be released on 26th September! • Will support gradient of gradient (higher order differentiation) • Will add the official Windows support ensured by Microsoft The release schedule after v2.0.1 (4th July)→
  • 67. ● Started as a fork of Chainer. API is similar to Chainer. ● Codebase is written in a combination of Python, C, C++ -> More complicated codebase than Chainer but can also be faster in some cases. ● Uses familiar Torch tensor API. Easy for torch users but requires extra learning curve for NumPy users. Conversion functions sometimes needed. Comparison of Chainer with PyTorch ● Codebase is written entirely in Python and therefore accessible to researchers who are not familiar with C/C++ -> Keep codebase in pure Python if possible. Value an easily understandable and maintainable codebase over extreme performance in all cases. ● Uses familiar Numpy API: Numpy (CPU) or CuPy (GPU). ● Easy to write custom GPU kernels in Chainer using CuPy. ● CuPy makes it easy for researchers to write custom CUDA kernels. Chainer PyTorch So, the differences are mostly in the array libraries and different philosophies on how to achieve the best balance between performance optimizations and maintainability and flexibility of the core codebase.