SlideShare a Scribd company logo
1 of 27
Download to read offline
深層学習フレームワーク概要と
Chainerの事例紹介
Stapy x 理研AIP オープンソース研究会
@理化学研究所 ⾰新知能統合研究センター
2017年5⽉19⽇
株式会社Preferred Networks ⼤野健太
oono@preferred.jp
大野 健太
• twitter: @delta2323_
• 経歴
• 数学専攻(修⼠) → 2011.4 PFI → 2014.10 PFN
• 担当分野
• バイオプロジェクト
• Chainerコアチーム
• インターン・採⽤チーム
2
Chainer Meetup #05
2017年6⽉10⽇
@⽇本マイクロソフト株式会社
品川オフィス
3
• 2014年3⽉設⽴
• 本社:東京 アメリカ⼦会社:カリフォルニア州サンマテオ
• 社員数:約80名(8割以上はエンジニア・リサーチャー)
• 事業内容:深層学習の産業、特に産業⽤ロボット・交通・バイオ
ヘルスケアへの応⽤
4
YouTube Channel
preferredjp
Factory
Robot
Healthcare
Automotive
Warring states period of deep learning
frameworks
5
Technology stack of a DL framework
6
name functions example
Graphical visualization DIGITS, TensorBoard
Machine learning workflow
management
Dataset prep, Save/Load
Training loop
Keras, TF slim
Computational graph(CG)
management
Build/Optimize CGs
Forward/Back prop
Theano, TensorFlow
Torch.nn
Multi-dimensional
array processing
High-level array
manipulation
NumPy, CuPy
Eigen, Torch (core)
Numerical computation Matrix operation
Convolution
BLAS(OpenBLAS, MKL),
cuBLAS, cuDNN, MKL
DNN
Computational device CPU, GPU, TPU, FPGA
üconstructing NNs as a Python programming
üdynamic NN construction
üCPU/GPU agnostic code with CuPy
7
Mission
Speed up research and development of
deep learning and its applications.
Features
Flexible and intuitive description of complex NNs by
http://chainer.org
*NN = Neural Network
Software stack
8
CPU
CuPy
NVIDIA GPU
CUDA
cuDNN
BLAS
NumPy
Chainer
MKL
f g
x f g
Static
graph
construction
Data feed
x yf zg
Dynamic
graph
construction
Define-And-Run
(Most frameworks)
Define-By-Run
(Chainer)
9
x yf
x
z
Static Dynamic
Optimization ✓ △
Flexibility △ ✓
10
(MinPy)
Era of dynamic graph frameworks
Construct NNs as
Python programming
class MLP(Link):
def __int__(self):
super(MLP, self).__init__(
l1=Linear(784, 1000),
l2=Linear(1000, 1000),
l3=Linear(1000, 10))
def __call__(x):
h1 = F.relu(self.l1(x))
h2 = F.relu(self.l2(l1))
return self.l3(h2)
Linear l1
x
W bias
ReLU
Linear l2
h1
W bias
ReLU
Linear l3
h2
W bias
11
Release history
• 2015/06: v1.0.0
• 2015/09: v1.3.0 (CuPy)
• 2015/11: v1.5.0 (Link/Chain, CuPy in Cython)
• 2016/06: [MinPy]
• 2016/07: v1.11.0 (Trainer)
• 2017/01: [PyTorch, TensorFlow Fold]
• 2017/02: v2.0.0a
• 2017/04: v2.0.0b
• 2017/05: v1.24.0 (Last v1 release)
Chainer v2
• First major version up that breaks backward compatibility.
• Important features (almost fixed)
• CuPy separation
• Unified configuration (chainer.config, esp. train mode)
• train argument is removed from many functions
• Variable updated: Parameter class, uninitialized var, volatile removed
• Funcion.retain_inputs and retain_outputs to reduce memory usage
• New-style parameter/child link registration (just setting them as an attribute)
• UpdateRule customized for each parameter
• Extention.initialize added, invoke_before_training removed
• No duplicated memory between training graph and evaluation graph
• Input size is made optional in many links (L.Linear(100))
• wscale option is removed from many links
13
Will release on May 30th 2017
Libraries on top of Chainer
ChainerRL (beta): Reinforcement learning
ChainerMN (v0.1.0): Multi-node distributed learning
ChainerCV (v0.4.5): Computer vision
ChainerRL
15
ü Implement latest deep reinforcement learning algorithms
ü Work with OpenAI Gym
ChainerMN
16
ü Distributed deep learning with MPI, NCCL
ü approx. 100x speed with 128 GPUs
ChainerCV
17
ü Dataset Wrapper For well-known CV datasets (CUB, Pascal VOC)
ü Dataset transformer (random crop, random flop)
ü Implements typical workflow in CV
Application of Chainer
PaintsChainer : Line draw colorization tool
PonanzaChainer : Shogi AI with Deep Learning
PaintsChainer
19https://paintschainer.preferred.tech
Ponanza Chainer
20
NumPy-like API accelerated with CUDA
# CPU
x_cpu = numpy.array([1, 2, 3])
l2_cpu = numpy.linalg.norm(x_cpu)
# GPU
x_gpu = cupy.array([1, 2, 3])
l2_gpu = cupy.linalg.norm(x_gpu)
CuPy will be an independent project from Chainer from Chainer v2.
>150 NumPy functions are supported
Development team (as of May 2017)
• Chainer, CuPy
• Core development team: approx. 10 members
• Reviewer team: approx. 10 members
• Chainer user group: approx. 5 members
• Chainer RL, Chainer MN, Chainer CV: 2, 3 members for each
• Paints Chainer: approx. 10 members
22
* some members overlap
CI
23
Travis CI
Run all CPU tests of all PRs
Jenkins installation test
CI
24
Jenkins dairy test (internal): Run all tests with various configuration
Community activities
25
Chainer meetup (#0 - #4)
#5 will be held in June 10th 2017
Deep learning mini course @UCSF
application for biology
Google group (ja, en), Slack (ja, en), Twitter (ja, en)
Conclusion
• Chainer is a Python-based deep learning framework that
leverages flexible and intuitive description of NNs.
• Many libraries and services are being developed on top of
Chainer (ChainerRL/MN/CV, PaintsChainer, PonanzaChainer,
CuPy).
• Introduced the development and the user-group teams of
Chainer
26
Try Chainer ! http://chainer.org
27
Seiya
Tokui
Kenta
Oono
Yuya
Unno
Ryosuke
Okuta
Chainer core development team
Brian
Vogel
Gentaro
Watanabe
Shunta
Saito
Daisuke
Nishino
and many
contributors !
Contact: oono@preferred.jp
Google Group: Chainer User Group

More Related Content

What's hot

What's hot (20)

Intro to TensorFlow and PyTorch Workshop at Tubular Labs
Intro to TensorFlow and PyTorch Workshop at Tubular LabsIntro to TensorFlow and PyTorch Workshop at Tubular Labs
Intro to TensorFlow and PyTorch Workshop at Tubular Labs
 
TensorFlow Dev Summit 2017 요약
TensorFlow Dev Summit 2017 요약TensorFlow Dev Summit 2017 요약
TensorFlow Dev Summit 2017 요약
 
TensorFlow
TensorFlowTensorFlow
TensorFlow
 
Introduction to Neural Networks in Tensorflow
Introduction to Neural Networks in TensorflowIntroduction to Neural Networks in Tensorflow
Introduction to Neural Networks in Tensorflow
 
TensorFlow and Keras: An Overview
TensorFlow and Keras: An OverviewTensorFlow and Keras: An Overview
TensorFlow and Keras: An Overview
 
Tensorflow
TensorflowTensorflow
Tensorflow
 
TensorFlow in Context
TensorFlow in ContextTensorFlow in Context
TensorFlow in Context
 
MN-3, MN-Core and HPL - SC21 Green500 BOF
MN-3, MN-Core and HPL - SC21 Green500 BOFMN-3, MN-Core and HPL - SC21 Green500 BOF
MN-3, MN-Core and HPL - SC21 Green500 BOF
 
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
 
Tensorflow 101 @ Machine Learning Innovation Summit SF June 6, 2017
Tensorflow 101 @ Machine Learning Innovation Summit SF June 6, 2017Tensorflow 101 @ Machine Learning Innovation Summit SF June 6, 2017
Tensorflow 101 @ Machine Learning Innovation Summit SF June 6, 2017
 
Intro to Python
Intro to PythonIntro to Python
Intro to Python
 
PyData.Tokyo Meetup #21 講演資料「Optuna ハイパーパラメータ最適化フレームワーク」太田 健
PyData.Tokyo Meetup #21 講演資料「Optuna ハイパーパラメータ最適化フレームワーク」太田 健PyData.Tokyo Meetup #21 講演資料「Optuna ハイパーパラメータ最適化フレームワーク」太田 健
PyData.Tokyo Meetup #21 講演資料「Optuna ハイパーパラメータ最適化フレームワーク」太田 健
 
[AI07] Revolutionizing Image Processing with Cognitive Toolkit
[AI07] Revolutionizing Image Processing with Cognitive Toolkit[AI07] Revolutionizing Image Processing with Cognitive Toolkit
[AI07] Revolutionizing Image Processing with Cognitive Toolkit
 
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
Rajat Monga, Engineering Director, TensorFlow, Google at MLconf 2016
 
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
 
Tensorflow presentation
Tensorflow presentationTensorflow presentation
Tensorflow presentation
 
Deep Learning with PyTorch
Deep Learning with PyTorchDeep Learning with PyTorch
Deep Learning with PyTorch
 
Introduction To TensorFlow
Introduction To TensorFlowIntroduction To TensorFlow
Introduction To TensorFlow
 
Tensorflow vs MxNet
Tensorflow vs MxNetTensorflow vs MxNet
Tensorflow vs MxNet
 
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
 

Similar to 深層学習フレームワーク概要とChainerの事例紹介

Similar to 深層学習フレームワーク概要とChainerの事例紹介 (20)

Keynote at Converge 2019
Keynote at Converge 2019Keynote at Converge 2019
Keynote at Converge 2019
 
from ai.backend import python @ pycontw2018
from ai.backend import python @ pycontw2018from ai.backend import python @ pycontw2018
from ai.backend import python @ pycontw2018
 
Machine_learning_internship_report_facemaskdetection.pptx
Machine_learning_internship_report_facemaskdetection.pptxMachine_learning_internship_report_facemaskdetection.pptx
Machine_learning_internship_report_facemaskdetection.pptx
 
Python for Data Science with Anaconda
Python for Data Science with AnacondaPython for Data Science with Anaconda
Python for Data Science with Anaconda
 
Python ml
Python mlPython ml
Python ml
 
Array computing and the evolution of SciPy, NumPy, and PyData
Array computing and the evolution of SciPy, NumPy, and PyDataArray computing and the evolution of SciPy, NumPy, and PyData
Array computing and the evolution of SciPy, NumPy, and PyData
 
Standardizing arrays -- Microsoft Presentation
Standardizing arrays -- Microsoft PresentationStandardizing arrays -- Microsoft Presentation
Standardizing arrays -- Microsoft Presentation
 
Scallable Distributed Deep Learning on OpenPOWER systems
Scallable Distributed Deep Learning on OpenPOWER systemsScallable Distributed Deep Learning on OpenPOWER systems
Scallable Distributed Deep Learning on OpenPOWER systems
 
2016 dg2
2016 dg22016 dg2
2016 dg2
 
TensorFlow meetup: Keras - Pytorch - TensorFlow.js
TensorFlow meetup: Keras - Pytorch - TensorFlow.jsTensorFlow meetup: Keras - Pytorch - TensorFlow.js
TensorFlow meetup: Keras - Pytorch - TensorFlow.js
 
Deep learning with Keras
Deep learning with KerasDeep learning with Keras
Deep learning with Keras
 
The road ahead for scientific computing with Python
The road ahead for scientific computing with PythonThe road ahead for scientific computing with Python
The road ahead for scientific computing with Python
 
Strata parallel m-ml-ops_sept_2017
Strata parallel m-ml-ops_sept_2017Strata parallel m-ml-ops_sept_2017
Strata parallel m-ml-ops_sept_2017
 
Image Classification (20230411)
Image Classification (20230411)Image Classification (20230411)
Image Classification (20230411)
 
Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"Travis Oliphant "Python for Speed, Scale, and Science"
Travis Oliphant "Python for Speed, Scale, and Science"
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...AI powered emotion recognition: From Inception to Production - Global AI Conf...
AI powered emotion recognition: From Inception to Production - Global AI Conf...
 
Deep learning with Keras
Deep learning with KerasDeep learning with Keras
Deep learning with Keras
 
AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
 

More from Kenta Oono

提供AMIについて
提供AMIについて提供AMIについて
提供AMIについて
Kenta Oono
 
Caffeインストール
CaffeインストールCaffeインストール
Caffeインストール
Kenta Oono
 

More from Kenta Oono (20)

Minimax statistical learning with Wasserstein distances (NeurIPS2018 Reading ...
Minimax statistical learning with Wasserstein distances (NeurIPS2018 Reading ...Minimax statistical learning with Wasserstein distances (NeurIPS2018 Reading ...
Minimax statistical learning with Wasserstein distances (NeurIPS2018 Reading ...
 
Deep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistryDeep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistry
 
Overview of Machine Learning for Molecules and Materials Workshop @ NIPS2017
Overview of Machine Learning for Molecules and Materials Workshop @ NIPS2017Overview of Machine Learning for Molecules and Materials Workshop @ NIPS2017
Overview of Machine Learning for Molecules and Materials Workshop @ NIPS2017
 
20170422 数学カフェ Part2
20170422 数学カフェ Part220170422 数学カフェ Part2
20170422 数学カフェ Part2
 
20170422 数学カフェ Part1
20170422 数学カフェ Part120170422 数学カフェ Part1
20170422 数学カフェ Part1
 
情報幾何学の基礎、第7章発表ノート
情報幾何学の基礎、第7章発表ノート情報幾何学の基礎、第7章発表ノート
情報幾何学の基礎、第7章発表ノート
 
GTC Japan 2016 Chainer feature introduction
GTC Japan 2016 Chainer feature introductionGTC Japan 2016 Chainer feature introduction
GTC Japan 2016 Chainer feature introduction
 
On the benchmark of Chainer
On the benchmark of ChainerOn the benchmark of Chainer
On the benchmark of Chainer
 
Tokyo Webmining Talk1
Tokyo Webmining Talk1Tokyo Webmining Talk1
Tokyo Webmining Talk1
 
VAE-type Deep Generative Models
VAE-type Deep Generative ModelsVAE-type Deep Generative Models
VAE-type Deep Generative Models
 
Introduction to Chainer and CuPy
Introduction to Chainer and CuPyIntroduction to Chainer and CuPy
Introduction to Chainer and CuPy
 
Stochastic Gradient MCMC
Stochastic Gradient MCMCStochastic Gradient MCMC
Stochastic Gradient MCMC
 
Chainer Contribution Guide
Chainer Contribution GuideChainer Contribution Guide
Chainer Contribution Guide
 
2015年9月18日 (GTC Japan 2015) 深層学習フレームワークChainerの導入と化合物活性予測への応用
2015年9月18日 (GTC Japan 2015) 深層学習フレームワークChainerの導入と化合物活性予測への応用 2015年9月18日 (GTC Japan 2015) 深層学習フレームワークChainerの導入と化合物活性予測への応用
2015年9月18日 (GTC Japan 2015) 深層学習フレームワークChainerの導入と化合物活性予測への応用
 
Introduction to Chainer (LL Ring Recursive)
Introduction to Chainer (LL Ring Recursive)Introduction to Chainer (LL Ring Recursive)
Introduction to Chainer (LL Ring Recursive)
 
日本神経回路学会セミナー「DeepLearningを使ってみよう!」資料
日本神経回路学会セミナー「DeepLearningを使ってみよう!」資料日本神経回路学会セミナー「DeepLearningを使ってみよう!」資料
日本神経回路学会セミナー「DeepLearningを使ってみよう!」資料
 
提供AMIについて
提供AMIについて提供AMIについて
提供AMIについて
 
Chainerインストール
ChainerインストールChainerインストール
Chainerインストール
 
Caffeインストール
CaffeインストールCaffeインストール
Caffeインストール
 
ディープラーニング最近の発展とビジネス応用への課題
ディープラーニング最近の発展とビジネス応用への課題ディープラーニング最近の発展とビジネス応用への課題
ディープラーニング最近の発展とビジネス応用への課題
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 

Recently uploaded (20)

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdf
 
Buy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptxBuy Epson EcoTank L3210 Colour Printer Online.pptx
Buy Epson EcoTank L3210 Colour Printer Online.pptx
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 

深層学習フレームワーク概要とChainerの事例紹介

  • 1. 深層学習フレームワーク概要と Chainerの事例紹介 Stapy x 理研AIP オープンソース研究会 @理化学研究所 ⾰新知能統合研究センター 2017年5⽉19⽇ 株式会社Preferred Networks ⼤野健太 oono@preferred.jp
  • 2. 大野 健太 • twitter: @delta2323_ • 経歴 • 数学専攻(修⼠) → 2011.4 PFI → 2014.10 PFN • 担当分野 • バイオプロジェクト • Chainerコアチーム • インターン・採⽤チーム 2
  • 4. • 2014年3⽉設⽴ • 本社:東京 アメリカ⼦会社:カリフォルニア州サンマテオ • 社員数:約80名(8割以上はエンジニア・リサーチャー) • 事業内容:深層学習の産業、特に産業⽤ロボット・交通・バイオ ヘルスケアへの応⽤ 4 YouTube Channel preferredjp Factory Robot Healthcare Automotive
  • 5. Warring states period of deep learning frameworks 5
  • 6. Technology stack of a DL framework 6 name functions example Graphical visualization DIGITS, TensorBoard Machine learning workflow management Dataset prep, Save/Load Training loop Keras, TF slim Computational graph(CG) management Build/Optimize CGs Forward/Back prop Theano, TensorFlow Torch.nn Multi-dimensional array processing High-level array manipulation NumPy, CuPy Eigen, Torch (core) Numerical computation Matrix operation Convolution BLAS(OpenBLAS, MKL), cuBLAS, cuDNN, MKL DNN Computational device CPU, GPU, TPU, FPGA
  • 7. üconstructing NNs as a Python programming üdynamic NN construction üCPU/GPU agnostic code with CuPy 7 Mission Speed up research and development of deep learning and its applications. Features Flexible and intuitive description of complex NNs by http://chainer.org *NN = Neural Network
  • 9. f g x f g Static graph construction Data feed x yf zg Dynamic graph construction Define-And-Run (Most frameworks) Define-By-Run (Chainer) 9 x yf x z Static Dynamic Optimization ✓ △ Flexibility △ ✓
  • 10. 10 (MinPy) Era of dynamic graph frameworks
  • 11. Construct NNs as Python programming class MLP(Link): def __int__(self): super(MLP, self).__init__( l1=Linear(784, 1000), l2=Linear(1000, 1000), l3=Linear(1000, 10)) def __call__(x): h1 = F.relu(self.l1(x)) h2 = F.relu(self.l2(l1)) return self.l3(h2) Linear l1 x W bias ReLU Linear l2 h1 W bias ReLU Linear l3 h2 W bias 11
  • 12. Release history • 2015/06: v1.0.0 • 2015/09: v1.3.0 (CuPy) • 2015/11: v1.5.0 (Link/Chain, CuPy in Cython) • 2016/06: [MinPy] • 2016/07: v1.11.0 (Trainer) • 2017/01: [PyTorch, TensorFlow Fold] • 2017/02: v2.0.0a • 2017/04: v2.0.0b • 2017/05: v1.24.0 (Last v1 release)
  • 13. Chainer v2 • First major version up that breaks backward compatibility. • Important features (almost fixed) • CuPy separation • Unified configuration (chainer.config, esp. train mode) • train argument is removed from many functions • Variable updated: Parameter class, uninitialized var, volatile removed • Funcion.retain_inputs and retain_outputs to reduce memory usage • New-style parameter/child link registration (just setting them as an attribute) • UpdateRule customized for each parameter • Extention.initialize added, invoke_before_training removed • No duplicated memory between training graph and evaluation graph • Input size is made optional in many links (L.Linear(100)) • wscale option is removed from many links 13 Will release on May 30th 2017
  • 14. Libraries on top of Chainer ChainerRL (beta): Reinforcement learning ChainerMN (v0.1.0): Multi-node distributed learning ChainerCV (v0.4.5): Computer vision
  • 15. ChainerRL 15 ü Implement latest deep reinforcement learning algorithms ü Work with OpenAI Gym
  • 16. ChainerMN 16 ü Distributed deep learning with MPI, NCCL ü approx. 100x speed with 128 GPUs
  • 17. ChainerCV 17 ü Dataset Wrapper For well-known CV datasets (CUB, Pascal VOC) ü Dataset transformer (random crop, random flop) ü Implements typical workflow in CV
  • 18. Application of Chainer PaintsChainer : Line draw colorization tool PonanzaChainer : Shogi AI with Deep Learning
  • 21. NumPy-like API accelerated with CUDA # CPU x_cpu = numpy.array([1, 2, 3]) l2_cpu = numpy.linalg.norm(x_cpu) # GPU x_gpu = cupy.array([1, 2, 3]) l2_gpu = cupy.linalg.norm(x_gpu) CuPy will be an independent project from Chainer from Chainer v2. >150 NumPy functions are supported
  • 22. Development team (as of May 2017) • Chainer, CuPy • Core development team: approx. 10 members • Reviewer team: approx. 10 members • Chainer user group: approx. 5 members • Chainer RL, Chainer MN, Chainer CV: 2, 3 members for each • Paints Chainer: approx. 10 members 22 * some members overlap
  • 23. CI 23 Travis CI Run all CPU tests of all PRs Jenkins installation test
  • 24. CI 24 Jenkins dairy test (internal): Run all tests with various configuration
  • 25. Community activities 25 Chainer meetup (#0 - #4) #5 will be held in June 10th 2017 Deep learning mini course @UCSF application for biology Google group (ja, en), Slack (ja, en), Twitter (ja, en)
  • 26. Conclusion • Chainer is a Python-based deep learning framework that leverages flexible and intuitive description of NNs. • Many libraries and services are being developed on top of Chainer (ChainerRL/MN/CV, PaintsChainer, PonanzaChainer, CuPy). • Introduced the development and the user-group teams of Chainer 26
  • 27. Try Chainer ! http://chainer.org 27 Seiya Tokui Kenta Oono Yuya Unno Ryosuke Okuta Chainer core development team Brian Vogel Gentaro Watanabe Shunta Saito Daisuke Nishino and many contributors ! Contact: oono@preferred.jp Google Group: Chainer User Group