2. Contents
• Machine Learning?
• Artificial Neural
Network?
• Open Source based
Artificial Intelligence
Softwares
• Open Source A.I
Software Applications
3. Mario (manseok) Cho
Development Experience
Image Recognition using Neural Network
Bio-Medical Data Processing
Human Brain Mapping on High Performance Computing
Medical Image Reconstruction(Computer Tomography)
Enterprise System Architect & consuliting
Artificial Intelligence for medicine decision support
Open Source Software Developer
Committer: (Cloud NFV/SDN)
Contribute:
TensorFlow (Deep Learning)
OpenStack (Cloud compute)
LLVM (compiler)
Kernel (Linux)
Book
Unix V6 Kernel
Lablup Inc.
Mario Cho
hephaex@gmail.com
4. The Future of Jobs
“The Fourth Industrial Revolution, which
includes developments in previously
disjointed fields such as
artificial intelligence & machine-learning,
robotics, nanotechnology, 3-D printing,
and genetics & biotechnology,
will cause widespread disruption not only
to business models but also to labor
market over the next five years, with
enormous change predicted in the skill
sets needed to thrive in the new
landscape.”
6. What is the Machine Learning ?
• Field of Computer Science that evolved from the
study of pattern recognition and computational
learning theory into Artificial Intelligence.
• Its goal is to give computers the ability to learn
without being explicitly programmed.
• For this purpose, Machine Learning uses
mathematical / statistical techniques to construct
models from a set of observed data rather than
have specific set of instructions entered by the
user that define the model for that set of data.
13. Neural Network as a Computational Graph
• In Most Machine Learning Frameworks,
• Neural Network is conceptualized as a
Computational Graph
• The simple form of Computational Graph,
• Directed Acyclic Graph consist Data Nodes
and Operator Nodes
Y = x1 * x2
Z = x3 – y
Data node
Opeator node
16. Multi layer perceptron
X
W1 b1
h1Affine
a1
W2 b2
h2Affine
ReLU
ReLU
a2
W3 b3
h3Affine Softmax
t
Cross
Entropy
prob loss
17. WFO Discovery Advisor
• Researches can‟t innovate fast enough to create truly breakthrough therapies
• To anticipate the safety profile of new treatments
WFO Corpus
Over 1TB of data
Over 40m documents
Over 100m entities
& relationships
Chemical
12M+ Chemical Structures
Genomics
20,000+ genes
MD Text
50+ books
Medline
23M+ abstracts
Journals
100+ journals
FDA drugs
11,000+ drugs
Patents
16M+ patents
23. Traditional learning vs Deep Machine Learning
Eiffel Tower
Eiffel Tower
RAW data
RAW data
Deep
Learning
Network
Feature
Extraction
Vectored Classification
Traditional Learning
Deep Learning
28. Open Source Software for Machine Learning
Caffe
Theano
Convnet.js
Torch7
Chainer
DL4J
TensorFlow
Neon
SANOA
Summingbird
Apache SA
Flink ML
Mahout
Spark MLlib
RapidMiner
Weka
Knife
Scikit-learn
Amazon ML
BigML
DataRobot
FICO
Google
prediction API
HPE haven
OnDemand
IBM Watson
PurePredictive
Yottamine
Deep
Learning
Stream
Analytics
Big Data
Machine Learning
Data
Mining
Machine Learning
As a Service
Pylearn2
29. • Created by
Yangqing Jia (http://daggerfs.com/)
UC Berkerey Computer Science Ph.D. / Trevor Darrell, BAIR
Google BrainLab.TensorFlow join
Facebook research Scientest
Evan Shellhamer (http://imaginarynumber.net/)
• Maintained by
BAIR(Berkeley Artificial Intelligence Research, http://bair.berkeley.edu/)
• Release
„2013: DeCAF (https://arxiv.org/abs/1310.1531)
Dec. „2013: Caffe v0
• Application
Facebook, Adobe, Microsoft, Samsung, Flickr, Tesla, Yelp, Pinterest, etc.
• Motivation
„2012 ILSVRC, AlexNet
DNN define/training/deploy implementation by F/W
Caffe
http://caffe.berkeleyvision.org/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL Multi GPU
Parallel
Executi
on
Caffe BAIR
Linux,
Mac
- C++
Python,
MATLAB
Y
Y
- Y
30. • Created & Maintained by
Preferred Networks, Inc.
(https://www.preferred-networks.jp/ja/)
• Release
Jun. „2015
• Application
Toyota motors, Panasonic
(https://www.wsj.com/articles/japan-seeks-tech-revival-with-artificial-intelligence-
1448911981)
FANUC
(http://www.fanucamerica.com/FanucAmerica-news/Press-
releases/PressReleaseDetails.aspx?id=79)
• Motivation
Define-by-Run Architecture
Chainer
http://docs.chainer.org/en/latest/index.html
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL Multi GPU
Parallel
Executi
on
Chainer
Preferred
Networks
Linux - Python Python -
Y
- Y Y
[Define-and-Run (TensorFlow)] [Define-by-Run (Chainer, PyTorch)]
31. • Created & Maintained by
Microsoft Research
• Release
Jan. „2016
• Applications
Microsoft‟s speech recognition engine
Skype‟s Translator
• Motivation
Efficient performance on distributed environments
CNTK
https://www.microsoft.com/en-us/research/product/cognitive-toolkit/
https://www.microsoft.com/en-us/research/blog/microsoft-computational-network-toolkit-offers-most-efficient-distributed-deep-learning-computational-performance/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL
Multi
GPU
Parallel
Execution
CNTK Microsoft
Linux,
Windows
- C++ Python, C++ Y Y - Y Y
32. • 주체
• Created by
Adam Gibson @Skymind (CTO)
Chris Nicholson @Skymind (CEO)
• Maintained by
Skymind (https://skymind.ai/)
• Release
Jun. „2014
• Application
Finatial Fraud Detection Research Partnership with Nextremer in Japan
(https://skymind.ai/press/nextremer)
DL4J
https://deeplearning4j.org/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL
Multi
GPU
Parallel
Execution
DL4J SkyMind
Cross-
platform
(JVM)
Android Java
Java, Scala,
Python
Y Y
- Y
Y
(Spark)
33. • Created & Maintained by
Francois Chollet @Google
• Release
Mar. „2015
• Appliation
TensorFlow (http://www.fast.ai/2017/01/03/keras)
• Motivation
Provide a high-level interface based on deep learning framework like Theano, TensorFlow
Easy to use
Simple Modular
Various Deep-learning framework support
Keras
https://keras.io/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL
Multi
GPU
Parallel
Execution
Keras
François
Chollet
Linux,
Mac,
Windows
- Python Python
Y(Thean
o)
N(TF)
Y
- Y
34. • Created by
CMU (http://www.cs.cmu.edu/~muli/file/mxnet-learning-sys.pdf)
• Maintained by
DMLC(Distributed Machine Learning Community)
CMU, NYU, NVIDIA, Baidu, Amazon, etc.
• Release
Oct. „2015
• Application
AWS (https://www.infoq.com/news/2016/11/amazon-mxnet-deep-learning)
• Motivation
Support for Mixed Programming Model: Imperative & Symbolic
Support for Portability: Desktops, Clusters, Mobiles, etc.
Support for Multiple Languages: C++, R, Python, Matlab, Javascript, etc.
MXNet
http://mxnet.io/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL
Multi
GPU
Parallel
Execution
MXNet DMLC
Linux,
Mac,
Windows,
Javascript
Android,
iOS
C++
C++, Python,
Julia,
MATLAB,
JavaScript,
Go, R, Scala,
Perl
Y Y - Y Y
35. • Created by
James Bergstra, Frederic Bastien, etc. (http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf_
Maintained by
LISA lab @ Université de Montréal
• Release
Nov „2010
• Application
Keras
Lasagne
Blocks
• Motivation
There‟s any.
Theano
http://deeplearning.net/software/theano/index.html
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL
Multi
GPU
Parallel
Execution
Theano
Université
de
Montréal
Linux,
Mac,
Windows
- Python Python
Y Y
- Y
36. • Created & Maintained by
Ronan Collobert: Research Scientist @ Facebook
Clément Farabet: Senior Software Engineer @ Twitter
Koray Kavukcuoglu: Research Scientist @ Google DeepMind
Soumith Chinatala: Research Engineer @ Facebook
• Release
Jul. „2014
• Application
Facebook, Google, Twitter, Element Inc., etc.
• Motivation
Unlike Caffe, for research rather than mass market
Unlike Theano, easy to use based on imperative model rather than symbolic model
Torch
http://torch.ch/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL
Multi
GPU
Parallel
Execution
Torch
Ronan,
Clément,
Koray,
Soumith
Linux,
Mac,
Windows
Android,
iOS
C, Lua Lua Y
Y
Y Y
Not
officially
37. • Created & Maintained by
Google Brain
• Release
Nov. „2015
• Application
Google
Search Signals (https://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-
web-search-over-to-ai-machines)
Email auto-responder (https://research.googleblog.com/2015/11/computer-respond-to-this-
email.html)
Photo Search (https://techcrunch.com/2015/11/09/google-open-sources-the-machine-learning-
tech-behind-google-photos-search-smart-reply-and-more/#.t38yrr8:fUIZ)
• Motivation
It‟s Google
TensorFlow
https://www.tensorflow.org/
S/W Creator Platform Mobile
Langua
ge
Interface OpenMP CUDA OpenCL Multi GPU
Parallel
Executi
on
TensorFlow Google
Linux,
Mac,
Windows
Android,
iOS
C++,
Python
Python,
C/C++, Java,
Go
N
Y
- Y Y
39. * Source: Oriol Vinyals – Research Scientist at Google Brain
40. Expressing High-Level ML Computations
• Core in C++
• Different front ends for specifying/driving the computation
• Python and C++ today, easy to add more
* Source: Jeff Dean– Research Scientist at Google Brain
52. Human-Level Face Recognition
• Convolutional neural networks based
face recognition system is dominant
• 99.15% face verification accuracy on
LFW dataset in DeepID2 (2014)
Beyond human-level recognition
Source: Taigman et al. DeepFace: Closing the Gap to Human-Level Performance in Face Verification, CVPR’14