SlideShare a Scribd company logo
Intro to Deep Learning w/ Clojure
It’s Difficult to Make Predictions.
Especially About the Future.
@JulioBarros
Consultant
E-String.com
@JulioBarros http://E-String.com 1
Hypothesis
Given that Python dominates ML/DL;
Is there a valid use case for Clojure?
TL;DR --It depends
@JulioBarros http://E-String.com 2
I Hope So
...and I'm not the only one.
@JulioBarros http://E-String.com 3
Hype or Reality?
@JulioBarros http://E-String.com 4
"It is a renaissance, it is a golden age,"
"MachinelearningandAIisahorizontalenablinglayer.Itwillempowerand
improveeverybusiness,everygovernmentorganization,everyphilanthropy
—basicallythere'snoinstitutionintheworldthatcannotbeimprovedwith
machinelearning."
—Bezos
@JulioBarros http://E-String.com 5
Every industry can expect to be transformed by Artificial
Intelligence
@JulioBarros http://E-String.com 6
Image Classification
Justin Johnson,Andrej Karpathy,Li Fei-Fei-Stanford
@JulioBarros http://E-String.com 7
My dog Rex
@JulioBarros http://E-String.com 8
Object Detection
@JulioBarros http://E-String.com 9
Image Captioning
@JulioBarros http://E-String.com 10
Dense Captioning
@JulioBarros http://E-String.com 11
Healthcare
Near or better than human level performance.
@JulioBarros http://E-String.com 12
Business
— Law & Finance
— Text,audio,image,video understanding
— Churn prediction,customer segmentation
— Product recommendations
— Manufacturing,maintenance and control
— Many many more
@JulioBarros http://E-String.com 13
Artificial Intelligence
Artificial Intelligence (AI)-the study of"intelligent agents".Reasoning,
knowledge representation,planning,robotics,etc.
— Artificial Narrow Intelligence (ANI)
— Artificial General Intelligence (AGI)
— Artificial Superintelligence (ASI)
@JulioBarros http://E-String.com 14
Machine Learning
Machine Learning (ML)-Programs that learn from the data and make
predictions.
@JulioBarros http://E-String.com 15
Types of prediction
Regression-continuous values
Classification-discreet values/labels
@JulioBarros http://E-String.com 16
Types of ML Algorithms
Supervised-trained on labeled data (regression or classification).
Unsupervised-trained on unlabeled data (clustering,segmentation).
Reinforcement-learns based on outcomes/results of actions.
@JulioBarros http://E-String.com 17
Deep Learning
Deep Learning (DL)-ML/AIusingartificialneuralnetworks(ANNs)
You might be thinking...
What?Isn'tthatjustaneuralnet?
Didn'tweshowthosedon'twork.
Twice.
@JulioBarros http://E-String.com 18
Third time is a charm
Capabilities of neural nets have changed dramatically due to
advancements in:
— Data
— Algorithms
— Hardware
@JulioBarros http://E-String.com 19
Neurons: Biologically inspired
1942 McCulloch and Pitts
1957 Rosenblatt
(A (+ b (apply + (map * x w)))
@JulioBarros http://E-String.com 20
Activation Function
Introduces non linearity
— Historicaly: Identity,Step,Tanh,Sigmoid
— Currently: Rectified linear Unit (relu),Softmax
@JulioBarros http://E-String.com 21
Universal Approximation Theorem
(1989)
...a feed-forward network with a single
hidden layer containing a finite number of
neurons can approximate continuous
functions...
@JulioBarros http://E-String.com 22
Deep Neural Nets
A net with more than one hidden layer.
@JulioBarros http://E-String.com 23
VGG16
@JulioBarros http://E-String.com 24
GoogleLeNet
@JulioBarros http://E-String.com 25
Training
1) Initialize weights randomly
2) Make prediction
3) Measure error (loss)
4) Adjust weights in the right direction
5) GOTO 2
Repeat over and over and over again with your training data
@JulioBarros http://E-String.com 26
Backpropagation
Finding the right direction.
@JulioBarros http://E-String.com 27
Backpropagation
To know the right direction calculate the
gradient of the loss function with respect
to each weight.Multiply by the error and a
learning rate to get the delta.
Don'tworry.Thelibrariesdoitforyou.
@JulioBarros http://E-String.com 28
Millions of Knobs Parameters
@JulioBarros http://E-String.com 29
A Simple Neural Net in Cortex
(def network-architecture
[(layers/input 11 1 1 :id :x)
(layers/linear 64)
(layers/relu)
(layers/linear 1 :id :y)])
@JulioBarros http://E-String.com 30
Wine Quality Data
11 features,1 target column
@JulioBarros http://E-String.com 31
UCI Machine Learning Repository
https://archive.ics.uci.edu/ml/datasets/Wine+Quality
Source: Paulo Cortez, University of Minho, Guimarães, Portugal, http://www3.dsi.uminho.pt/pcortez A.Cerdeira,
F.Almeida, T.Matos and J.Reis, Viticulture Commission of the Vinho Verde Region(CVRVV), Porto, Portugal
@2009
@JulioBarros http://E-String.com 32
Normalized / Scaled data (Standardized)
@JulioBarros http://E-String.com 33
Demo
https://github.com/thinktopic/cortex
https://github.com/JulioBarros/clj-dl-demo
@JulioBarros http://E-String.com 34
How Do We Work With Images
Well,images are just numbers/data.
Though numbers close to each other are more related.
@JulioBarros http://E-String.com 35
Convolutional Layers
Similar to correlations from signal
processing or filters from photoshop.
A small NxN filter is slid over and
convolved/correlated with the image.
Learns to find features.
Then lower level features are combined
into higher level features.
@JulioBarros http://E-String.com 36
Common Types of Layers
— input / output
— fully connected (dense)
— activation-relu,softmax
— convolutional
— maxpool
— flatten
— drop out
@JulioBarros http://E-String.com 37
Types of ANN
1. Dense Neural Net (DNN)
2. Convolutional Neural Net (CNN)
3. Recurrent Neural Net (RNN)
4. Everything else
@JulioBarros http://E-String.com 38
Programming Abstractions
Level Python CLJ/JVM
DSL/API Keras,Lasagne,TF-Slim Cortex,???
Computation Graph,
Backprop,Autograd
Tensorflow,Theano,
Torch,Pytorch,Caffe,
MXNet,CNTK
Cortex,dl4j,Java
bindings
Matrix Math CUDA (cuDNN),Eigen3 Neandertal,
core.matrix,vectorz-clj
@JulioBarros http://E-String.com 39
Production Considerations
— Target hardware environment
— GPU(s)
— Powerful multicore server
— Mobile device
— Embedded
— Desired latency / scalability
— Data pipeline
— Software engineering practices
@JulioBarros http://E-String.com 40
Don't Underestimate the Last Two
We need to consider running and maintaining the models in production.
@JulioBarros http://E-String.com 41
Approaches
— Python based environment
— Clojure based environment
— Generate C++ binaries
— API calls to third party
— API calls to microservices
@JulioBarros http://E-String.com 42
Challenges with DL
— Needs lots of data.Labeled data is expensive.
— Lacks explain-ability
— Performance requirements-training and inference
— Max performance unclear
— Best architecture unclear
@JulioBarros http://E-String.com 43
Benefits of DL
— Handles much of the feature engineering
— Handles complex (non linear) problems
— Advancements coming quickly
@JulioBarros http://E-String.com 44
Recommendations
Do not be intimidated by ANNs or the math.
Start with Keras (and Tensorflow or Theano) tutorials (or maybe Pytorch).
Later choose language/framework as needs dictate.
@JulioBarros http://E-String.com 45
Resources
Andrew Ng's Coursera Course and Fast.AI Mooc
Deep Learning Book-Goodfellow,Bengio and Courville
Meetups
-Portland-Data-Science-Group
-Portland-Machine-Learning-Meetup
-Portland-Deep-Learning
@JulioBarros http://E-String.com 46
Thank you! Questions?
Julio@E-String.com
@JulioBarros
@JulioBarros http://E-String.com 47

More Related Content

Similar to Intro to Deep Learning w/ Clojure

Getting Started with Machine Learning
Getting Started with Machine LearningGetting Started with Machine Learning
Getting Started with Machine Learning
Humberto Marchezi
 
How machine learning is changing the world
How machine learning is changing the worldHow machine learning is changing the world
How machine learning is changing the world
Emilio Garcia
 
How DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of GoHow DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of Go
Tim Riser
 
BWB Meetup: Storm - distributed realtime computation system
BWB Meetup: Storm - distributed realtime computation systemBWB Meetup: Storm - distributed realtime computation system
BWB Meetup: Storm - distributed realtime computation system
Andrii Gakhov
 
Deep Learning Demystified
Deep Learning DemystifiedDeep Learning Demystified
Deep Learning Demystified
Affine Analytics
 
Neural Nets Deconstructed
Neural Nets DeconstructedNeural Nets Deconstructed
Neural Nets Deconstructed
Paul Sterk
 
Artificial Intelligence (and the telecom industry)
Artificial Intelligence (and the telecom industry)Artificial Intelligence (and the telecom industry)
Artificial Intelligence (and the telecom industry)
Samuel Dratwa
 
FreddyAyalaTorchDomineering
FreddyAyalaTorchDomineeringFreddyAyalaTorchDomineering
FreddyAyalaTorchDomineering
FAYALA1987
 
introduction to deeplearning
introduction to deeplearningintroduction to deeplearning
introduction to deeplearning
Eyad Alshami
 
Transfer Learning _ Monk AI _ GCOEN.pptx
Transfer Learning _ Monk AI _ GCOEN.pptxTransfer Learning _ Monk AI _ GCOEN.pptx
Transfer Learning _ Monk AI _ GCOEN.pptx
GDSCGCOEN
 
[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo
JaeJun Yoo
 
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
inside-BigData.com
 
2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!
2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!
2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!
Bruno Capuano
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and Hype
Siby Jose Plathottam
 
Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)
Julien SIMON
 
Machine Learning @NECST
Machine Learning @NECSTMachine Learning @NECST
Machine Learning @NECST
NECST Lab @ Politecnico di Milano
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
Zeynep Su Kurultay
 
Sneaky computation
Sneaky computationSneaky computation
Sneaky computation
Juan J. Merelo
 
Artificial Intelligence for Undergrads
Artificial Intelligence for UndergradsArtificial Intelligence for Undergrads
Artificial Intelligence for Undergrads
Jose Berengueres
 
Introduction to Apache Storm
Introduction to Apache StormIntroduction to Apache Storm
Introduction to Apache Storm
Tiziano De Matteis
 

Similar to Intro to Deep Learning w/ Clojure (20)

Getting Started with Machine Learning
Getting Started with Machine LearningGetting Started with Machine Learning
Getting Started with Machine Learning
 
How machine learning is changing the world
How machine learning is changing the worldHow machine learning is changing the world
How machine learning is changing the world
 
How DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of GoHow DeepMind Mastered The Game Of Go
How DeepMind Mastered The Game Of Go
 
BWB Meetup: Storm - distributed realtime computation system
BWB Meetup: Storm - distributed realtime computation systemBWB Meetup: Storm - distributed realtime computation system
BWB Meetup: Storm - distributed realtime computation system
 
Deep Learning Demystified
Deep Learning DemystifiedDeep Learning Demystified
Deep Learning Demystified
 
Neural Nets Deconstructed
Neural Nets DeconstructedNeural Nets Deconstructed
Neural Nets Deconstructed
 
Artificial Intelligence (and the telecom industry)
Artificial Intelligence (and the telecom industry)Artificial Intelligence (and the telecom industry)
Artificial Intelligence (and the telecom industry)
 
FreddyAyalaTorchDomineering
FreddyAyalaTorchDomineeringFreddyAyalaTorchDomineering
FreddyAyalaTorchDomineering
 
introduction to deeplearning
introduction to deeplearningintroduction to deeplearning
introduction to deeplearning
 
Transfer Learning _ Monk AI _ GCOEN.pptx
Transfer Learning _ Monk AI _ GCOEN.pptxTransfer Learning _ Monk AI _ GCOEN.pptx
Transfer Learning _ Monk AI _ GCOEN.pptx
 
[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo[PR12] Inception and Xception - Jaejun Yoo
[PR12] Inception and Xception - Jaejun Yoo
 
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
 
2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!
2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!
2019 05 11 Chicago Codecamp - Deep Learning for everyone? Challenge Accepted!
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and Hype
 
Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)
 
Machine Learning @NECST
Machine Learning @NECSTMachine Learning @NECST
Machine Learning @NECST
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Sneaky computation
Sneaky computationSneaky computation
Sneaky computation
 
Artificial Intelligence for Undergrads
Artificial Intelligence for UndergradsArtificial Intelligence for Undergrads
Artificial Intelligence for Undergrads
 
Introduction to Apache Storm
Introduction to Apache StormIntroduction to Apache Storm
Introduction to Apache Storm
 

Recently uploaded

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
James Anderson
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Aggregage
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
Pixlogix Infotech
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 

Recently uploaded (20)

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionGenerative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 

Intro to Deep Learning w/ Clojure

  • 1. Intro to Deep Learning w/ Clojure It’s Difficult to Make Predictions. Especially About the Future. @JulioBarros Consultant E-String.com @JulioBarros http://E-String.com 1
  • 2. Hypothesis Given that Python dominates ML/DL; Is there a valid use case for Clojure? TL;DR --It depends @JulioBarros http://E-String.com 2
  • 3. I Hope So ...and I'm not the only one. @JulioBarros http://E-String.com 3
  • 4. Hype or Reality? @JulioBarros http://E-String.com 4
  • 5. "It is a renaissance, it is a golden age," "MachinelearningandAIisahorizontalenablinglayer.Itwillempowerand improveeverybusiness,everygovernmentorganization,everyphilanthropy —basicallythere'snoinstitutionintheworldthatcannotbeimprovedwith machinelearning." —Bezos @JulioBarros http://E-String.com 5
  • 6. Every industry can expect to be transformed by Artificial Intelligence @JulioBarros http://E-String.com 6
  • 7. Image Classification Justin Johnson,Andrej Karpathy,Li Fei-Fei-Stanford @JulioBarros http://E-String.com 7
  • 8. My dog Rex @JulioBarros http://E-String.com 8
  • 12. Healthcare Near or better than human level performance. @JulioBarros http://E-String.com 12
  • 13. Business — Law & Finance — Text,audio,image,video understanding — Churn prediction,customer segmentation — Product recommendations — Manufacturing,maintenance and control — Many many more @JulioBarros http://E-String.com 13
  • 14. Artificial Intelligence Artificial Intelligence (AI)-the study of"intelligent agents".Reasoning, knowledge representation,planning,robotics,etc. — Artificial Narrow Intelligence (ANI) — Artificial General Intelligence (AGI) — Artificial Superintelligence (ASI) @JulioBarros http://E-String.com 14
  • 15. Machine Learning Machine Learning (ML)-Programs that learn from the data and make predictions. @JulioBarros http://E-String.com 15
  • 16. Types of prediction Regression-continuous values Classification-discreet values/labels @JulioBarros http://E-String.com 16
  • 17. Types of ML Algorithms Supervised-trained on labeled data (regression or classification). Unsupervised-trained on unlabeled data (clustering,segmentation). Reinforcement-learns based on outcomes/results of actions. @JulioBarros http://E-String.com 17
  • 18. Deep Learning Deep Learning (DL)-ML/AIusingartificialneuralnetworks(ANNs) You might be thinking... What?Isn'tthatjustaneuralnet? Didn'tweshowthosedon'twork. Twice. @JulioBarros http://E-String.com 18
  • 19. Third time is a charm Capabilities of neural nets have changed dramatically due to advancements in: — Data — Algorithms — Hardware @JulioBarros http://E-String.com 19
  • 20. Neurons: Biologically inspired 1942 McCulloch and Pitts 1957 Rosenblatt (A (+ b (apply + (map * x w))) @JulioBarros http://E-String.com 20
  • 21. Activation Function Introduces non linearity — Historicaly: Identity,Step,Tanh,Sigmoid — Currently: Rectified linear Unit (relu),Softmax @JulioBarros http://E-String.com 21
  • 22. Universal Approximation Theorem (1989) ...a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions... @JulioBarros http://E-String.com 22
  • 23. Deep Neural Nets A net with more than one hidden layer. @JulioBarros http://E-String.com 23
  • 26. Training 1) Initialize weights randomly 2) Make prediction 3) Measure error (loss) 4) Adjust weights in the right direction 5) GOTO 2 Repeat over and over and over again with your training data @JulioBarros http://E-String.com 26
  • 27. Backpropagation Finding the right direction. @JulioBarros http://E-String.com 27
  • 28. Backpropagation To know the right direction calculate the gradient of the loss function with respect to each weight.Multiply by the error and a learning rate to get the delta. Don'tworry.Thelibrariesdoitforyou. @JulioBarros http://E-String.com 28
  • 29. Millions of Knobs Parameters @JulioBarros http://E-String.com 29
  • 30. A Simple Neural Net in Cortex (def network-architecture [(layers/input 11 1 1 :id :x) (layers/linear 64) (layers/relu) (layers/linear 1 :id :y)]) @JulioBarros http://E-String.com 30
  • 31. Wine Quality Data 11 features,1 target column @JulioBarros http://E-String.com 31
  • 32. UCI Machine Learning Repository https://archive.ics.uci.edu/ml/datasets/Wine+Quality Source: Paulo Cortez, University of Minho, Guimarães, Portugal, http://www3.dsi.uminho.pt/pcortez A.Cerdeira, F.Almeida, T.Matos and J.Reis, Viticulture Commission of the Vinho Verde Region(CVRVV), Porto, Portugal @2009 @JulioBarros http://E-String.com 32
  • 33. Normalized / Scaled data (Standardized) @JulioBarros http://E-String.com 33
  • 35. How Do We Work With Images Well,images are just numbers/data. Though numbers close to each other are more related. @JulioBarros http://E-String.com 35
  • 36. Convolutional Layers Similar to correlations from signal processing or filters from photoshop. A small NxN filter is slid over and convolved/correlated with the image. Learns to find features. Then lower level features are combined into higher level features. @JulioBarros http://E-String.com 36
  • 37. Common Types of Layers — input / output — fully connected (dense) — activation-relu,softmax — convolutional — maxpool — flatten — drop out @JulioBarros http://E-String.com 37
  • 38. Types of ANN 1. Dense Neural Net (DNN) 2. Convolutional Neural Net (CNN) 3. Recurrent Neural Net (RNN) 4. Everything else @JulioBarros http://E-String.com 38
  • 39. Programming Abstractions Level Python CLJ/JVM DSL/API Keras,Lasagne,TF-Slim Cortex,??? Computation Graph, Backprop,Autograd Tensorflow,Theano, Torch,Pytorch,Caffe, MXNet,CNTK Cortex,dl4j,Java bindings Matrix Math CUDA (cuDNN),Eigen3 Neandertal, core.matrix,vectorz-clj @JulioBarros http://E-String.com 39
  • 40. Production Considerations — Target hardware environment — GPU(s) — Powerful multicore server — Mobile device — Embedded — Desired latency / scalability — Data pipeline — Software engineering practices @JulioBarros http://E-String.com 40
  • 41. Don't Underestimate the Last Two We need to consider running and maintaining the models in production. @JulioBarros http://E-String.com 41
  • 42. Approaches — Python based environment — Clojure based environment — Generate C++ binaries — API calls to third party — API calls to microservices @JulioBarros http://E-String.com 42
  • 43. Challenges with DL — Needs lots of data.Labeled data is expensive. — Lacks explain-ability — Performance requirements-training and inference — Max performance unclear — Best architecture unclear @JulioBarros http://E-String.com 43
  • 44. Benefits of DL — Handles much of the feature engineering — Handles complex (non linear) problems — Advancements coming quickly @JulioBarros http://E-String.com 44
  • 45. Recommendations Do not be intimidated by ANNs or the math. Start with Keras (and Tensorflow or Theano) tutorials (or maybe Pytorch). Later choose language/framework as needs dictate. @JulioBarros http://E-String.com 45
  • 46. Resources Andrew Ng's Coursera Course and Fast.AI Mooc Deep Learning Book-Goodfellow,Bengio and Courville Meetups -Portland-Data-Science-Group -Portland-Machine-Learning-Meetup -Portland-Deep-Learning @JulioBarros http://E-String.com 46