Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
DEEP LEARNING GLOSSARY
DRIVERLESS CARS, BETTER PREVENTATIVE HEALTHCARE, AND
EVEN BETTER FASHION RECOMMENDATIONS ARE ALL POSSIBLE
TODAY BECAUSE OF...
OUR FRIENDS AT RE-WORK PUBLISHED AN “A” TO “Z” DEEP
LEARNING GLOSSARY. HERE ARE THE MOST IMPORTANT TERMS
LINKED WITH RESOU...
“PROCESSING DEVICES THAT ARE
LOOSELY MODELLED AFTER THE
NEURONAL STRUCTURE OF THE
HUMAN BRAIN.”
SOURCE: UNIVERSITY OF WISC...
“DESCRIBES A LARGE VOLUME OF DATA
– BOTH STRUCTURED AND
UNSTRUCTURED – THAT INUNDATES A
BUSINESS ON A DAY-TO-DAY BASIS.”
S...
“COMPRISED OF ONE OR MORE
CONVOLUTIONAL LAYERS AND THEN
FOLLOWED BY ONE OR MORE FULLY
CONNECTED LAYERS AS IN A
STANDARD MU...
“FORM OF MACHINE LEARNING THAT
ENABLES COMPUTERS TO LEARN FROM
EXPERIENCE AND UNDERSTAND THE
WORLD IN TERMS OF A HIERARCHY...
“IS A REPRESENTATION OF INPUT, OR
AN ENCODING. FOR EXAMPLE, A
NEURAL WORD EMBEDDING IS A
VECTOR THAT REPRESENTS THAT
WORD....
“ALLOW SIGNALS TO TRAVEL ONE WAY
ONLY; FROM INPUT TO OUTPUT.
THERE IS NO FEEDBACK (LOOPS) I.E.
THE OUTPUT OF ANY LAYER DOE...
“TYPE OF AI ALGORITHMS USED IN
UNSUPERVISED MACHINE LEARNING,
IMPLEMENTED BY A SYSTEM OF TWO
NEURAL NETWORKS COMPETING
AGA...
“ARE AN ARCHITECTURE TO LET
INFORMATION FLOW UNHINDERED
ACROSS SEVERAL RNN LAYERS ON SO-
CALLED “INFORMATION HIGHWAYS.”
SO...
“TRAINING DEEP MODELS ARE SUFFICIENTLY DIFFICULT TASKS. MOST
ALGORITHMS ARE STRONGLY AFFECTED BY THE CHOICE OF
INITIALIZAT...
“AN ARTIFICIAL NOISE ADDED TO THE
INPUTS DURING TRAINING USED AS
ANOTHER METHOD FOR REGULARIZATION
AND IMPROVING GENERALIZ...
“IS A TYPE OF UNSUPERVISED LEARNING, WHICH IS USED WHEN
YOU HAVE UNLABELED DATA (I.E., DATA WITHOUT DEFINED
CATEGORIES OR ...
“FOR EACH PREDICTION, THERE IS AN
ASSOCIATED NUMBER WHICH IS THE
LOSS. FOR A TRUE PREDICTION, THE
LOSS WILL BE SMALL AND F...
“IS A FEEDFORWARD NEURAL NETWORK WITH MULTIPLE FULLY-
CONNECTED LAYERS THAT USE NONLINEAR ACTIVATION FUNCTIONS
TO DEAL WIT...
“IS THE COMPREHENSION BY COMPUTERS OF THE STRUCTURE AND
MEANING OF HUMAN LANGUAGE (E.G., ENGLISH, SPANISH,
JAPANESE), ALLO...
“ONE-SHOT LEARNING IS WHEN AN
ALGORITHM LEARNS FROM ONE OR A FEW
NUMBER OF TRAINING EXAMPLES, CONTRAST
TO THE TRADITIONAL ...
“TYPE OF LAYER COMMONLY FOUND
IN CONVOLUTIONAL NEURAL
NETWORKS, WHICH INTEGRATES
INFORMATION FROM NEURONS WITH
NEARBY RECE...
“A NOVEL ARTIFICIAL AGENT, TERMED A DEEP Q-NETWORK, THAT
CAN LEARN SUCCESSFUL POLICIES DIRECTLY FROM HIGH-
DIMENSIONAL SEN...
“A BRANCH OF MACHINE LEARNING THAT
IS GOAL ORIENTATED; THAT IS,
REINFORCEMENT LEARNING ALGORITHMS
HAVE AS THEIR OBJECT TO ...
“IS A FUNCTION USED AS THE OUTPUT LAYER OF A NEURAL NETWORK
THAT CLASSIFIES INPUT. IT CONVERTS VECTORS INTO CLASS
PROBABIL...
“ALLOWS US TO [TRAIN NEW MODELS] BY LEVERAGING THE ALREADY
EXISTING LABELED DATA OF SOME RELATED TASK OR DOMAIN. WE TRY
TO...
“IS A TYPE OF MACHINE LEARNING
ALGORITHM USED TO DRAW INFERENCES
FROM DATASETS CONSISTING OF INPUT
DATA WITHOUT LABELED RE...
“IS A DIRECTED MODEL THAT USES
LEARNED APPROXIMATE INFERENCE
AND CAN BE TRAINED PURELY WITH
GRADIENT-BASED METHODS.”
SOURC...
“TO PENALIZE LARGE WEIGHTS USING
PENALTIES OR CONSTRAINTS ON THEIR
SQUARED VALUES (L2 PENALTY) OR
ABSOLUTE VALUES (L1 PENA...
“THE PROCESS OF INITIALIZING WEIGHTS THAT THE VARIANCE
REMAINS THE SAME FOR “X” AND “Y”. THIS INITIALIZATION
PROCESS IS KN...
“AS PIONEERS OF DEEP LEARNING, YOSHUA & YANN HAVE MADE
SIGNIFICANT CONTRIBUTIONS TO THE ADVANCEMENTS OF THIS FIELD.
THEY W...
“AN EXTREME FORM OF TRANSFER
LEARNING, WHERE NO LABELED
EXAMPLES ARE GIVEN AT ALL FOR THE
ZERO-SHOT LEARNING TASK.”
SOURCE...
Explore Now
DISCOVER THE LATEST NEWS AND
UPDATES ON DEEP LEARNING AND AI …
Upcoming SlideShare
Loading in …5
×

The Deep Learning Glossary

17,336 views

Published on

Learn the most important terminology from "A" to "Z" utilized in deep learning linked with resources for more in-depth exploration in our glossary.

Published in: Technology

The Deep Learning Glossary

  1. 1. DEEP LEARNING GLOSSARY
  2. 2. DRIVERLESS CARS, BETTER PREVENTATIVE HEALTHCARE, AND EVEN BETTER FASHION RECOMMENDATIONS ARE ALL POSSIBLE TODAY BECAUSE OF DEEP LEARNING.
  3. 3. OUR FRIENDS AT RE-WORK PUBLISHED AN “A” TO “Z” DEEP LEARNING GLOSSARY. HERE ARE THE MOST IMPORTANT TERMS LINKED WITH RESOURCES FOR MORE IN-DEPTH EXPLORATION …
  4. 4. “PROCESSING DEVICES THAT ARE LOOSELY MODELLED AFTER THE NEURONAL STRUCTURE OF THE HUMAN BRAIN.” SOURCE: UNIVERSITY OF WISCONSIN-MADISON ARTIFICIAL NEURAL NETWORKS (ANN’S) NVIDIA Applied Research Compressing DNA Engine Facial Performance Video Classification
  5. 5. “DESCRIBES A LARGE VOLUME OF DATA – BOTH STRUCTURED AND UNSTRUCTURED – THAT INUNDATES A BUSINESS ON A DAY-TO-DAY BASIS.” SOURCE: SAS INSIGHTS BIG DATA AI-Accelerated Analytics For Industries Finance Telco IoT
  6. 6. “COMPRISED OF ONE OR MORE CONVOLUTIONAL LAYERS AND THEN FOLLOWED BY ONE OR MORE FULLY CONNECTED LAYERS AS IN A STANDARD MULTILAYER NEURAL NETWORK.” SOURCE: UFLDL CONVOLUTIONAL NEURAL NETWORKS NVIDIA Applied Research Detection & Classification of Hand Gestures Resource Efficient Inference
  7. 7. “FORM OF MACHINE LEARNING THAT ENABLES COMPUTERS TO LEARN FROM EXPERIENCE AND UNDERSTAND THE WORLD IN TERMS OF A HIERARCHY OF CONCEPTS.” SOURCE: GOODFELLOW, I., BENGIO, T., COURVILLE, A. DEEP LEARNING Deep Learning Applications Transforming Finance Detecting Cancer Deep Learning In Self-Driving Cars
  8. 8. “IS A REPRESENTATION OF INPUT, OR AN ENCODING. FOR EXAMPLE, A NEURAL WORD EMBEDDING IS A VECTOR THAT REPRESENTS THAT WORD.” SOURCE: DEEPLEARNING4J EMBEDDING Research Publication Generating An Embedded Neural Network
  9. 9. “ALLOW SIGNALS TO TRAVEL ONE WAY ONLY; FROM INPUT TO OUTPUT. THERE IS NO FEEDBACK (LOOPS) I.E. THE OUTPUT OF ANY LAYER DOES NOT AFFECT THAT SAME LAYER.” SOURCE: IMPERIAL COLLEGE FEEDFORWARD NETWORK Tutorial Creating a Feed-Forward Network With DIGITS
  10. 10. “TYPE OF AI ALGORITHMS USED IN UNSUPERVISED MACHINE LEARNING, IMPLEMENTED BY A SYSTEM OF TWO NEURAL NETWORKS COMPETING AGAINST EACH OTHER IN A ZERO-SUM GAME FRAMEWORK.” SOURCE: IAN GOODFELLOW GENERATIVE ADVERSARIAL NETWORKS Origin of Generative Adversarial Networks Ian Goodfellow Podcast GTC 17 Talk – Generative Adversarial Networks
  11. 11. “ARE AN ARCHITECTURE TO LET INFORMATION FLOW UNHINDERED ACROSS SEVERAL RNN LAYERS ON SO- CALLED “INFORMATION HIGHWAYS.” SOURCE: DEEPLEARNING4J HIGHWAY NETWORKS Research Publication Highway Networks for Visual Questions Answering
  12. 12. “TRAINING DEEP MODELS ARE SUFFICIENTLY DIFFICULT TASKS. MOST ALGORITHMS ARE STRONGLY AFFECTED BY THE CHOICE OF INITIALIZATION. THE INITIAL POINT CAN DETERMINE WHETHER THE ALGORITHM CONVERGES AT ALL, WITH SOME INITIAL POINTS BEING SO UNSTABLE THAT THE ALGORITHM ENCOUNTERS NUMERICAL DIFFICULTIES AND FAILS ALTOGETHER.” SOURCE: IAN GOODFELLOW INITIALIZATION Article Weight Initialization In Deep Neural Networks
  13. 13. “AN ARTIFICIAL NOISE ADDED TO THE INPUTS DURING TRAINING USED AS ANOTHER METHOD FOR REGULARIZATION AND IMPROVING GENERALIZATION OF A NEURAL NETWORK.” SOURCE: MICHAEL BRAGISNKEY, CTO AT AIDOC JITTER Tutorial What is Jitter?
  14. 14. “IS A TYPE OF UNSUPERVISED LEARNING, WHICH IS USED WHEN YOU HAVE UNLABELED DATA (I.E., DATA WITHOUT DEFINED CATEGORIES OR GROUPS). THE GOALS OF THIS ALGORITHM IS TO FIND GROUPS IN THE DATA, WITH THE NUMBER OF GROUPS REPRESENTED BY THE VARIABLE K.” SOURCE: DATA SCIENCE K-MEANS ALGORITHM Tutorial CUDA Implementation of the K-Means Clustering Algorithm
  15. 15. “FOR EACH PREDICTION, THERE IS AN ASSOCIATED NUMBER WHICH IS THE LOSS. FOR A TRUE PREDICTION, THE LOSS WILL BE SMALL AND FOR A TOTALLY WRONG PREDICTION THE LOSS WILL BE HIGH.” SOURCE: MICHAEL BRAGINSKY, CTO AT AIDOC LOSS FUNCTION NVIDIA Applied Research Loss Function for Image Restoration
  16. 16. “IS A FEEDFORWARD NEURAL NETWORK WITH MULTIPLE FULLY- CONNECTED LAYERS THAT USE NONLINEAR ACTIVATION FUNCTIONS TO DEAL WITH DATA WHICH IS NOT LINEARLY SEPARABLE. AN MLP IS THE MOST BASIC FORM OF A MULTILAYER NEURAL NETWORK, OR A DEEP NEURAL NETWORKS IF IT HAS MORE THAN 2 LAYERS.” SOURCE: DEEPLEARNING.NET MULTILAYER PERCEPTRON (MLP) Research Publication Multi-Layer Perceptron On A GPU
  17. 17. “IS THE COMPREHENSION BY COMPUTERS OF THE STRUCTURE AND MEANING OF HUMAN LANGUAGE (E.G., ENGLISH, SPANISH, JAPANESE), ALLOWING USERS TO INTERACT WITH THE COMPUTER USING NATURAL SENTENCES.” SOURCE: GARTNER RESEARCH NATURAL LANGUAGE PROCESSING Blog NVIDIA Developer – Natural Language Processing
  18. 18. “ONE-SHOT LEARNING IS WHEN AN ALGORITHM LEARNS FROM ONE OR A FEW NUMBER OF TRAINING EXAMPLES, CONTRAST TO THE TRADITIONAL MACHINE-LEARNING MODELS WHICH USES THOUSANDS EXAMPLES IN ORDER TO LEARN..” SOURCE: SUSHOVAN HALDAR ONE-SHOT LEARNING Research Publication One-Shot Imitation Learning with OpenAI & Berkeley
  19. 19. “TYPE OF LAYER COMMONLY FOUND IN CONVOLUTIONAL NEURAL NETWORKS, WHICH INTEGRATES INFORMATION FROM NEURONS WITH NEARBY RECEPTIVE FIELDS.” SOURCE: MICHAEL BRAGINSKY, CTA AT AIDOC POOLING Blog ParallelForall – Deep Learning In a Nutshell Core Concepts
  20. 20. “A NOVEL ARTIFICIAL AGENT, TERMED A DEEP Q-NETWORK, THAT CAN LEARN SUCCESSFUL POLICIES DIRECTLY FROM HIGH- DIMENSIONAL SENSORY INPUTS USING END-TO-END REINFORCEMENT LEARNING.” SOURCE: PETERSEN, S. (2015) HUMAN-LEVEL CONTROL THROUGH DEEP REINFORCEMENT LEARNING. Q-NETWORKS Blog Q-Network Trained to Play Breakout on OpenAI Gym
  21. 21. “A BRANCH OF MACHINE LEARNING THAT IS GOAL ORIENTATED; THAT IS, REINFORCEMENT LEARNING ALGORITHMS HAVE AS THEIR OBJECT TO MAXIMIZE A REWARD, OFTEN OVER THE COURSE OF MANY DECISIONS.” SOURCE: DEEPLEARNING4J REINFORCEMENT LEARNING Blog Deep Learning in a Nutshell: Reinforcement Learning
  22. 22. “IS A FUNCTION USED AS THE OUTPUT LAYER OF A NEURAL NETWORK THAT CLASSIFIES INPUT. IT CONVERTS VECTORS INTO CLASS PROBABILITIES. SOFTMAX NORMALIZES THE VECTOR OF SCORES BY FIRST EXPONENTIATING AND THEN DIVIDING BY A CONSTANT.” SOURCE: DEEPLEARNING4J SOFTMAX REGRESSION Tutorial Stanford – Softmax Regression
  23. 23. “ALLOWS US TO [TRAIN NEW MODELS] BY LEVERAGING THE ALREADY EXISTING LABELED DATA OF SOME RELATED TASK OR DOMAIN. WE TRY TO STORE THIS KNOWLEDGE GAINED IN SOLVING THE SOURCE TASK IN THE SOURCE DOMAIN AND APPLY IT TO OUR PROBLEM OF INTEREST.” SOURCE: SEBASTIAN RUDER TRANSFER LEARNING Research Publication Transfer Learning From Deep Features for Remote Sensing and Poverty Mapping
  24. 24. “IS A TYPE OF MACHINE LEARNING ALGORITHM USED TO DRAW INFERENCES FROM DATASETS CONSISTING OF INPUT DATA WITHOUT LABELED RESPONSES.” SOURCE: MATHWORKS UNSUPERVISED LEARNING Blog Using Unsupervised Learning For Artistic Style
  25. 25. “IS A DIRECTED MODEL THAT USES LEARNED APPROXIMATE INFERENCE AND CAN BE TRAINED PURELY WITH GRADIENT-BASED METHODS.” SOURCE: GOODFELLOW, IAN VARIATIONAL AUTOENCODER Blog Auto-Encoder Model Querying a Computer To Design Clothing
  26. 26. “TO PENALIZE LARGE WEIGHTS USING PENALTIES OR CONSTRAINTS ON THEIR SQUARED VALUES (L2 PENALTY) OR ABSOLUTE VALUES (L1 PENALTY).” SOURCE: HINTON, . G. NEURAL NETWORKS FOR MACHINE LEARNING WEIGHT DECAY Research Publication Weight Decay Can Improve Generalization
  27. 27. “THE PROCESS OF INITIALIZING WEIGHTS THAT THE VARIANCE REMAINS THE SAME FOR “X” AND “Y”. THIS INITIALIZATION PROCESS IS KNOWN AS XAVIER INITIALIZATION.” SOURCE: PRATEEK JOSHI XAVIER INITIALIZATION Research Publications Understanding the Difficulty of Training Deep Feedforward Neural Networks
  28. 28. “AS PIONEERS OF DEEP LEARNING, YOSHUA & YANN HAVE MADE SIGNIFICANT CONTRIBUTIONS TO THE ADVANCEMENTS OF THIS FIELD. THEY WILL BOTH BE JOINING THE DEEP LEARNING SUMMIT, MONTREAL IN OCTOBER, SPEAKING ON THE “PANEL OF PIONEERS.” SOURCE: RE-WORK GLOSSARY YOSHUA BENGIO & Their Work Advancing Robotics, Physics, and Medicine with AI – Yann LeCunn YANN LECUN The Rise of AI Through Deep Learning – Yoshua Bengio
  29. 29. “AN EXTREME FORM OF TRANSFER LEARNING, WHERE NO LABELED EXAMPLES ARE GIVEN AT ALL FOR THE ZERO-SHOT LEARNING TASK.” SOURCE: GOODFELLOW, IAN ZERO-SHOT LEARNING Research Publication Zero-Shot Learning Through Cross-Modal Transfer
  30. 30. Explore Now DISCOVER THE LATEST NEWS AND UPDATES ON DEEP LEARNING AND AI …

×