The document introduces deep learning, highlighting its capabilities in achieving human-like performance in tasks such as natural language understanding and video recognition through neural networks. It discusses the construction of classifiers to predict outcomes using linear equations and the need for non-linear functions to enhance model efficiency, leading to deep neural networks. The text emphasizes the advantages of using deep networks, particularly in handling complex problems like language and vision, and briefly describes convolutional networks for visual recognition.

1. An introduction to Deep Learning

2. Who am I?
• David Rostcheck
• I am a data science
consultant
• Follow my articles on
LinkedIn

3. DEEP LEARNING

4. in some tests, Deep Learning has already
shown abilities at the same level as humans

5. These include:
• computers that understand natural
language
• autonomous vehicles
• programs that can identify what is
occurring in a video

6. It’s notable that
these solutions to diverse
in very different
use the same powerful technology

7. NEURAL NET

8. a neural net is a
simulation
of the brain,
a mathematical abstraction

9. in the real brain,
the neurons send signals with
fre cuen cies
not discrete signals

10. tools exist that try to simulate the brain in a
way that’s
more accurate
to the real brain

11. Example: Numenta NuPIC, a type of Hierarchical
Temporal Memory (HTM)

12. but the techniques of neural nets
are sufficient
to deliver results
similar or better than humans
in specific cognitive tests

13. therefore:
Deep Learning
what is it?

14. common point of view:
a with
neural distinct
net levels
is correct, but…

15. there is another point of view,
maybe more useful,
that we are going to present here

16. it comes from Vincent Vanhoucke, Principal
Research Scientist at Google.
the following
comes from
his course on
Deep
Learning, on
Udacity

17. He thinks about Deep Learning as
a framework for calculating
linear and almost linear
equations in an efficient way

18. to develop this framework,
we are going to construct a
classifier
the simplest (and worst)
possible

19. but wait a minute…
why
a classifier?

20. Because classification (or more
generally prediction) is a central
technique in Machine Learning
with this, we can achieve ranking,
regression, detection, reinforcement
learning, and more…

21. we start with a linear equation, in vector
form…

22. Think about constructing a simple classifier to
predict, for each occurrence of X, which is:

23. to do this, we must learn the values of W and b

24. Does it work well?

25. No.
It’s the worst.

26. Why?

27. there are two problems…

28. No. 1:
it gives values,
and what we want
are probabilities

29. we can fix it with the“softmax” function:

30. we express the
correct values in a
vector of values 1
(correct) and 0 (the
others).
we call this“one-hot
encoding”

31. to evaluate errors, we
compare the probabilities
with the correct values

32. using what we call“cross-entropy”

33. better, but…
there remains the second problem:
our equation is linear
and doesn’t represent non-linear
equations well

35. this problem killed the perceptron (single
level neural net)

36. it doesn’t help to just add levels to the network
because we can represent whatever combination
of linear operations as another linear
operation – we can reduce the new network to
another WX + b with the same problem 

37. What do we do?

38. without another option,
we have to introduce non-linear
functions
logistic
function

39. but it’s expensive to calculate – we can use a
simplified approximation called a “Rectified
Linear Unit” , o ReLU

40. now we can construct our neural net, in a way
that’s efficient to calculate

41. we can express this in a modular , with a
series of linear or almost linear operations with
a matrix ... that allows us to us the power of a
GPU

42. this is good, but we are still lacking
something…
to improve our estimation, we must
minimize the error,
and this requires us to calculate the
derivative of the function

43. think about the chain rule of calculus:
d f(x) = d du f(x)
dx du dx

44. that can convert a derivative into a
product (of other derivatives):

45. that fits in our modular framework 

46. now we have it! a general, modular
framework that incorporates
everything we need!

47. and we can construct deep neural nets,
adding more levels as we need them
…but wait a minute:
why do we like deep networks?

48. the most interesting problems,
like language and vision,
have very complex rules
we need a lot of parameters to represent
them

49. yes, but why don’t we use wider
networks?
why is it better to have deep ones?

50. are more efficient and better capture the structure
inherent in many problems

51. CONVNETS

52. the convolutional network, or convnet,
transforms the input
so that the translation
of the input does not matter
we use it for visual recognition

53. Let’s start with a photo:

54. We use a region (kernel) of a photo like an input to
another small neural net, with K as the output

55. we slice the window across the photo

56. this transforms the photo into another new one,
with K color channels, and different dimensions

57. this operation is called
a convolution

58. if the region (the “kernel”) has
the same size as the original,
what did we obtain?
?

59. in this case,
we recover the original photo

60. Questions?
Contact: drostcheck@leopardllc.com, twitter: @davidrostcheck
Articles: http://linkedin.com/in/davidrostcheck