it is about neural network no linear hypothisis

1. Neural Networks:
Representation
Non-linear
hypotheses
Machine Learning

2. Andrew Ng
Non-linear Classification
x1
x2
size
# bedrooms
# floors
age

3. Andrew Ng
You see this:
But the camera sees this:
What is this?

4. Andrew Ng
Computer Vision: Car detection
Testing:
What is this?
Not a car
Cars

5. Andrew Ng
Learning
Algorithm
pixel 1
pixel 2
pixel 1
pixel 2
Raw image
Cars
“Non”-Cars

6. Andrew Ng
pixel 1
pixel 2
Raw image
Cars
“Non”-Cars
Learning
Algorithm
pixel 1
pixel 2

7. Andrew Ng
pixel 1
pixel 2
Raw image
Cars
“Non”-Cars
50 x 50 pixel images→ 2500 pixels
(7500 if RGB)
pixel 1 intensity
pixel 2 intensity
pixel 2500 intensity
Quadratic features ( ): ≈3 million
features
Learning
Algorithm
pixel 1
pixel 2

9. Neural Networks:
Representation
Neurons and
the brain
Machine Learning

10. Andrew Ng
Neural Networks
Origins: Algorithms that try to mimic the brain.
Was very widely used in 80s and early 90s; popularity
diminished in late 90s.
Recent resurgence: State-of-the-art technique for many
applications

11. Andrew Ng
Auditory cortex learns to see
Auditory Cortex
The “one learning algorithm” hypothesis
[Roe et al., 1992]

12. Andrew Ng
Somatosensory cortex learns to see
Somatosensory Cortex
The “one learning algorithm” hypothesis
[Metin & Frost, 1989]

13. Andrew Ng
Seeing with your tongue Human echolocation (sonar)
Haptic belt: Direction sense Implanting a 3rd eye
Sensor representations in the brain
[BrainPort; Welsh & Blasch, 1997; Nagel et al., 2005; Constantine-Paton & Law, 2009]

15. Neural Networks:
Representation
Machine Learning
Model
representation I

16. Andrew Ng
Neuron in the brain

17. Andrew Ng
Neurons in the brain
[Credit: US National Institutes of Health, National Institute on Aging]

18. Andrew Ng
Neuron model: Logistic unit
Sigmoid (logistic) activation function.

19. Andrew Ng
Neural Network
Layer 3
Layer 1 Layer 2

20. Andrew Ng
Neural Network
“activation” of unit in layer
matrix of weights controlling
function mapping from layer to
layer
If network has units in layer , units in layer , then
will be of dimension .

22. Neural Networks:
Representation
Model
representation II
Machine Learning

23. Andrew Ng
Add .
Forward propagation: Vectorized implementation

24. Andrew Ng
Layer 3
Layer 1 Layer 2
Neural Network learning its own features

25. Andrew Ng
Layer 3
Layer 1 Layer 2
Other network architectures
Layer 4

27. Neural Networks:
Representation
Examples and
intuitions I
Machine Learning

28. Andrew Ng
Non-linear classification example: XOR/XNOR
, are binary (0 or 1).
x1
x2
x1
x2

29. Andrew Ng
Simple example: AND
0 0
0 1
1 0
1 1
1.0

30. Andrew Ng
Example: OR function
0 0
0 1
1 0
1 1
-10
20
20

32. Neural Networks:
Representation
Examples and
intuitions II
Machine Learning

33. Andrew Ng
Negation:
0
1

34. Andrew Ng
Putting it together:
0 0
0 1
1 0
1 1
-30
20
20
10
-20
-20
-10
20
20

35. Andrew Ng
Neural Network intuition
Layer 3
Layer 1 Layer 2 Layer 4

36. Andrew Ng
Handwritten digit classification
[Courtesy of Yann LeCun]

37. Andrew Ng
Handwritten digit classification
[Courtesy of Yann LeCun]

38. Andrew Ng

39. Andrew Ng
Neural Networks:
Representation
Multi-class
classification
Machine Learning

40. Andrew Ng
Multiple output units: One-vs-all.
Pedestrian Car Motorcycle Truck
Want , , , etc.
when pedestrian when car when motorcycle

41. Andrew Ng
Multiple output units: One-vs-all.
Want , , , etc.
when pedestrian when car when motorcycle
Training set:
one of , , ,
pedestrian car motorcycle truck

42. Andrew Ng