1.
Outline
• Class organization
• Artificial Neural networks
– Introduction
– History of Artificial Neural Networks
– Applications
• Survey
2.
1/29/2015 CAP 5615: Introduction to Neural Networks 2
Programming Approaches
• Programming approaches
– Abstract the problem
– Design an algorithm to solve the problem
• An algorithm is a precise step-by-step procedure to solve
the problem
– Implement the algorithm
• Artificial neural network approaches for solving
problems are different than typical programming
approaches in computer science
3.
1/29/2015 CAP 5615: Introduction to Neural Networks 3
An Example
• No hands across America
– Sponsored by Delco Electronics, AssistWare
Technology, and Carnegie Mellon University
– Navlab 5 drove from Pittsburgh, PA to San
Diego, CA, using the RALPH computer
program.
– The trip was 2849 miles of which 2797 miles
were driven automatically with no hands
• Which is 98.2%
4.
1/29/2015 CAP 5615: Introduction to Neural Networks 4
An Example – cont.
5.
1/29/2015 CAP 5615: Introduction to Neural Networks 5
An Example – cont.
6.
1/29/2015 CAP 5615: Introduction to Neural Networks 6
An Example – cont.
• RALPH
– Rapidly Adapting Lateral Position Handler
– Uses video images to determine the location of
the road ahead and the appropriate steering
direction to keep the vehicle on the road
– It uses a multi-layer perceptron with
backpropagation as the learning algorithm
7.
1/29/2015 CAP 5615: Introduction to Neural Networks 7
An Example – cont.
8.
1/29/2015 CAP 5615: Introduction to Neural Networks 8
Artificial Neural Network Approaches
• Artificial neural network approaches
– Motivated by the biological counterpart
• The brain, which has about 1011 neurons
• All biological neural functions including learning and
memory are stored in the neurons and the connections
between them
• The brain is also called “biological neural network”
– Try to “learn” from examples
9.
1/29/2015 CAP 5615: Introduction to Neural Networks 9
Artificial Neural Networks – cont.
• Inspiration from neuroscience
– The brain consists of 1011 highly interconnected
neurons
10.
1/29/2015 CAP 5615: Introduction to Neural Networks 10
Artificial Neural Networks – cont.
• Each neuron is a complicated process
– The best available mathematical model is called
Hodgkin-Huxley Model
• We are not interested in recreating the
neurons but trying to abstract their main
functionalities
11.
1/29/2015 CAP 5615: Introduction to Neural Networks 11
McCulloch and Pitts Model
• McCulloch and Pitts Model
– It is a simple model as a binary threshold unit
– The model neuron first computes a weighted sum
of its inputs
– It outputs one if the weighted sum is above a
threshold and zero otherwise
otherwise0
0if1
(x)
))(()1(
x
tnwtn
j
ijiji
12.
1/29/2015 CAP 5615: Introduction to Neural Networks 12
McCulloch and Pitts Model – cont.
wi1
win
13.
1/29/2015 CAP 5615: Introduction to Neural Networks 13
McCulloch and Pitts Model – cont.
• McCulloch and Pitts networks (consisting of
multiple layers of McCulloch and Pitts
neurons) are Turing-machine equivalent in
turns of computation capability
– In other words, a synchronous assembly of
McCulloch-Pitts neurons can perform any
computation that an ordinary digital computer
can, though they can be slow or cumbersome.
14.
1/29/2015 CAP 5615: Introduction to Neural Networks 14
Digital Computers
• All digital computers consist of digital gates
to control the dataflow and perform
computation
– How many different types of logic gates do we
need in order to be able to build any computer?
15.
1/29/2015 CAP 5615: Introduction to Neural Networks 15
McCulloch and Pitts Model – cont.
16.
1/29/2015 CAP 5615: Introduction to Neural Networks 16
History of Artificial Neural Networks
• General theories of learning
– Late 19th and early 20th centuries
– Interdisciplinary in physics, psychology, and
neurophysiology
– Hermann Helmholtz, Ernst Mack, Ivan Pavlov
17.
1/29/2015 CAP 5615: Introduction to Neural Networks 17
History of Artificial Neural Networks – cont.
• McCulloch and Pitts Model
– 1940s
– Artificial neural networks in principle could
compute any arithmetic and logical functions
• Origin of the modern neural networks
– Warren McCulloch and Walter Pitts
– McCulloch and Pitts network cannot learn
• A network must be designed for each function
• For computers, we write a program for each function
18.
1/29/2015 CAP 5615: Introduction to Neural Networks 18
History of Artificial Neural Networks – cont.
• Hebbian learning rule
– 1949
– A mechanism for learning in biological neurons
• The first learning algorithm
– Donald Hebb
19.
1/29/2015 CAP 5615: Introduction to Neural Networks 19
History of Artificial Neural Networks – cont.
• Perceptron
– 1950s
– A network can learn to solve pattern recognition
problems
– Frank Rosenblatt
– Unfortunately, it only solve a limited class of
problems
• Linear separability
20.
1/29/2015 CAP 5615: Introduction to Neural Networks 20
History of Artificial Neural Networks – cont.
• Widrow-Hoff learning rule
– 1960s
– Similar to perceptron
• Widely used in adaptive signal processing
• Most long distance phone lines use Widrow-Hoff for
echo cancellation
21.
1/29/2015 CAP 5615: Introduction to Neural Networks 21
History of Artificial Neural Networks – cont.
• Minsky and Papert’s book
– 1969
– Pointed out the intrinsic limitations of
Perceptrons and neural networks at that time
• The neural network area became quiet for almost
twenty years
22.
1/29/2015 CAP 5615: Introduction to Neural Networks 22
History of Artificial Neural Networks – cont.
• New Stage
– 1980s and 1990s
– New powerful computers
– Two new concepts
• Statistical mechanics to analyze neural network
behaviors
• Backpropagation algorithm
– Overcome the limitation of perceptron
23.
1/29/2015 CAP 5615: Introduction to Neural Networks 23
History of Artificial Neural Networks – cont.
• Current stage
– A standard tool to solve many practical problems
– Theoretical and statistical foundations of neural
networks
– New neural network architectures and learning
algorithms
• Support vector machines
• Adaboost
24.
1/29/2015 CAP 5615: Introduction to Neural Networks 24
Applications
• Aerospace
• Banking
• Defense
• Financial
• Medical
• Securities
• Transportation
• See http://www.calsci.com/
25.
1/29/2015 CAP 5615: Introduction to Neural Networks 25
Financial Forecasting
• Use neural networks to predict stock market
trends
• Use neural networks to perform credit
scoring
• Use neural networks to perform fraud
detection
• See http://www.calsci.com/Applications.html#Stock%20Applications
26.
1/29/2015 CAP 5615: Introduction to Neural Networks 26
Online Neural Network Resources
• Neural Java
– Neural network tutorial with Java
• Neural network resources
– http://www.makhfi.com/resources.htm
27.
1/29/2015 CAP 5615: Introduction to Neural Networks 27
Topics to Be Covered
• Basic neural network architectures and
learning algorithms
– Perceptron
– Multiple layer perceptron and backpropagation
– Associate learning
– Competitive networks
– Hopfield network
– These topics will be on the exams
28.
1/29/2015 CAP 5615: Introduction to Neural Networks 28
Topics to Be Covered – cont.
• Advanced topics
– Will NOT be on the exams but very important to
have a comprehensive understanding of the area
– Radial basis neural networks
– Support vector machines
– AdaBoost
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.
Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.
Be the first to comment