This presentation guide you through Neural Networks, use neural networksNeural Networks v/s Conventional
Computer, Inspiration from Neurobiology, Types of neural network, The Learning Process, Hetero-association recall mechanisms and Key Features,
For more topics stay tuned with Learnbay.
An Neural Network (NN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information.
It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems.
An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process.
An artificial neuron is a device with many inputs and one output. The neuron has two modes of operation; the training mode and the using mode. In the training mode, the neuron can be trained to fire (or not), for particular input patterns.
In the using mode, when a taught input pattern is detected at the input, its associated output becomes the current output.
Basics of Neural networks and its image recognition and its applications of engineering fields and medicines and how it detect those images and give the results of those images....
This presentation guide you through Neural Networks, use neural networksNeural Networks v/s Conventional
Computer, Inspiration from Neurobiology, Types of neural network, The Learning Process, Hetero-association recall mechanisms and Key Features,
For more topics stay tuned with Learnbay.
An Neural Network (NN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information.
It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems.
An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process.
An artificial neuron is a device with many inputs and one output. The neuron has two modes of operation; the training mode and the using mode. In the training mode, the neuron can be trained to fire (or not), for particular input patterns.
In the using mode, when a taught input pattern is detected at the input, its associated output becomes the current output.
Basics of Neural networks and its image recognition and its applications of engineering fields and medicines and how it detect those images and give the results of those images....
Artificial Neural Network and its Applicationsshritosh kumar
Abstract
This report is an introduction to Artificial Neural
Networks. The various types of neural networks are
explained and demonstrated, applications of neural
networks like ANNs in medicine are described, and a
detailed historical background is provided. The
connection between the artificial and the real thing is
also investigated and explained. Finally, the
mathematical models involved are presented and
demonstrated.
Neural Network and Artificial Intelligence.
Neural Network and Artificial Intelligence.
WHAT IS NEURAL NETWORK?
The method calculation is based on the interaction of plurality of processing elements inspired by biological nervous system called neurons.
It is a powerful technique to solve real world problem.
A neural network is composed of a number of nodes, or units[1], connected by links. Each linkhas a numeric weight[2]associated with it. .
Weights are the primary means of long-term storage in neural networks, and learning usually takes place by updating the weights.
Artificial neurons are the constitutive units in an artificial neural network.
WHY USE NEURAL NETWORKS?
It has ability to Learn from experience.
It can deal with incomplete information.
It can produce result on the basis of input, has not been taught to deal with.
It is used to extract useful pattern from given data i.e. pattern Recognition etc.
Biological Neurons
Four parts of a typical nerve cell :• DENDRITES: Accepts the inputs• SOMA : Process the inputs• AXON : Turns the processed inputs into outputs.• SYNAPSES : The electrochemical contactbetween the neurons.
ARTIFICIAL NEURONS MODEL
Inputs to the network arerepresented by the x1mathematical symbol, xn
Each of these inputs are multiplied by a connection weight , wn
sum = w1 x1 + ……+ wnxn
These products are simplysummed, fed through the transfer function, f( ) to generate a result and then output.
NEURON MODEL
Neuron Consist of:
Inputs (Synapses): inputsignal.Weights (Dendrites):determines the importance ofincoming value.Output (Axon): output toother neuron or of NN .
Artificial Neural Network and its Applicationsshritosh kumar
Abstract
This report is an introduction to Artificial Neural
Networks. The various types of neural networks are
explained and demonstrated, applications of neural
networks like ANNs in medicine are described, and a
detailed historical background is provided. The
connection between the artificial and the real thing is
also investigated and explained. Finally, the
mathematical models involved are presented and
demonstrated.
Neural Network and Artificial Intelligence.
Neural Network and Artificial Intelligence.
WHAT IS NEURAL NETWORK?
The method calculation is based on the interaction of plurality of processing elements inspired by biological nervous system called neurons.
It is a powerful technique to solve real world problem.
A neural network is composed of a number of nodes, or units[1], connected by links. Each linkhas a numeric weight[2]associated with it. .
Weights are the primary means of long-term storage in neural networks, and learning usually takes place by updating the weights.
Artificial neurons are the constitutive units in an artificial neural network.
WHY USE NEURAL NETWORKS?
It has ability to Learn from experience.
It can deal with incomplete information.
It can produce result on the basis of input, has not been taught to deal with.
It is used to extract useful pattern from given data i.e. pattern Recognition etc.
Biological Neurons
Four parts of a typical nerve cell :• DENDRITES: Accepts the inputs• SOMA : Process the inputs• AXON : Turns the processed inputs into outputs.• SYNAPSES : The electrochemical contactbetween the neurons.
ARTIFICIAL NEURONS MODEL
Inputs to the network arerepresented by the x1mathematical symbol, xn
Each of these inputs are multiplied by a connection weight , wn
sum = w1 x1 + ……+ wnxn
These products are simplysummed, fed through the transfer function, f( ) to generate a result and then output.
NEURON MODEL
Neuron Consist of:
Inputs (Synapses): inputsignal.Weights (Dendrites):determines the importance ofincoming value.Output (Axon): output toother neuron or of NN .
Examinations of humans' central nervous systems inspired the concept of artificial neural networks. In an artificial neural network, simple artificial nodes, known as "neurons", "neurodes", "processing elements" or "units", are connected together to form a network which mimics a biological neural network
A neural network is a series of algorithms that endeavors to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial in nature....
اسلایدهای درس شبکه عصبی و یادگیری عمیق که در دانشگاه شیراز توسط استاد اقبال منصوری تدریس می شود.
Neural network and deep learning course slide taught by Professor Iqbal Mansouri at Shiraz University.
Invited talk at Tsinghua University on "Applications of Deep Neural Network". As the tech. lead of deep learning task force at NIO USA INC, I was invited to give this colloquium talk on general applications of deep neural network.
This PPT contains entire content in short. My book on ANN under the title "SOFT COMPUTING" with Watson Publication and my classmates can be referred together.
These simplified slides by Dr. Sidra Arshad present an overview of the non-respiratory functions of the respiratory tract.
Learning objectives:
1. Enlist the non-respiratory functions of the respiratory tract
2. Briefly explain how these functions are carried out
3. Discuss the significance of dead space
4. Differentiate between minute ventilation and alveolar ventilation
5. Describe the cough and sneeze reflexes
Study Resources:
1. Chapter 39, Guyton and Hall Textbook of Medical Physiology, 14th edition
2. Chapter 34, Ganong’s Review of Medical Physiology, 26th edition
3. Chapter 17, Human Physiology by Lauralee Sherwood, 9th edition
4. Non-respiratory functions of the lungs https://academic.oup.com/bjaed/article/13/3/98/278874
micro teaching on communication m.sc nursing.pdfAnurag Sharma
Microteaching is a unique model of practice teaching. It is a viable instrument for the. desired change in the teaching behavior or the behavior potential which, in specified types of real. classroom situations, tends to facilitate the achievement of specified types of objectives.
Title: Sense of Taste
Presenter: Dr. Faiza, Assistant Professor of Physiology
Qualifications:
MBBS (Best Graduate, AIMC Lahore)
FCPS Physiology
ICMT, CHPE, DHPE (STMU)
MPH (GC University, Faisalabad)
MBA (Virtual University of Pakistan)
Learning Objectives:
Describe the structure and function of taste buds.
Describe the relationship between the taste threshold and taste index of common substances.
Explain the chemical basis and signal transduction of taste perception for each type of primary taste sensation.
Recognize different abnormalities of taste perception and their causes.
Key Topics:
Significance of Taste Sensation:
Differentiation between pleasant and harmful food
Influence on behavior
Selection of food based on metabolic needs
Receptors of Taste:
Taste buds on the tongue
Influence of sense of smell, texture of food, and pain stimulation (e.g., by pepper)
Primary and Secondary Taste Sensations:
Primary taste sensations: Sweet, Sour, Salty, Bitter, Umami
Chemical basis and signal transduction mechanisms for each taste
Taste Threshold and Index:
Taste threshold values for Sweet (sucrose), Salty (NaCl), Sour (HCl), and Bitter (Quinine)
Taste index relationship: Inversely proportional to taste threshold
Taste Blindness:
Inability to taste certain substances, particularly thiourea compounds
Example: Phenylthiocarbamide
Structure and Function of Taste Buds:
Composition: Epithelial cells, Sustentacular/Supporting cells, Taste cells, Basal cells
Features: Taste pores, Taste hairs/microvilli, and Taste nerve fibers
Location of Taste Buds:
Found in papillae of the tongue (Fungiform, Circumvallate, Foliate)
Also present on the palate, tonsillar pillars, epiglottis, and proximal esophagus
Mechanism of Taste Stimulation:
Interaction of taste substances with receptors on microvilli
Signal transduction pathways for Umami, Sweet, Bitter, Sour, and Salty tastes
Taste Sensitivity and Adaptation:
Decrease in sensitivity with age
Rapid adaptation of taste sensation
Role of Saliva in Taste:
Dissolution of tastants to reach receptors
Washing away the stimulus
Taste Preferences and Aversions:
Mechanisms behind taste preference and aversion
Influence of receptors and neural pathways
Impact of Sensory Nerve Damage:
Degeneration of taste buds if the sensory nerve fiber is cut
Abnormalities of Taste Detection:
Conditions: Ageusia, Hypogeusia, Dysgeusia (parageusia)
Causes: Nerve damage, neurological disorders, infections, poor oral hygiene, adverse drug effects, deficiencies, aging, tobacco use, altered neurotransmitter levels
Neurotransmitters and Taste Threshold:
Effects of serotonin (5-HT) and norepinephrine (NE) on taste sensitivity
Supertasters:
25% of the population with heightened sensitivity to taste, especially bitterness
Increased number of fungiform papillae
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists Saeid Safari
Preoperative Management of Patients on GLP-1 Receptor Agonists like Ozempic and Semiglutide
ASA GUIDELINE
NYSORA Guideline
2 Case Reports of Gastric Ultrasound
Flu Vaccine Alert in Bangalore Karnatakaaddon Scans
As flu season approaches, health officials in Bangalore, Karnataka, are urging residents to get their flu vaccinations. The seasonal flu, while common, can lead to severe health complications, particularly for vulnerable populations such as young children, the elderly, and those with underlying health conditions.
Dr. Vidisha Kumari, a leading epidemiologist in Bangalore, emphasizes the importance of getting vaccinated. "The flu vaccine is our best defense against the influenza virus. It not only protects individuals but also helps prevent the spread of the virus in our communities," he says.
This year, the flu season is expected to coincide with a potential increase in other respiratory illnesses. The Karnataka Health Department has launched an awareness campaign highlighting the significance of flu vaccinations. They have set up multiple vaccination centers across Bangalore, making it convenient for residents to receive their shots.
To encourage widespread vaccination, the government is also collaborating with local schools, workplaces, and community centers to facilitate vaccination drives. Special attention is being given to ensuring that the vaccine is accessible to all, including marginalized communities who may have limited access to healthcare.
Residents are reminded that the flu vaccine is safe and effective. Common side effects are mild and may include soreness at the injection site, mild fever, or muscle aches. These side effects are generally short-lived and far less severe than the flu itself.
Healthcare providers are also stressing the importance of continuing COVID-19 precautions. Wearing masks, practicing good hand hygiene, and maintaining social distancing are still crucial, especially in crowded places.
Protect yourself and your loved ones by getting vaccinated. Together, we can help keep Bangalore healthy and safe this flu season. For more information on vaccination centers and schedules, residents can visit the Karnataka Health Department’s official website or follow their social media pages.
Stay informed, stay safe, and get your flu shot today!
New Directions in Targeted Therapeutic Approaches for Older Adults With Mantl...i3 Health
i3 Health is pleased to make the speaker slides from this activity available for use as a non-accredited self-study or teaching resource.
This slide deck presented by Dr. Kami Maddocks, Professor-Clinical in the Division of Hematology and
Associate Division Director for Ambulatory Operations
The Ohio State University Comprehensive Cancer Center, will provide insight into new directions in targeted therapeutic approaches for older adults with mantle cell lymphoma.
STATEMENT OF NEED
Mantle cell lymphoma (MCL) is a rare, aggressive B-cell non-Hodgkin lymphoma (NHL) accounting for 5% to 7% of all lymphomas. Its prognosis ranges from indolent disease that does not require treatment for years to very aggressive disease, which is associated with poor survival (Silkenstedt et al, 2021). Typically, MCL is diagnosed at advanced stage and in older patients who cannot tolerate intensive therapy (NCCN, 2022). Although recent advances have slightly increased remission rates, recurrence and relapse remain very common, leading to a median overall survival between 3 and 6 years (LLS, 2021). Though there are several effective options, progress is still needed towards establishing an accepted frontline approach for MCL (Castellino et al, 2022). Treatment selection and management of MCL are complicated by the heterogeneity of prognosis, advanced age and comorbidities of patients, and lack of an established standard approach for treatment, making it vital that clinicians be familiar with the latest research and advances in this area. In this activity chaired by Michael Wang, MD, Professor in the Department of Lymphoma & Myeloma at MD Anderson Cancer Center, expert faculty will discuss prognostic factors informing treatment, the promising results of recent trials in new therapeutic approaches, and the implications of treatment resistance in therapeutic selection for MCL.
Target Audience
Hematology/oncology fellows, attending faculty, and other health care professionals involved in the treatment of patients with mantle cell lymphoma (MCL).
Learning Objectives
1.) Identify clinical and biological prognostic factors that can guide treatment decision making for older adults with MCL
2.) Evaluate emerging data on targeted therapeutic approaches for treatment-naive and relapsed/refractory MCL and their applicability to older adults
3.) Assess mechanisms of resistance to targeted therapies for MCL and their implications for treatment selection
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...kevinkariuki227
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Verified Chapters 1 - 19, Complete Newest Version.pdf
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Verified Chapters 1 - 19, Complete Newest Version.pdf
Basavarajeeyam is an important text for ayurvedic physician belonging to andhra pradehs. It is a popular compendium in various parts of our country as well as in andhra pradesh. The content of the text was presented in sanskrit and telugu language (Bilingual). One of the most famous book in ayurvedic pharmaceutics and therapeutics. This book contains 25 chapters called as prakaranas. Many rasaoushadis were explained, pioneer of dhatu druti, nadi pareeksha, mutra pareeksha etc. Belongs to the period of 15-16 century. New diseases like upadamsha, phiranga rogas are explained.
Knee anatomy and clinical tests 2024.pdfvimalpl1234
This includes all relevant anatomy and clinical tests compiled from standard textbooks, Campbell,netter etc..It is comprehensive and best suited for orthopaedicians and orthopaedic residents.
2. What are connectionist neural
networks?
• Connectionism refers to a computer modeling
approach to computation that is loosely based
upon the architecture of the brain.
• Many different models, but all include:
– Multiple, individual “nodes” or “units” that operate at
the same time (in parallel)
– A network that connects the nodes together
– Information is stored in a distributed fashion among the
links that connect the nodes
– Learning can occur with gradual changes in connection
strength
3. Neural Network History
• History traces back to the 50’s but became popular in the 80’s
with work by Rumelhart, Hinton, and Mclelland
– A General Framework for Parallel Distributed Processing in Parallel
Distributed Processing: Explorations in the Microstructure of
Cognition
• Peaked in the 90’s. Today:
– Hundreds of variants
– Less a model of the actual brain than a useful tool, but still some
debate
• Numerous applications
– Handwriting, face, speech recognition
– Vehicles that drive themselves
– Models of reading, sentence production, dreaming
• Debate for philosophers and cognitive scientists
– Can human consciousness or cognitive abilities be explained by a
connectionist model or does it require the manipulation of symbols?
4. Comparison of Brains and Traditional
Computers
• 200 billion neurons, 32
trillion synapses
• Element size: 10-6
m
• Energy use: 25W
• Processing speed: 100 Hz
• Parallel, Distributed
• Fault Tolerant
• Learns: Yes
• Intelligent/Conscious:
Usually
• 1 billion bytes RAM but
trillions of bytes on disk
• Element size: 10-9
m
• Energy watt: 30-90W (CPU)
• Processing speed: 109
Hz
• Serial, Centralized
• Generally not Fault Tolerant
• Learns: Some
• Intelligent/Conscious:
Generally No
5. Biological Inspiration
“My brain: It's my second favorite organ.”
- Woody Allen, from the movie Sleeper
Idea : To make the computer more robust, intelligent, and learn, …
Let’s model our computer software (and/or hardware) after the brain
6. Neurons in the Brain
• Although heterogeneous, at a low level
the brain is composed of neurons
– A neuron receives input from other neurons
(generally thousands) from its synapses
– Inputs are approximately summed
– When the input exceeds a threshold the neuron
sends an electrical spike that travels that
travels from the body, down the axon, to the
next neuron(s)
7. Learning in the Brain
• Brains learn
– Altering strength between neurons
– Creating/deleting connections
• Hebb’s Postulate (Hebbian Learning)
– When an axon of cell A is near enough to excite a cell B and repeatedly
or persistently takes part in firing it, some growth process or metabolic
change takes place in one or both cells such that A's efficiency, as one
of the cells firing B, is increased.
• Long Term Potentiation (LTP)
– Cellular basis for learning and memory
– LTP is the long-lasting strengthening of the connection between two
nerve cells in response to stimulation
– Discovered in many regions of the cortex
8. Perceptrons
• Initial proposal of connectionist networks
• Rosenblatt, 50’s and 60’s
• Essentially a linear discriminant composed of
nodes, weights
I1
I2
I3
W1
W2
W3
θ O
>+
= ∑
otherwise
Iw
O i
ii
:0
0:1 θ
I1
I2
I3
W1
W2
W3
θ
O
or
1
Activation Function
9. Perceptron Example
2
1
.5
.3 θ =-1
2(0.5) + 1(0.3) + -1 = 0.3 , O=1
Learning Procedure:
Randomly assign weights (between 0-1)
Present inputs from training data
Get output O, nudge weights to gives results toward our
desired output T
Repeat; stop when no errors, or enough epochs completed
10. Perception Training
)()()1( twtwtw iii ∆+=+
ii IOTtw )()( −=∆
Weights include Threshold. T=Desired, O=Actual output.
5.1)2)(10(5.0)1(1 −=−+=+tw
Example: T=0, O=1, W1=0.5, W2=0.3, I1=2, I2=1,Theta=-1
7.0)1)(10(3.0)1(2 −=−+=+tw
2)1)(10(1)1( −=−+−=+twθ
If we present this input again, we’d output 0 instead
11. How might you use a perceptron
network?
• This (and other networks) are generally used to learn how
to make classifications
• Say you have collected some data regarding the diagnosis
of patients with heart disease
– Age, Sex, Chest Pain Type, Resting BPS, Cholesterol, …,
Diagnosis (<50% diameter narrowing, >50% diameter narrowing)
– 67,1,4,120,229,…, 1
– 37,1,3,130,250,… ,0
– 41,0,2,130,204,… ,0
• Train network to predict heart disease of new patient
12. Perceptrons
• Can add learning rate to speed up the learning process; just
multiply in with delta computation
• Essentially a linear discriminant
• Perceptron theorem: If a linear discriminant exists that can
separate the classes without error, the training procedure is
guaranteed to find that line or plane.
Class1 Class2
13. 10
1 0
XOR Problem: Not Linearly Separable!
Exclusive Or (XOR) Problem
Input: 0,0 Output: 0
Input: 0,1 Output: 1
Input: 1,0 Output: 1
Input: 1,1 Output: 0
We could however construct multiple layers of perceptrons to get around
this problem. A typical multi-layered system minimizes LMS Error,
14. LMS Learning
LMS = Least Mean Square learning Systems, more general than the
previous perceptron learning rule. The concept is to minimize the total
error, as measured over all training examples, P. O is the raw output,
as calculated by
( )∑ −=
P
PP OTLMSceDis
2
2
1
)(tan
E.g. if we have two patterns and
T1=1, O1=0.8, T2=0, O2=0.5 then D=(0.5)[(1-0.8)2
+(0-0.5)2
]=.145
We want to minimize the LMS:
E
W
W(old)
W(new)
C-learning rate
∑ +
i
ii Iw θ
15. LMS Gradient Descent
• Using LMS, we want to minimize the error. We can do this by finding
the direction on the error surface that most rapidly reduces the error
rate; this is finding the slope of the error function by taking the
derivative. The approach is called gradient descent (similar to hill
climbing).
k
k
w
Error
cw
δ
δ
−=∆
To compute how much to change weight for link k:
k
j
jk w
O
O
Error
w
Error
δ
δ
δ
δ
δ
δ
∗=
Chain rule:
)(
)(
2
1)(
2
1 22
jj
j
jj
j
P
PP
j
OT
O
OT
O
OT
O
Error
−−=
−
=
−
=
∑
δ
δ
δ
δ
δ
δ
We can remove the sum since we are taking the partial derivative wrt Oj
( ))('
kkk
k
j
WIFunctionActivationfI
w
O
=
δ
δ
( ) ( )FunctionActivationfIOTcw kjjk ')( −−−=∆
)( WIfOj =
16. Activation Function
• To apply the LMS learning rule, also
known as the delta rule, we need a
differentiable activation function.
( ) ( )FunctionActivationfOTcIw jjkk '−=∆
>+
=
∑
otherwise
Iw
O i
ii
:0
0:1 θ
Old:
θ−
New:
∑
+
= Θ+−
i
ii Iw
e
O
1
1
17. LMS vs. Limiting Threshold
• With the new sigmoidal function that is differentiable, we
can apply the delta rule toward learning.
• Perceptron Method
– Forced output to 0 or 1, while LMS uses the net output
– Guaranteed to separate, if no error and is linearly separable
• Otherwise it may not converge
• Gradient Descent Method:
– May oscillate and not converge
– May converge to wrong answer
– Will converge to some minimum even if the classes are not
linearly separable, unlike the earlier perceptron training method
18. Backpropagation Networks
• Attributed to Rumelhart and McClelland, late 70’s
• To bypass the linear classification problem, we can
construct multilayer networks. Typically we have fully
connected, feedforward networks.
I1
I2
1
Hidden Layer
H1
H2
O1
O2
Input Layer Output Layer
Wi,j
Wj,k
1’s - bias
∑
+
= −
j
jxj Hw
e
xO
,
1
1
)(
I3
1
∑
+
= −
i
ixi Iw
e
xH
,
1
1
)(
19. Backprop - Learning
Learning Procedure:
Randomly assign weights (between 0-1)
Present inputs from training data, propagate to outputs
Compute outputs O, adjust weights according to the delta
rule, backpropagating the errors. The weights will be
nudged closer so that the network learns to give the
desired output.
Repeat; stop when no errors, or enough epochs completed
20. Backprop - Modifying Weights
( ) ( )
+
=−=∆ −sumjjkk
e
fFunctionActivationfOTcIw
1
1
;'
We had computed:
For the Output unit k, f(sum)=O(k). For the output units, this is:
( ) )1(, kkkkjkj OOOTcHw −−=∆
( )( ))(1)(( sumfsumfOTcIw jjkk −−=∆
I H O
Wi,j Wj,k
For the Hidden units (skipping some math), this is:
kj
k
kkkkijjji wOOOTIHcHw ,, )1()()1( ∑ −−−=∆
21. Backprop
• Very powerful - can learn any function, given enough
hidden units! With enough hidden units, we can generate
any function.
• Have the same problems of Generalization vs.
Memorization. With too many units, we will tend to
memorize the input and not generalize well. Some schemes
exist to “prune” the neural network.
• Networks require extensive training, many parameters to
fiddle with. Can be extremely slow to train. May also fall
into local minima.
• Inherently parallel algorithm, ideal for multiprocessor
hardware.
• Despite the cons, a very powerful algorithm that has seen
widespread successful deployment.
23. Unsupervised Learning
• We just discussed a form of supervised learning
– A “teacher” tells the network what the correct output is
based on the input until the network learns the target
concept
• We can also train networks where there is no
teacher. This is called unsupervised learning.
The network learns a prototype based on the
distribution of patterns in the training data. Such
networks allow us to:
– Discover underlying structure of the data
– Encode or compress the data
– Transform the data
24. Unsupervised Learning – Hopfield
Networks
• A Hopfield network is a type of content-
addressable memory
– Non-linear system with attractor points that
represent concepts
– Given a fuzzy input the system converges to the
nearest attractor
• Possibility to have “spurious” attractors that is a blend
of multiple stored patterns
• Also possible to have chaotic patterns that never
converge
25. Standard Binary Hopfield Network
• Recurrent; Every unit is connected to every other unit
• Weights connecting units are symmetrical
– wij = wji
• If the weighted sum of the inputs exceeds a threshold, its
output is 1 otherwise its output is -1
• Units update themselves asynchronously as their inputs
change
B
C
A
D
wAD
wABwAC
wBC wBD
wCD
26. Hopfield Memories
• Setting the weights:
– A pattern is a setting of on or off for each unit
– Given a set of Q patterns to store
• For every weight connecting units i and j:
• This is a form of a Hebbian rule which makes the weight
strength proportional to the product of the firing rates of the
two interconnected units
p
j
Q
p
p
iij xxw ∑=
=
1
27. Hopfield Network Demo
• http://www.cbu.edu/~pong/ai/hopfield/hopfieldap
plet.html
• Properties
– Settles into a minimal energy state
– Storage capacity low, only 13% of number of units
– Can retrieve information even in the presence of noisy
data, similar to associative memory of humans
28. Unsupervised Learning – Self
Organizing Maps
• Self-organizing maps (SOMs) are a data
visualization technique invented by Professor
Teuvo Kohonen
– Also called Kohonen Networks, Competitive Learning,
Winner-Take-All Learning
– Generally reduces the dimensions of data through the
use of self-organizing neural networks
– Useful for data visualization; humans cannot visualize
high dimensional data so this is often a useful technique
to make sense of large data sets
29. Basic “Winner Take All” Network
• Two layer network
– Input units, output units, each input unit is connected to each
output unit
I1
I2
O1
O2
Input Layer
Wi,j
I3
Output Layer
30. Basic Algorithm
– Initialize Map (randomly assign weights)
– Loop over training examples
• Assign input unit values according to the values in the current
example
• Find the “winner”, i.e. the output unit that most closely
matches the input units, using some distance metric, e.g.
• Modify weights on the winner to more closely match the input
( )
2
1
∑=
−
n
i
iij IW
For all output units j=1 to m
and input units i=1 to n
Find the one that minimizes:
)(1 tt
i
t
WXcW −=∆ +
where c is a small positive learning constant
that usually decreases as the learning proceeds
31. Result of Algorithm
• Initially, some output nodes will randomly be a
little closer to some particular type of input
• These nodes become “winners” and the weights
move them even closer to the inputs
• Over time nodes in the output become
representative prototypes for examples in the input
• Note there is no supervised training here
• Classification:
– Given new input, the class is the output node that is the
winner
32. Typical Usage: 2D Feature Map
• In typical usage the output nodes form a 2D “map”
organized in a grid-like fashion and we update weights in a
neighborhood around the winner
I1
I2
Input Layer
I3
Output Layers
O11 O12 O13 O14 O15
O21 O22 O23 O24 O25
O31 O32 O33 O34 O35
O41 O42 O43 O44 O45
O51 O52 O53 O54 O55
…
33. Modified Algorithm
– Initialize Map (randomly assign weights)
– Loop over training examples
• Assign input unit values according to the values in the current
example
• Find the “winner”, i.e. the output unit that most closely
matches the input units, using some distance metric, e.g.
• Modify weights on the winner to more closely match the input
• Modify weights in a neighborhood around the winner so
the neighbors on the 2D map also become closer to the
input
– Over time this will tend to cluster similar items closer on the
map
34. Updating the Neighborhood
• Node O44 is the winner
– Color indicates scaling to update neighbors
Output Layers
O11 O12 O13 O14 O15
O21 O22 O23 O24 O25
O31 O32 O33 O34 O35
O41 O42 O43 O44 O45
O51 O52 O53 O54 O55
)(1 tt
i
t
WXcW −=∆ +
c=1
c=0.75
c=0.5
Consider if O42
is winner for
some other
input; “fight”
over claiming
O43,O33, O53
35. Selecting the Neighborhood
• Typically, a “Sombrero Function” or Gaussian
function is used
• Neighborhood size usually decreases over time to
allow initial “jockeying for position” and then
“fine-tuning” as algorithm proceeds
39. SOM for Classification
• A generated map can also be used for classification
• Human can assign a class to a data point, or use the
strongest weight as the prototype for the data point
• For a new test case, calculate the winning node and
classify it as the class it is closest to
• Handwriting recognition example: http://fbim.fh-
regensburg.de/~saj39122/begrolu/kohonen.html
40. Psychological and Biological
Considerations of Neural Networks
• Psychological
– Neural network models learn, exhibit some behavior similar to
humans, based loosely on brains
– Create their own algorithms instead of being explicitly programmed
– Operate under noisy data
– Fault tolerant and graceful degradation
– Knowledge is distributed, yet still some localization
• Lashley’s search for engrams
• Biological
– Learning in the visual cortex shortly after birth seems to correlate
with the pattern discrimination that emerges from Kohonen Networks
– Criticisms of the mechanism to update weights; mathematically
driven; feedforward supervised network unrealistic
41. Connectionism
• What’s hard for neural networks? Activities beyond
recognition, e.g.:
– Variable binding
– Recursion
– Reflection
– Structured representations
• Connectionist and Symbolic Models
– The Central Paradox of Cognition (Smolensky et al., 1992):
– "Formal theories of logical reasoning, grammar, and other higher
mental faculties compel us to think of the mind as a machine for rule-
based manipulation of highly structured arrays of symbols. What we
know of the brain compels us to think of human information
processing in terms of manipulation of a large unstructured set of
numbers, the activity levels of interconnected neurons. Finally, the full
richness of human behavior, both in everyday environments and in the
controlled environments of the psychological laboratory, seems to defy
rule-based description, displaying strong sensitivity to subtle statistical
factors in experience, as well as to structural properties of information.
To solve the Central Paradox of Cognition is to resolve these
contradictions with a unified theory of the organization of the mind, of
the brain, of behavior, and of the environment."
42. Possible Relationships?
• Symbolic systems implemented via
connectionism
– Possible to create hierarchies of networks with
subnetworks to implement symbolic systems
• Hybrid model
– System consists of two separate components;
low-level tasks via connectionism, high-level
tasks via symbols
43. Proposed Hierarchical Model
• Jeff Hawkins
• Founder: Palm Computing, Handspring
• Deep interest in the brain all his life
• Book: “On Intelligence”
– Variety of neuroscience research as input
– Includes his own ideas, theories, guesses
– Increasingly accepted view of the brain
44. The Cortex
• Hawkins’s point of interest in the brain
– “Where the magic happens”
• Hierarchically-arranged in regions
• Communication up the hierarchy
– Regions classify patterns of their inputs
– Regions output a ‘named’ pattern up the hierarchy
• Communication down the hierarchy
– A high-level region has made a prediction
– Alerts lower-level regions what to expect
45. Hawkins Quotes
“The human cortex is particularly large and
therefore has a massive memory capacity.
It is constantly predicting what you will see,
hear and feel, mostly in ways you are
unconscious of. These predictions are our
thoughts, and when combined with sensory
inputs, they are our perceptions. I call this
view of the brain the memory-prediction
framework of intelligence.”
46. Hawkins Quotes
“Your brain constantly makes predictions
about the very fabric of the world we live
in, and it does so in a parallel fashion. It
will just as readily detect an odd texture, a
misshapen nose, or an unusual motion. It
isn’t obvious how pervasive these mostly
unconscious predictions are, which is
perhaps why we missed their importance.”
47. Hawkins Quotes
“Your brain constantly makes predictions
about the very fabric of the world we live
in, and it does so in a parallel fashion. It
will just as readily detect an odd texture, a
misshapen nose, or an unusual motion. It
isn’t obvious how pervasive these mostly
unconscious predictions are, which is
perhaps why we missed their importance.”
48. Hawkins Quotes
“Suppose when you are out, I sneak over to
your home and change something about
your door. It could be almost anything. I
could move the knob over by and inch,
change a round knob into a thumb latch, or
turn it from brass to chrome…. When you
come home that day and attempt to open the
door, you will quickly detect that
something is wrong.”
49. Prediction
• Prediction means that the neurons involved
in sensing your door become active in
advance of them actually receiving sensory
input.
– When the sensory input does arrive, it is
compared with what is expected.
– Two way communication; classification up the
hierarchy, prediction down the hierarchy
50. Prediction
• Prediction is not limited to patterns of low-level
sensory information like hearing and seeing
• Mountcastle’s principle : we have lots of
different neurons, but they basically do the
same thing (particularly in the neocortex)
– What is true of low-level sensory areas must be true
for all cortical areas. The human brain is more
intelligent than that of other animals because it can
make predictions about more abstract kinds of
patterns and longer temporal pattern sequences.”
51. Visual Hierarchies
• Lowest visual level inputs pixels
• Second level recognizes edges, lines, etc
from known patterns of pixels
• Third level recognizes shapes from known
patterns of edges, lines, etc
• Fourth level recognizes objects from known
patterns of shapes
53. Not there yet…
• Many issues remain to be addressed by
Hawkins’ model
– Missing lots of details on how his model could
be implemented in a computer
– Creativity?
– Evolution?
– Planning?
– Rest of the brain, not just neocortex?
Recognizing pictures, making decisions, functioning with motor skills in reaction to environment, etc…
Most abstract concepts at the top (the big ideas, concepts, etc) and the features, details, pixels of images at the bottom
More on the naming shortly