This document provides an overview of single layer perceptrons and their use in classification tasks. It discusses the McCulloch-Pitts neuron model and how networks of these neurons can be connected to implement logic gates. It then introduces the perceptron as a single layer feedforward network and describes how to train perceptrons using supervised learning and the perceptron learning rule to classify data. Finally, it provides an example classification task of distinguishing trucks using their mass and length attributes.
The slide covers the basic concepts and designs of artificial neural networks. It explains and justifies the use of McCulloh Pitts Model, Adaline network, Perceptron algorithm, Backpropagation algorithm, Hopfield network and Kohonen network; along with its practical applications.
1
The Perceptron and its Learning Rule
Carlo U. Nicola, SGI FH Aargau
With extracts from publications of :
M. Minsky, MIT, Demuth, U. of Colorado,
D.J. C. MacKay, Cambridge University
WBS WS06-07 2
Perceptron
(i) Single layer ANN
(ii) It works with continuous or binary inputs
(iii) It stores pattern pairs (Ak,Ck) where: Ak = (a1
k, …, an
k)
and Ck = (c1
k, …, cn
k) are bipolar valued [-1, +1].
(iv) It applies the perceptron error-correction procedure, which
always converges.
(v) A perceptron is a classifier.
Bias b is sometimes called θ
The slide covers the basic concepts and designs of artificial neural networks. It explains and justifies the use of McCulloh Pitts Model, Adaline network, Perceptron algorithm, Backpropagation algorithm, Hopfield network and Kohonen network; along with its practical applications.
1
The Perceptron and its Learning Rule
Carlo U. Nicola, SGI FH Aargau
With extracts from publications of :
M. Minsky, MIT, Demuth, U. of Colorado,
D.J. C. MacKay, Cambridge University
WBS WS06-07 2
Perceptron
(i) Single layer ANN
(ii) It works with continuous or binary inputs
(iii) It stores pattern pairs (Ak,Ck) where: Ak = (a1
k, …, an
k)
and Ck = (c1
k, …, cn
k) are bipolar valued [-1, +1].
(iv) It applies the perceptron error-correction procedure, which
always converges.
(v) A perceptron is a classifier.
Bias b is sometimes called θ
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
2. 2
Recap: McCulloch-Pitts Neuron
• This vastly simplified model of real neurons is also known as a Threshold
Logic Unit:
1. A set of synapses (i.e. connections) brings in activations from other
neurons
2. A processing unit sums the inputs, and then applies a non-linear
activation function
3. An output line transmits the result to other neurons
I1
∑
Wj1
Wjn
I2
I3
In
Aj Yj
3. 3
Networks of McCulloch-Pitts Neurons
One neuron can’t do much on its own. Usually we will have many neurons
labelled by indices k, i, j and activation flows between via synapses with
strengths wki, wij:
)
sgn(
1
i
n
k
ki
i I
Y θ
−
= ∑
=
ki
k
ki w
Y
I ⋅
= ij
i
ij w
Y
I ⋅
=
I1i
I2i
I3i
Iki
Neuron i
Yi
Synapse ij
Iij
Neuron j
θi
wij
∑
Neuron k
W1i
4. 4
The Perceptron
We can connect any number of McCulloch-Pitts neurons together in
any way we like
An arrangement of one input layer of McCulloch-Pitts neurons feeding
forward to one output layer of McCulloch-Pitts neurons is known as
a Perceptron.
)
sgn(
1
j
ij
n
i
i
j w
Y
Y θ
−
⋅
= ∑
=
:
:
:
:
i j
wij
1
2
N
1
2
M
θ1
θ2
θM
5. 5
Implementing Logic Gates with MP Neurons
We can use McCulloch-Pitts neurons to implement the basic logic
gates (e.g. AND, OR, NOT).
It is well known from logic that we can construct any logical function
from these three basic logic gates.
All we need to do is find the appropriate connection weights and neuron
thresholds to produce the right outputs for each set of inputs.
We shall see explicitly how one can construct simple networks that
perform NOT, AND, and OR.
6. 6
Implementation of Logical NOT, AND, and OR
0
1
1
0
out
in
NOT
NOT
1
1
1
0
0
1
0
1
0
0
0
0
out
in2
in1
AND
AND
1
1
1
1
0
1
1
1
0
0
0
0
out
in2
in1
OR
OR
?
? ?
Problem: Train network to calculate the appropriate weights and thresholds in
order to classify correctly the different classes (i.e. form decision boundaries
between classes).
7. 7
Decision Surfaces
Decision surface is the surface at which the output of the unit is
precisely equal to the threshold, i.e. ∑wiIi=θ
In 1-D the surface is just a point:
I1=θ/w1
Y=0 Y=1
I1
In 2-D, the surface is
0
2
2
1
1 =
−
⋅
+
⋅ θ
w
I
w
I
which we can re-write as
1
2
1
2
2 I
w
w
w
I −
=
θ So, in 2-D the decision boundaries are
always straight lines.
8. 8
Decision Boundaries for AND and OR
We can now plot the decision boundaries of our logic gates
1
1
1
0
0
1
0
1
0
0
0
0
out
I2
I1
AND
AND
I1
I2
(0, 0) (0, 1)
(1, 0)
(1, 1)
I1
I2
(1, 1)
(0, 1)
(0, 0)
(1, 0)
1
1
1
1
0
1
1
1
0
0
0
0
out
I2
I1
OR
OR
AND
AND
w1=1, w2=1, θ=1.5
OR
OR
w1=1, w2=1, θ=0.5
9. 9
Decision Boundary for XOR
The difficulty in dealing with XOR is rather obvious. We need two straight
lines to separate the different outputs/decisions:
I1
I2
0
1
1
1
0
1
1
1
0
0
0
0
out
I2
I1
XOR
XOR
I1
I2
Solution: either change the transfer function so that it has more than one
decision boundary, or use a more complex network that is able to generate
more complex decision boundaries.
10. 10
ANN Architectures
Mathematically, ANNs can be represented as weighted directed graphs. The
most common ANN architectures are:
Single-Layer Feed-Forward NNs: One input layer and one output layer of
processing units. No feedback connections (e.g. a Perceptron)
Multi-Layer Feed-Forward NNs: One input layer, one output layer, and one or
more hidden layers of processing units. No feedback connections (e.g. a
Multi-Layer Perceptron)
Recurrent NNs: Any network with at least one feedback connection. It may, or
may not, have hidden units
Further interesting variations include: sparse connections, time-delayed
connections, moving windows, …
11. 11
Examples of Network Architectures
Single Layer Multi-Layer Recurrent
Feed-Forward Feed-Forward Network
12. 12
Types of Activation/Transfer Function
Threshold Function
<
≥
=
0
0
0
1
)
(
x
if
x
if
x
f
f(x)
x
Piecewise-Linear Function
Sigmoid Function
≤
≤
≤
−
+
≥
=
5
.
0
0
5
.
0
5
.
0
5
.
0
5
.
0
1
)
(
x
if
x
if
x
x
if
x
f
f(x)
x
x
e
x
f −
+
=
1
1
)
(
f(x)
x
13. 13
The Threshold as a Special Kind of Weight
The basic Perceptron equation can be simplified if we consider that the
threshold is another connection weight:
j
nj
n
j
j
j
ij
n
i
i w
I
w
I
w
I
w
I θ
θ −
⋅
+
+
⋅
+
⋅
=
−
⋅
∑
=
K
2
2
1
1
1
If we define w0j=-θj and I0=1 then
ij
n
i
i
j
nj
n
j
j
j
ij
n
i
i w
I
w
I
w
I
w
I
w
I
w
I ⋅
=
⋅
+
⋅
+
+
⋅
+
⋅
=
−
⋅ ∑
∑
=
= 0
0
0
2
2
1
1
1
K
θ
The Perceptron equation then becomes
)
sgn(
)
sgn(
0
1
ij
n
i
i
j
ij
n
i
i
j w
I
w
I
Y ⋅
=
−
⋅
= ∑
∑
=
=
θ
So, we only have to compute the weights.
14. 14
Example: A Classification Task
A typical neural network application is classification. Consider the simple example of
classifying trucks given their masses and lengths:
How do we construct a neural network that can classify any Lorry and Van?
Lorry
9
5.0
Lorry
8
15.0
Lorry
7
10.0
Lorry
6
3.0
Van
5
2.0
Van
5
2.0
Van
4
5.0
Lorry
5
20.0
Lorry
6
10.0
Class
Length
Mass
15. 15
Cookbook Recipe for Building Neural Networks
Formulating neural network solutions for particular problems is a multi-stage
process:
1. Understand and specify the problem in terms of inputs and required
outputs
2. Take the simplest form of network you think might be able to solve your
problem
3. Try to find the appropriate connection weights (including neuron
thresholds) so that the network produces the right outputs for each input
in its training data
4. Make sure that the network works on its training data and test its
generalization by checking its performance on new testing data
5. If the network doesn’t perform well enough, go back to stage 3 and try
harder
6. If the network still doesn’t perform well enough, go back to stage 2 and
try harder
7. If the network still doesn’t perform well enough, go back to stage 1 and
try harder
8. Problem solved – or not
16. 16
Building a Neural Network (stages 1 & 2)
For our truck example, our inputs can be direct encodings of the masses and
lengths. Generally we would have one output unit for each class, with
activation 1 for ‘yes’ and 0 for ‘no’. In our example, we still have one output
unit, but the activation 1 corresponds to ‘lorry’ and 0 to ‘van’ (or vice versa).
The simplest network we should try first is the single layer Perceptron. We
can further simplify things by replacing the threshold by an extra weight as
we discussed before. This gives us:
1 Mass Length
w0
w2
w1
Class=sgn(w0+w1.Mass+w2.Length)
17. 17
Training the Neural Network (stage 3)
Whether our neural network is a simple Perceptron, or a much
complicated multi-layer network, we need to develop a systematic
procedure for determining appropriate connection weights.
The common procedure is to have the network learn the appropriate
weights from a representative set of training data.
For classifications a simple Perceptron uses decision boundaries (lines
or hyperplanes), which it shifts around until each training pattern is
correctly classified.
The process of “shifting around” in a systematic way is called learning.
The learning process can then be divided into a number of small steps.
18. 18
Supervised Training
1. Generate a training pair or pattern:
- an input x = [ x1 x2 … xn]
- a target output ytarget (known/given)
2. Then, present the network with x and allow it to generate an
output y
3. Compare y with ytarget to compute the error
4. Adjust weights, w, to reduce error
5. Repeat 2-4 multiple times
19. 19
Perceptron Learning Rule
1. Initialize weights at random
2. For each training pair/pattern (x, ytarget)
- Compute output y
- Compute error, δ=(ytarget – y)
- Use the error to update weights as follows:
∆w = w – wold=η*δ*x or wnew = wold + η*δ*x
where η is called the learning rate or step size and it determines how
smoothly the learning process is taking place.
3. Repeat 2 until convergence (i.e. error δ is zero)
The Perceptron Learning Rule is then given by
wnew = wold + η*δ*x
where
δ=(ytarget – y)