This presentation introduces the Self-Organizing Incremental Neural Network (SOINN), which can represent the topological structure of input data through online incremental learning without predefined network structure or size. It then discusses Self-Organizing Maps (SOM) and issues with other related algorithms. The detailed SOINN algorithm is presented, including adaptive threshold updating, weight updates, and handling new data. An Adjusted SOINN Classifier (ASC) is also introduced for supervised learning tasks. Experimental results demonstrate SOINN's ability to learn prototypical representations of artificial and real-world data.

1. Self-organizing incremental
neural network
By : F.Noorbehbahani
Agust 2013
1

2. Contents of this presentation
 What is SOINN
 Why SOINN
 SOM
 Detail algorithm of SOINN
 ASOINN
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
2

3. What is SOINN
 SOINN: Self-organizing incremental neural network
 Represent the topological structure of the input data
 Realize online incremental learning
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
3

4. Background: Networks for topology
representation
 SOM(Self-Organizing Map): predefine structure and size of the
network
 NG(Neural Gas): predefine the network size
 GNG(Growing Neural Gas): predefine the network size; constant
learning rate leads to non-stationary result.
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
4

5. Self-Organizing Maps (SOMs)
 Self-Organizing Map (SOM) is an unsupervised learning algorithm.
 SOM is a visualization method to represent higher dimensional data in
an usually 1-D, 2-D or 3-D manner.
 SOMs have two phases:
 Learning phase: map is built, network organizes using a competitive process
using training set.
 Prediction phase: new vectors are quickly given a location on the
converged map, easily classifying or categorizing the new data.
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
5

6. F.Noorbehbahani - Isfahan University of Technology - Agust 2013
6

7. F.Noorbehbahani - Isfahan University of Technology - Agust 2013
7

8. Determining the BMU Neighborhood
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
8

9. F.Noorbehbahani - Isfahan University of Technology - Agust 2013
9

10. Modifying Nodes’ Weights
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
10

11. Modifying Nodes’ Weights
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
11

12. Example
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
12

13. Characteristics of SOINN
 Neurons are self-organized with no predefined network structure
and size
 Adaptively find suitable number of neurons for the network
 Realize online incremental learning without any priori condition
 Find typical prototypes for large-scale data set
 Robust to noise
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
13

14. Detail algorithm of
SOINN
• Two-layer competitive network
• First layer: Competitive for input
data
• Second layer: Competitive for
output of first-layer
• Output topology structure and
weight vector of second layer
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
14

15. Training flowchart of
SOINN
• Adaptively updated threshold
• Between-class insertion
• Update weight of nodes
• Within-class insertion
• Remove noise nodes
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
15

16. First layer: adaptively updating
threshold Ti
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
16

17. Update weights
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
17

18. Artificial data set: topology
representation
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
18

19. SOINN for supervised learning (ASC)
 Automatically learn the number of prototypes needed to represent
every class
 Only the prototypes used to determine the decision boundary will
be remained
 Realize both types of incremental learning
 Robust to noise
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
19

20. F.Noorbehbahani - Isfahan University of Technology - Agust 2013
20

21. Adjusted SOINN
classifier (ASC)
• SOINN learns k for k-means
• Noise-reduction removes
noisy prototypes
• Center-cleaning removes
prototypes unuseful for
decision
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
21

22. Noise reduction-Center cleaning
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
22

23. Experiment results: artificial data
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
23

24. Experiment results: artificial data
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
24

25. http://haselab.info/index-e.html
F.Noorbehbahani - Isfahan University of Technology - Agust 2013
25