The document compares the performance of a Fuzzy ARTMAP neural network and a multilayer perceptron on the task of handwritten digit recognition using data from the Unipen dataset. It finds that while the recognition performance of both networks is similar, the Fuzzy ARTMAP converges much faster during training and can be trained in a single pass with little performance loss using fast learning mode. The Fuzzy ARTMAP also exhibits stable incremental learning.
Ce cours présente comment définir de nouveaux objets en définissant des classes. Un objet est une instance d'une classe qui définit les variables d'instances (attributs) et méthodes (fonctionnalités) que les objets créés à partir de la classe auront.
Ce cours présente comment définir de nouveaux objets en définissant des classes. Un objet est une instance d'une classe qui définit les variables d'instances (attributs) et méthodes (fonctionnalités) que les objets créés à partir de la classe auront.
This presentation is intended for giving an introduction to Genetic Algorithm. Using an example, it explains the different concepts used in Genetic Algorithm. If you are new to GA or want to refresh concepts , then it is a good resource for you.
Presentation is about genetic algorithms. Also it includes introduction to soft computing and hard computing. Hope it serves the purpose and be useful for reference.
ARTIFICIAL NEURAL NETWORK APPROACH TO MODELING OF POLYPROPYLENE REACTORijac123
This paper shows modeling of highly nonlinear polymerization process using the artificial neural network approach for the model predictive purposes. Polymerization occurs in a fluidized bed polypropylene reactor using Ziegler - Natta catalyst and the main objective was modeling of the reactor production rate.
The data set used for an identification of the model is a real process data received from an existing polypropylene plant and the identified model is a nonlinear autoregressive neural network with the exogenous input. Performance of a trained network has been verified using the real process data and the
ability of the production rate prediction is shown in the conclusion.
A New Classifier Based onRecurrent Neural Network Using Multiple Binary-Outpu...iosrjce
IOSR Journal of Computer Engineering (IOSR-JCE) is a double blind peer reviewed International Journal that provides rapid publication (within a month) of articles in all areas of computer engineering and its applications. The journal welcomes publications of high quality papers on theoretical developments and practical applications in computer technology. Original research papers, state-of-the-art reviews, and high quality technical notes are invited for publications.
A Threshold Logic Unit (TLU) is a mathematical function conceived as a crude model, or abstraction of biological neurons. Threshold logic units are the constitutive units in an artificial neural network. In this paper a positive clock-edge triggered T flip-flop is designed using Perceptron Learning Algorithm, which is a basic design algorithm of threshold logic units. Then this T flip-flop is used to design a two-bit up-counter that goes through the states 0, 1, 2, 3, 0, 1… Ultimately, the goal is to show how to design simple logic units based on threshold logic based perceptron concepts.
DESIGN AND IMPLEMENTATION OF BINARY NEURAL NETWORK LEARNING WITH FUZZY CLUSTE...cscpconf
In this paper, Design and Implementation of Binary Neural Network Learning with Fuzzy
Clustering (DIBNNFC), is proposed to classify semisupervised data, it is based on the
concept of binary neural network and geometrical expansion. Parameters are updated
according to the geometrical location of the training samples in the input space, and each
sample in the training set is learned only once. It’s a semisupervised based approach, the
training samples are semi-labelled i.e. for some samples, labels are known and for some
samples data labels are not known. The method starts with classification, which is done by
using the concept of ETL algorithm. In classification process various classes are formed.
These classes classify samples in to two classes after that considers each class as a region and calculates the average of the entire region separately. This average is centres of the region which is used for the purpose of clustering by using FCM algorithm. Once clustering process over labelling of semi supervised data is done, then whole samples would be classify by (DIBNNFC). The method proposes here is exhaustively tested with different benchmark datasets and it is found that, on increasing value of training parameters number of hidden neurons and training time both are getting decrease. The result reported, using real character recognition data set and result will compare with existing semi-supervised classifier, the proposed approach learned with semi-supervised leads to higher classification accuracy.
Secured transmission through multi layer perceptron in wireless communication...ijmnct
In this paper, a multilayer perceptron guided encryption/decryption (STMLP) in wireless communication
has been proposed for exchange of data/information. Multilayer perceptron transmitting systems at both
ends generate an identical output bit and the network are trained based on the output which is used to
synchronize the network at both ends and thus forms a secret-key at end of synchronizations of the
networks. Weights or hidden units of the hidden layer help to form a secret session key. The plain text is
encrypted through chaining , cascaded xoring of multilayer perceptron generated session key. If size of the
final block of plain text is less than the size of the key then this block is kept unaltered. Receiver will use
identical multilayer perceptron generated session key for performing deciphering process for getting the
plain text. Parametric tests have been done and results are compared in terms of Chi-Square test, response
time in transmission with some existing classical techniques, which shows comparable results for the
proposed technique. Variation numbers of input vectors and hidden layers will increase the confusion
/diffusion of the schemeand hence increase the security. As a result variable energy based techniques may
be achieved which may be applicable devices/interface of the heterogeneous sizes of the network/device.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Dr. Vinod Kumar Kanvaria
Exploiting Artificial Intelligence for Empowering Researchers and Faculty,
International FDP on Fundamentals of Research in Social Sciences
at Integral University, Lucknow, 06.06.2024
By Dr. Vinod Kumar Kanvaria
1. UNIVERSITÉ
LA AL
V
A Comparison of Fuzzy ARTMAP
and Multilayer Perceptron for
Handwritten Digit Recognition
Martin Busque Marc Parizeau
October 31, 1997
Computer Vision and Systems Laboratory
Universit Laval
e
Sainte-Foy (Qubec), Canada
e
G1K 7P4
Tel: (418) 656-2131, ext. 7912
Fax: (418) 656-3159
Email: parizeau@gel.ulaval.ca
2. A Comparison of Fuzzy ARTMAP and Multilayer
Perceptron for Handwritten Digit Recognition
Abstract
This paper explores the features of the Fuzzy ARTMAP neural net-
work classi
3. er in the context of on-line handwritten recognition. A Fuzzy
ARTMAP classi
4. er is compared with a multilayer Perceptron trained using
the standard backpropagation algorithm. Results show that although the
recognition performance of both networks is similar, the convergence of the
Fuzzy ARTMAP is both much faster and incrementaly stable than that of
the Perceptron. In fact, it is shown experimentally that a Fuzzy ARTMAP
can be trained in a single pass with almost no performance penalty when
using the fast learning mode. The recognition experiments were conducted
on handwritten digits extracted from the Unipen training dataset.
Keywords: Neural Networks; Fuzzy ARTMAP; Multilayer Perceptron; Hand-
written Digit Recognition; Unipen Dataset.
1 Introduction
There are many ways to model a system for handwriting recognition. One of
them is to use a neural network. Still, there exist many dierent kinds of neural
networks. Among them, the multilayer Perceptron and the Fuzzy ARTMAP.
The Perceptron is now a well-known architecture [1]. It is able to model
a system, based on a set of inputs with the correponding desired outputs. It
1
5. appeared in the 1960's and has much evolved, especially since the development
of the back-propagation algorithm in the 1980's. This supervised learning neural
network has proven in practice to be quite ecient and has been used on a number
of diverse problems.
What particularly interests us in this discussion is the Fuzzy ARTMAP. This
network, introduced much more recently (1991), is a supervised network like the
Perceptron. But it has some very interesting additional properties, such as a
very fast convergence and a capacity for incremental learning. Our objective
is to compare the Fuzzy ARTMAP with the Perceptron on handwritten digit
recognition.
First, in Section 2, the Fuzzy ART will be presented. The Fuzzy ART is an
unsupervised network which is the essential component of the Fuzzy ARTMAP.
Then the Fuzzy ARTMAP itself will be explained in Section 3. We will discuss
in Section 4 the experimental results obtained on handwritten digit recognition
with both the Perceptron and the Fuzzy ARTMAP.
2 Fuzzy ART
The Fuzzy ART is a neural network introduced by Carpenter, Grossberg and
Rosen in 1991 [2]. It is a modi
6. ed version of the binary ART1 [3], which is
notably able to accept analog fuzzy input patterns, i.e. vectors whose components
are real numbers between 0 and 1. The Fuzzy ART is an unsupervised neural
network capable of incremental learning, that is it can learn continouously without
2
7. F2
F1
Complement
0.4 0.8 0.6 0.2 coding
0.4 0.8
Input vector
Figure 1: Sample Fuzzy ART network.
forgetting what it has previously learned.
A Fuzzy ART network is formed of two layers of neurons, the input layer F1
and the output layer F2 , as illustrated in Figure 1. Both layers have an activity
pattern, schematized on the
8. gure with vertical bars of varying height. The layers
are fully interconnected, each neuron being connected to every neuron on the other
layer. Every connection is weighted by a number lying between 0 and 1. A neuron
of F2 represents one category formed by the network and is caracterized by its
weight vector wj (j is the index of the neuron) . The weight vector's size is equal to
the dimension M of layer F1 . Initially all the weight vectors' components are
10. ed, we say that it is uncommited.
Inversely, once a neuron's weights have been modi
11. ed, this neuron is said to be
commited.
The network uses a form of normalization called complement coding. The
operation consists on taking the input vector and concatenating it with its com-
3
12. plement. The resulting vector is presented to layer F1 . Therefore, the dimension
M of layer F1 is the double of the input vector's dimension. Complement coding
can be deactivated, in this case layer F1 will have the same dimension as the in-
put vector. Unless speci
13. ed otherwise, we will always suppose that complement
coding is active.
The Fuzzy ART learns by placing hyperboxes in the M {dimensions hyperspace,
2
M being the size of layer F1 . As said earlier, each neuron of layer F2 represents
a category formed by the network, and this category is de
14. ned by a box1 [2,
section 5]. The position of the box in the space is encoded in the weight vector
of the neuron. Because of complement coding and of the learning process we will
explain later, the
15. rst half of the weight vector memorizes one corner of the box
(the closest to the origin) and the other half memorizes the opposite one.
Learning process
When an input vector is presented to the network, it is
16. rst preprocessed into
complement coding form. The resulting vector, I, represents a point in the space.
The activity pattern on layer F1 is set equal to I. The choice function is then
evaluated for each neuron of F2 . This function is de
17. ned by
Tj = I+ jwj jj
j ^
w (1)
j
where j j is the L1 norm of the vector, i.e. the sum of its components (jpj
PM pi), ^ is the fuzzy AND operator ((p ^ q)i min(pi; qi)) and is the choice
i =1
1 To simplify the discussion, we will sometimes drop the hyper pre
18. x and use the terms
box and space to refer respectively to hyperbox and hyperspace.
4
19. parameter. The choice parameter must be greater than zero. It is usually chosen
close to zero for a good performance [4, p.701]. This parameter ensures that when
a point is enclosed in more than one hyperbox, the smallest one is selected. This
is in fact the objective of the choice function: to choose the smallest box in which
the point is included. A point is said to be in a box when the weight vector wj
corresponding to this box is a fuzzy subset of the input vector I. If no such box
exist, either the one which needs to be the less expanded to enclose the point is
selected or a new one is created, i.e. the
21. ne the size of a box as the sum of its lengths in each dimension.
Once a neuron is selected (the winning neuron ), a vigilance criteron is eval-
uated. The vigilance criterion causes the network to choose another box (neuron)
if the one chosen is already too large. Thus the vigilance parameter controls the
maximum size of hyperboxes [2, section 5]. Mathematically, the vigilance criterion
is
jI ^ wJ j (2)
jIj
where J is the index of the winning neuron on F2 and is the vigilance parameter,
which lies in the interval [0; 1]. If the criterion is respected, the Fuzzy ART learns
the input vector, else it selects the next neuron with the highest choice function
and reevaluates the vigilance criterion. These two steps are repeated until the
vigilance criterion is respected. At that moment, it is said that the network is in
resonance. All components of the activity pattern on F2 , which is also the output
vector of the Fuzzy ART, are set to 0 except for the index of the winning neuron
which is set to 1.
5
22. When the Fuzzy ART enters resonance, it proceeds with the learning of the
input vector I by modifying the weight vector wJ of the neuron as follows
wJ =
26. = 1, we are in fast learning. We may note that the weights are nonin-
creasing through time, which means that a hyperbox can never shrink but only
expand itself. With fast learning, equation 3 means that the box associated with
the neuron is enlarged just enough to include the point represented by the input
vector. If
27. is smaller than 1, the box will expand towards the point, its enlarge-
ment being directly proportional to the size of
29. 1, we use
what is called the fast-commit slow-recode option. This consists in setting
30. = 1
when the neuron is not commited and using the real value of
31. ( 1) after it is
commited.
3 Fuzzy ARTMAP
The Fuzzy ARTMAP, introduced by Carpenter et al. in 1992 [4] is a supervised
network which is composed of two Fuzzy ARTs. The Fuzzy ARTs are identi
32. ed as
ARTa and ARTb . The parameters of these networks are designated respectively
by the subscripts a and b. The two Fuzzy ARTs are interconnected by a series
of connections between the F2 layers of ARTa and ARTb . The connections are
weighted, i.e. a weight wij between 0 and 1 is associated with each one of them.
These connections form what is called the map
35. xab
F ab
ab
wJ
ARTa ARTb
ya yb
xa xb
A 0.8 0.4 0.2 0.6 0.6 0.4 B
a 0.8 0.4
Figure 2: Sample Fuzzy ARTMAP network.
parameters (
36. ab and ab ) and an output vector xab .
The input vector a of ARTa is put in complement coding form, resulting in
vector A. Complement coding is not necessary in ARTb so we present the input
vector B directly to this network. Figure 2 presents the structure of the Fuzzy
ARTMAP. The weights of the map
37. eld's connections are illustrated by a vertical
bar with a height proportional to the size of the weight. The weights of the map
38. eld are all initialized to 1.
Training
In order to train the Fuzzy ARTMAP, we present to ARTa a vector representing
a data pattern and to ARTb a vector which is the desired output corresponding
to this pattern. The network uses a form of hypothesis testing. When it receives
the
39. rst vector, it deduces to which category it should belong. With the second
7
41. rm or reject the hypothesis, in which
case the process is repeated with a new one.
During the training process, learning is deactivated in ARTa (
42. a = 0) and is
controlled by the Fuzzy ARTMAP. The vigilance parameter of ARTa , a , varies
during learning. We identify the initial value of a as the baseline vigilance a .
The vigilance of ARTb, b, is set to 1 to perfectly distinguish the desired output
vectors.
When vectors A and B are presented to ARTa and ARTb , both networks soon
enter resonance. A vigilance criterion is then evaluated to verify if the winning
neuron of ARTa corresponds to the desired output vector presented to ARTb. This
criterion is
jyb ^ wJ j
ab
(4)
jybj ab
where yb is the output vector of ARTb (the activity pattern on F2b), J is the index
of the winning neuron on F2a, wJ corresponds to the weights of the connections
ab
to the J th neuron of F2a and ab 2]0; 1] is the vigilance parameter of the map
44. eld uses fast learning, the value of this fraction will either
be 0 or 1. If the criterion is not respected, the vigilance of ARTa is increased just
enough to select another winning neuron (a jA ^ wJ j=jAj, see equation 2) and
the vector A is repropagated in ARTa .
When the vigilance criterion is respected, the map
45. eld learns the association
between vectors A and B by modifying its weigths as follows
wJ =
49. ed as in equation 3. The vigilance of ARTa
is set back to its baseline value a .
In terms of hypexboxes, the map
50. eld of the Fuzzy ARTMAP associates a
category number to each box created by ARTa . This number is the index of
the associated neuron of F2b. Consequently, each category learned by the Fuzzy
ARTMAP is represented by many rectangles. In other words it learna to distin-
guish the data by mapping boxes into the space, enclosing each category with a
certain number of these boxes.
Classifying
Once the Fuzzy ARTMAP is trained, it can be used as a classi
51. er. In this case,
the ARTb network is not used. We present a single input vector to ARTa which is
propagated until resonance, with a (temporary) vigilance a = 0. Thus, the
52. rst
category selected by the choice function is accepted. Learning in this network is
also temporarily deactivated (
56. ab = 1), the output vector contains only 0's except a 1. The index of
this component is the number of the category in which the input vector A has
been classi
58. eld is thus to associate a category number
to each neuron of ARTa 's F2 layer (F2a), i.e. to each box in the hyperspace.
9
59. If fast learning was not used, one neuron of F2a can correspond to many cat-
egories with dierent degrees. One way to determine the category number could
be to select the index of the maximal component. If needed, the desired output
vector can be restored from the weights of the neuron of F2b whose index is the
category number.
4 Experimental results
We used the multilayer Perceptron and the Fuzzy ARTMAP to recognize hand-
written digits taken from the
60. fth release of the Unipen training set, section 1a2.
All the samples in the database were identi
61. ed with one of the following labels
depending on their quality : Good, Ok and Bad. Good samples are perfectly
recognizable, Ok samples are of a lower quality but should still be recognizable
and Bad samples are either mis-labelled3, mis-segmented4, mis-scaled5 or would
be very dicult to recognize by a human.
The data set for our experiments is composed of a training set and a test set.
We
62. rst selected randomly 300 Good and Ok samples of each digit (0-9) to form
the 3000 samples training set. The test set consists of the rest of the database
(3519 samples), approximately 11% of them being Bad samples.
To transform the data into a vectorial space, we used a fuzzy representation
based on a segmentation of the digits into circular arcs and straight lines [5].
2 For more information on the Unipen database, see http://hwr.nici.kun.nl/unipen/.
3 For example a 1 labelled as a 0.
4 A single digit is segmented into two dierent digits.
5 Two dierent scales are applied on the X and Y axis.
10
63. Each vector representing a data sample has 49 components. Brie
y, each digit
is divided in 6 regions having 7 fuzzy characteristics related to the orientation
and curvature of the segments in those regions. There are 7 more fuzzy variables
related to the whole digit.
Multilayer Perceptron
We will now describe our results with the multilayered Perceptron using the de-
scribed vectorial representation and data sets. We used a Perceptron with one
hidden layer of 100 neurons. The ouput layer has 10 neurons, one neuron for
each digit class. We trained this Perceptron using the standard back-propagation
algorithm with a learning rate = 0:5 and a momentum = 0:5 [1].
We
64. rst presented to the network the data set in a random order. After 200
epochs, an RMS error of 0.0613 was obtained and the training set was correctly
classi
65. ed at 99.97% (single error). The training took a CPU time of 1494 seconds
(24.9 minutes) on a Sun Ultra-sparc 30. We then obtained a classi
66. cation rate
on the test set of 93.0%. We measured the CPU time to compare this result with
the performance of the Fuzzy ARTMAP.
We also trained the Perceptron with the data in order, from digits 0 to 9. After
100 training epochs, an RMS error of 0.0212 was obtained. When we used the
network to classify the test set, the rate of the data that was correctly classi
67. ed
was extremely low, 9.21%. This is not surprising because it is well known that
the multilayer Perceptron is not capable of incremental learning. In fact it only
remembers the few last data it was trained with, in this case the samples of nines.
11
77. b = 1, b = 1. Thus the two varying parameters in our
experiments are
78. a and a , the learning rate and baseline vigilance of the ARTa
12
79. module.
We trained the Fuzzy ARTMAP by presenting the training set in order, from
the zeros to the nines. The learning rate of ARTa was set to
80. a = 1. The
network was trained until it perfectly classi
81. ed the training set. Table 1 presents
the results with dierent baseline vigilances a . These results show that values
of a between 0.75 and 0.90 are the ones yielding the best correct classi
82. cation
rate on the test set. We can note that increasing the baseline vigilance a causes
the Fuzzy ARTMAP to form a higher number of ARTa categories. Because the
maximal size of the boxes formed by a Fuzzy ART is smaller when the vigilance
is higher, ARTa has to create a greater number of boxes, or categories.
A remarkable point in those results is that the number of training epochs and
the corresponding CPU time are both much lower for the Fuzzy ARTMAP than
for the multilayer Perceptron. Still, the best classi
83. cation rate obtained with both
networks are very close to one another (93.0% vs. 92.5%). Moreover, it is not
needed to perfectly classify the training set to obtain good rates on the test set.
In fact, the test set correct classi
84. cation rate is almost always maximal in only
one epoch.
Changing the order of the training set has a small but non negligible in
uence
on the results. The test set classi
85. cation rates may vary by about 1%. We were
able to obtain, by randomly ordering the training set, a correct classi
86. cation rate
on the test set of 92.7% (with a = 0:85) in one epoch (and a CPU time of 184 s).
Table 2 presents results obtained when varying the learning rate of ARTa ,
87. a.
We used a = 0:85, which yielded the best result with
93. a Epochs categories Training set Test set CPU time (s)
0.8 4 1137 100 93.0 621
0.6 4 1077 100 92.1 570
0.4 11 963 100 93.1 967
0.2 11 958 100 93.0 905
data in a random order at each training epoch. The test set correct classi
94. cation
rate was increased up to 93.1%. However, this implies more training epochs and,
since the number of ARTa categories is higher, a larger ARTa network.
Finally, to verify that Fuzzy ARTMAP can do incremental learning, we trained
it in two separate steps, using fast learning and again a = 0:85. First, we per-
formed training with the samples of all digits except the nines. After two epochs,
the training set was perfectly learned and the test set correct classi
95. cation rate
on digits 0 to 8 was 93.1%. Secondly, we trained the same network exclusively on
the nines. It then correctly classi
96. ed the whole test set at 92.4%. It is now clear
that this network can learn new data without forgetting what it was previously
trained with. In fact, only 7 more samples of digits between 0 and 8 were incor-
rectly classi
97. ed (on a total of 3197 samples). Thus the Fuzzy ARTMAP is capable
of incremental learning, which is not the case for the multilayer Perceptron.
14
98. 5 Conclusion
We have seen the structure of the Fuzzy ARTMAP and its essential component,
the Fuzzy ART. We performed handwritten digit recognition with the multilayer
Perceptron and the Fuzzy ARTMAP and compared the performance of these two
neural networks. We saw that they yield similar recognition rates, but Fuzzy
ARTMAP learning process is a lot faster. Moreover, Fuzzy ARTMAP is capable
of incrementaly stable learning.
We presented the best results we obtained so far but we believe the perfor-
mance of the Fuzzy ARTMAP may still be improved. Nevertheless, it is clear
that Fuzzy ARTMAP is a neural network that has attractive features for use in
pattern recognition.
15
99. References
[1] Simon Haykin. Neural Networks: A comprehensive foundation. IEEE Com-
puter Society Press, 1994.
[2] Gail A. Carpenter, Stephen Grossberg, and David B. Rosen. Fuzzy ART: Fast
stable learning and categorization of analog patterns by an adaptive resonance
system. Neural Networks, 4:759{771, 1991.
[3] Gail A. Carpenter and Stephen Grossberg. A massively parallel architecture
for a self-organizing neural pattern recognition machine. Computer Vision,
Graphics and Image Processing, 37:54{115, 1987.
[4] Gail A. Carpenter, Stephen Grossberg, Natalya Markuzon, John H. Reynolds,
and David B. Rosen. Fuzzy ARTMAP: A neural network architecture for in-
cremental supervised learning of analog multidimensional maps. IEEE Trans-
actions on Neural Networks, 3:698{713, 1992.
[5] Jean-Franois Hbert, Marc Parizeau, and Nadia Ghazzali. A new fuzzy ge-
c e
ometric representation for isolated character recognition. Submitted to the
14th International Conference on Pattern Recognition (ICPR '98).
16