ISSN: 2277 – 9043                             International Journal of Advanced Research in Computer Science and Electroni...
ISSN: 2277 – 9043                              International Journal of Advanced Research in Computer Science and Electron...
ISSN: 2277 – 9043                              International Journal of Advanced Research in Computer Science and Electron...
ISSN: 2277 – 9043                               International Journal of Advanced Research in Computer Science and Electro...
ISSN: 2277 – 9043                                   International Journal of Advanced Research in Computer Science and Ele...
Upcoming SlideShare
Loading in...5
×

88 92

990

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
990
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "88 92"

  1. 1. ISSN: 2277 – 9043 International Journal of Advanced Research in Computer Science and Electronics Engineering Volume 1, Issue 5, July 2012 ANN Implementation for Classification of Noisy Numeral Corrupted By Salt and Pepper Noise Smita K. Chaudhari* G.A.Kulkarni Dept. Of E& C Engg SSGBCOET,Bhusawal Dept. Of E& C Engg SSGBCOET,Bhusawal smita.c20@gmail.com girish227252@rediffmail.com of as an "expert" in the category of information it has been given toAbstract— Neural Network (NN) is information processing analyse.paradigm that is inspired by the way biological nervous Neural networks take a different approach to problem solvingsystems, such as the brain, process information. Neural than that of conventional computers. Conventional computers useNetworks are known to be capable of providing good an algorithmic approach i.e. the computer follows a set ofrecognition rate in presence of noise. Neural Network with instructions in order to solve a problem. Unless the specific stepsvarious architectures and Training algorithms have successfully that the computer needs to follow are known the computer cannotbeen applied for letter or character recognition [1]. Numerals solve the problem. That restricts the problem solving capability ofRecognition is one of the artificial intelligence applications conventional computers to problems that we already understand andwhich provide an important fundamental for various advanced know how to solve. But computers would be so much more useful ifapplications, including information retrieval and they could do things that we dont exactly know how to do. Neuralhuman-computer interaction applications. The neural networks networks process information in a similar way the human brainare also able to extract meaningful features of the digits, such as does.edges. Handwritten recognition is complex due to large Object recognition is the study of how machines can observe thevariation of handwritten style whereas printed character environment, learn to distinguish patterns of interest and makerecognition is also difficult due to increase number fonts. reasonable decisions about the categories of patterns. TheThis paper uses hamming netwok to recognize noisy numerals. performance of a machine may be better than the performanceThe proposed algorithm will design a system which associates of a human in a noisy environment due to the factors: humanevery fundamental pattern with itself. That is, when presented performance degrades with increasing number of targets; where aswith xi as input, the system should produce xi at the output. In the performance of a machine does not depend on the size of theaddition, when presented with a noisy (corrupted) version of xi set of targets. The performance of a machine does not degradeat the input, the system should also produce xi at the output. due to fatigue caused by prolonged effort. A knowledge basedThe recognition results of the noisy numeral showed that the system is desirable for reliable, quick and accurate recognition ofnetwork could recognize normal numerals with 100% accuracy, objects from noisy and partial input images [3].numerals added with salt and pepper noise at average of 89%. The McCulloch and Pitts model was utilized in the development of the first artificial neural network by Rosenblatt in 1959 [11]. This network was based on a unit called the perceptron, which produces an output scaled as 1 or -1 depending upon the weighted, linear Index Terms— Character Recognition, Hamming Network, combination of inputs.Noisy Numeral, Salt & pepper Noise, Neural Network. The optical character recognition system for hand printed numerals of noisy and low-resolution measurement consists of the two-stage feature extraction process. In the first stage a set of I. INTRODUCTION primary features insensitive to the quality and format of a Recently, neural network becomes more popular as a black-white bit pattern are extracted. In the second stage, a set oftechnique to perform character recognition. It has been reported properties capable of discriminating the character classes is derivedthat neural networks could produce high recognition accuracy. from primary features. The system is simple and reliable in that onlyNeural networks are capable of providing good recognition at three kinds of primary features are needed to be detected. Thethe present of noise that other methods normally fail. recognition is based on the decision tree which tests the logic An Artificial Neural Network (ANN) is information processing statements of secondary features. [12]paradigm that is inspired by the way biological nervous systems, The importance of using a hierarchical network is shownsuch as the brain, process information. The key element of this in literature [16] Seong-Whan Lee finds a new scheme for off-lineparadigm is the novel structure of the information processing system. recognition of totally unconstrained handwritten numerals using aIt is composed of a large number of highly interconnected simple multilayer cluster neural network trained with the backprocessing elements (neurons) working in unison to solve specific propagation algorithm which avoids the problem of finding localproblems. ANNs, like people, learn by example. An ANN is minima & improves the recognition rates [10]configured for a specific application, such as pattern recognition or H. K. Kwan introduced multilayer recurrent neuraldata classification, through a learning process. Learning in networks in the form of 3-layer bidirectional symmetrical andbiological systems involves adjustments to the synaptic connections asymmetrical associative memories are presented. The networksthat exist between the neurons. This is true of ANNs as well. Neural possess the features of both a multilayer feedforward neural networknetworks, with their remarkable ability to derive meaning from and a bidirectional associative memory. These networks can havecomplicated or imprecise data, can be used to extract patterns and two modes of recalling, namely, recalling by one pattern anddetect trends that are too complex to be noticed by either humans or recalling by a pattern pair in[12]other computer techniques. A trained neural network can be thought Recognition of Noisy Numerals using Neural Network by Mohd Yusoff Mashor and Siti Noraini Sulaiman.This paper uses MLP network trained using Levenberg-Marquardt algorithm to recognise noisy numerals. The recognition results of the noisy 88 All Rights Reserved © 2012 IJARCSEE
  2. 2. ISSN: 2277 – 9043 International Journal of Advanced Research in Computer Science and Electronics Engineering Volume 1, Issue 5, July 2012numeral showed that the network could recognize normal indicate the correlation between the prototype patterns and the inputnumerals, blended numerals[15]. matrix. The neurons will compete each other to determine a winner. When II. BACKGROUND & TERMINOLOGY. the processes are finished, there will be only one neuron with nonzero output. This neuron indicates the prototype pattern that is The Hamming network method was developed by a closest to the input.mathematician, Richard W. Hamming. He has many contributions The processes in the recurrent layer will be divided into iterations.not only in the mathematical field, but also for computer science and When one iteration is finished, a function will check whether theretelecommunication [5]. He was also the founder and has been the is only one nonzero output. If available so , the process in this layerpresident of Association for Computing Machinery. Hamming will be stopped, and the process will continue to generate the output.network method is developed to solve pattern recognition problems In Figure D is the function to check whether there is only onewhich use binary format, such as a matrix with only two possible nonzero output. W2 is the weight matrix for this layer with thevalues, 0 and 1. In the Hamming network, there is a matrix which dimension of S x S. The iteration number will be given as t, and itstores the patterns of all objects, called the prototype data matrix. will be added by one until the iteration stopped. The activationThe patterns will not be learned by the system, but rather to be function which is used is the positive linear transfer functionstored as a matrix data. The matrix will be used to define the output (poslin). This function is linear for positive values and zero forof the network. The objective of the Hamming network is to decide negative values.which prototype matrix is closest to the input matrix. It calculatesthe similarities between the prototype matrix of all objects and theinput. C. Salt and Pepper Noise. It is designed explicitly to solve binary pattern recognition Salt and pepper noise is an impulse type of noise, which is alsoproblems. It has both feed forward and recurrent layer. The number referred to as intensity spikes. This is caused generally due to errorsof neuron in the first layer is the same as the number of neurons in in data transmission. It has only two possible values, a and b. Thethe second layer. The objective of the hamming network is to probability of each is typically less than 0.1. The corrupted pixelsdecide which prototype vector is closest to the input vector. are set alternatively to the minimum or to the maximum value,This decision is indicated by the output of the recurrent layer. When giving the image a “salt and pepper” like appearance. Unaffectedthe network converges, there will be only one nonzero output. This pixels remain unchanged. For an 8-bit image, the typical value forindicates the prototype pattern that is closest to the input vector. pepper noise is 0 and for salt noise 255. The salt and pepper noise is generally caused by malfunctioning of pixel elements in the camera sensors, faulty memory locations, or timing errors in the digitization process. The probability density function for this type of noise is shown in Figure 2. Salt and pepper noise with a variance of 0.05 is shown in Image 3 Fig.1 Hamming Network A. Feedforward Layer Fig 2 . PDF for salt and pepper noise Feedforward layer calculates the correlation between eachpatterns of the prototype matrix and the input matrix (figure 1). Thecalculation results will be processed to generate the output neuronsfor this layer. As shown in the figure 1, the layer has the input matrix from p,which has the dimension as R x 1. This input matrix goes to theweight matrix (W1) with the dimension of S x R. The net of thislayer (n1) will be the sum of the W1p and the bias input b. Theweight matrix of W1 will be the matrix of the prototype data whichinclude the patterns of all objects. The element of the bias b will begiven as the number of R. The transfer function which is used in this Fig 3. Salt & Pepper Noise.layer is the linear transfer function (purelin). This function will notchange the value so the output of this feedforward layer (a1) will begiven as: a1 = purelin (W1p + b1) The output neurons of this layerwill be used as the initial input for the recurrent layer. III. LITERATURE SURVEY.B.Recurrent Layer The neural networks were also able to extract meaningful features of the digits, such as edges. The cascade correlation The recurrent layer is also called as a competitive layer. In this network was the least successful, possibly because the network waslayer, there is a neuron for each prototype pattern. The neurons are committing itself to poor results on in training when few hiddeninitialized with the output neurons of the feedforward layer, which units were present. It was found that an elaborate conjugate gradient minimization technique yielded little improvement in generalization 89 All Rights Reserved © 2012 IJARCSEE
  3. 3. ISSN: 2277 – 9043 International Journal of Advanced Research in Computer Science and Electronics Engineering Volume 1, Issue 5, July 2012performance and resulted in six times longer training time than H. K. Kwan introduced multilayer recurrent neuralordinary backpropagation. The importance of using a hierarchical networks in the form of 3-layer bidirectional symmetrical andnetwork is shown in literature [9] Seong-Whan Lee finds a new asymmetrical associative memories are presented. The networksscheme for off-line recognition of totally unconstrained handwritten possess the features of both a multilayer feedforward neural networknumerals using a simple multilayer cluster neural network trained and a bidirectional associative memory. These networks can havewith the back propagation algorithm which avoids the problem of two modes of recalling, namely, recalling by one pattern andfinding local minima & improves the recognition rates [10] recalling by a pattern pair in[12]. Le Cun et al. [19] achieved excellent results with a back Recognition of totally unconstrained handwritten numeralpropagation network using size-normalized images as direct input. strings. The system is built upon a number of components, named, aTheir solution consists of a network architecture which is highly presegmentation module, an isolated numeral recognizer, aconstrained and specifically designed for the task. There are four segmentation-free module and a merging module. Presegmentationinternal layers, two layers made of independent groups of feature consists in dividing continuous numeral string image into groups ofextractors and two layers which perform averaging/sub-sampling. numerals, each of which represents an integer number of numerals.The last internal layer is fully connected to the ten-element output, For each group, the actual number of numerals and their identity arebut all other connections are local and use shared weights. In total, then determined by a cascade of two recognition-based tests:there are 4,635 units and 98,442 connections but only 2,578 isolated numeral and segmentation-free. The last one is able toindependent parameters. recognize a numeral group of any length. All results from all groups A modified quadratic classifier based scheme was used for the are eventually merged yielding the final interpretation of the inputrecognition of off-line handwritten numerals of six popular Indian numeral string. The concept of dummy symbol in order to overcomescripts such as Devnagari, Bangla, Telugu, Oriya, Kannada and the problem o f noisy parts that cannot be eliminated by standardTamil scripts. The features used in the classifier are obtained from filtering algorithms.[13]the directional information of the numerals. For featurecomputation, the bounding box of a numeral is segmented into IV. DESIGN AND IMPLEMENTATION OF THEblocks and the directional features are computed in each of the SYSTEM.blocks. These blocks are then down sampled by a Gaussian filterand the features obtained from the down sampled blocks are fed to amodified quadratic classifier for recognition.[20] The system designed in this paper associates everyAmit Choudhary analyzes the performance of back-propagation fundamental pattern with itself. That is, when presented with xi asfeed-forward algorithm using various different activation functions input, the system should produce xi at the output. In addition, whenfor the neurons of hidden and output layers. For sample creation, presented with a noisy (corrupted) version of xi at the input, the250 numerals were gathered form 35 people. After binarization, system should also produce xi at the output. The system which isthese numerals were clubbed together to form training patterns for developed is a system that gets an input of digit, process it throughthe neural network. Network was trained to learn its behavior by the network, and generates the result. The digit which are used in theadjusting the connection strengths at every iteration. The conjugate development are limited to printed digit from 1 to 9. The system hasgradient descent of each presented training pattern was calculated to some prototype data that consists of the pattern of digits, from 1 toidentify the minima on the error surface for each training pattern. 9. This prototype data is used as the weight matrix for the process inExperiments were performed by selecting different combinations of the feedforward layer of the Hamming network. The system is builttwo activation functions out of the three activation functions using the MATLAB and the images are processed using the„logsig‟, „tansig‟ and „purelin‟ for the neurons of the hidden and Microsoft Paint.output layers and the results revealed that the percentage The type of the image file is bitmap (.bmp) . The image is read &recognition accuracy of the neural network was observed to be converted into 64×64 matrix form. This matrix is converted to 8×8optimum when „tansig‟-„tansig‟ combination of activation functions matrix to reduce the computations. Since two dimensional inputwas used for neurons of hidden and output layers.[16] can‟t be given to neural network then it is converted to 64×1 Handwritten Numeral recognition plays a vital role in column vector and this column vector is the prototype pattern Thepostal automation services especially in countries like India where system will have a function that simulates the Hamming network.multiple languages and scripts are used. Because of intermixing of The function will act as the network and process the input data tothese languages; it is very difficult to understand the script in which generate the output. The output neuron of the network indicates thethe pin code is written. Objective of this paper is to resolve this result of the recognition process.problem through Multilayer feed-forward back-propagation In the feedforward layer, p is the input matrix. It will be the matrixalgorithm using two hidden layer. This work has been tested on five of the input image which size is 8 x 8. Therefore, the input p will bedifferent popular Indian scripts namely Devnagri, English, Urdu, a matrix of 64 x 1. The R number is 64, which is the number of inputTamil and Telugu. Network was trained to learn its behavior by neuron, while S is the number of the output neuron for this network,adjusting the connection strengths on every iteration. The resultant which is 9. The weight matrix W1 will be generated using theof each presented training pattern was calculated to identify the prototype data. It will take the prototype data matrix of the 9 digits,minima on the error surface for each training pattern. Experiments so the weight matrix will be a matrix of 9 x 64. The bias b will be awere performed on samples by using two hidden layers and as the matrix of 9 x 1.number of hidden layers is increased, more accuracy is achieved in In the recurrent layer, there are 9 output neurons which representlarge number of epochs.[17] the number of digits. The recognition of machine printed and handwritten The salt & pepper noise having different density is added in thenumerals has been the subject of much attention in pattern image by using the MATLAB function & then it is processed &recognition because of its number of applications such as bank recognized by using the designed system.check processing, interpretation of ID numbers, vehicle registrationnumbers and pin codes for mail sorting. Promising feature V. PERFORMANCE ANALYSIS.extraction methods have been identified in the literature forrecognition of characters and numerals of many different scripts. The task was to design a system which associates every iThese include template matching, projection histograms, geometric fundamental pattern with itself. That is, when presented with x as imoments, Zernike moments, contour profile, Fourier descriptors, input, the system should produce x at the output. In addition, when iand unitary transforms. A brief review of these feature extraction presented with a noisy (corrupted) version of x at the input, the imethods is found in [21] system should also produce x at the output. 90 All Rights Reserved © 2012 IJARCSEE
  4. 4. ISSN: 2277 – 9043 International Journal of Advanced Research in Computer Science and Electronics Engineering Volume 1, Issue 5, July 2012 Let the Hamming distance between two binary vectors x andy (of the same dimension) be denoted as d(x, y). The design phase 100of the Hamming memory involves simply storing all the patterns ofthe fundamental memory set. In the recall phase, for a given input 80 %Accuracy Nmemory key x є (0, 1) , the retrieved pattern is obtained as follows (1) Compute the Hamming distances dk = d(x, xk), k = 1, 2, 60 ……., m. (2) Select the minimum such distance dk = min {d1, d2, 40 …..dm} % Accuracy k 20 (3) Output the fundamental memory y = x (closest match) (4) Input: storage patterns for Hamming network. 0 (5) Input prototype images for digits 1-9 from .bmp format. (6) Example: p = 64*64 matrix of prototype input image of 1 2 3 4 5 6 7 8 9 digit 1. Input Pattern Fig. 5. Reconstruction Efficiency for Salt & Pepper Noise Reconstruction is done by Hopfield network which gives which gives maximum % accuracy for digit 1. Figure 5 shows the graph of % accuracy of reconstruction and input pattern Fig. 4. Prototype ImagesScale data and display as image to use the full colormap. Colormap VI. CONCLUSION.(gray) sets the current figure‟s colormap to gray. The values are inthe range from 0 to 1. A colormap matrix may have any number of Pattern recognition can be done in normal computers androws, but it must have exactly 3 columns. Each row is interpreted as neural networks. Computers use conventional arithmetic algorithmsa color, with the first element specifying the intensity of Red light, to detect whether the given pattern matches an existing one. It willthe second Green light, and the third Blue. Color intensity can be say either yes or no. It does not tolerate noisy patterns. On the otherspecified on the interval 0.0 to 1.0. For example, [0 0 0] is black, [1 hand, neural networks can tolerate noise and, if trained properly,1 1] is white, [1 0 0] is pure Red, [.5 .5 .5] is gray, and [127/255 1 will respond correctly for unknown patterns. Neural networks212/255] is aquamarine. Resizes a matrix map image to an 8*8 constructed with the proper architecture and trained correctly withmatrix to reduce computations. i.e. Convert and compression of good data give amazing results, not only in pattern recognition butimage. also in other scientific and commercial applications. Example: p2 = resize (p, [8, 8]). The model hamming is used for image pattern classificationTable 1: Classification Efficiency/ Output digit for salt & paper noise this algorithm supply the prototype images in the model memory and then use the memory later to identify the stored Input Noise Density/ Recognised output % patterns; when distorted input is given as input to the model . Efficiency of both models varies according to the noise. The Pattern 0.01 0.05 0.1 0.5 1 Accuracy hamming network could recognize the input numerals added with 1 1 1 1 1 4 80 salt and pepper noise at average of 89% . The developed system can be used in car plate recognition. In future we can consider alphabets 2 2 2 2 2 3 80 for recognition. 3 3 3 3 3 1 80 REFERENCES 4 4 4 4 4 1 80 [1] Hagan M.T., Demuth H. B., Beale M., “Neural Network Design”, 5 5 5 5 5 4 80 Thomson Learning Vikas Publishing House, 1996. [2] R C.Gonzalez, R. E. Woods, “Digital Image Processing”, Pearson 6 6 6 6 1 4 60 Education, Inc. and Dorling Kindersley Publishing, Inc.2008. 7 7 7 7 7 9 80 [3] S Jayaraman, S Esakkirajan, T Veerakumar, “Digital Image Processing”, Tata McGraw Hill Education, 2009 8 8 8 8 8 4 80 [4] Earl, Gose, Richard Johnsonbaugh, Steve Jost, “Pattern recognition and Image analysis”, Asoke K. Ghosh, Prentice Hall, 1997. 9 9 9 9 9 4 80 [5] “Statistical Pattern Recognition: A Review”, Anil K. Jain,Fellow, IEEE, Robert P.W.Duin, and Jianchang Mao, Senior Member, IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 22,Classification efficiency of the system is different for each digit for No. 1, January 2000.the increased noise density. It gives maximum 80% accuracy for [6] Zurada, J. M., “Introduction to Artificial Neural Systems”, Jaiconumeral 1 to 5 and 7 to 9 publishing House, Mumbai, 2002. [7] The IEEE website. [Online]. Available: http://www.ieee.org/ “PDCA12-70 data sheet,” Opto Speed SA, Mezzovico, Switzerland. [8 ] McCulloch, W.S., and Pitts, W. (1943), "A Logical Calculus of the Ideas Immanent in Nervous Activity," Bulletin of Mathematical Biophysics, 5, 115-133. [9] Brion IC. Dolenko and Howard C. Card, “Handwritten Digit Feature Extraction and Classification Using Neural Networks", „CCECE’, 1993, IEEE 0-7803-1443, PP 88-91. 91 All Rights Reserved © 2012 IJARCSEE
  5. 5. ISSN: 2277 – 9043 International Journal of Advanced Research in Computer Science and Electronics Engineering Volume 1, Issue 5, July 2012[10] Seong-Whan Lee, “off-line recognition of totally unconstrained handwritten numerals using a simple multilayer cluster neural network”, IEEE transactions on pattern analysis and machine intelligence, vol. 18, no. 6, June 1996, 648-652[11] Rosenblatt, F. (1959), "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review 65:386-408.[12] “Recognition of handprinted numerals by two-stage feature extraction”, IEEE transactions on systems science and cybernetics, April 1970[13] “Handwritten alphabet recognition using hamming network”, Arnold Aribowo, Samuel Lukas, Handy, Seminar National Aplikasi Teknologi Informasion, 2007 (SNATI 2007) ISSN: 1907-5022 Yogyakarta, 16 June 2007[14] A Two-Level Hamming Network for High Performance Associative Memory”, by Nobuhiko Ikeda, Paul Watta, Metin Artiklar and Mohamad H. Hassoun.[15] “Recognition of Noisy Numerals using Neural Network”, Mohd Yusoff Mashor and Sitz Noraini Sulaiman Centre for Electronic Intelligent System (CELIS), School of Electrical and Electronic Engineering, University Sains Malaysia Engineering Campus, 14300 Nibong Tebal Pulau Pinang, Malaysia.[16] Amit Choudhary, Rahul Rishi, Savita Ahlawat, “Performance Analysis of Feed Forward MLP with various Activation Functions for Handwritten Numerals Recognition” IEEE, Volume 5, 2010,pp 852-856,[17] Stuti Asthana, Farha Haneef, Rakesh K Bhujade, “Handwritten Multiscript Numeral Recognition using Artificial Neural Networks” IJSCE 2231-2307, Volume-1, Issue-1, March 2011 pp[18] Leah Bar, Nir Sochen, and Nahum Kiryati “Image eblurring in the Presence of Salt-and-Pepper Noise”IEEE IMAGE PROCESSING, VOL. 16, NO. 4, APRIL 2007[19] Y. Le Cun, et al., ”Constrained Neural Network for Unconstrained Handwritten Digit Recognition,” Proc. of First Int. Workshop on Frontiers in Handwriting Recognition, Montreal, Canada, 1990, pp. 145-154,. [20] U. Pal, T. Wakabayashi and F. Kimura, “Handwritten numeral recognition of six popular scripts,” Ninth International conference on Document Analysis and Recognition ICDAR 07, Vol.2, pp.749-753, 2007.[21] Øivind Due Trier, Anil K. Jain and Torfinn Taxt, Feature Extraction Methods for Character Recognition- A survey, Pattern Recognition, Volume 29, Issue 4, April 1996, pp 641-662. Smita K. Chaudhari is presently Pursuing ME in Electronics & Communication Engineering from SSGB College of Engg. & Technology, North Maharashta University- Jalgaon, Maharashtra, India. She received the BE degree from Godavari College of Engg.& Technology, North Maharashta University, Jalgaon. Her interested area is Image processing, Neural Network,VLSI. G.A.Kulkarni is presently Associate Professor & Head of Electronics & Communication Engg. Department SSGB College of Engg. & Technology, affiliated to North Maharashtra University- Jalgaon, Maharashtra, India.. He received the M.E degree from the Dr.B.A.M. University Aurangabad and presently he is persuing PhD degree from Dr.B.A.M. University Aurangabad. His research interests include communication systems & electromagnetic engg., Neural network. 92 All Rights Reserved © 2012 IJARCSEE

×