Computer Engineering and Intelligent Systems                                                  www.iiste.orgISSN 2222-1719 ...
Computer Engineering and Intelligent Systems                                                   www.iiste.orgISSN 2222-1719...
Computer Engineering and Intelligent Systems                                                   www.iiste.orgISSN 2222-1719...
Computer Engineering and Intelligent Systems                                                   www.iiste.orgISSN 2222-1719...
Computer Engineering and Intelligent Systems                                                    www.iiste.org ISSN 2222-17...
Computer Engineering and Intelligent Systems                                                   www.iiste.orgISSN 2222-1719...
Computer Engineering and Intelligent Systems                                                   www.iiste.orgISSN 2222-1719...
Computer Engineering and Intelligent Systems                                                     www.iiste.orgISSN 2222-17...
Computer Engineering and Intelligent Systems                                                  www.iiste.orgISSN 2222-1719 ...
Computer Engineering and Intelligent Systems                                                  www.iiste.orgISSN 2222-1719 ...
Computer Engineering and Intelligent Systems                                                    www.iiste.orgISSN 2222-171...
Computer Engineering and Intelligent Systems                                             www.iiste.orgISSN 2222-1719 (Pape...
Computer Engineering and Intelligent Systems                                         www.iiste.orgISSN 2222-1719 (Paper) I...
Computer Engineering and Intelligent Systems                                            www.iiste.orgISSN 2222-1719 (Paper...
Computer Engineering and Intelligent Systems                                              www.iiste.orgISSN 2222-1719 (Pap...
Computer Engineering and Intelligent Systems                                             www.iiste.orgISSN 2222-1719 (Pape...
Computer Engineering and Intelligent Systems                                               www.iiste.orgISSN 2222-1719 (Pa...
International Journals Call for PaperThe IISTE, a U.S. publisher, is currently hosting the academic journals listed below....
Upcoming SlideShare
Loading in …5
×

11.development of a writer independent online handwritten character recognition system using modified hybrid

980 views

Published on

IISTE international journals call for paper http://www.iiste.org/Journals

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
980
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

11.development of a writer independent online handwritten character recognition system using modified hybrid

  1. 1. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012 Development of a Writer-Independent Online Handwritten Character Recognition System Using Modified Hybrid Neural Network Model Fenwa O. D.* Department of Computer Science and Engineering, Ladoke Akintola University of Technology, P.M.B 4000, Ogbomoso, Nigeria. *E-mail of the corresponding author: odfenwa@lautech.edu.ng Omidiora E. O. Department of Computer Science and Engineering, Ladoke Akintola University of Technology, P.M.B 4000, Ogbomoso, Nigeria. E-mail: omidiorasayo@yahoo.co.uk Fakolujo O. A. Department of Computer Science and Engineering, Ladoke Akintola University of Technology, P.M.B 4000, Ogbomoso, Nigeria. E-mail: ola@fakolujo.com Ganiyu R. A. Department of Computer Science and Engineering, Ladoke Akintola University of Technology, P.M.B 4000, Ogbomoso, Nigeria. E-mail: ganiyurafiu@yahoo.comAbstractRecognition of handwritten characters has become a difficult problem because of the high variability andambiguity in the character shapes written by individuals. Some of the problems encountered by researchersinclude selection of efficient feature extraction method, long network training time, long recognition timeand low recognition accuracy. However, many feature extraction techniques have been proposed inliterature to improve overall recognition rate although most of the techniques used only one property of thehandwritten character. This research focuses on developing a feature extraction technique that combinedthree characteristics (stroke information, contour pixels and zoning) of the handwritten character to create aglobal feature vector. A hybrid feature extraction algorithm was developed to alleviate the problem of poorfeature extraction algorithm of online character recognition system. Besides, this research work alsofocused on alleviating the problem of standard backpropagation algorithm based on ‘error adjustment’. Ahybrid of modified Counterpropagation and modified optical backpropagation neural network model wasdeveloped to enhance the performance of the proposed character recognition system. Experiments were 89
  2. 2. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012performed with 6200 handwriting character samples (English uppercase, lowercase and digits) collectedfrom 50 subjects using G-Pen 450 digitizer and the system was tested with 100 character samples writtenby people who did not participate in the initial data acquisition process. The performance of the system wasevaluated based on different learning rates, different image sizes and different database sizes. Thedeveloped system achieved better performance with no recognition failure, 99% recognition rate and anaverage recognition time of 2 milliseconds.Keywords: Character recognition, Feature extraction, Neural Network, Counterpropagation, OpticalBackpropagation, Learning rate.1. IntroductionThe use of neural network for handwriting recognition is a field that is attracting a lot of attention. As thecomputing technology advances, the benefits of using Artificial Neural Network (ANN) for the purpose ofhandwriting recognition become more obvious. Hence, new ANN approaches geared toward the task ofhandwriting recognition are constantly being studied. Character recognition is the process of applyingpattern-matching methods to character shapes that has been read into a computer to determine which alpha-numeric character, punctuation marks, and symbols the shapes represent. The classes of recognitionsystems that are usually distinguished are online systems for which handwriting data are captured duringthe writing process (which makes available the information on the ordering of the strokes) and offlinesystems for which recognition takes place on a static image captured once the writing process is over(Anoop and Anil, 2004; Liu et al., 2004; Mohamad and Zafar, 2004; Naser et al., 2009; Pradeep et al.,2011). The online methods have been shown to be superior to their offline counterpart in recognizinghandwriting characters due the temporal information available with the former (Pradeep et al., 2011).Handwriting recognition system can further be broken down into two categories: writer independentrecognition system which recognizes wide range of possible writing styles and a writer dependentrecognition system which recognizes writing styles only from specific users (Santosh and Nattee, 2009).Online handwriting recognition today has special interest due to increased usage of the hand held devices.The incorporation of keyboard being difficult in the hand held devices demands for alternatives, and in thisrespect, online method of giving input with stylus is gaining quite popularity (Gupta et al., 2007).Recognition of handwritten characters with respect to any language is difficult due to variability of writingstyles, state of mood of individuals, multiple patterns to represent a single character, cursive representationof character and number of disconnected and multi-stroke characters (Shanthi and Duraiswamy, 2007).Current technology supporting pen-based input devices include: Digital Pen by Logitech, Smart Pad byPocket PC, Digital Tablets by Wacom and Tablet PC by Compaq (Manuel and Joaquim, 2001). Althoughthese systems with handwriting recognition capability are already widely available in the market, furtherimprovements can be made on the recognition performances for these applications.The challenges posed by the online handwritten character recognition systems are to increase therecognition accuracy and to reduce the recognition time (Rejean and Sargurl, 2000; Gupta et. al., 2007).Various approaches that have been used by many researchers to develop character recognition systems,these include; template matching approach, statistical approach, structural approach, neural networksapproach and hybrid approach. Hybrid approach (combination of multiple classifiers) has become a veryactive area of research recently (Kittler and Roli, 2000; 2001). It has been demonstrated in a number ofapplications that using more than a single classifier in a recognition task can lead to a significantimprovement of the system’s overall performance. Hence, hybrid approach seems to be a promisingapproach to improve the recognition rate and recognition accuracy of current handwriting recognitionsystems (Simon and Horst, 2004). However, Selection of a feature extraction method is probably the singlemost important factor in achieving high recognition performance in character recognition system (Pradeepet. al., 2011). No matter how sophisticated the classifiers and learning algorithms, poor feature extractionwill always lead to poor system performance (Marc et. al., 2001). In furtherance, Fenwa et al.(2012) 90
  3. 3. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012developed a feature extraction technique for online character recognition system using hybrid ofgeometrical and statistical features. Thus, through the integration of geometrical and statistical features,insights were gained into new character properties, since these types of features were considered to becomplementary.2. Research MethodologyThe five stages involved in developing the proposed character recognition system, which include dataacquisition, pre-processing, character processing that comprises feature extraction and character digitization,training and classification using hybrid neural network model and testing, are as shown in Figure 2.2.Experiments were performed with 6200 handwriting character samples (English uppercase, lowercase anddigits) collected from 50 subjects using G-Pen 450 digitizer and the system was tested with 100 charactersamples written by people who did not participate in the initial data acquisition process. The performanceof the system was evaluated based on different learning rates, different image sizes and different databasesizes.2.1 Data AcquisitionThe data used in this work were collected using Digitizer tablet (G-Pen 450) shown in Figure 2.3. It has anelectric pen with sensing writing board. An interface was developed using C# to acquire data (characterinformation) such as stroke number, stroke pressure, etc from different subjects using the digitizer tablet. 26Upper case (A-Z), 26 lower case (a-z) English alphabets and 10 digits (0-9) making a total number of 62characters. 6,200 characters (62 x 2 x 50) were collected from 50 subjects as each individual was requestedto write each of the characters 2 times (this is done to allow the network learn various possible variations ofa single character and become adaptive in nature). This serves as the training data set which was the inputdata that was fed into the neural network.2.2 Data PreprocessingPre-processing is done prior to the application of feature extraction algorithms. Pre-processing aims toproduce clean character images that are easy for the character recognition systems to operate moreaccurately. Feature extraction stage relies on the output of this process. The pre processing techniques usedin this research work is Grid Resizing.2.3 Resizing GridFrom the interface that was provided, there wasn’t any degree of measurement to determine how small/bigthe input character should be. Hence, the character written is resized to a matrix size of 5 by 7, 10 by 14and 20 by 28 for all input characters. This is used to get the universe of discourse which is the shortestmatrix that fit the entire character skeleton. The universe of discourse is measured to easily get a uniformmatrix size in multiple of 5 by 7, 10 by 14 and 20 by 28 respectively.Any character measured smaller than the required size is considerably resized to the multiple of 5 by 7, 10by 14 and 20 by 28 conversely any character measured larger than the required size will also be resized tothe multiple of 5 by 7, 10 by 14 and 20 by 28. This implies that rows and columns of Zero’s are added orsubtracted from the resized image matrix to achieve the required multiple of 5 by 7, 10 by 14 and 20 by 28.2.4 Feature Extraction DevelopmentThe goal of feature extraction is to extract a set of features, which maximizes the recognition rate with theleast amount of elements. Many feature extraction techniques have been proposed to improve overallrecognition rate; however, most of the techniques used only one property of the handwritten character. Thisresearch focuses on a feature extraction technique that combined three characteristics (stroke information,contour pixels and zoning) of the handwritten character to create a global feature vector. Hence, a hybridfeature extraction algorithm was developed using Geometrical and Statistical features as shown in Figure 91
  4. 4. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 20122.4. Integration of Geometrical and Statistical features was used to highlight different character properties,since these types of features are considered to be complementary.2.4.1 The Developed Hybrid (Geom-Statistical) Feature Extraction Algorithm The Hybrid feature extraction model adopted in this work is as shown in Figure 2.4. Stages ofdevelopment of the proposed hybrid feature extraction algorithm are as follow:Stage 1: Get the stroke information of the input characters from the digitizer (G- pen 450). These include: (i) Pressure used in writing the strokes of the characters (ii) Number (s) of strokes used in writing the characters (iii) Number of junctions and the location in the written characters (iv) The horizontal projection count of the character.Stage 2: Apply Contour tracing algorithm to trace out the contour of the characters:Stage 3: Develop a modified hybrid zoning algorithm and RUN it on the contours of the characters: Two zoning algorithms were proposed by Vanajah and Rajashekararadhya in 2008 for the recognition of four popular Indian scripts (Kannada, Telugu, Tamil and Malayalam) numeral. These were: Image Centroid and Zone-based (ICZ) Distance Metric Feature Extraction System (Vanajah and Rajashekararadhya, 2008a) and Zone Centroid and Zone-based (ZCZ) Distance Metric Feature Extraction System (Rajashekararadhya and Vanajah, 2008b). The Two Algorithms were modified in terms of: (i) Number of zones being used (25 Zones) as shown in Figure 2.1. (ii) Measurement of the distances from both the Image Centroid and Zone Centroid to the pixels present in each zone. (iii) The area of application (uppercase (A-Z), lowercase (a-z) English alphabet and digit (0- 9)).Few zones are adopted in this research but emphasis is laid on how to effectively measure the pixeldensities in each zone. However, pixels pass through the zones at varied distances that is why we measuredfive distances for each zone at an angle of 200 .Hybrid of Modified ICZ and Modified ZCZ based Distance Metric Feature Extraction Algorithm.Input: Pre processed character imageOutput: Features for classification and recognitionMethod BeginsStep1: Divide the input image into 25 equal zones.Step2: Compute the input image centroidStep3: Compute the distance between the image centroid to each pixel present in the zone measured atangle 200Step4: Repeat step 3 for the entire pixel present in the zone.Step5: Compute average distance between these points. 92
  5. 5. Computer Engineering and Intelligent Systems www.iiste.org ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online) Vol 3, No.4, 2012 Step6: Compute the zone centroid Step7: Compute the distance between the zone centroid to each pixel present in the zone measured at angle 200. Step8: Repeat step 7 for the entire pixel present in the zone Step9: Compute average distance between these points. Step10: Repeat the steps 3-9 sequentially for the entire zones. Step11: Finally, 2xn (50) such features were obtained for classification and recognition. Method EndsStage 4: Feed the outputs of the extracted features of the characters into the digitization stage in order to convert all the extracted features into digital forms. 2.4.2 Development of Hybrid Neural Network Model In this research work, hybrid approach with serial combination scheme was adopted. Hybrid of modified Counterpropagation and modified Optical Backpropagation neural network was developed as shown in Figure 2.5. The architecture of the neural network with three layers is illustrated in Figure 2.5. The output of ith layer is given by (2.1) except the output layer which uses maxsoft function: ai = logsig(wi ai-1 + bi) (2.1) where i = (2, 3) and a0 = P, a1 = E where E is the Euclidean distance between the weight vector and the input vector. wi = Weight vector of ith layer ai = Output of ith layer bi = Bias vector for ith layer. The input vector ‘P’ is represented by the solid vertical bar at the left. The dimensions of ‘P’ are displayed as 35 x 1, indicating that the input is a single vector of 35 elements (i.e. the image size). These inputs go to weight matrix ‘W1’, which has 86 rows (i.e. 86 neurons in the first hidden layer) and 35 columns. A constant ‘1’ enters the neuron as input and is multiplied by a bias ‘b1’. The net input to the transfer function (Euclidean distance) in the Kohonen (hidden layer) is ‘n1’, which is given as the Euclidean distance between the weight vector wi and the input vector P. The neurons output ‘a1’ serves as inputs to the second hidden layer. These inputs go to weight matrix ‘W2’, which has 86 rows (i.e.86 neurons in the first hidden layer) and 35 columns. A constant ‘1’ enters the neuron as input and is multiplied by a bias ‘b2’. The net input to the transfer function (log sigmoid) in the second (hidden layer) is ‘n2’, which is the sum of the bias ‘b2’ and the product ‘W2a1’. The output ‘a2’ is a single vector of 86 elements and this serves as inputs to the output layer ‘a3’. These inputs go to weight matrix ‘W3’, which has 86 rows (i.e. 86 neurons from the second hidden layer) and 62 columns (26 uppercase + 26 lowercase + 10 digits). The net input to the transfer function (maxsoft) in the output layer is ‘n3’, which is the sum of the bias ‘b3’ and the product ‘W3a2’. The neurons output ‘a3’ is a single vector of 62 elements and this serves as the final output of the neural network. This research work adopted the modified Counterpropagation network (CPN) developed by Jude, Vijila, and Anitha (2010). The training algorithm involves the following two phases: (i) Weight adjustment between the input layer and the hidden layer (Kohonen layer) The weight adjustment procedure for the hidden layer weights is same as that of the conventional CPN. It follows the unsupervised methodology to obtain the stabilized weights. After convergence, the weights between the hidden layer and the output layer are calculated. 93
  6. 6. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012(ii) Weight adjustment between the hidden layer and its output layerThe weight adjustment procedure employed in this work is significantly different from the conventionalCPN. The weights are calculated in the reverse direction without any iterative procedures. Normally, theweights are calculated based on the criteria of minimizing the error. But in this work, a minimum errorvalue is specified initially and the weights are estimated based on the error value. Thus without any trainingmethodology, the weight values are estimated. This technique accounts for higher convergence rate sinceone set of weights are estimated directly. It is this output that served as the input to the modified opticalbackpropagation algorithm.2.4.2.1 Modified Optical Backpropagation Neural NetworkIn the standard backpropagation, the error at a single output unit is defined according to Equation as:δo pk = (Ypk - Opk ) . fo k (neto pk) (2.2)where the subscript “p” refers to the pth training vector, and “k” refers to the kth output unit, Ypk is thedesired output value, Opk is the actual output from kth unit, then δopk will propagate backward to update theoutput-layer weights and the hidden-layer weights. In the Optical backpropagation (OBP), error at a singleoutput unit is adjusted according to Otair and Salameh (2005) as:New δopk = (1+e (Ypk - Opk )2 . fok (neto pk)), if (Y – O) >= zero(2.3a)New δopk = - (1+e (Ypk - Opk )2 . fok (neto pk)), if (Y – O) < zero(2.3b)The error function defined in Optical Backpropagation (Otair and Salameh, 2005) earlier is proportional tothe square of the Euclidean distance between the desired output and the actual output of the network for aparticular input pattern. As an alternative, other error functions whose derivatives exist and can becalculated at the output layer can replace the traditional square error criterion (Haykin, 2003). In thisresearch work, error of the third order (Cubic error) has been adopted to replace the traditional square errorcriterion used in Optical backpropagation. The Equation of the cubic error is given as: δopk = -3(Ypk – Opk)2 . fok(netopk)(2.4)The cubic error in Equation (2.4) was manipulated mathematically in order to maximize the error of eachoutput unit which will be transmitted backward from the output layer to each unit in the intermediate layer.These are as shown in Equations (2.5a) and (2.5b) below:Modified δopk = 3((1+ et)2 . fok(netopk)) If (Ypk – Opk)2 >= 0(2.5a)Modified δopk = -3((1+ et)2 . fok(netopk)) If (Ypk – Opk)2 <=0(2.5b) where Ypk = Target or Desired output Opk = Network output t = (Ypk – Opk)2However, one of the ways to reduce the training time is through the use of momentum, as it enhances thestability of the training process. The momentum is used to keep the training process going in the samegeneral direction (Haykin, 2003). In the modified Optical Backpropagation network, momentum wasintroduced. Hence, the weight update for the output unit is:Wokj(t+1) = Wokj(t) + µWokj(t) + (η . Modified δopk. ipj)(2.6)where µ is the momentum coefficient typically about 0.9 and η is the learning rate.2.4.2.1 The Modified Optical Backpropagation AlgorithmModifications of the algorithm are in terms of: 94
  7. 7. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012(i) Error signal function(ii) Application areaWith the introduction of Cubic error function and Momentum, the modified Optical Backpropagation isgiven as:1. Apply the input example to the input units.2. Calculate the net-input values to the hidden layer units.3. Calculate the outputs from the hidden layer.4. Calculate the net-input values to the output layer units5. Calculate the outputs from the output units6. Calculate the error term for the output units, using Equations (2.5a) and (2.5b)7. Calculate the error term for the hidden units, through applying Modified δopk also Modified δh pj = fhj(neth pj) .( δopk . Wokj) (2.7)8. Update weights on the output layer. Wokj(t+1) = Wokj(t) + µWokj(t) + (η . Modified δopk . ipj) (2.8)9. Update weights on the hidden layer. Whji(t+1) = Whji(t) + (η . Modified δhpj . Xi) (2.9)Repeat steps from step 1 to step 9 until the error (Ypk – Opk) is acceptably small for each of the trainingvector pair. The proposed algorithm as in OBP is stopped when the cubes of the differences between theactual and target values summed over units and all patterns are acceptably small.2.4.3 The Hybrid Neural Network AlgorithmThis research work employed hybrid of modified Counterpropagation and modified OpticalBackpropagation neural networks for the training and classification of the input pattern. The trainingalgorithm involves the following two stages:Stage A: Performs the training of the weights from the input nodes to the Kohonen hidden node.Step 1: Weight adjustment between the input layer and the hidden layerThe weight adjustment procedure for the hidden layer weights is same as that of the conventional CPN. Itfollows the unsupervised methodology to obtain the stabilized weights. This process is repeated for asuitable number of iterations and the stabilized set of weights are obtained. After convergence, the weightsbetween the Kohonen hidden layer and the output layer are calculated.Step 2: Weight adjustment between the hidden layer and the output layer The weight adjustment procedure employed in this work is significantly different from theconventional CPN. The weights are calculated in the reverse direction without any iterative procedures.Normally, the weights are calculated based on the criteria of minimizing the error. But in this work, aminimum error value is specified initially and the weights are estimated based on the error value. Thedetailed steps of the modified algorithm are given below.Step 1: The stabilized weight values are obtained when the error value (target output) is equal to zero (or) apredefined minimum value. The error value used for convergence in this work is 0.1. The followingprocedure uses this concept for weight matrices calculation.Step 2: Supply the target vectors t1 to the output layer neuronsStep 3: Since ( t1 – y1 ) = 0.1 for convergence(2.10) 95
  8. 8. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012The output of the output layer neurons is set equal to the target values as: y1 = t1 − 0.1(2.11)Step 4: Once the output value is calculated, the sum of the weighted input signals. Thus without anytraining methodology, the weight values are estimated. This technique accounts for higher convergence ratesince one set of weights are estimated directly.STAGE B: Performs the training of the weights from the second hidden node to the output nodes.1. Calculate the net-input values from the Kohonen to the second hidden layer units.2. Calculate the outputs from the second hidden layer.3. Calculate the net-input values to the output layer units4. Calculate the outputs from the output units5. Calculate the error term for the output units, using Equations (2.5a) and (2.5b)6. Calculate the error term for the hidden units, through applying modified δopk as in Equation (2.7)7. Update weights on the output layer using (2.8)8. Update weights on the hidden layer using Equation (2.9)Repeat steps from step 1 to step 8 until the error (Ypk – Opk) is acceptably small for each of the trainingvector pair. The proposed algorithm as classical BP is stopped when the cubes of the differences betweenthe actual and target values summed over units and all patterns are acceptably small. The description of notation used in the training procedure is as given below: Xpi : net input to the ith input unit nethpj: net input to the jth hidden unit Whji: weight on the connection from the ith input unit to jth hidden unit ipj: net input to the jth hidden unit netopk: net input to the kth output unit Wokj: weight on the connection from the jth hidden unit to kth output unit Opk: actual output for the kth output unit3. Software ImplementationThis section discussed the phases of developing the proposed character recognition system. All thealgorithms have been implemented using C# programming language and RUN under Windows7 operatingsystem on Pentium (R) 2.00GB RAM, 1.83GH Processor and Pentium (R) 4.00GB RAM, 2.13GHProcessor respectively. Different interfaces representing the phases of the system development are shownfrom Figures 2.6 to 2.11.3.1 Character AcquisitionCharacter acquisition interface as in Figures 2.7 and 2.8 displayed a drawing area to serve as a platform toacquire characters from users. Any drawn character on this platform must be saved into the developedsystem’s database by clicking the ‘Add Botton’. It also captured number of strokes and pressure used inwriting a character. A total 6,200 character samples were acquired using G-Pen 450 as shown in Figure 2.2 96
  9. 9. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012and stored in the database.3.2 Network TrainingBefore the network can be trained, the network parameters such as the learning rate, epoch value, the quiterror and character image size must be specified in the ‘Application setting’ interface. The next thing is tospecify the total number of the character to be loaded from the database for training. The database size canbe varied by specifying the maximum number of character to be selected from character category(uppercase, lowercase and digit). This can be accomplished with the interfaces shown in Figure 2.9 andFigure 2.10. On clicking the ‘Training phase button’, the neural network trains itself with alphanumericcharacters of A – Z, a – z and 0 – 9 by performing several iterations based on the error value, learning rateand the epoch value stipulated for the Neural Network. Epoch increases from 1 to 10,000 and the Errorreduces from a particular value depending on the value specified by the user. The training is terminatedwhen either the epoch reaches 10,000 or the error reaches the minimum value specified. Network trainingresults such the training time, the epoch value were displayed as shown in Figure 2.10.3.3 Recognition PhaseRecognition of the character can be accomplished by clicking the ‘Data acquisition/ Recognition phasebotton’. User is required to write a character and click the ‘Recognize botton’. The results such as the‘detected character’, ‘recognition rate’ and ‘recognition time’ will be displayed as shown in Figure 2.11.4. Performance EvaluationThe performance metrics adopted in evaluating the developed system are: different learning rate parameters,different image sizes, different database sizes and system configurations. The results are as given in Figure2.12 to Figure 2.17. Figure 2.12 shows the graph of learning rate versus epoch. Learning rate parametervariation has a positive effect on the network performance. The smaller the value of learning rate, the lowerthe value with which the network updates its weights. This intuitively implies that it will be less likely toface the over learning difficulty since it will be updating its links slowly and in a more refined manner.However, this would also imply more number of iterations is required to reach its optimal state. Figure 2.13indicates the graph of image size versus epoch. The image size is directly proportional to epoch. Usually,the complex and large sized input sets require a large topology network with more number of iterations andthis has an adverse effect on the recognition time. Figure 2.14 shows the graph of variation of database sizeand recognition rates. Increase in database size has a positive proportionality relation to the recognitionrates due to the fact that network is able to attribute the test character to larger character samples in thevector space. However, rate of increment in recognition rate with respect to database size is considerablysmall. Moreover, it can be shown from Figure 2.15 that the more the dimensional input vector (imagematrix size), the less the recognition performance. Three different image sizes (5 by 7, 10 by 14 and 20 by28) were considered; it was shown from the results that the higher the image size, the higher the percentageof recognition, although, the rate of change was small. Furthermore, increase in image size also increasesthe recognition time.System configuration is also another factor influencing the performance of the recognition system. Both thetraining time and recognition time are measured in CPU (processor) seconds, the higher the speed of thesystem (processor speed), the lower the training time and the recognition time. This is due to the fact thatmore time will be used by 1.8GH system to reach the required epochs (iteration) than the system with2.13GH speed. The result is as shown in Figure 2.16. Figure 2.17 is the graph showing comparison ofperformance of the developed system with related works in literature. It can be shown from the graph thatthe accuracy of the system developed by Muhammad et. al. (2005) is higher than that of Muhammad et. al.(2006). This shows that the performance of Counterpropagation neural network is better than that ofStandard Backpropagation. The best recognition rate was achieved in the developed system with no 97
  10. 10. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012recognition failure. This means that the developed system was able to recognise characters irrespective ofthe writing styles.5. Conclusion and Future WorkIn this paper, we have been able to develop an effective feature extraction technique for the proposed onlinecharacter recognition system using hybrid of geometrical and statistical features. The hybrid featureextraction was developed to alleviate the problem of poor feature extraction algorithm of online characterrecognition system. However, a hybrid of modified counter propagation and modified opticalbackpropagation neural network model was developed for the proposed system. The performance of theonline character recognition system under consideration was evaluated based on different learning rates,image sizes and database sizes. The results from the study were compared with some works in the literatureusing counterpropagation and standard backpropagation respectively. The results showed thatcounterpropagation performed better than standard backpropagation in terms of correct recognition, falserecognition, recognition failure and the achieved recognition rates were 94% and 81% respectively. Thedeveloped system achieved better performance with no recognition failure, 99% recognition rate and anaverage recognition time of 2 milliseconds. Future work could be geared towards integrating anoptimization algorithm into the learning algorithms to further enhance the convergence of the neuralnetwork.ReferencesAnoop, M. N. and Anil K.J. (2004): “Online Handwritten Script Recognition’’, IEEE Trans. PAMI, 26(1):124130.Liu, C.L., Nakashima, K., Sako, H. and Fujisawa, H. (2004): “Handwritten Digit Recognition:Investigation ofNormalization and Feature Extraction Techniques’’, Pattern Recognition, 37(2): 265-279.Mohamad, D. and Zafar, M.F. (2004): “Comparative Study of Two Novel Feature Vectors for ComplexImageMatching Using Counterpropagation Neural Network’’, Journal of Information Technology, FSKSM, UTM,16(1): 2073-2081.Naser, M.A., Adnan, M., Arefin, T.M., Golam, S.M. and Naushad, A. (2009): “Comparative Analysis ofRadon and Fan-beam based Feature Extraction Techniques for Bangla Character Recognition”, IJCSNSInternational Journal of Computer Science and Network Security, 9(9): 120-135.Pradeep, J., Srinivasan, E. and Himavathi, S. (2011): “Diagonal Based Feature Extraction for HandwrittenAlphabets Recognition using Neural Network’’, International Journal of Computer Science and InformationTechnology (IJCS11), 3(1): 27-37.Shanthi, N., and Duraiwamy, K. (2007): “Performance Comparison of Different Image Sizes forRecognizing Unconstrained Handwritten Tamil Characters Using SVM’’, Journal of Science, 3(9): 760-764.Santosh, K.C. and Nattee, C. (2009): “A Comprehensive Survey on Online Handwriting RecognitionTechnology and Its Real Application to the Nepalese Natural Handwriting’’, Kathmandu University Journalof Science, Engineering Technology, 5(1): 31-55.Simon G. and Horst B. (2004): “Feature Selection Algorithms for the Generalization of Multiple ClassifierSystems and their Application to Handwritten Word Recognition’’, Pattern Recognition Letters, 25(11):1323-1336. 98
  11. 11. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012Gupta, K., Rao, S.V., and Viswanath (2007): “Speeding up Online Character Recognition’’, Proceedings ofImage and Vision Computing Newzealand, Hamilton: 41-45.Manuel, J., Fonseca, and Joaquim, A.J., (2001): “Experimental Evolution of an Online ScribbleRecognizer’’, Pattern Recognition Letters, 22(12): 1311-1319.Rejean, P. and Sargur, S.N. (2000): “On-line and Off-line Recognition: A Comprehensive Survey’’, IEEETransaction on Pattern Analysis and Machine Intelligence, 22(1): 63-84.Kittler, J. and Roli, F. (2000): “1st International Workshop on Multiple Classifier Systems”, Cagliari, Italy.Kittler, J. and Roli, F. (2001): “2nd International Workshop on Multiple Classifier Systems”, Cagliari, Italy.Marc, P., Alexandre, L., and Christian, G. (2001): “Character Recognition Experiment using Unipen Data’’,Parizeau and al., Proceeding of ICDAR, Seattle: 10-13.Muhammad, F.Z., Dzuulkifli, M., and Razib, M.O. (2006): “Writer Independent Online HandwrittenCharacter Recognition Using Simple Approach’’, Information Technology Journal, 5(3): 476-484.Freeman, J.A. and Skapura, D.M. (1992): “Backpropagation Neural Networks Algorithm Applications andProgramming Techniques”, Addison-Wesley Publishing Company: 89-125.Minai, A.A. and Williams, R.D. (1990): “Acceleration of Backpropagation through learning RateMomentum Adaptation’’, Proceeding of the International Joint Conference on Neural Networks: 1676-1679.Riedmiller, M. and Braun, H. (1993): “A Direct Adaptive Method for Faster Backpropagation learning thePROP Algorithm’’, Proceedings of the IEEE International Conference on Neural Networks (ICNN), 1: 586-591, Francisco.Otair, M.A. and Salameh, W.A. (2005a): “Online Handwritten Character Recognition using an OpticalBackpropagation Neural Network’’, Issues in Informing Science and Information Technology, 2: 787-797.Rajashekararadhya, S.V. and Vanaja, P.R. (2008a): “Handwritten numeral recognition of three popularSouth Indian Scripts: A Novel Approach:’’, Proceedings of the Second International Conference oninformation processing ICIP: 162-167.Rajashekararadhya S.V. and Vanaja, P.R (2008b): “Isolated Handwritten Kannada Digit Recognition: ANovel Approach’’, Proceedings of the International Conference on Cognition and Recognition: 134-140.Fenwa, O. D., Omidiora, E. O. and Fakolujo O. A. (2012): “Development of a Feature ExtractionTechnique for Online Character Recognition System, Journal of Innovative Systems Design andEngineering, International Institute of Science, Technology and Education,Vol. 3, No.3, pp. 10-23. 99
  12. 12. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012 Figure 2.1: Character ‘n’ in 5 by 5 (25 equal zones) Figure 2.2: Block Diagram of the Developed Character Recognition System Figure 2.3: The snapshot of Genius Pen (G-Pen 450) Digitizer for character acquisition 100
  13. 13. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012 Figure 2.4: The Developed Hybrid Feature Extraction Model a1 = E (W1 – P), a2 = logsig (W2a1 + b2), a3 = Maxsoft (W3 + b3) Figure 2.5: The Hybrid Neural Network Model 101
  14. 14. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012 Figure 2.6: Graphic user interface of the developed Online Character Recognition System Figure 2.7: Acquisition of character ’A’ using 3 strokes 102
  15. 15. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012 Figure 2.8: Acquisition of character ’A’ using 2 strokes Figure 2.9: Network parameter setting and loading data to be trained from the database Figure 2.10: Network Training in progress Figure 2.11: Result of recognition process displaying recognition status 103
  16. 16. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012 Figure 2.12: Graph showing the effect of variation in Learning Rate and Database size on Epochs Figure 2.13: Graph showing the effect of variation of Image size and Database size on Epoch Figure 2.14: Graph showing the effect of variation in Database size and Epoch on the Recognition Rate 104
  17. 17. Computer Engineering and Intelligent Systems www.iiste.orgISSN 2222-1719 (Paper) ISSN 2222-2863 (Online)Vol 3, No.4, 2012Figure 2.15: Graph showing the effect of variation in Image size and Database size on the Recognition Rate Figure 2.16: Graph showing the effect of variation in system configurations and Database size on Epochs CR = Correct Recognition; FR = False Recognition; RF = Recognition Failure Figure 2.17: Graph showing performance evaluation of the developed system with related works 105
  18. 18. International Journals Call for PaperThe IISTE, a U.S. publisher, is currently hosting the academic journals listed below. The peer review process of the following journalsusually takes LESS THAN 14 business days and IISTE usually publishes a qualified article within 30 days. Authors shouldsend their full paper to the following email address. More information can be found in the IISTE website : www.iiste.orgBusiness, Economics, Finance and Management PAPER SUBMISSION EMAILEuropean Journal of Business and Management EJBM@iiste.orgResearch Journal of Finance and Accounting RJFA@iiste.orgJournal of Economics and Sustainable Development JESD@iiste.orgInformation and Knowledge Management IKM@iiste.orgDeveloping Country Studies DCS@iiste.orgIndustrial Engineering Letters IEL@iiste.orgPhysical Sciences, Mathematics and Chemistry PAPER SUBMISSION EMAILJournal of Natural Sciences Research JNSR@iiste.orgChemistry and Materials Research CMR@iiste.orgMathematical Theory and Modeling MTM@iiste.orgAdvances in Physics Theories and Applications APTA@iiste.orgChemical and Process Engineering Research CPER@iiste.orgEngineering, Technology and Systems PAPER SUBMISSION EMAILComputer Engineering and Intelligent Systems CEIS@iiste.orgInnovative Systems Design and Engineering ISDE@iiste.orgJournal of Energy Technologies and Policy JETP@iiste.orgInformation and Knowledge Management IKM@iiste.orgControl Theory and Informatics CTI@iiste.orgJournal of Information Engineering and Applications JIEA@iiste.orgIndustrial Engineering Letters IEL@iiste.orgNetwork and Complex Systems NCS@iiste.orgEnvironment, Civil, Materials Sciences PAPER SUBMISSION EMAILJournal of Environment and Earth Science JEES@iiste.orgCivil and Environmental Research CER@iiste.orgJournal of Natural Sciences Research JNSR@iiste.orgCivil and Environmental Research CER@iiste.orgLife Science, Food and Medical Sciences PAPER SUBMISSION EMAILJournal of Natural Sciences Research JNSR@iiste.orgJournal of Biology, Agriculture and Healthcare JBAH@iiste.orgFood Science and Quality Management FSQM@iiste.orgChemistry and Materials Research CMR@iiste.orgEducation, and other Social Sciences PAPER SUBMISSION EMAILJournal of Education and Practice JEP@iiste.orgJournal of Law, Policy and Globalization JLPG@iiste.org Global knowledge sharing:New Media and Mass Communication NMMC@iiste.org EBSCO, Index Copernicus, UlrichsJournal of Energy Technologies and Policy JETP@iiste.org Periodicals Directory, JournalTOCS, PKPHistorical Research Letter HRL@iiste.org Open Archives Harvester, Bielefeld Academic Search Engine, ElektronischePublic Policy and Administration Research PPAR@iiste.org Zeitschriftenbibliothek EZB, Open J-Gate,International Affairs and Global Strategy IAGS@iiste.org OCLC WorldCat, Universe Digtial Library ,Research on Humanities and Social Sciences RHSS@iiste.org NewJour, Google Scholar.Developing Country Studies DCS@iiste.org IISTE is member of CrossRef. All journalsArts and Design Studies ADS@iiste.org have high IC Impact Factor Values (ICV).

×