Dissertation character recognition - Report


Published on

MCA Academic Work for project report on small application and survey

Published in: Technology
1 Comment
  • Bro please mail me this at sagardutt08@gmail.com
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Dissertation character recognition - Report

  1. 1. DISSERTATION ON CHARACTER RECOGNITION USING NEURAL NETWORK IN PARTIAL FULFILLMENT OF SURVEY ON RESEARCHER’S TOPIC AS A PART OF CURRICULUM IN MASTER’S DEGREE IN COMPUTER APPLICATION (M.C.A) SEMESTER – v Gujarat University (2010-2011) Submitted By Sachinkumar M. Bharadva Dhara Solanki Internal Guide Mr. Sandeep R. Vasant A.E.S Institute Of Computer Studies (AESICS) B++ ACCREDITATION BY NAAC OF UGC THE AHMEDABAD EDUCATION SOCIETY (Affiliated to Gujarat University, M.C.A Programme) H.L. College Campus, P.B. 4206, Navarangpura, AHMEDABAD - 380009
  2. 2. CHARACTER RECOGNITION USING NEURAL NETWORK INDEX ___________________________________________________ SR.NO._______ 1. 2. 3. 4. _CONTENTS_____ __PAGE NO. Introduction to Neural Networks 1.1 What is a Neural Network? 1.2 Historical background 1.2.1 History 1940’s to 1970’s 1.2.2 History 1980’s to the Present 1.3 Why use Neural Networks? 1.4 Neural Networks versus Conventional Computers 12 13 Human and Artificial Neurones - Investigating the Similarities 2.1 The Human Brain, Neural Network and Computers 2.2 The Artificial Neural Network and Artificial Intelligence 2.3 Neural Network and Neuro Science 15 17 20 Architecture 3.1 Architecture of Neural Networks 3.2 Feed-forward (associative) networks 3.3 Feedback (autoassociative) networks 3.4 Network Layers 3.5 Acyclic Network 3.6 Modular Neural Network 3.7 Perceptrons 2 6 6 10 22 24 24 25 26 26 27 Applications of Neural Networks 4.1 Neural Networks in Practice 30 4.2 Neural Networks in Medicine 31 4.2.1 Modelling and Diagnosing the Cardiovascular System 4.2.2 Electronic Noses - Detection and Reconstruction 4.2.3 Instant Physician - a commercial neural net - diagnostic program 4.3 Neural Networks in Business 33 4.4 Marketing 33 4.5 Credit Evaluation 34 4.6 Signal Processing 34 4.7 Speech Recognition 34 4.8 Intelligent Control 35 4.9 Financial Forecasting 35 4.10Condition Monitoring 36 4.11 Neuro Forecasting 36 4.12 Pattern Analysis[B35] 36 4.13 Classification 37 AES Institute Of Computer Studies
  3. 3. CHARACTER RECOGNITION USING NEURAL NETWORK INDEX ___________________________________________________ SR.NO._______ 5. _CONTENTS_____ 6. Introduction to Character Recognition 5.1 What is Character Recognition? 5.2 History for Character Recognition 5.3 Steps for the Character Recognition How to Recognize Character 7. __PAGE NO. Matlab 7.1 What is Matlab? 7.2 History and Present 7.3 Neural Network Toolbox 8. 39 40 41 42 47 48 49 Literature Survey 8.1 Kevin Larson Paper and Application 8.2 Ketil Hunn Paper and Application 8.3 Current Market Scenario 8.4 Current Research 54 59 62 64 Proposed Work 9.1 Work Introduction 9.2 Application 9.3 Accuracy and Result 9.4 Recognize Digit and Success/Failure Phase 66 66 71 72 10. Enhancements 75 11. Conclusion 78 12. Sources and Bibliography 80 9. AES Institute Of Computer Studies
  4. 4. CHARACTER RECOGNITION USING NEURAL NETWORK ACKNOWLEDGEMENT I sincerely successful thank to completion of all my the persons whoever played a vital role in the Dissertation under titled CHARACTER RECOGNITION USING NEURAL NETWORK. We are grateful to the whole staff of the AES Institute of Computer Studies under whose guidance I completed my successful Dissertation, who devoted their precious time. In spite of, their in busy schedules they always came forward to guide us in our work whenever needed. We are also thankful to Mr. Bipin Mehta for his constant support. We accept this opportunity to acknowledgement all other individuals for their direct or indirect contribution to our work. We are also very thankful to Prof. Sandeep Vasant and to our staff members without whom it would have been impossible for us to carry out the Dissertation work. Again with a word of thanks to all we represent this Dissertation. AES Institute Of Computer Studies
  6. 6. CHARACTER RECOGNITION USING NEURAL NETWORK 1.1 What is a Neural Network? An artificial neural network is an information processing system that has been developed as a generalization of the mathematical model of human cognition (faculty of knowing). A neural network is a network of interconnected neurons, inspired from the studies of the biological nervous system. Neural network functions in a way similar to the human brain. The function of a neural network is to produce an output pattern when presented with an input pattern. An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurones) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurones. This is true of ANNs as well. A neural network is the study of networks consisting of nodes connected by adaptable weights, which store experimental knowledge from ktask examples through a process of learning. The human brain provides proof of the existence of massive neural networks that can succeed at those cognitive, perceptual, and control tasks in which humans are successful. The brain is capable of computationally demanding perceptual acts (e.g. recognition of faces, speech) and control activities (e.g. body movements and body functions). The advantage of the brain is its effective use of massive parallelism, the highly parallel computing structure, and the imprecise information processing capability. AES Institute Of Computer Studies 2
  7. 7. CHARACTER RECOGNITION USING NEURAL NETWORK The human brain is a collection of more than 10billion interconnected neurons. Each neuron is a cell (Figure1) that uses biochemical reactions to receive, process, and transmit information. Treelike networks of nerve fibers called dendrites are connected to the cell body or soma, where the cell nucleusis located. Extending from the cell body is a single long fiber called the axon, which eventually branches into strands and substrands, a dare connected to other neurons through synaptic terminals or synapses. The transmission of signals from one neural onto another at synapsesisa complex chemical process in which specific transmitter substances are released from the sending end of the junction. The effect is to raise or lower the electrical potential in side the body of the receiving cell. If the potential reaches a threshold, a pulse is sent down the axon and the cell is ‘fired’. AES Institute Of Computer Studies 3
  8. 8. CHARACTER RECOGNITION USING NEURAL NETWORK Traditionally, the term neural network had been used to refer to a network or circuit of biological neurons; the modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes. Thus the term has two distinct usages: Biological neural networks are made up of real biological neurons that are connected or functionally related in the peripheral nervous system or the central nervous system. In the field of neuroscience, they are often identified as groups of neurons that perform a specific physiological function in laboratory analysis. Artificial neural networks are made up of interconnecting artificial neurons (programming constructs that mimic the properties of biological neurons). Artificial neural networks may either be used to gain an understanding of biological neural networks, or for solving artificial intelligence problems without necessarily creating a model of a real biological system. The real, biological nervous system is highly complex and includes some features that may seem superfluous based on an understanding of artificial networks. In general a biological neural network is composed of a group or groups of chemically connected or functionally associated neurons. A single neuron may be connected to many other neurons and the total number of neurons and connections in a network may be extensive. Connections, called synapses, are usually formed from AES Institute Of Computer Studies 4
  9. 9. CHARACTER RECOGNITION USING NEURAL NETWORK axons to dendrites, though dendrodendritic microcircuits and other connections are possible. Apart from the electrical signaling, there are other forms of signaling that arise from neurotransmitter diffusion, which have an effect on electrical signaling. As such, neural networks are extremely complex. Artificial intelligence and cognitive modeling try to simulate some properties of neural networks. While similar in their techniques, the former has the aim of solving particular tasks, while the latter aims to build mathematical models of biological neural systems. In the artificial intelligence field, artificial neural networks have been applied successfully to speech recognition, image analysis and adaptive control, in order to construct software agents (in computer and video games) or autonomous robots. Most of the currently employed artificial neural networks for artificial intelligence are based on statistical estimation, optimization and control theory. The cognitive modeling field involves the physical or mathematical modeling of the behavior of neural systems; ranging from the individual neural level (e.g. modeling the spike response curves of neurons to a stimulus), through the neural cluster level (e.g. modeling the release and effects of dopamine in the basal ganglia) to the complete organism (e.g. behavioral modeling of the organism's response to stimuli). Artificial intelligence, cognitive modeling, and neural networks are information processing paradigms inspired by the way biological neural systems process data. AES Institute Of Computer Studies 5
  10. 10. CHARACTER RECOGNITION USING NEURAL NETWORK 1.2 Historical Background The concept of neural networks started in the late 1800s as an effort to describe how the human mind performed. Neural network simulations appear to be a recent development. However, this field was established before the advent of computers, and has survived at least one major setback and several eras. The Austrian School of economics theory of spontaneous order was explained by Murray Rothbard to have been first realised by Zhuangzi (Chuang Tzu) who said, "Good order results spontaneously when things are let alone." Many important advances have been boosted by the use of inexpensive computer emulations. Following an initial period of enthusiasm, the field survived a period of frustration and disrepute. During this period when funding and professional support was minimal, important advances were made by relatively few researchers. These pioneers were able to develop convincing technology which surpassed the limitations identified by Minsky and Papert. 1.2.1 History 1940’s to 1970’s 1940 – Donald Hebb In the late 1940s Donald Hebb made one of the first hypotheses of learning with a mechanism of neural plasticity called Hebbian learning. Hebbian learning is considered to be a 'typical' unsupervised learning rule and it and later variants were early models for long term potentiation. These ideas started being applied to computational models in 1948 with Turing's B-type machines and the perceptron. In 1943 - Warren McCulloch The first artificial neuron was produced in 1943 by the neurophysiologist Warren McCulloch and the logician Walter Pits. But the technology available at that time did not allow them to do too much. AES Institute Of Computer Studies 6
  11. 11. CHARACTER RECOGNITION USING NEURAL NETWORK Walter Pitts wrote a paper on how neurons might work. In order to describe how neurons in the brain might work, they modeled a simple neural network using electrical circuits. 1949 - Donald Hebb In 1949, Donald Hebb wrote The Organization of Behavior, a work which pointed out the fact that neural pathways are strengthened each time they are used, a concept fundamentally essential to the ways in which humans learn. If two nerves fire at the same time, he argued, the connection between them is enhanced. 1950 - Nathanial Rochester In 1950’s when computers became more advanced, It was finally possible to simulate a hypothetical neural network. So Nathanial Rochester from the IBM research laboratories trying to implement on it but unfortunately for him, the first attempt to do so failed. 1959 - Bernard Widrow and Marcian Hoff In 1959, Bernard Widrow and Marcian Hoff of Stanford developed models called "ADALINE" and "MADALINE." In a typical display of Stanford's love for acronymns, the names come from their use of Multiple ADAptive LINear Elements. ADALINE was developed to recognize binary patterns so that if it was reading streaming bits from a phone line, it could predict the next bit. MADALINE was the first neural network applied to a real world problem, using an adaptive filter that eliminates echoes on phone lines. While the system is as ancient as air traffic control systems, like air traffic control systems, it is still in commercial use. 1962 - Widrow & Hoff In 1962, Widrow & Hoff developed a learning procedure that examines the value before the weight adjusts it (i.e. 0 or 1) according to the rule: Weight Change = (Pre-Weight line value) * (Error / (Number of Inputs)). It is based on the idea that while one active perceptron may have a big error, one can adjust the weight values to AES Institute Of Computer Studies 7
  12. 12. CHARACTER RECOGNITION USING NEURAL NETWORK distribute it across the network, or at least to adjacent perceptrons. Applying this rule still results in an error if the line before the weight is 0, although this will eventually correct itself. If the error is conserved so that all of it is distributed to all of the weights than the error is eliminated. 1969 - Minsky and Papert Minsky and Papert, published a book (in 1969) in which they summed up a general feeling of frustration (against neural networks) among researchers, and was thus accepted by most without further analysis. Currently, the neural network field enjoys a resurgence of interest and a corresponding increase in funding. About 15 years after the publication of McCulloch and Pitt's pioneer paper, a new approach to the area of neural network research was introduced. In 1958 Frank Rosenblatt, a neuro-biologist at Cornell University began working on the Perceptron. The perceptron was the first "practical" artificial neural network. It was built using the somewhat primitive and "ancient" hardware of that time. The perceptron is based on research done on a fly's eye. The processing which tells a fly to flee when danger is near is done in the eye. One major downfall of the perceptron was that it had limited capabilities and this was proven by Marvin Minsky and Seymour Papert's book of 1969 entitled, "Perceptrons". 1972 - Kohonen and Anderson In 1972, Kohonen and Anderson developed a similar network independently, They both used matrix mathematics to describe their ideas but did not realize that what they were doing was creating as array of analog ADALINE circuits. The neurons are supposed to activate a set o outputs instead of just one. The later success of the neural network, traditional con Neumann architecture took over the computing scene, and neural research was let behind, Neumann himself suggested the imitation of neural functions by using telegraph relays or vacuum tubes. AES Institute Of Computer Studies 8
  13. 13. CHARACTER RECOGNITION USING NEURAL NETWORK In the same time paper was written that suggested of some neural networks led to an exaggeration of the potential of neural networks, especially practical technology at the time. The idea of a computer which programs itself is very appealing. If microsoft’s windows 2000 could reprogram itself, it might be able to repair the thousands of bugs that the programming staff made. Such ideas were appealing but very difficult to implement After that first multilayered network was developed in 1975, an unsupervised network. AES Institute Of Computer Studies 9
  14. 14. CHARACTER RECOGNITION USING NEURAL NETWORK 1.2.2 History 1980’s to the Present After 1975 when first multilayered network was developed the scientist interest renewed and trying to making something new in the neural network field. After 1980’s they gave new way to the neural network 1982 - John Hopfield In 1982, John Hopfield of Caltech presented a paper to the National Academy of Sciences. His approach was to create more useful machine by using bidirectional lines. Previously, the connections between neurons was only one way. Reilly and Cooper In the same year 1982, Reilly and Cooper used a “Hybrid network” with multiple layers, each layer using a different problem-solving strategy. There was a joint US-Japan conference on Cooperative/Competitive Neural Networks. Japan announced a new Fifth Generation effort on neural networks, and US papers generated worry that the US could be left behind in the field. (Fifth generation computing involves artificial intelligence. First generation used switches and wires, second generation used the transister, third state used solid-state technology like integrated circuits and higher level programming languages, and the fourth generation is code generators.) As a result, there was more funding and thus more research in the field. 1986 - Based on Widrow-Hoff Rule In this year multiple layered neural networks flash in news, the problem occur was how to extend the Widrow-Hoff rule to multiple layers. Three independent groups of researchers, One of which included David Rumelhart, a former member of Stanford’s psychology department, came up with similar ideas which are now called back propagation networks because it distributes pattern recognition errors throughout the AES Institute Of Computer Studies 10
  15. 15. CHARACTER RECOGNITION USING NEURAL NETWORK network. Hybrid networks used just two layers, these back-propagation network use many. The result is that back-propagation networks are “slow learners,” needing possibly thousands of iterations to learn. After that neural networks are used in several applications, some o which we will describe in below. The fundamental idea behind the nature o neural networks is that if it works in nature, it must be able to work in computers. The future of neural networks, through, lies in the development of hardware. Much like the advance chessplaying machines like Deep Blue, fast, efficient neural networks depend on hardware being specified for its eventual use. AES Institute Of Computer Studies 11
  16. 16. CHARACTER RECOGNITION USING NEURAL NETWORK 1.3 Why use Neural Networks? Neural networks, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. A trained neural network can be thought of as an "expert" in the category of information it has been given to analyse. This expert can then be used to provide projections given new situations of interest and answer "what if" questions. Other advantages include: 1. Adaptive learning: An ability to learn how to do tasks based on the data given for training or initial experience. 2. Self-Organization: An ANN can create its own organization or representation of the information it receives during learning time. 3. Real Time Operation: ANN computations may be carried out in parallel, and special hardware devices are being designed and manufactured which take advantage of this capability. 4. Fault Tolerance via Redundant Information Coding: Partial destruction of a network leads to the corresponding degradation of performance. However, some network capabilities may be retained even with major network damage. AES Institute Of Computer Studies 12
  17. 17. CHARACTER RECOGNITION USING NEURAL NETWORK 1.4 Neural Networks versus Conventional Computer Neural networks take a different approach to problem solving than that of conventional computers. Conventional computers use an algorithmic approach i.e. the computer follows a set of instructions in order to solve a problem. Unless the specific steps that the computer needs to follow are known the computer cannot solve the problem. That restricts the problem solving capability of conventional computers to problems that we already understand and know how to solve. But computers would be so much more useful if they could do things that we don't exactly know how to do. Neural networks process information in a similar way the human brain does. The network is composed of a large number of highly interconnected processing elements (neurones) working in parallel to solve a specific problem. Neural networks learn by example. They cannot be programmed to perform a specific task. The examples must be selected carefully otherwise useful time is wasted or even worse the network might be functioning incorrectly. The disadvantage is that because the network finds out how to solve the problem by itself, its operation can be unpredictable. On the other hand, conventional computers use a cognitive approach to problem solving; the way the problem is to solved must be known and stated in small unambiguous instructions. These instructions are then converted to a high level language program and then into machine code that the computer can understand. These machines are totally predictable; if anything goes wrong is due to a software or hardware fault. Neural networks and conventional algorithmic computers are not in competition but complement each other. There are tasks are more suited to an algorithmic approach like arithmetic operations and tasks that are more suited to neural networks. Even more, a large number of tasks, require systems that use a combination of the two approaches (normally a conventional computer is used to supervise the neural network) in order to perform at maximum efficiency. Neural networks do not perform miracles. But if used sensibly they can produce some amazing results. AES Institute Of Computer Studies 13
  19. 19. CHARACTER RECOGNITION USING NEURAL NETWORK 2.1 The Human Brain, Neural Networks and Computers A function approximator like an ANN can be viewed as a black box and when it comes to FANN, this is more or less all you will need to know. However, basic knowledge of how the human brain operates is needed to understand how ANNs work. The human brain is a highly complicated system which is capable to solve very complex problems. The brain consists of many different elements, but one of its most important building blocks is the neuron, of which it contains approximately 1011. These neurons are connected by around 1015 connections, creating a huge neural network. Neurons send impulses to each other through the connections and these impulses make the brain work. The neural network also receives impulses from the fiber nerve senses and sends out impulses to muscles to achieve motion or speech. The individual neuron can be seen as an input-output machine which waits for impulses from the surrounding neurons and, when it has received enough impulses, it sends out an impulse to other neurons. Neural networks, as used in artificial intelligence, have traditionally been viewed as simplified models of neural processing in the brain, even though the relation between this model and brain biological architecture is debated. AES Institute Of Computer Studies 15
  20. 20. CHARACTER RECOGNITION USING NEURAL NETWORK A subject of current research in theoretical neuroscience is the question surrounding the degree of complexity and the properties that individual neural elements should have to reproduce something resembling animal intelligence. Historically, computers evolved from the von Neumann architecture, which is based on sequential processing and execution of explicit instructions. On the other hand, the origins of neural networks are based on efforts to model information processing in biological systems, which may rely largely on parallel processing as well as implicit instructions based on recognition of patterns of 'sensory' input from external sources. In other words, at its very heart a neural network is a complex statistical processor (as opposed to being tasked to sequentially process and execute). Neural coding is concerned with how sensory and other information is represented in the brain by neurons. The main goal of studying neural coding is to characterize the relationship between the stimulus and the individual or ensemble neuronal responses and the relationship among electrical activity of the neurons in the ensemble. It is thought that neurons can encode both digital and analog information. AES Institute Of Computer Studies 16
  21. 21. CHARACTER RECOGNITION USING NEURAL NETWORK 2.2 The Artificial Neural Networks and Artificial Intelligence Artificial neurons are similar to their biological counter parts. They have input connections which are summed together to determine the strength of their output, which is the result of the sum being fed into an activation function. Though many activation functions exist, the most common is the sigmoid activation function, which outputs a number between 0 (for low input values) and 1 (for high input values). The resultant of this function is then passed as the input to other neurons through more connections, each of which are weighted. These weights determine the behavior of the network. A neural network (NN), in the case of artificial neurons called artificial neural network (ANN) or simulated neural network (SNN), is an interconnected group of natural or artificial neurons that uses a mathematical or computational model for information processing based on a connectionistic approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network. In more practical terms neural networks are non-linear statistical data modeling or decision making tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data. However, the paradigm of neural networks - i.e., implicit, not explicit , learning is stressed - seems more to correspond to some kind of natural intelligence than to the traditional Artificial Intelligence, which would stress, instead, rule-based learning. (1) Background An artificial neural network involves a network of simple processing elements (artificial neurons) which can exhibit complex global behavior, determined by the connections between the processing elements and element parameters. Artificial neurons were first proposed in 1943 by Warren McCulloch, a neurophysiologist, and AES Institute Of Computer Studies 17
  22. 22. CHARACTER RECOGNITION USING NEURAL NETWORK Walter Pitts, an MIT logician. One classical type of artificial neural network is the recurrent Hopfield net. In a neural network model simple nodes, which can be called variously "neurons", "neurodes", "Processing Elements" (PE) or "units", are connected together to form a network of nodes — hence the term "neural network". While a neural network does not have to be adaptive per se, its practical use comes with algorithms designed to alter the strength (weights) of the connections in the network to produce a desired signal flow. In modern software implementations of artificial neural networks the approach inspired by biology has more or less been abandoned for a more practical approach based on statistics and signal processing. In some of these systems, neural networks, or parts of neural networks (such as artificial neurons), are used as components in larger systems that combine both adaptive and non-adaptive elements. The concept of a neural network appears to have first been proposed by Alan Turing in his 1948 paper "Intelligent Machinery". (2) Applications of natural and of artificial neural networks The utility of artificial neural network models lies in the fact that they can be used to infer a function from observations and also to use it. This is particularly useful in applications where the complexity of the data or task makes the design of such a function by hand impractical. The tasks to which artificial neural networks are applied tend to fall within the following broad categories: Function approximation, or regression analysis, including time series prediction and modeling. Classification, including pattern and sequence recognition, novelty detection and sequential decision making. Data processing, including filtering, clustering, blind signal separation and compression. AES Institute Of Computer Studies 18
  23. 23. CHARACTER RECOGNITION USING NEURAL NETWORK Application areas of ANNs include system identification and control (vehicle control, process control), game-playing and decision making (backgammon, chess, racing), pattern recognition (radar systems, face identification, object recognition, etc.), sequence recognition (gesture, speech, handwritten text recognition), medical diagnosis, financial applications, data mining (or knowledge discovery in databases, "KDD"), visualization and e-mail spam filtering. Moreover, some brain diseases, e.g. Alzheimer, are apparently, and essentially, diseases of the brain's natural NN by damaging necessary prerequisites for the functioning of the mutual interconnections between neurons. AES Institute Of Computer Studies 19
  24. 24. CHARACTER RECOGNITION USING NEURAL NETWORK 2.3 Neural Networks and Neuro Science Theoretical and computational neuroscience is the field concerned with the theoretical analysis and computational modeling of biological neural systems. Since neural systems are intimately related to cognitive processes and behavior, the field is closely related to cognitive and behavioral modeling. The aim of the field is to create models of biological neural systems in order to understand how biological systems work. To gain this understanding, neuroscientists strive to make a link between observed biological processes (data), biologically plausible mechanisms for neural processing and learning (biological neural network models) and theory (statistical learning theory and information theory). Classification of Neural Networks Artificial neural networks can be classified on the basis of: 1. Pattern of connection between neurons, (architecture of the network) 2. Activation function applied to the neurons 3. Methods of determining weights on the connection AES Institute Of Computer Studies 20
  26. 26. CHARACTER RECOGNITION USING NEURAL NETWORK 3.1 Architecture of Neural Network The neurons are assumed to be arranged in layers, and the neurons in the same layer behave in the same manner. All the neurons in a layer usually have the same activation function. Within each layer, the neurons are either fully interconnected or not connected at all. The arrangement of neurons into layers and the connection pattern within and between layers is known as network architecture. Input layer: The neurons in this layer receive the external input signals and perform no computation, but simply transfer the input signals to the neurons on another layer. Output layer: The neurons in this layer receive signals from neurons either in the input layer or in the hidden layer. Hidden layer: The layer of neurons that are connected in between the input layer and the output layer is known as hidden layer. Neural nets are often classified as single layer networks or multilayer networks. The number of layers in a net can be classified as the number of layers of weighted interconnection links between various layers. AES Institute Of Computer Studies 22
  27. 27. CHARACTER RECOGNITION USING NEURAL NETWORK Fully Connected N Networks An artificial neural network architecture in which every node is connected to every node, and these connections may be either excitatory (positive weights) inhibitory (negative weights), or irrelevant (almost zero weights). This is the most general neural net architecture imaginable, and every other architecture can be seen to be its special case, obtained by setting some weights to zeroes. In a fully connected asymmet network, the connection from one node to asymmetric , another may carry a different weight than the connection from the second node to the first. This architecture is seldom used deposit its generality and conceptual simplicity, due to the large numbers of parameters. In a network there are n2 weights. It is difficult to devise fast learning schemes that can produces fully connected networks that generalize well. It is practically never the case that every node has the direct influence on every other node. AES Institute Of Computer Studies 23
  28. 28. CHARACTER RECOGNITION USING NEURAL NETWORK 3.2 Feed Forward Network Feed-forward ANNs (figure 1) allow signals to travel one way only; from input to output. There is no feedback (loops) i.e. the output of any layer does not affect that same layer. Feed-forward ANNs tend to be straight forward networks that associate inputs with outputs. They are extensively used in pattern recognition. This type of organisation is also referred to as bottom-up or top-down. Figure 1. This is a subclass of acyclic networks in which a connection is allowed from a node in layer i only to node in layer i+1. These networks, generally with no more than four layers, are among the most common neural nets in use. So much so that some users identify the phrase “neural networks” to mean only feedforward networks. Conceptually, nodes in successively higher layers abstract successively higher level features from preceding layers. 3.3 Feedback Network Feedback networks (figure 1) can have signals travelling in both directions by introducing loops in the network. Feedback networks are very powerful and can get extremely complicated. Feedback networks are dynamic; their 'state' is changing continuously until they reach an equilibrium point. They remain at the equilibrium point until the input changes and a new equilibrium needs to be found. Feedback architectures are also referred to as interactive or recurrent, although the latter term is often used to denote feedback connections in single-layer organisations. AES Institute Of Computer Studies 24
  29. 29. CHARACTER RECOGNITION USING NEURAL NETWORK 3.4 Network Layers The commonest type of artificial neural network consists of three groups, or layers, of units: a layer of "input" units is connected to a layer of "hidden" units, which is connected to a layer of "output" units. (See below Figure) The activity of the input units represents the raw information that is fed into the network. The activity of each hidden unit is determined by the activities of the input units and the weights on the connections between the input and the hidden units. The behavior of the output units depends on the activity of the hidden units and the weights between the hidden and output units. This simple type of network is interesting because the hidden units are free to construct their own representations of the input. The weights between the input and hidden units determine when each hidden unit is active, and so by modifying these weights, a hidden unit can choose what it represents. Figure : An example of a simple feedforward network We also distinguish single-layer and multi-layer architectures. The singlelayer organization, in which all units are connected to one another, constitutes the most general case and is of more potential computational power than hierarchically AES Institute Of Computer Studies 25
  30. 30. CHARACTER RECOGNITION USING NEURAL NETWORK structured multi-layer organizations. In multi-layer networks, units are often numbered by layer, instead of following a global numbering. These are networks in which nodes are partitioned into subsets called layers, with no connections that lead from layer j to layer k if j > k. We adopt the convention that a single input arrives at and is distributed to other nodes by each node of the “input layer” or “layer 0”; no other computation occurs at nodes in layer 0, and there are no intra-layer connections among nodes in nodes layer. Connections with arbitrary weights, may exits from any node in layer i to any node in layer j for j >= i; intra-layer connections may exits. 3.5 Acyclic Network There is a subclass of layered networks in which there are no intra-layered connections. In other words, a connection may exits between any node in layer i and any nude in layer j for i < j, but a connection is not allowed for i = j. The computation processes in acyclic networks are much simpler than those in networks with exhaustive, cyclic or intra-layer connections. Networks that are not acyclic are referred to as recurrent networks. 3.6 Modular Neural Network Many problems are best solved using neural networks whose architecture consist of several modules, with spare interconnections between modules. Modularity allows the neural network developer to solve smaller tasks separately using small modules and then combine these modules in a logical manner. AES Institute Of Computer Studies 26
  31. 31. CHARACTER RECOGNITION USING NEURAL NETWORK 3.7 Perceptron The perceptron is a type of artificial neural network invented in 1958 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier. The perceptron is essentially a linear classifier for classifying data specified by parameters and an output function f = w'x + b. Its parameters are adapted with an adhoc rule similar to stochastic steepest gradient descent. Because the inner product is a linear operator in the input space, the perceptron can only perfectly classify a set of data for which different classes are linearly separable in the input space, while it often fails completely for non-separable data. While the development of the algorithm initially generated some enthusiasm, partly because of its apparent relation to biological mechanisms, the later discovery of this inadequacy caused such models to be abandoned until the introduction of non-linear models into the field. The cognitron (1975) designed by Kunihiko Fukushima was an early multilayered neural network with a training algorithm. The actual structure of the network and the methods used to set the interconnection weights change from one neural strategy to another, each with its advantages and disadvantages. Networks can propagate information in one direction only, or they can bounce back and forth until selfactivation at a node occurs and the network settles on a final state. The ability for bidirectional flow of inputs between neurons/nodes was produced with the Hopfield's network (1982), and specialization of these node layers for specific purposes was introduced through the first hybrid network. The parallel distributed processing of the mid-1980s became popular under the name connectionism. The rediscovery of the backpropagation algorithm was probably the main reason behind the repopularisation of neural networks after the publication of "Learning Internal Representations by Error Propagation" in 1986 (Though backpropagation itself dates from 1969). AES Institute Of Computer Studies 27
  32. 32. CHARACTER RECOGNITION USING NEURAL NETWORK The original network utilized multiple layers of weight-sum units of the type f = g(w'x + b), where g was a sigmoid function or logistic function such as used in logistic regression. Training was done by a form of stochastic Gradient descent. The employment of the chain rule of differentiation in deriving the appropriate parameter updates results in an algorithm that seems to 'backpropagate errors', hence the nomenclature. However it is essentially a form of gradient descent. Determining the optimal parameters in a model of this type is not trivial, and steepest gradient descent methods cannot be relied upon to give the solution without a good starting point. In recent times, networks with the same architecture as the backpropagation network are referred to as Multi-Layer Perceptrons. This name does not impose any limitations on the type of algorithm used for learning. The backpropagation network generated much enthusiasm at the time and there was much controversy about whether such learning could be implemented in the brain or not, partly because a mechanism for reverse signalling was not obvious at the time, but most importantly because there was no plausible source for the 'teaching' or 'target' signal. AES Institute Of Computer Studies 28
  34. 34. CHARACTER RECOGNITION USING NEURAL NETWORK APPLICATIONS OF ARTIFICIAL NEURAL NETWORK There have been many impressive demonstrations of artificial neural networks. A few areas where neural networks are currently applied are mentioned below. 4.1 Neural Networks in Practice Given this description of neural networks and how they work, what real world applications are they suited for? Neural networks have broad applicability to real world business problems. In fact, they have already been successfully applied in many industries. Since neural networks are best at identifying patterns or trends in data, they are well suited for prediction or forecasting needs including: sales forecasting industrial process control customer research data validation risk management target marketing But to give you some more specific examples : ANN are also used in the following specific paradigms: recognition of speakers in communications; diagnosis of hepatitis; recovery of telecommunications from faulty software; interpretation of multimeaning Chinese words; undersea mine AES Institute Of Computer Studies 30
  35. 35. CHARACTER RECOGNITION USING NEURAL NETWORK detection; texture analysis; three-dimensional object recognition; hand-written word recognition; and facial recognition. 4.2. Neural Networks in Medicine Artificial Neural Networks (ANN) are currently a 'hot' research area in medicine and it is believed that they will receive extensive application to biomedical systems in the next few years. At the moment, the research is mostly on modelling parts of the human body and recognizing diseases from various scans (e.g. cardiograms, CAT scans, ultrasonic scans, etc.). Neural networks are ideal in recognizing diseases using scans since there is no need to provide a specific algorithm on how to identify the disease. Neural networks learn by example so the details of how to recognize the disease are not needed. What is needed is a set of examples that are representative of all the variations of the disease. The quantity of examples is not as important as the 'quantity'. The examples need to be selected very carefully if the system is to perform reliably and efficiently. 4.2.1 Modelling and Diagnosing the Cardiovascular System Neural Networks are used experimentally to model the human cardiovascular system. Diagnosis can be achieved by building a model of the cardiovascular system of an individual and comparing it with the real time physiological measurements taken from the patient. If this routine is carried out regularly, potential harmful medical conditions can be detected at an early stage and thus make the process of combating the disease much easier. A model of an individual's cardiovascular system must mimic the relationship among physiological variables (i.e., heart rate, systolic and diastolic blood pressures, and breathing rate) at different physical activity levels. If a model is adapted to an individual, then it becomes a model of the physical condition of that individual. The simulator will have to be able to adapt to the features of any individual without the supervision of an expert. This calls for a neural network. AES Institute Of Computer Studies 31
  36. 36. CHARACTER RECOGNITION USING NEURAL NETWORK Another reason that justifies the use of ANN technology, is the ability of ANNs to provide sensor fusion which is the combining of values from several different sensors. Sensor fusion enables the ANNs to learn complex relationships among the individual sensor values, which would otherwise be lost if the values were individually analysed. In medical modelling and diagnosis, this implies that even though each sensor in a set may be sensitive only to a specific physiological variable, ANNs are capable of detecting complex medical conditions by fusing the data from the individual biomedical sensors. 4.2.2 Electronic Noses ANNs are used experimentally to implement electronic noses. Electronic noses have several potential applications in telemedicine. Telemedicine is the practice of medicine over long distances via a communication link. The electronic nose would identify odours in the remote surgical environment. These identified odours would then be electronically transmitted to another site where an door generation system would recreate them. Because the sense of smell can be an important sense to the surgeon, telesmell would enhance telepresent surgery. 4.2.3 Instant Physician An application developed in the mid-1980s called the "instant physician" trained an auto associative memory neural network to store a large number of medical records, each of which includes information on symptoms, diagnosis, and treatment for a particular case. After training, the net can be presented with input consisting of a set of symptoms; it will then find the full stored pattern that represents the "best" diagnosis and treatment. AES Institute Of Computer Studies 32
  37. 37. CHARACTER RECOGNITION USING NEURAL NETWORK 4.3 Neural Networks in Business Business is a diverted field with several general areas of specialization such as accounting or financial analysis. Almost any neural network application would fit into one business area or financial analysis. There is some potential for using neural networks for business purposes, including resource allocation and scheduling. There is also a strong potential for using neural networks for database mining, that is, searching for patterns implicit within the explicitly stored information in databases. Most of the funded work in this area is classified as proprietary. Thus, it is not possible to report on the full extent of the work going on. Most work is applying neural networks, such as the Hopfield-Tank network for optimization and scheduling. 4.4 Marketing There is a marketing application which has been integrated with a neural network system. The Airline Marketing Tactician (a trademark abbreviated as AMT) is a computer system made of various intelligent technologies including expert systems. A feedforward neural network is integrated with the AMT and was trained using back-propagation to assist the marketing control of airline seat allocations. The adaptive neural approach was amenable to rule expression. Additionally, the application's environment changed rapidly and constantly, which required a continuously adaptive solution. The system is used to monitor and recommend booking advice for each departure. Such information has a direct impact on the profitability of an airline and can provide a technological advantage for users of the system. While it is significant that neural networks have been applied to this problem, it is also important to see that this intelligent technology can be integrated with expert systems and other approaches to make a functional system. Neural networks were used to discover the influence of undefined interactions by the various variables. While these interactions were not defined, they were used by the neural system to develop useful conclusions. It is also noteworthy to see that neural networks can influence the bottom line. AES Institute Of Computer Studies 33
  38. 38. CHARACTER RECOGNITION USING NEURAL NETWORK 4.5 Credit Evaluation The HNC Company, founded by Robert Hecht-Nielsen, has developed several neural network applications. One of them is the Credit Scoring system which increases the profitability of the existing model up to 27%. The HNC neural systems were also applied to mortgage screening. A neural network automated mortgage insurance underwriting system was developed by the Nestor Company. This system was trained with 5048 applications of which 2597 were certified. The data related to property and borrower qualifications. In a conservative mode the system agreed on the underwriters on 97% of the cases. In the liberal model the system agreed 84% of the cases. This is system run on an Apollo DN3000 and used 250K memory while processing a case file in approximately 1 sec. 4.6 Signal Processing In digital communication systems, distorted signals cause inter-signal interference. One of the first commercial applications of neural networks was to suppress noise cancellation and it was implemented by widrow using ADALINE. The ADALINE is trained to remove the noise from the telephone line signal. 4.7 Speech Recognition In recent years, speech recognition has received enormous attention. It involves three modules namely, the front end which samples the speech signals and extracts the data the word processor which is used for finding the probability of words of words in the vocabulary that match the features of spoken words and the sentence processor which determines if the recognized word makes sense in the sentence. Neural networks remove the noise in the speech. AES Institute Of Computer Studies 34
  39. 39. CHARACTER RECOGNITION USING NEURAL NETWORK 4.8 Intelligent Control A problem of concern in industrial motor control is the ability to predict system failure. Motor failure depends on specific motor parameters, transient currents and motor positioning. Neural networks are employed to learn the motor-current variants as well as installation characteristics. 80% to 90% accurate prediction rate has been reported. Many of the neural networks applications merge in Robotics. The simplest tasks performed are arm movement, object recognition and object manipulation. Neural networks have been used in many vehicular applications including trains and automatic gear transmission in cars. Neural networks had been used in steel rolling machines to control the strip thickness within very tight tolerances. Function Approximation Many computational models can be described as functions mapping the input vectors to numerical outputs. Neural networks are employed to construct the function that generates approximately the same output fro a given input vector based on the available training data. 4.9 Financial Forecasting One of the most sought after applications of neural networks have been in financial application. The idea that if one dealer can predict market values or assess risk better than his competitors even by a very small amount, leads to an expectation of huge financial benefits. Much of the work in this area is based on the so called ‘time series prediction’. In contrast with the time series prediction, neural networks have also been used in financial modeling which aims at testing theories about the dynamics of market. AES Institute Of Computer Studies 35
  40. 40. CHARACTER RECOGNITION USING NEURAL NETWORK 4.10 Condition Monitoring The ability of neural nets to receive input from a large number of sensors and to learn to assess the significance of these is a major advantage in condition monitoring. The sensitivity to novelty is almost an emergent property of most neural systems. This has obvious applications in the detection of departures from normality. Time dependent behavior of nets can also be used in predicating the occurrence of malfunction ahead of time. Applications under this range from health monitoring of turbo machinery to gear boxes, jets and internal combustion engines. Process Monitoring and Control Neurocontrol principles can be applied to various areas of the process control industry. Neural techniques come into their own when the link between the measurement and operation parameter selection cannot be obtained analysis. These techniques can be applied in processes such as obtained by analysis. These techniques can be applied in processes such as chemical and biochemical, pharmaceutical, water purification, food and beverage production, power distribution and oil and gas industries. 4.11 Neuro Forecasting A major application of neural nets is the prediction of prospective long term foreign exchange rates. Neural networks provide greater flexibility in this regard compared to the traditional methods. 4.12 Pattern Analysis [B35] This application works in various areas such as online industrial plant monitoring, smart sensor systems and industrial image analysis as may be required in inspection and control tasks. Other classical problems of pattern analysis such as real time audio bandwidth systems, images and data management can also be tackled with the help of neural networks. AES Institute Of Computer Studies 36
  41. 41. CHARACTER RECOGNITION USING NEURAL NETWORK 4.13 Classification A neural network can discover the distinguishing features needed to perform a classification task. Classification is the assignment of each object to a specific class, which is an important aspect in image classification. Neural networks have been used successfully in a large number of classification tasks which includes: a. Recognition of printed or handwritten characters. b. Classification of SONAR and RADAR signals. AES Institute Of Computer Studies 37
  43. 43. CHARACTER RECOGNITION USING NEURAL NETWORK 5.1 What is Character Recognition? Character recognition is the mechanical or electronic translation of images of handwritten or printed text into machine-editable text. It is widely used to convert a books and documents into electronic files, to computerize a record-keeping system in an office, or to publish the text on a website. It is often useful to have a machine perform pattern recognition. In particular, machines that can read symbols are very cost effective. A machine that reads banking checks can process many more checks than a human being in the same time. This kind of application saves time and money, and eliminates the requirement that a human perform such a repetitive task. Many scientist and major company implement on character recognition from 1945 to till the date with use of neural network. And most of them are very successful with 99% accuracy. AES Institute Of Computer Studies 39
  44. 44. CHARACTER RECOGNITION USING NEURAL NETWORK 5.2 History for Character Recognition 1915 : U.S. Patent on handwriting recognition user interface with a stylus. 1957 : Stylator tablet: Tom Dimond demonstrateselectronic tablet with pen for Computer input and handwriting recognition. 1961 : RAND Tablet inveted : better known than earlier Stylator system. 1962 : Computer recognition of connected/script handwriting. 1969 : GRAIL system: handwriting recognition with electronic ink display, gesture Commands. 1973 : Application CAD/CAM computer system using the Ledeen recognizer for Handwriting recognition. 1980 : Retail handwriting-recognition systems: Pencept and CIC both offer PC Computers for the consumer market using a tablet and handwriting recognition Instead of a keyboard and mouse. Cadre System markets Inforite point-of-sale terminal using handwriting Recognition and a small electronic tablet and pen. 1989 : Portable handwriting recognition computer : GRIDPAD from Grid System. 1997 : First handwritten address interpretation system(HWAI) deployed by United State Postal Service. 2007 : First automatic writer recognition system : CEDAR-FOX. AES Institute Of Computer Studies 40
  45. 45. CHARACTER RECOGNITION USING NEURAL NETWORK 5.3 Steps for The Character Recognition There are different steps which should be follow for recognize the character which are given below. 1. The input neurons receive the pixel data from the image. 2. Each input neuron calculates its output. 3. This output is fed directly to the neurons in the hidden layer. 4. The neurons in the hidden layer calculate their output. 5. This output is fed to the neurons in the output layer. 6. The neurons in the output layer calculate their output. 7. The output of the output layer is our result. We interpret it the way we want. Practically how its work and which functions we can use in the character Recognition using MATLAB, all the details are given in next chapter. MATLAB is a mathematical software which cointain Artificial Neural Network toolbox and mathematical functions as well as neural network functions like back propagation network, logsig, tansig, linear methods, and also many image processing tools. We will also discuss about MATLAB in next chapter. AES Institute Of Computer Studies 41
  47. 47. CHARACTER RECOGNITION USING NEURAL NETWORK HOW TO RECOGNIZE CHARACTER A network is to be designed and trained to recognize the 26 letters of the alphabet. An imaging system that digitizes each letter centered in the system's field of vision is available. The result is that each letter is represented as a 5 by 7 grid of Boolean values. For example, here is the letter A. However, the imaging system is not perfect, and the letters can suffer from noise. AES Institute Of Computer Studies 43
  48. 48. CHARACTER RECOGNITION USING NEURAL NETWORK Perfect classification of ideal input vectors is required, and reasonably accurate classification of noisy vectors. The twenty-six 35-element input vectors are defined in the function prprob as a matrix of input vectors called alphabet. The target vectors are also defined in this file with a variable called targets. Each target vector is a 26-element vector with a 1 in the position of the letter it represents, and 0's everywhere else. For example, the letter A is to be represented by a 1 in the first element (as A is the first letter of the alphabet), and 0's in elements two through twenty-six. When Insert in network The network receives the 35 Boolean values as a 35-element input vector. It is then required to identify the letter by responding with a 26-element output vector. The 26 elements of the output vector each represent a letter. To operate correctly, the network should respond with a 1 in the position of the letter being presented to the network. All other values in the output vector should be 0. In addition, the network should be able to handle noise. In practice, the network does not receive a perfect Boolean vector as input. Specifically, the network should make as few mistakes as possible when classifying vectors with noise of mean 0 and standard deviation of 0.2 or less. The neural network needs 35 inputs and 26 neurons in its output layer to identify the letters. The network is a two-layer log-sigmoid/log-sigmoid network. The AES Institute Of Computer Studies 44
  49. 49. CHARACTER RECOGNITION USING NEURAL NETWORK log-sigmoid transfer function was picked because its output range (0 to 1) is perfect for learning to output Boolean values. The hidden (first) layer has 10 neurons. This number was picked by guesswork and experience. If the network has trouble learning, then neurons can be added to this layer. The network is trained to output a 1 in the correct position of the output vector and to fill the rest of the output vector with 0's. However, noisy input vectors can result in the network's not creating perfect 1's and 0's. After the network is trained the output is passed through the competitive transfer function compet. This makes sure that the output corresponding to the letter most like the noisy input vector takes on a value of 1, and all others have a value of 0. The result of this postprocessing is the output that is actually used. After applying training with noise and without noise we can get the following result as a character recognition. AES Institute Of Computer Studies 45
  51. 51. CHARACTER RECOGNITION USING NEURAL NETWORK 7.1 WHAT IS MATLAB ? MATLAB (for matrix laboratory) is a numerical computing environment and fourth-generation programming language. Developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, and Fortran.It provides many readymade functionality of solving difficult area like processing and recognition etc in a simple way. It contain different formula and mathematical functions and many valid tool box for easily maintain and overcome from difficult processing with a simplest way. And also its much faster and successful result in many area. There are different types of versions available in market with new upgraded tools and functionality. The full name of Matlab - the language of technical computing. We can also get a 3d module like figure and mapping of two or more than two things in a different way with different tools. The nnet Toolbox extends the basic capabilities of Matlab by providing a number of functions for character recognize from images. It also recognize character from image either it is good or bad condition. Matlab also provide Toolbox for image processing. This toolbox provide number of input output functions and control for image processing. E.e image blur – deblur , image scaling, image binding, resizing pixels and arrange pixel for making a good quality picture. Although MATLAB is intended primarily for numerical computing, an optional toolbox uses the MuPAD symbolic engine, allowing access to symbolic computing capabilities. An additional package, Simulink, adds graphical multidomain simulation and Model-Based Design for dynamic and embedded systems. In 2004, MATLAB had around one million users across industry and academia. MATLAB users come from various backgrounds of engineering, science, and economics. MATLAB is widely used in academic and research institutions as well as industrial enterprises. AES Institute Of Computer Studies 47
  52. 52. CHARACTER RECOGNITION USING NEURAL NETWORK 7.2 History AND Present MATLAB was created in the late 1970s by Cleve Moler, the chairman of the computer science department at the University of New Mexico.[3] He designed it to give his students access to LINPACK and EISPACK without having to learn Fortran. It soon spread to other universities and found a strong audience within the applied mathematics community. Jack Little, an engineer, was exposed to it during a visit Moler made to Stanford University in 1983. Recognizing its commercial potential, he joined with Moler and Steve Bangert. They rewrote MATLAB in C and founded MathWorks in 1984 to continue its development. These rewritten libraries were known as JACKPAC.[4] In 2000, MATLAB was rewritten to use a newer set of libraries for matrix manipulation. There are diferent types of version available each new version contain much powerful and successful toolbox and methods which are very useful in natural science and new implementation. MATLAB release its first version in 1984.when they launch its 3.5 version it higher then 1.0 so it required 386 processor and math coprocessor.and it ran on MSDOS. After that they realese last version R2007a and R2007b for wondows 2000 and PowerPC Mac. MATLAB release 7.8 for 32-bit and 64-bit window–7, 64-bit Mac and Solaris SPARC. And finally last version for intel 32-bit Mac in year 2010. AES Institute Of Computer Studies 48
  53. 53. CHARACTER RECOGNITION USING NEURAL NETWORK 7.3 NEURAL NETWORK TOOLBOX Neural networks are composed of simple elements operating in parallel. These elements are inspired by biological nervous systems. As in nature, the connections between elements largely determine the network function. You can train a neural network to perform a particular function by adjusting the values of the connections (weights) between elements. Typically, neural networks are adjusted, or trained, so that a particular input leads to a specific target output. The next figure illustrates such a situation. There, the network is adjusted, based on a comparison of the output and the target, until the network output matches the target. Typically, many such input/target pairs are needed to train a network. Neural networks have been trained to perform complex functions in various fields, including pattern recognition, identification, classification, speech, vision, and control systems. Neural networks can also be trained to solve problems that are difficult for conventional computers or human beings. The toolbox emphasizes the use of neural network paradigms that build up to—or are themselves used in— engineering, financial, and other practical applications. The next sections explain how AES Institute Of Computer Studies 49
  54. 54. CHARACTER RECOGNITION USING NEURAL NETWORK to use three graphical tools for training neural networks to solve problems in function fitting, pattern recognition, and clustering. We can solve following application by using Neural Network Toolbox (Nnet). Aerospace High-performance aircraft autopilot, flight path simulation, aircraft control systems, autopilot enhancements, aircraft component simulation, and aircraft component fault detection Automotive Automobile automatic guidance system, and warranty activity analysis Banking Check and other document reading and credit application evalution Defence Weapon steering, target tracking, object discrimination, facial recognition, new kinds of sensors, sonar, radar and image signal processing including data compression, feature extraction and noise suppression, and signal/image identification Electronics Code sequence prediction, integrated circuit chip layout, process control, chip failure analysys, machine vision, voice synthesis, and nonlinear modeling Entertainment Animation, special effects, and market forecasting Financial Real estate appraisal, loan advising, mortgage screening, corporate bond rating, credit-line use analysis, credit card activity tracking, portfolio trading program, corporate financial analysis, and currency price prediction AES Institute Of Computer Studies 50
  55. 55. CHARACTER RECOGNITION USING NEURAL NETWORK Industrial Prediction of industrial processes, such as the output gases of furnances, replacing complex and costly equipment used for this purpose in the past Insurance Policy application evaluation and product optimization Manufacturing Manufacturing process control, product design and analysis, process and machine diagnosis, real-time particle identification, visual quality inspection systems, beer testing, welding quality analysis, paper quality prediction, computer-chip quality analysis, analysis of grinding operations, chemical product design analysis, project bidding, planning and management, and dynamic modeling of chemical process system Medical Breast cancer cell analysis, EEG and ECG analysis, prosthesis design, optimization of transplant times, hospital expense reduction, hospital quality improvement, and emergency-room test advisement Oil and gas Exploration Robotics Trajectory control, forklift robot, manipulator controllers, and vision systems Speech Speech recognition, speech compression, vowel classification, and text-to-speech synthesis Securities Market analysis, automatic bond rating, and stock trading advisory systems Telecommunications Image and data compression, automated AES Institute Of Computer Studies 51
  56. 56. CHARACTER RECOGNITION USING NEURAL NETWORK information services, real-time translation of spoken language, and customer payment processing systems Transportation Truck brake diagnosis systems, vehicle scheduling, and routing systems AES Institute Of Computer Studies 52
  58. 58. CHARACTER RECOGNITION USING NEURAL NETWORK 8.1Kevin Larson paper and Application Kevin Larson from microsoft corporation,he research on Advance Reading Technology in july 2004. According to him, Evidence from the last 20 years of work in cognitive psychology indicate that we use the letters within a word to recognize a word. Many typographers and other text enthusiasts I’ve met insist that words are recognized by the outline made around the word shape. Some have used the term bouma as a synonym for word shape, though He was unfamiliar with the term. The term bouma appears in Paul Saenger’s 1997 book Space Between Words: The Origins of Silent Reading. There he learned to his chagrin that we recognize words from their word shape and that “Modern psychologists call this image the ‘Bouma shape.’” In his paper he did dozen of experiments and all come from peer reviewed journal where the experiments are well specified so that anyone could reproduce the experiment and expect to achieve the same result. The paper was originally presented as a talk at the ATypI conference in Vancouver in September ,2003. The main goal of his paper was to review the history of why psychologists moved from a word shape model of word recognition to a letter recognition model, and to help others to come to the same conclusion. Paper covered many topics in relatively few pages. Along the way he presents experiments and models that he couldn’t hope to cover completely without boring the reader. STRATEGY In his paper he described three major categories of word recognition models that are as below. Word shape model Serial model Parallel model for letter recognition. Model - 1: Word Shape The word recognition model that says words are recognized as complete units is the oldest model in the psychological literature, and is likely much older than the psychological literature. The general idea is that we see words as a complete patterns AES Institute Of Computer Studies 54
  59. 59. CHARACTER RECOGNITION USING NEURAL NETWORK rather than the sum of letter parts. Some claim that the information used to recognize a word is the pattern of ascending, descending, and neutral characters. Another formulation is to use the envelope created by the outline of the word. The word patterns are recognizable to us as an image because we have seen each of the patterns many times before. James Cattell (1886) was the first psychologist to propose this as a model of word recognition. Cattell is recognized as an influential founder of the field of psycholinguistics, which includes the scientific study of reading. Figure 1: Word shape recognition using the pattern of ascending, descending, and neutral characters. Characters Figure 2: Word shape recognition using the envelope around the word Cattell’s study was sloppy by modern standards, but the same effect was replicated in 1969 by Reicher. He presented strings of letters – half the time real words, half the time not – for brief periods. The subjects were asked if one of two letters were contained in the string, for example D or K. Reicher found that subjects were more accurate at recognizing D when it was in the context of WORD than when in the context of ORWD. This supports the word shape model because the word allows the subject to quickly recognize the familiar shape. Once the shape has been recognized, then the AES Institute Of Computer Studies 55
  60. 60. CHARACTER RECOGNITION USING NEURAL NETWORK subject can deduce the presence of the correct letter long after the stimulus presentation. The second key piece of experimental data to support the word shape model is that lowercase text is read faster than uppercase text. Woodworth (1938) was the first to report this finding in his influential textbook Experimental Psychology. This finding has been confirmed more recently by Smith (1969) and Fisher (1975). Participants were asked to read comparable passages of text, half completely in uppercase text and half presented in standard lowercase text. In each study, participants read reliably faster with the lowercase text by a 5-10% speed difference. This supports the word shape model because lowercase text enables unique patterns of ascending, descending, and neutral characters. When text is presented in all uppercase, all letters have the same text size and thus are more difficult and slower to read. Model - 2: Serial Letter Recognition The shortest lived model of word recognition is that words are read letter-byletter serially from left to right. Gough (1972) proposed this model because it was easy to understand, and far more testable than the word shape model of reading. In essence, recognizing a word in the mental lexicon was analogous to looking up a word in a dictionary. You start off by finding the first letter, than the second, and so on until you recognize the word. This model is consistent with Sperling’s (1963) finding that letters can be recognized at a rate of 10-20ms per letter. Sperling showed participants strings of random letters for brief periods of time, asking if a particular letter was contained in the string. He found that if participants were given 10ms per letter, they could successfully complete the task. For example, if the target letter was in the fourth position and the string was presented for 30ms, the participant couldn’t complete the task successfully, but if string was presented for 40ms, they could complete the task successfully. Gough AES Institute Of Computer Studies 56
  61. 61. CHARACTER RECOGNITION USING NEURAL NETWORK noted that a rate of 10ms per letter would be consistent with a typical reading rate of 300 wpm. The serial letter recognition model is also able to successfully predict that shorter words are recognized faster than longer words. It is a very robust finding that word recognition takes more time with longer words. It takes more time to recognize a 5-letter word than a 4-letter word, and 6-letter words take more time to recognize than 5-letter words. The serial letter recognition model predicts that this should happen, while a word shape model does not make this prediction. In fact, the word shape model should expect longer words with more unique patterns to be easier to recognize than shorter words. The serial letter recognition model fails because it cannot explain the Word Superiority Effect. Model - 3: Parallel Letter Recognition The model that most psychologists currently accept as most accurate is the parallel letter recognition model. This model says that the letters within a word are recognized simultaneously, and the letter information is used to recognize the words. AES Institute Of Computer Studies 57
  62. 62. CHARACTER RECOGNITION USING NEURAL NETWORK Above Figure shows a generic activation based parallel letter recognition model. In this example, the reader is seeing the word work. Each of the stimulus letters are processed simultaneously. The first step of processing is recognizing the features of the individual letters, such as horizontal lines, diagonal lines, and curves. The details of this level are not critical for our purposes. These features are then sent to the letter detector level, where each of the letters in the stimulus word are recognized simultaneously. The letter level then sends activation to the word detector level. The W in the first letter detector position sends activation to all the words that have a W in the first position (WORD and WORK). The O in the second letter detector position sends activation to all the words that have an O in the second position (FORK, WORD, and WORK). While FORK and WORD have activation from three of the four letters, WORK has the most activation because it has all four letters activated, and is thus the recognized word. AES Institute Of Computer Studies 58
  63. 63. CHARACTER RECOGNITION USING NEURAL NETWORK 8.2 Ketil Hunn Paper and Application Ketil Hunn created Character Recognition application using back propagation in neural network. He used Parallel distributed Processing in University of Pittsburgh, Solving Problem Reducing the resolution of each character to 15x9 pixels was necessary to decrease the use of memory and learning time. By letting the user draw characters directly in the program we eliminate the problem with noise described earlier. Noise, if desirable, can be added by drawing arbitrary patterns around the drawn character. Letting the user draw characters directly in the program also simplifies demonstrations since a demonstration can be created on the fly without having to create input files etc. Each character will be drawn in a separate box eliminating the problem of separating characters. Above figure shows a snapshot of his application. In this we just have to enter character. Here we can see that four character ‘H’,’E’,’L’ and ‘O’. He Train network and learn the network via backpropagation algorithm. And in this how many iteration fired at hidden layer and pattern graph are given below. AES Institute Of Computer Studies 59
  64. 64. CHARACTER RECOGNITION USING NEURAL NETWORK Learning Character New characters can be added to the database by drawing the character in the box , selecting which character it will map onto and then pressing Add character. When done entering all new characters, pressing Learn characters will make the program learn the characters in the database using backprop. The program does this by first initializing all weights and biases with random values between -1.0 and 1.0. Then for each pattern it chooses randomly, it then computes the hidden and response activities as described in the previous chapter before it backpropagates the error in the network. The error in each response unit is calculated using douti=(Ti-ri)*ri(1-ri)*dt and then the error in the hidden units are calculated using dhidi=Sum((kdoutk*vki*hi(1-hi)*dt). Finally the program updates the weights and biases; dci=douti*dt, dbi=dhidi*dt, dvij=douti*hj*dt and dwij=dhidi*sj*dt. The procedure of choosing a random pattern, performing feedforward and then backprop continues until the total error of the network is less than the error limit or until the user interrupts the learning. The total error of the network is calculated as the sum square error between the target and the response value, e=(T-r)². The error shown during the learning procedure is the greatest error of all the patterns. AES Institute Of Computer Studies 60
  65. 65. CHARACTER RECOGNITION USING NEURAL NETWORK Result : There were several problems to take into consideration when performing backprop on a neural network, such as the number of hidden units to use, setting the learning rate etc. The results were surprisingly good, almost 90% accuracy! Normally it will take a real character recognition program years to get to this level of accuracy. Suggetion He also suggest the following things in his paper for more accuracy and more clear result. Character scaling Since a neural network only receives data in form of input stimuli it cannot recognize different sizes of a character without having to learn all possible sizes. By letting the program scale characters this problem would be reduced to a minimum. Character centering Another problem appears when the user draws characters with a different alignment than the one learned, i.e. drawing a left aligned I when all I’s that have been learned are centered. This can easily be solved by letting the program center all characters before they are stored in the database and before interpreting them. AES Institute Of Computer Studies 61
  66. 66. CHARACTER RECOGNITION USING NEURAL NETWORK 8.3 Current Market Scenario There are many huge Software company that have implement on such a software and also available on the web(on Line). Among them some application are costly and some are much cheaper. We can have brief idea from below chart. AES Institute Of Computer Studies 62
  68. 68. CHARACTER RECOGNITION USING NEURAL NETWORK 8.4 Current Research While initially research had been concerned mostly with the electrical characteristics of neurons, a particularly important part of the investigation in recent years has been the exploration of the role of neuromodulators such as dopamine, acetylcholine, and serotonin on behaviour and learning. Biophysical models, such as BCM theory, have been important in understanding mechanisms for synaptic plasticity, and have had applications in both computer science and neuroscience. Research is ongoing in understanding the computational algorithms used in the brain, with some recent biological evidence for radial basis networks and neural backpropagation as mechanisms for processing data. Computational devices have been created in CMOS for both biophysical simulation and neuromorphic computing. More recent efforts show promise for creating nanodevices for very large scale principal components analyses and convolution. If successful, these efforts could usher in a new era of neural computing that is a step beyond digital computing, because it depends on learning rather than programming and because it is fundamentally analog rather than digital even though the first instantiations may in fact be with CMOS digital devices. AES Institute Of Computer Studies 64
  69. 69. CHARACTER RECOGNITION USING NEURAL NETWORK Proposed Work AES Institute Of Computer Studies 65
  70. 70. CHARACTER RECOGNITION USING NEURAL NETWORK 9.1 Work Introduction In research work me and my coleague try to understand how its application work. In this work we create a character recognition application using MATLAB.more detailed review and understanding about this application are as given below. 9.2 Application Here is first screen of the Application.in which we can see the main image screen and other processing buttons with processed image view. AES Institute Of Computer Studies 66
  71. 71. CHARACTER RECOGNITION USING NEURAL NETWORK Open Image and Select character AES Institute Of Computer Studies 67
  72. 72. CHARACTER RECOGNITION USING NEURAL NETWORK Image Processing In this stage we can see the original image in selected character part. From that image we conver image in to gray scale and build the image in binary. we can see these image in Process On Image. After converting it in to binary and build it we find the blank space area from top side then bottom,right and at last left and cropping image and convert image into 5 * 7 single vector Image. AES Institute Of Computer Studies 68
  73. 73. CHARACTER RECOGNITION USING NEURAL NETWORK PLOTTING IMAGE IN CHARVEC After cropping binary image we mapping pixels and pust into 35 element array in the network and create a character vector. We can see that charvec in the below stage. Here is the charvec with some noisy environment for more clear appearance we have to train network much higher rate. From this charvec it recognize the character and finally show it in to recognize frame. AES Institute Of Computer Studies 69
  74. 74. CHARACTER RECOGNITION USING NEURAL NETWORK Recognize Character AES Institute Of Computer Studies 70
  75. 75. CHARACTER RECOGNITION USING NEURAL NETWORK 9.3 Accuracy and Results Actually we have create this application for character recognition but unforunately its work for digit only.it doesn’t able to recognize Alphabet due to some network problem. But suprisingly its accuracy is around 85- 90 % of recognize the character. Due to some reason like more noisy image or dull image or may be the scale of image. Here some snapshot of success and failure of the recognition of the image. AES Institute Of Computer Studies 71
  76. 76. CHARACTER RECOGNITION USING NEURAL NETWORK 9.4 Recognize Digit – 5 Success Phase AES Institute Of Computer Studies 72
  77. 77. CHARACTER RECOGNITION USING NEURAL NETWORK 9.4 Recognize Digit– 3 Failure Phase AES Institute Of Computer Studies 73
  78. 78. CHARACTER RECOGNITION USING NEURAL NETWORK 9.4 Recognize Digit– 5 Failure Phase AES Institute Of Computer Studies 74
  80. 80. CHARACTER RECOGNITION USING NEURAL NETWORK Enhancements Recent advances and future applications of NNs Integration of fuzzy logic into neural networks • Fuzzy logic is a type of logic that recognizes more than simple true and false values, hence better simulating the real world. For example, the statement today is sunny might be 100% true if there are no clouds, 80% true if there are a few clouds, 50% true if it's hazy, and 0% true if rains all day. Hence, it takes into account concepts like -usually, somewhat, and sometimes. • Fuzzy logic and neural networks have been integrated for uses as diverse as automotive engineering, applicant screening for jobs, the control of a crane, and the monitoring of glaucoma. Pulsed neural networks • "Most practical applications of artificial neural networks are based on a computational model involving the propagation of continuous variables from one processing unit to the next. In recent years, data from neurobiological experiments have made it increasingly clear that biological neural networks, which communicate through pulses, use the timing of the pulses to transmit information and perform computation. This realization has stimulated significant research on pulsed neural networks, including theoretical analyses and model development, neurobiological modeling, and hardware implementation." Hardware specialized for neural networks Some networks have been hardcoded into chips or analog devices ? this technology will become more useful as the networks we use become more complex. The primary benefit of directly encoding neural networks onto chips or specialized analog devices is SPEED! AES Institute Of Computer Studies 76
  81. 81. CHARACTER RECOGNITION USING NEURAL NETWORK NN hardware currently runs in a few niche areas, such as those areas where very high performance is required (e.g. high energy physics) and in embedded applications of simple, hardwired networks (e.g. voice recognition). Many NNs today use less than 100 neurons and only need occasional training. In these situations, software simulation is usually found sufficient When NN algorithms develop to the point where useful things can be done with 1000's of neurons and 10000's of synapses, high performance NN hardware will become essential for practical operation. Improvement of existing technologies • All current NN technologies will most likely be vastly improved upon in the future. Everything from handwriting and speech recognition to stock market prediction will become more sophisticated as researchers develop better training methods and network architectures. NNs might, in the future, allow: robots that can see, feel, and predict the world around them improved stock prediction common usage of self-driving cars composition of music handwritten documents to be automatically transformed into formatted word processing documents trends found in the human genome to aid in the understanding of the data compiled by the Human Genome Project self-diagnosis of medical problems using neural networks and much more! AES Institute Of Computer Studies 77
  83. 83. CHARACTER RECOGNITION USING NEURAL NETWORK Conclusion The computing world has a lot to gain from neural networks. Their ability to learn by example makes them very flexible and powerful. Furthermore there is no need to devise an algorithm in order to perform a specific task; i.e. there is no need to understand the internal mechanisms of that task. They are also very well suited for real time systems because of their fast responseand computational times which are due to their parallel architecture. Neural networks also contribute to other areas of research such as neurology and psychology. They are regularly used to model parts of living organisms and to investigate the internal mechanisms of the brain. Perhaps the most exciting aspect of neural networks is the possibility that some day 'consious' networks might be produced. There is a number of scientists arguing that conciousness is a 'mechanical' property and that 'consious' neural networks are a realistic possibility. Finally, I would like to state that even though neural networks have a huge potential we will only get the best of them when they are integrated with computing, AI, fuzzy logic and related subjects. AES Institute Of Computer Studies 79
  85. 85. CHARACTER RECOGNITION USING NEURAL NETWORK REFERENCES [1]. S.N.SIVANANADAM and M.PAULRAJ = Introduction to ARTIFICIAL NEURAL NETWORKS [2]. CHILUKURI K.MOHAN and SANJAY RANKA = ELEMENTS OF Artificial Neural Network [3]. Ajith Abraham Oklahoma State University, Stillwater, OK, USA = Artificial Neural Networks. [4]. Kevin Larson from Microsoft Corporation, = Advance Reading Technology – July 2004 [5]. Ketil Hunn Paper and Application using Parallel Distributed processing in University of Pittsburgh. BIBILOGRAPHY www.google.com www.wikipedia.org www.wikipedia.org/wiki/Neural_network www.library.thinkquest.org/C007395/tqweb/history.html http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html http://www-cs-faculty.stanford.edu http://www.emsl.pnl.gov:2080/docs/cie/neural/neural.homepage.html http://en.wikipedia.org/wiki/Character_recognition http://www.microsoft.com/typography/ctfonts/wordrecognition.aspx http://home.eunet.no/~khunn/papers/2039.html AES Institute Of Computer Studies 81
  86. 86. CHARACTER RECOGNITION USING NEURAL NETWORK http://en.wikipedia.org/wiki/List_of_optical_character_recognition_software AES Institute Of Computer Studies 82