The document presents a method for integrated offline and online handwritten character recognition using artificial neural networks and its implementation on ARM processors. Horizontal and vertical scanning are used to extract features from characters to generate feature vectors for recognition. An accuracy of 99.12% was achieved for uppercase English alphabets using this integrated offline-online recognition method. The approach was implemented on an ARM-based e-book prototype for recognizing handwritten characters.

1. ICCIC120527: Integrated Approach To Handwritten
Character Recognition Using ANN And It’s
Implementation On ARM
Amol Mahurkar (presenter)
Sinhgad College of Engineering
Pune, India
Ganesh Rakate
Vishwakarma Institute of Technology
Pune, India
Authors:

2. Hardware Setup
• Resistive touch-screen
• We extract X-Y coordinates
of the sketched character
December 17, 2012 2

3. Extracting features: Horizontal Scanning
Step 1: Creating 2-D image of scanned character
Step 2: Horizontal scanning & Feature extraction
Why horizontal scanning?
• Works for tilted ‘E’ as well
Normal ‘E’ Tilted ‘E’
Feature Vector (FV) = [42, 7, 32, 7, 36]
X(t) = [1, 2, 4, 6, 7, ….]
Y(t) = [200, 201, 198, 200, …]
December 17, 2012 3

4. Sample histograms
Letter ‘E’ Letter ‘T’
December 17, 2012 4

5. Character Uniqueness: Introducing Vertical Scanning as well
Normal ‘E’ Tilted ‘E’ Accounting tilt of character by Vertical Scanning
Feature Vector (FV) = [42, 7, 32, 7, 36]
Feature Vector (FV) = [50, 21, 27, 14, 12]
Horizontal Scanning
December 17, 2012 5

6. Feature Vector Generation
Feature Vector (FV) = [20 22 35 23 21 27 24 23 25 28]
December 17, 2012 6

7. Performance – Offline recognition
Accuracy of offline recognition: 98.29%
December 17, 2012 7

8. Problem with offline recognition???
Yes!!
These following characters has similar FV,
December 17, 2012 8

9. Online recognition
X coords vs length of char
Same as X(t)
Y coords vs length of char
Same as Y(t)
[X Y] coords vs length of char
Same as [X(t) Y(t)]
December 17, 2012 9

10. Performance – Online recognition
Accuracy of online recognition: 96.15% ???
December 17, 2012 10

11. Integrated offline-online recognition
Accuracy of integrated method: 99.12%
December 17, 2012 11

12. MATLAB Coder: Savior
December 17, 2012 12

13. Intelligent e-book
December 17, 2012 13

14. Intelligent e-book in action
December 17, 2012 14

15. Conclusion
• We balanced a tradeoff between length of FV and accuracy, to
make implementation feasible.
• We achieved a accuracy of 99.12% on uppercase English
alphabets.
• By choosing right tradeoff factor, we managed to run our code
faster.
Future Work
• We intend to introduce all different set of characters into
device (such as small case characters, special characters)
• We also want our device to recognize देवनागर� font as well.
December 17, 2012 15

16. References
• H.Kamada, K.Fujimoto, K.Kurokawa: "High-speed, Highaccuracy Document Recognition System for Languages Containing Many Characters",
Proc. 3rd IAPR Workshop on Document Analysis Systems (DAS'98), 177-180 (1998.11)
• S.N.Srihari, “High Performance Reading Machines”, Proc. IEEE, Vol. 80., pp.1,120-1,132,1992.
• G.Seni, R.K.Srihari, and N.Nasrabadi, “Large Vocabulary Recognition of On-Line Handwritten Cursive Words”, IEEE Transactions on Pattern
Analysis and Machine Intelligence, Vol 18, no.7, pp. 757-762, July 1996.
• M.Nakagawa, K.Akiyama: "A Linear-time Elastic Matching for Stroke Number Free Recognition if On-line Handwritten Characters", Proc. 4th
IWFHR, 48-56 (1994.12)
• K.Akiyama, M.Nakagawa: "A Linear-Time Elastic Matching Algorithm for On-line Recognition of Handwritten Japanese Characters" (in
Japanese), Trans.IEICEJ, vol.81-DII, No.4, pp.651-659(1998.4)
• M. Nakai, N. Akira, H. Shimodaira, and S. Sagayama, “Substroke Approach to HMM-based On-line Kanji Handwriting Recognition,” Proc. of
ICDAR'01, pp.491-495, 2001
• Shruthi Kubatur, Maher Sid-Ahmed, Majid Ahmadi, " A Neural Network Approach to Online Devanagari Handwritten Character Recognition ",
High Performance Computing and Simulation (HPCS), pp-209-214, 2012
• S. Hangai, S. Yamanaka and T. Hamamoto, “On-Line Signature Verificatoin Based on Altitude and Direction of pen Movement,” IEEE
International Conference on Multimedia and Expo, ICME2000,(2000).
• J. -J. Lee and J. -H. Kim and M. Nakajima, “A Hierarchical HMM Network-Based Approach for On-Line Recognition of Multilingual Cursive
Handwritings,” IEICE Trans, E-81-D-8, 881-888, 1998
• N. Joshi, G. Sita, A. G. Ramakrishnan, Deepu V., S. Madhvanath, “Machine Recognition of Online Handwritten Devanagari Characters”, ICDAR
2005 International Conference on Document Analysis and Recognition, Vol. 2, 2005.
• Rabiner L. R., “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proc. IEEE, vol 77(2),pp.257-286,1989.
• G. E. Kopec and P. A. Chou, “Document Image Decoding Using Malkov Source Models,” IEEE Trans. on Pattern Analysis and Machine
Intelligence, 16(6), pp. 602-617, 1994.
• H. Nishimura, M. Kobayashi, M. Maruyama, Y. Nakano, ``Off-line Character Recognition Using HMM by Multiple Directional Feature Extraction
and Voting with Bagging Algorithm,'' Proc. ICDAR'99,pp.49-52,1999
• Hiroshi Tanaka, Kenji Nakajima, Kazushi Ishigaki, Katsuhiko Akiyama, Masaki Nakagawa, "Hybrid Pen-Input Character Recognition System
Based on Integration of Online-Offline Recognition", ICDAR '99. Proceedings of the Fifth International Conference on Digital Object Identifier,
pp-209-212, 1999
• Hiromitsu NISHIMURA, Takehiko TIMIKAWA, "Off-line Character Recognition using On-line Character Writing Information", Proc. ICDAR’03,
2003
December 17, 2012 16

17. Thank you

18. Bill of material
Component Cost (in Rs.)
Resistive Touch Screen 300
Graphic Liquid Crystal Display 550
LPC2138 stack board 1100
Battery + Charger 400
PCB 200
Others 50
Total 2600
* All prices are indicative. In actual can vary by mass prodution
“Necessity is the mother of invention”
December 17, 2012 18