Your SlideShare is downloading. ×
A SYSTEM FOR THE RECOGNITITON OF HANDWRITTEN YORÙBÁ  CHARACTERS BY ABDULRAHMAN O. IBRAHEEM AND ODETUNJI A. ODEJOBI AGIS ...
In this presentation.. <ul><li>Provide an overview of the issues  in Yoruba handwritten  character recognition </li></ul><...
YORUBA ORTHOGRAPHY <ul><li>Yorùbá  orthography comprises roman letters </li></ul><ul><ul><li>With diacritical marks to i...
SCOPE <ul><li>The Yoruba letters considered in this work are: </li></ul><ul><li>À  Á  È  É  Ẹ  Ẹ̀  Ẹ́  Ì  Í  </li></...
REVIEW OF LITERATURE <ul><li>Pattern recognition systems, including character recognition ones, include two main modules: ...
LITERATURE REVIEW OF FEATURES <ul><li>Hu (1962) derived seven invariant geometric features (called Hu moment invariants) i...
LITERATURE REVIEW OF FEATURES (CONTD) <ul><li>AMIs were shown to be superior to Hu moments in experiments that involved th...
LITERATURE REVIEW OF FEATURES (CONTD) <ul><li>We concluded that OFMM leads to lower noise to signal ratio.  </li></ul><ul>...
LITERATURE REVIEW OF FEATURES (CONTD) <ul><li>This approach is to essentially augment low order geometric moments with int...
THE PROPOSED SYSTEM <ul><li>The system we have developed comprises six stages: </li></ul><ul><ul><li>a pre-processing stag...
ILLUSTRATION OF THE SYSTEM Output class label roman letter  diacritical marks Pre-processing stage Input Yoruba character ...
DATA PREPARATION AND THE PRE-PROCESSING STAGE <ul><li>The Bayesian Stage of our system is data-driven. </li></ul><ul><li>T...
DATA PREPARATION AND THE PRE-PROCESSING STAGE (CONTD) <ul><li>Our volunteer writers were male and female students of the O...
DATA PREPARATION AND THE PRE-PROCESSING STAGE (CONTD) <ul><li>The A4 sheets were scanned, and with the aid of Paint graphi...
SAMPLES CROPPED SAMPLES CROPPED SAMPLES DIACRITIC REMOVE
DATA PREPARATION AND THE PRE-PROCESSING STAGE (CONTD) <ul><li>In the Matlab environment, we pre-processed each of the  sam...
CHARACTER DATA PROCESSING Cropped character Character after Binarisation Character after Noise removal Character and its b...
THE SEGMENTATION STAGE <ul><li>The segmentation stage attempts to isolate the roman letter from the diacritical marks. </l...
A Connected-at-core Yoruba character. An under dot (tail) in the lower region of a Yoruba character. The lower region  ( L...
A BROKEN-AT-CORE YORUBA CHARACTER A tail in the LR of a Yoruba character.  The largest object in  a  Yoruba character. The...
SOME TREMINOLOGIES USED IN FIGURE <ul><li>UR: Upper Region </li></ul><ul><li>MR: Middle Region </li></ul><ul><li>LR:  Lowe...
THE SEGMENTATION ALGORITHM <ul><li>Identify the largest object in the image by finding the object with the maximum  area. ...
THE SEGMENTATION ALGORITHM (CONTD) <ul><li>If the slope in step 4 above is less than 0.35, </li></ul><ul><li>release this ...
THE DECISION TREE STAGE <ul><li>The segmentation stage isolates the roman letter from the diacritical marks. </li></ul><ul...
CHARACTER DECISION TREE GWT Grave With Tail;  AWT Acute With Tail GDT Grave Devoid of Tail;  ADT  Acute Devoid of Tail;  M...
TERMINOLOGIES IN THE DECISION TREE <ul><li>GWT: Grave With Tail </li></ul><ul><li>AWT: Acute With Tail </li></ul><ul><li>G...
THE FEATURE EXTRACTION STAGE <ul><li>The segmentation isolates the roman letter of the Yoruba character from the diacritic...
THE EIGHT GEOMETRIC FEATURES WE EMPLOYED <ul><li>Mean of character widths </li></ul><ul><li>Mean of character heights </li...
THE EIGHT GEOMETRIC FEATURES WE EMPLOYED <ul><li>Second order normalized central moment </li></ul><ul><li>Haralick’s circu...
A DISPLAY OF THE FEATURES AND THEIR FORMULAE FEATURE  NO. FEATURE AND EXPLANATION OF FORMULA FORMULA 1 Mean character widt...
BAYESIAN DECISION THEORY (BDT) AND THE BAYESIAN CLASSIFIER STAGE. <ul><li>Bayesian Decision Theory is a fundamental tool i...
BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>Here is Baye’s formula : </li></ul><ul><li>c  is the number of Baye...
BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>is the class-conditional probability. We take it to be a Gaussian f...
BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>We can view Baye’s formula  as a decision rule formula with a zero-...
BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>The above equation is the form we employed in our Bayesian classifi...
BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>is the mean feature vector of the i-th class. </li></ul><ul><li>The...
THE RESULT FUSION STAGE <ul><li>Result fusion is handled in our system by the following equation: </li></ul><ul><li>is the...
THE RESULT FUSION STAGE (CONTD) <ul><li>is the label from the Bayesian classifier. This is an integer value between 1 and ...
THE RESULT FUSION STAGE (CONTD) <ul><li>is a function of  that  “pads the equation up” to achieve the neat one-to-one corr...
THE RESULT FUSION STAGE (CONTD) <ul><li>is defined in the table below: </li></ul>bdt_label f(bdt_label) 1 -1 2 0 3 4 4 5 5...
RESULTS <ul><li>We tested the system in two folds. </li></ul><ul><li>In the first fold, with eight independent samples fro...
RESULTS (CONTD) <ul><li>In the second fold, we tested the system with 3 non-independent samples from each of the six Bayes...
CONCLUSION AND FUTURE WORK <ul><li>In summary, we have developed a technique for the classification of diacritically marke...
Conclusion cont… <ul><li>We built the Bayesian stage using forty  samples per Bayesian class.  </li></ul><ul><li>We tested...
ONGOING WORK  <ul><li>We plan to extend this work in two directions: </li></ul><ul><ul><li>We wish to extend the system to...
Upcoming SlideShare
Loading in...5
×

A System for the Recognition of Handwritten Yorùbá Characters

1,073

Published on

© Abdulrahman O. Ibraheem & Odetunji A. Odejobi

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,073
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "A System for the Recognition of Handwritten Yorùbá Characters"

  1. 1. A SYSTEM FOR THE RECOGNITITON OF HANDWRITTEN YORÙBÁ CHARACTERS BY ABDULRAHMAN O. IBRAHEEM AND ODETUNJI A. ODEJOBI AGIS 2011 Ethiopia Obafemi Awolowo University, Ile-Ife, Nigeria
  2. 2. In this presentation.. <ul><li>Provide an overview of the issues in Yoruba handwritten character recognition </li></ul><ul><li>Discuss the literature in character recognition </li></ul><ul><li>Discuss an approach based on Bayesian classifier and a decision tree to recognition of the character </li></ul><ul><li>Discuss our results </li></ul><ul><li>State on-going work </li></ul>
  3. 3. YORUBA ORTHOGRAPHY <ul><li>Yorùbá orthography comprises roman letters </li></ul><ul><ul><li>With diacritical marks to indicate tone </li></ul></ul><ul><ul><li>under dot for some phone distinction </li></ul></ul><ul><li>We focus on Uppercase letter in this work </li></ul><ul><li>The uppercase letters of the Yoruba orthography are </li></ul><ul><ul><li>À A Á B D È E É Ẹ̀ Ẹ Ẹ́ F G GB H Ì I Í J K L M N Ò O Ó ̀Ọ Ọ Ọ́ P R Ş T Ù U Ú W Y. </li></ul></ul>AGIS 2011 Ethiopia
  4. 4. SCOPE <ul><li>The Yoruba letters considered in this work are: </li></ul><ul><li>À Á È É Ẹ Ẹ̀ Ẹ́ Ì Í </li></ul><ul><li>Ò Ó Ọ ̀Ọ Ọ́ Ş Ù Ú </li></ul><ul><li>Notice that they all come from just six roman letters ( termed the six Bayesian classes ): </li></ul><ul><li>A E I O U S </li></ul>AGIS 2011 Ethiopia
  5. 5. REVIEW OF LITERATURE <ul><li>Pattern recognition systems, including character recognition ones, include two main modules: </li></ul><ul><ul><li>a feature extraction module </li></ul></ul><ul><ul><li>classification module (Duda et. al 2000). </li></ul></ul><ul><li>This divides our literature review into two: a review of features, and a review of classifier techniques. </li></ul>
  6. 6. LITERATURE REVIEW OF FEATURES <ul><li>Hu (1962) derived seven invariant geometric features (called Hu moment invariants) in terms of a number of normalized central moments. </li></ul><ul><li>But they contain redundant information and that are not invariant under affine transformations such as slants. </li></ul><ul><li>Teague (1980) addressed the problem of information redundancy by proposing the use of Zernike moments (ZM) </li></ul><ul><li>The problem of affine transformations was addressed by Flusser and Suk (1994) who derived a class of moments called Affine Moment Invariants (AMI) . </li></ul>
  7. 7. LITERATURE REVIEW OF FEATURES (CONTD) <ul><li>AMIs were shown to be superior to Hu moments in experiments that involved the recognition of slanted characters. (Flusser and Suk; 1994)! </li></ul><ul><li>But AMIs are also not free from the information redundancy problem. </li></ul><ul><li>Although ZMs save on information redundancy, they are not very robust to noise. </li></ul><ul><li>Kan and Srinath (2002) compared ZMs with a class of moments known as Orthogonal Fourier-Mellin moments (OFMM). </li></ul>
  8. 8. LITERATURE REVIEW OF FEATURES (CONTD) <ul><li>We concluded that OFMM leads to lower noise to signal ratio. </li></ul><ul><li>BUT, it is computationally too involved. </li></ul><ul><li>We adopted the Frey and Slate (1991) approach which involves less computations. </li></ul><ul><li>This approach has been used successfully, e.g Schwenk and Bengio (1997), Breuel (2003), and Husnain and Naweed (2009). </li></ul>
  9. 9. LITERATURE REVIEW OF FEATURES (CONTD) <ul><li>This approach is to essentially augment low order geometric moments with intuitively defined features. </li></ul><ul><li>This is the direction we favor in this work. </li></ul>
  10. 10. THE PROPOSED SYSTEM <ul><li>The system we have developed comprises six stages: </li></ul><ul><ul><li>a pre-processing stage; </li></ul></ul><ul><ul><li>a segmentation stage; </li></ul></ul><ul><ul><li>a feature extraction stage; </li></ul></ul><ul><ul><li>a Bayesian classification stage; </li></ul></ul><ul><ul><li>a decision tree; </li></ul></ul><ul><ul><li>A stage for combining the results of the previous two stages into a single class label </li></ul></ul>
  11. 11. ILLUSTRATION OF THE SYSTEM Output class label roman letter diacritical marks Pre-processing stage Input Yoruba character Segmentation stage Feature extraction stage Decision tree stage Results fusion stage Bayesian classification stage
  12. 12. DATA PREPARATION AND THE PRE-PROCESSING STAGE <ul><li>The Bayesian Stage of our system is data-driven. </li></ul><ul><li>The Bayesian classifier serves to distinguish amongst the six Bayesian classes: </li></ul><ul><li> A E I O U S </li></ul><ul><li>For each of the six Bayesian classes, we collected 48 handwritten samples from volunteer writers. </li></ul>
  13. 13. DATA PREPARATION AND THE PRE-PROCESSING STAGE (CONTD) <ul><li>Our volunteer writers were male and female students of the Obafemi Awolowo University, Ile-Ife, Osun state Nigeria whose ages lied essentially between sixteen and thirty five years. </li></ul><ul><li>Volunteers wrote on plain white A4 sheets. </li></ul><ul><li>Each volunteer provided a single sample of each of the seventeen characters </li></ul>
  14. 14. DATA PREPARATION AND THE PRE-PROCESSING STAGE (CONTD) <ul><li>The A4 sheets were scanned, and with the aid of Paint graphics program, we cropped 40 samples of each of the Bayesian classes. </li></ul><ul><li>Of these, we used 40 samples per Bayesian class to train the Bayesian classifier. </li></ul><ul><li>We reserved eight samples from each of the seventeen classes for system evaluation. </li></ul>
  15. 15. SAMPLES CROPPED SAMPLES CROPPED SAMPLES DIACRITIC REMOVE
  16. 16. DATA PREPARATION AND THE PRE-PROCESSING STAGE (CONTD) <ul><li>In the Matlab environment, we pre-processed each of the samples that were used to train the Bayesian classifier using the following simple steps: </li></ul><ul><ul><li>conversion of color character image to binary form; removal of “salt and pepper” noise; computation of bounding box; and normalization of resulting image to fit a grid. </li></ul></ul>
  17. 17. CHARACTER DATA PROCESSING Cropped character Character after Binarisation Character after Noise removal Character and its bounding
  18. 18. THE SEGMENTATION STAGE <ul><li>The segmentation stage attempts to isolate the roman letter from the diacritical marks. </li></ul><ul><li>It is able to handle both connected-at-core characters and broken-at-core characters. </li></ul><ul><li>The roman letter of connected-at-core Yoruba characters do not contain breakages; the roman letter of broken-at-core Yoruba characters do. </li></ul>
  19. 19. A Connected-at-core Yoruba character. An under dot (tail) in the lower region of a Yoruba character. The lower region ( LR ) of a Yoruba character A grave mark in the upper Region of a Yoruba character. (Goes to the decision tree). Yoruba character. The upper region ( UR ) of a Yoruba character . The middle region ( MR )of a Yoruba character character. The roman letter of a Yoruba character.
  20. 20. A BROKEN-AT-CORE YORUBA CHARACTER A tail in the LR of a Yoruba character. The largest object in a Yoruba character. The character’s RCLO The character’s LR. ( This LR corresponds to the RBLO) The RBLO of a Yoruba character. ( This RBLO corresponds to the LR) A diacritical mark in the UR of a Yoruba character. A pseudo- diacritic in the RALO character. A breakage in the roman letter of a Yoruba character The RALO of a Yoruba character .( This RALO does not correspond to the UR) The UR of a Yoruba character .
  21. 21. SOME TREMINOLOGIES USED IN FIGURE <ul><li>UR: Upper Region </li></ul><ul><li>MR: Middle Region </li></ul><ul><li>LR: Lower Region </li></ul><ul><li>RALO: Region Above Largest Object </li></ul><ul><li>RCLO: Region Containing Largest Object. </li></ul><ul><li>RBLO: Region Below Largest Object. </li></ul>
  22. 22. THE SEGMENTATION ALGORITHM <ul><li>Identify the largest object in the image by finding the object with the maximum area. </li></ul><ul><li>2) Do a tentative segmentation of the image into regions: the region above the largest object (RALO); the region containing the largest object (RCLO); and the region below the largest object (RBLO). </li></ul><ul><li>3) If RALO contains two objects, release the lower one into RCLO. </li></ul><ul><li>4) If RALO contains a single object, calculate the absolute value of </li></ul><ul><li>the object’s slope. </li></ul>
  23. 23. THE SEGMENTATION ALGORITHM (CONTD) <ul><li>If the slope in step 4 above is less than 0.35, </li></ul><ul><li>release this single object into RCLO . </li></ul><ul><li>If RBLO contains two objects, release the upper one </li></ul><ul><li>into RCLO. </li></ul><ul><li>If RBLO contains a single object, calculate the </li></ul><ul><li>absolute value of the object’s slope. </li></ul><ul><li>If the slope in step 7 above is less than 2.0, release </li></ul><ul><li>this single object into RCLO . </li></ul>
  24. 24. THE DECISION TREE STAGE <ul><li>The segmentation stage isolates the roman letter from the diacritical marks. </li></ul><ul><li>The decision tree takes the diacritical marks and tries to recognize them. </li></ul>
  25. 25. CHARACTER DECISION TREE GWT Grave With Tail; AWT Acute With Tail GDT Grave Devoid of Tail; ADT Acute Devoid of Tail; MWT Macron With Tail MDT for macron devoid of tail. Number of Elements up 0 GDT Slope up MDT Slope up 0 Number of elements down 1 Number of elements down GWT ADT MWT +ve -ve 0 1 -ve +ve AWT
  26. 26. TERMINOLOGIES IN THE DECISION TREE <ul><li>GWT: Grave With Tail </li></ul><ul><li>AWT: Acute With Tail </li></ul><ul><li>GDT: Grave Devoid of Tail </li></ul><ul><li>ADT: Acute Devoid of Tail </li></ul><ul><li>MWT: Macron With Tail </li></ul><ul><li>MDT: Macron Devoid of tail </li></ul>
  27. 27. THE FEATURE EXTRACTION STAGE <ul><li>The segmentation isolates the roman letter of the Yoruba character from the diacritical marks. </li></ul><ul><li>The roman letter goes to a Bayesian classifier via a feature extraction stage. </li></ul><ul><li>The feature extraction stage extracts eight geometric features. </li></ul>
  28. 28. THE EIGHT GEOMETRIC FEATURES WE EMPLOYED <ul><li>Mean of character widths </li></ul><ul><li>Mean of character heights </li></ul><ul><li>Correlation of character heights with horizontal position. </li></ul><ul><li>Second order normalized central moment </li></ul>
  29. 29. THE EIGHT GEOMETRIC FEATURES WE EMPLOYED <ul><li>Second order normalized central moment </li></ul><ul><li>Haralick’s circularity </li></ul><ul><li>Third order normalized moment </li></ul><ul><li>Third order normalized moment </li></ul>
  30. 30. A DISPLAY OF THE FEATURES AND THEIR FORMULAE FEATURE NO. FEATURE AND EXPLANATION OF FORMULA FORMULA 1 Mean character width. stands for width at row . is the height of the character. 2 Mean character height. . stands for height at column . is the width of the character. 3 Correlation of height with horizontal distance. 4 Second order normalized central moment . is the raster coordinate of the row of character centroid. 5 Second order normalized central moment. is the raster coordinate of the column of character centroid. 6 Haralick’s circularity. is the distance from every point on the character to the centroid of character. is the mean of the s; is their standard deviation. 7 Third order normalized moment 8 Third order normalized moment
  31. 31. BAYESIAN DECISION THEORY (BDT) AND THE BAYESIAN CLASSIFIER STAGE. <ul><li>Bayesian Decision Theory is a fundamental tool in pattern recognition. </li></ul><ul><li>It has been applied with success in the character recognition domain (Husnain and Naweed; 2009). </li></ul><ul><li>Roughly speaking, a Bayesian classifier puts an object in the class it has the highest probability of belonging to. </li></ul>
  32. 32. BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>Here is Baye’s formula : </li></ul><ul><li>c is the number of Bayesian classes (six) </li></ul><ul><li>is the posterior probability </li></ul>
  33. 33. BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>is the class-conditional probability. We take it to be a Gaussian for handwritings (Duda et. al 2000). </li></ul><ul><li>is the prior probability </li></ul><ul><li>is called the evidence: it is the same across all classes. </li></ul>
  34. 34. BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>We can view Baye’s formula as a decision rule formula with a zero-one cost function (Duda et. al 2000). </li></ul><ul><li>Using the facts that decision rules are not affected by: addition of constants; multiplication by positive constants; and the effect of strictly increasing functions, such as the natural logarithm function, we can ultimately obtain a decision rule formula equivalent to Baye’s rule (Duda et. al 2000): </li></ul>
  35. 35. BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>The above equation is the form we employed in our Bayesian classifier. </li></ul><ul><li>is the extracted feature; it is 8-dimensional. </li></ul><ul><li>is the covariance of the i-th class, </li></ul><ul><li>being its determinant and its inverse. </li></ul>
  36. 36. BDT AND THE BAYESIAN CLASSIFIER STAGE (CONTD). <ul><li>is the mean feature vector of the i-th class. </li></ul><ul><li>The rule is to compute for all from 1 to and pick the class corresponding to the maximum. </li></ul><ul><li>The obtained result is fused with the result of the decision tree in a result fusion stage. </li></ul>
  37. 37. THE RESULT FUSION STAGE <ul><li>Result fusion is handled in our system by the following equation: </li></ul><ul><li>is the final class label produced by the system. It can take integer values between 1 and 17 </li></ul>
  38. 38. THE RESULT FUSION STAGE (CONTD) <ul><li>is the label from the Bayesian classifier. This is an integer value between 1 and 6. </li></ul><ul><li>tracks tonal marks. = 0 indicates macrons, = 1 indicates grave marks. </li></ul><ul><li>= 2 indicates acute marks. </li></ul><ul><li>tracks under dots = 0 indicates absence of under dot = 3 indicates </li></ul><ul><li>presence of under dot. </li></ul>
  39. 39. THE RESULT FUSION STAGE (CONTD) <ul><li>is a function of that “pads the equation up” to achieve the neat one-to-one correspondence: </li></ul>characters À Á È É Ẹ Ẹ̀ Ẹ́ Ì Í Ò Ó Ọ ̀Ọ Ọ́ Ş Ù Ú final_label 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
  40. 40. THE RESULT FUSION STAGE (CONTD) <ul><li>is defined in the table below: </li></ul>bdt_label f(bdt_label) 1 -1 2 0 3 4 4 5 5 9 6 9
  41. 41. RESULTS <ul><li>We tested the system in two folds. </li></ul><ul><li>In the first fold, with eight independent samples from each of the seventeen classes. </li></ul><ul><li>A recognition rate of 91.18% was obtained. </li></ul>
  42. 42. RESULTS (CONTD) <ul><li>In the second fold, we tested the system with 3 non-independent samples from each of the six Bayesian classes. </li></ul><ul><li>A recognition rate of 94.44% was obtained. </li></ul>
  43. 43. CONCLUSION AND FUTURE WORK <ul><li>In summary, we have developed a technique for the classification of diacritically marked uppercase Yorùbá handwritten charaters in offline mode. </li></ul><ul><li>Our system involves six stages: </li></ul><ul><ul><li>Pre-processing </li></ul></ul><ul><ul><li>Segmentation </li></ul></ul><ul><ul><li>Decision tree </li></ul></ul><ul><ul><li>Feature extraction </li></ul></ul><ul><ul><li>Bayesian classification </li></ul></ul><ul><ul><li>Result fusion </li></ul></ul>
  44. 44. Conclusion cont… <ul><li>We built the Bayesian stage using forty samples per Bayesian class. </li></ul><ul><li>We tested the system on independent and non-independent samples. </li></ul><ul><li>On independent samples, a recognition rate of 91.18% was obtained. </li></ul><ul><li>On non-independent samples, a recognition rate of 94.44 % was recorded. </li></ul>
  45. 45. ONGOING WORK <ul><li>We plan to extend this work in two directions: </li></ul><ul><ul><li>We wish to extend the system to cover the entire Yorùbá orthography, including the lower case letters and cursive scripts. </li></ul></ul><ul><ul><li>Deploy the system in Table environment </li></ul></ul>AGIS 2011 Ethiopia

×