Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Tesseract Tutorial: DAS 2014 Tours France
6. Character
Segmentation, Language
Models and Beam Search
The heart of Tesserac...
Tesseract Tutorial: DAS 2014 Tours France
Approaches to Segmentation
● Segment first using only geometry.
● Maximally chop...
Tesseract Tutorial: DAS 2014 Tours France
Over-Segmentation
● Aim is to maximize recall of chops with the compromise of
re...
Tesseract Tutorial: DAS 2014 Tours France
Segmentation Graph
● Segmentation possibilities and classifier results form a di...
Tesseract Tutorial: DAS 2014 Tours France
Searching the Segmentation Graph
l } I 1
m
u nh b
l } I 1 i l 1 e c r T
h b
i l ...
Tesseract Tutorial: DAS 2014 Tours France
Searching the Segmentation Graph
l } I 1
m
u nh b
l } I 1 i l 1 e c r T
h b
i l ...
Tesseract Tutorial: DAS 2014 Tours France
Integration of Language Models (General Methods)
● Implement Language Model as F...
Tesseract Tutorial: DAS 2014 Tours France
Segmentation Free = Extreme Over-Segmentation
● Slide over the word/textline wit...
Tesseract Tutorial: DAS 2014 Tours France
Tesseract Segmentation Approach based on observations:
● Initial segmentation is...
Tesseract Tutorial: DAS 2014 Tours France
Tesseract Segmentation Approach
Classify Initial Segmentation
Search Word: OK? Y...
Tesseract Tutorial: DAS 2014 Tours France
Types of Pain Point
● Initial: Join each adjacent pair
● Ambiguity: Eg m/rn
● Pa...
Tesseract Tutorial: DAS 2014 Tours France
Ratings MATRIX = Segmentation Graph
l } I 1
m
u nh b
l } I 1 i l 1 e c r T
h b
i...
Tesseract Tutorial: DAS 2014 Tours France
Live Demo of Segmentation Search
Chopper demo
api/tesseract unlv/mag.3B/2/8022_0...
Tesseract Tutorial: DAS 2014 Tours France
Evaluation of a WORD_CHOICE (no params-model)
Word Rating = word_factor ∑blob_ch...
Tesseract Tutorial: DAS 2014 Tours France
Evaluation of a WORD_CHOICE (with params-model)
Word Rating = word_factor ∑outli...
Tesseract Tutorial: DAS 2014 Tours France
Tesseract Word Recognizer
Tesseract Tutorial: DAS 2014 Tours France
Example of Chopping (unlv/mag.3B/2/8022_028.3B.tif
Col 2, line 6, word 1)
Word D...
Tesseract Tutorial: DAS 2014 Tours France
Example of Combining (unlv/doe3.3B/4/2214_007.3B.tif,
col 2, line 8, word 2)
Wor...
Tesseract Tutorial: DAS 2014 Tours France
Thanks for Listening!
Questions?
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
5 character classifiers
Next
Download to read offline and view in fullscreen.

0

Share

Download to read offline

6 char segmentation

Download to read offline

6 char segmentation

  • Be the first to like this

6 char segmentation

  1. 1. Tesseract Tutorial: DAS 2014 Tours France 6. Character Segmentation, Language Models and Beam Search The heart of Tesseract Ray Smith, Google Inc.
  2. 2. Tesseract Tutorial: DAS 2014 Tours France Approaches to Segmentation ● Segment first using only geometry. ● Maximally chop, then combine with a beam search. (Over-segmentation.) ● Sliding window to "avoid" segmentation altogether. ● Tesseract: Chop only as needed, then combine as needed.
  3. 3. Tesseract Tutorial: DAS 2014 Tours France Over-Segmentation ● Aim is to maximize recall of chops with the compromise of reduction in precision.
  4. 4. Tesseract Tutorial: DAS 2014 Tours France Segmentation Graph ● Segmentation possibilities and classifier results form a directed graph l } I 1 m u nh b l } I 1 i l 1 e c r T h b i l 1 i l 1 i l 1 l } I 1 m m g 9 q n u %
  5. 5. Tesseract Tutorial: DAS 2014 Tours France Searching the Segmentation Graph l } I 1 m u nh b l } I 1 i l 1 e c r T h b i l 1 i l 1 i l 1 l } I 1 m m g 9 q n u higher %
  6. 6. Tesseract Tutorial: DAS 2014 Tours France Searching the Segmentation Graph l } I 1 m u nh b l } I 1 i l 1 e c r T h b i l 1 i l 1 i l 1 l } I 1 m m g 9 q n u l } I 1 m u nh b l } I 1 i l 1 e c r T h b i l 1 i l 1 i l 1 l } I 1 m m g 9 q n u higher }uglier % %
  7. 7. Tesseract Tutorial: DAS 2014 Tours France Integration of Language Models (General Methods) ● Implement Language Model as Finite State Machine. ● Search Language Model and Segmentation Graph in parallel. ● Combine “probabilities” in some sensible way. ● Hidden Markov Model methods are good example.
  8. 8. Tesseract Tutorial: DAS 2014 Tours France Segmentation Free = Extreme Over-Segmentation ● Slide over the word/textline with a classifier/HMM. ● Beam search + shape model probs + language model probs solves the segmentation internally. ● Really just mega-over-segmentation.
  9. 9. Tesseract Tutorial: DAS 2014 Tours France Tesseract Segmentation Approach based on observations: ● Initial segmentation is often correct or close. ● Classifier generally doesn’t like incorrectly segmented text. ● Over-segmentation often leads to poor results., eg m->iii
  10. 10. Tesseract Tutorial: DAS 2014 Tours France Tesseract Segmentation Approach Classify Initial Segmentation Search Word: OK? Yes => Done while any Bad Blob has any Chops available Chop and classify pieces of Worst Choppable Blob Search Word: OK? Yes => Done while any fixable “Pain point” Associate adjacent blobs and classify Search Word: OK? Yes => Done
  11. 11. Tesseract Tutorial: DAS 2014 Tours France Types of Pain Point ● Initial: Join each adjacent pair ● Ambiguity: Eg m/rn ● Path: Neighbors of blobs in the current best path
  12. 12. Tesseract Tutorial: DAS 2014 Tours France Ratings MATRIX = Segmentation Graph l } I 1 m u nh b l } I 1 i l 1 e c r T h b i l 1 i l 1 i l 1 l } I 1 m m g 9 q n u % l}Ii hb il1 un il1 m nu il1 g9q l}I1 % l}I1 hb il1 m ec m rT Each entry holds a BLOB_CHOICE_LIST providing classifier choices with rating and certainty.
  13. 13. Tesseract Tutorial: DAS 2014 Tours France Live Demo of Segmentation Search Chopper demo api/tesseract unlv/mag.3B/2/8022_028.3B.tif test1 inter segdemo chopdebug3 Col 2, line 6, word 1 Associator demo api/tesseract unlv/doe3.3B/4/2214_007.3B.tif test1 inter segdemo Col 2, line 8, word 2
  14. 14. Tesseract Tutorial: DAS 2014 Tours France Evaluation of a WORD_CHOICE (no params-model) Word Rating = word_factor ∑blob_choice->rating() word_factor = segmentation Condition base word_factor Add-ons Frequent dawg word 1.0 Inconsistent case +0.1 Other dawg word 1.1 Inconsistent case +0.1 Non-dawg word 1.25 +0.01 for each char over 3. Inconsistent case +0.1 Inconsistent punc +0.2 Inconsistent chartype +0.3 Inconsistent script +0.5 Inconsisted char spacing +0.01 All except script +0.01 for each additional occurrence.
  15. 15. Tesseract Tutorial: DAS 2014 Tours France Evaluation of a WORD_CHOICE (with params-model) Word Rating = word_factor ∑outline length word_factor = weighted sum of word features: ● mean blob rating ● num inconsistent spaces ● num inconsistent char type ● num x-height inconsistencies ● num case inconsistencies ● word length (in type categories)
  16. 16. Tesseract Tutorial: DAS 2014 Tours France Tesseract Word Recognizer
  17. 17. Tesseract Tutorial: DAS 2014 Tours France Example of Chopping (unlv/mag.3B/2/8022_028.3B.tif Col 2, line 6, word 1) Word Distance Worst blob Momm 212.2 7.7 Mommn 186.3 8.3 Momtfln178.0 9.2 Momtain 124.9 5.3 Mounm 184.0 7.7 Mountain 80.6 3.1 ACCEPT!
  18. 18. Tesseract Tutorial: DAS 2014 Tours France Example of Combining (unlv/doe3.3B/4/2214_007.3B.tif, col 2, line 8, word 2) Word Distance lilllit- 77.58 limit57.1 Emit 89.7 Unfit 95.4 Hulk 122.7 Bulk 136.8 Word Distance HUM 120.6 MUM 127.0 BUM 134.7 1mm 112.8 Milk147.2 huh 137.1 fink 140.7 Emu 129.7 BMW 140.3
  19. 19. Tesseract Tutorial: DAS 2014 Tours France Thanks for Listening! Questions?

6 char segmentation

Views

Total views

526

On Slideshare

0

From embeds

0

Number of embeds

5

Actions

Downloads

20

Shares

0

Comments

0

Likes

0

×