Python & PerlPIL Basics by Example: OCR & Zone MatchingVladimir Kulyukinwww.youtube.com/vkedcowww.youtube.com/vkedco www.v...
Outline● OCR Background– Application Domains– Approaches● Zone Vector Matching in Pythonwww.youtube.com/vkedcowww.youtube....
OCR Backgroundwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
What is OCR● OCR stands for optical characterrecognition● OCR is automated conversion of printed,typewritten, or handwritt...
OCR Application Domains● Electronic document conversion: manybusinesses use OCR for data entry● Conversion of handwritten ...
OCR Approaches● Character Recognition: segment images intoblobs (glyphs) and recognize blobs as charactersone at a time● W...
OCR Techniques● Deskewing – realignment of skewed textsegments● Binarization – conversion of color or grayscale images int...
Zone Matchingwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
Zone Matching● Zone matching is a technique used in computer visionfor image matching● An image is divided into several su...
Example: Zone Stats & Feature Vectorswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogsp...
Example: 16 5x5 zones of 20x20 Imagewww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspo...
Example: Cosine Vector Similaritywww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.c...
Zone Vector Matching in OCR: Steps● Create a library of character images● For each character image, compute its zonefeatur...
Sample Character Imageswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comArchives...
Zone Vector Matching in Pythonsource code is herewww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.v...
Zone Specificationswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com## Each imag...
Counting Pixels in a Zonewww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef zo...
Building Zone Vectorswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef zone_v...
Cosine Similarity Functionwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef c...
Building a Zone Vector Mapwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef b...
Finding Best Matchwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef find_best...
Reading & References● http://en.wikipedia.org/wiki/Optical_character_recognition● http://vkedco.blogspot.com/2012/03/explo...
Upcoming SlideShare
Loading in …5
×

Python & Perl: PIL Basics: OCR & Zone Vector Matching

1,217 views
1,135 views

Published on

Python & Perl: PIL Basics: OCR & Zone Vector Matching

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,217
On SlideShare
0
From Embeds
0
Number of Embeds
527
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Python & Perl: PIL Basics: OCR & Zone Vector Matching

  1. 1. Python & PerlPIL Basics by Example: OCR & Zone MatchingVladimir Kulyukinwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  2. 2. Outline● OCR Background– Application Domains– Approaches● Zone Vector Matching in Pythonwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  3. 3. OCR Backgroundwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  4. 4. What is OCR● OCR stands for optical characterrecognition● OCR is automated conversion of printed,typewritten, or handwritten text into digitalformat● OCR is an intersection of three areas:pattern recognition, AI, and computervisionwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  5. 5. OCR Application Domains● Electronic document conversion: manybusinesses use OCR for data entry● Conversion of handwritten documents intoelectronic format● Conversion of images with text intosearchable documents● Translation of street signs on smart phoneswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  6. 6. OCR Approaches● Character Recognition: segment images intoblobs (glyphs) and recognize blobs as charactersone at a time● Word Recognition: segment images into blobsand recognize each blob as a word● Intelligent Recognition: use patterns and AItechniques (e.g., artificial neural networks) torecognize hard text (e.g., handwritten)www.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  7. 7. OCR Techniques● Deskewing – realignment of skewed textsegments● Binarization – conversion of color or grayscale images into black and white images● Line and word detection – detection oftext lines and segmentation of detectedlines into word blobs● Layout analysis – detection of columns,paragraphs, headings, etc.www.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  8. 8. Zone Matchingwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  9. 9. Zone Matching● Zone matching is a technique used in computer visionfor image matching● An image is divided into several sub-images, calledzones● For each zone, a specific statistic or a set of statistics iscomputed (e.g., number of horizontal lines, number ofpixels of a specific color, etc.)● Those statistics are placed into feature vectors and thefeature vectors are matched against each otherwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  10. 10. Example: Zone Stats & Feature Vectorswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com1) Image is divided into four zones (1, 2, 3, 4 movingclockwise)2) Statistic is the number of black pixels3) Result feature vector is [2, 3, 5, 4]
  11. 11. Example: 16 5x5 zones of 20x20 Imagewww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  12. 12. Example: Cosine Vector Similaritywww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com1) A and B are feature vectors2) Aiand Biare i-th elements of A and B
  13. 13. Zone Vector Matching in OCR: Steps● Create a library of character images● For each character image, compute its zonefeature vector and save it in a table of featurevectors● Given an input character image, compute itszone feature vector and match it with all featurevectors saved in the table of feature vectors● Return top N characters whose feature vectorsare closest to the feature vector of the inputcharacterwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  14. 14. Sample Character Imageswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comArchives with character images are here
  15. 15. Zone Vector Matching in Pythonsource code is herewww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com
  16. 16. Zone Specificationswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com## Each image is assumed to be a 20x20 bitmap divided into## 16 5 x 5 zones.## Each zone is specified as a 4-tuple: the first 2 integers are## the column and row of the zones top left corner; the## second two integers are the column and row coordinates## of the zones bottom right cornerzone_coords = ((0, 0, 4, 4), (5, 0, 9, 4), (10, 0, 14, 4), (15, 0, 19, 4),(0, 5, 4, 9), (5, 5, 9, 9), (10, 5, 14, 9), (15, 5, 19, 9),(0, 10, 4, 14), (5, 10, 9, 14), (10, 10, 14, 14), (15, 10, 19, 14),(0, 15, 4, 19), (5, 15, 9, 19), (10, 15, 14, 19), (15, 15, 19, 19))
  17. 17. Counting Pixels in a Zonewww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef zone_color_count(img, xyzw, pix_color=(0, 0, 0)):zone_color_count(img, xyzw, color=(0, 0, 0).- img is an RGB bitmap- xyzw is a four tuple that specifies the zone in img- pix_color specifies the color of the pixels that are countedin the zonecount = 0top_x, top_y, bottom_x, bottom_y = xyzwfor x in xrange(top_x, bottom_x+1):for y in xrange(top_y, bottom_y+1):if img.getpixel((x,y)) == pix_color:count += 1return count
  18. 18. Building Zone Vectorswww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef zone_vector(imgpath, zone_coords, pix_color=(0, 0, 0)):zone_vector(impgpath, zone_specs) -> list of zone counts.- imgpath is a path to an 20 x 20 bitmap with a single character.- zone_specs is a list of 4-tuples. Each 4-tuple specifiesthe top left and bottom right coordinates of a 5x5 zone.- zone_vector returns the list of black pixel counts in eachof the 16 zones.vector = []img = Image.open(imgpath)for zc in zone_coords:vector.append(zone_color_count(img, zc, pix_color))del imgreturn vector
  19. 19. Cosine Similarity Functionwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef cosine_similarity(zv1, zv2):cosine_similarity(zv1, zv2) -> floatthis function computes the cosine similarity between twovectors zv1 and zv2.dotp = sum(map(lambda x, y: x * y, zv1, zv2))zmagn1 = math.sqrt(sum([v1*v1 for v1 in zv1]))zmagn2 = math.sqrt(sum([v2*v2 for v2 in zv2]))return float(dotp)/(zmagn1 * zmagn2)
  20. 20. Building a Zone Vector Mapwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef build_zone_vector_map(dir, zone_coords):build_zone_vector_map(dir, zone_coords) -> dictionary- dir is a directory, zone_coords is a sequence of zone coords (4-tuples).- dir is assumped to store 20x20 bitmaps of 26 ASCII characters:A.bmp, B.bmp, C.bmp, etc.- build_zone_vector_map returns a dictionary that maps characters to theirzone vectors.dict = {}for letter in string.ascii_uppercase:dict[letter] = zone_vector(dir + letter + .bmp, zone_coords)return dict
  21. 21. Finding Best Matchwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.comdef find_best_zv_char_match(char_zv, zv_map, similarity=cosine_similarity):best_zv_match = 0.0best_char = Nonefor char, zone_vector in zv_map.items():curr_match = similarity(char_zv, zone_vector)if curr_match > best_zv_match:best_zv_match = curr_matchbest_char = charreturn (best_char, best_zv_match)
  22. 22. Reading & References● http://en.wikipedia.org/wiki/Optical_character_recognition● http://vkedco.blogspot.com/2012/03/exploring-basics-of-zone-vector.htmlwww.youtube.com/vkedcowww.youtube.com/vkedco www.vkedco.blogspot.comwww.vkedco.blogspot.com

×