• Save
MobiCom on Android: Word Segmentation & Matching in Vision-Based Nutrition Information Extraction
Upcoming SlideShare
Loading in...5
×
 

MobiCom on Android: Word Segmentation & Matching in Vision-Based Nutrition Information Extraction

on

  • 1,420 views

 

Statistics

Views

Total Views
1,420
Views on SlideShare
503
Embed Views
917

Actions

Likes
0
Downloads
0
Comments
0

50 Embeds 917

http://vkedco.blogspot.com 397
http://www.vkedco.blogspot.com 167
http://vkedco.blogspot.in 52
http://reader.aol.com 28
http://www.vkedco.blogspot.in 23
http://vkedco.blogspot.de 22
http://vkedco.blogspot.ca 16
http://vkedco.blogspot.sg 15
http://vkedco.blogspot.fr 14
http://vkedco.blogspot.co.uk 11
http://vkedco.blogspot.mx 10
http://vkedco.blogspot.com.br 10
http://vkedco.blogspot.tw 10
http://vkedco.blogspot.co.at 9
http://www.vkedco.blogspot.ru 9
http://vkedco.blogspot.kr 9
http://vkedco.blogspot.co.il 9
http://vkedco.blogspot.it 8
http://vkedco.blogspot.ru 8
http://translate.googleusercontent.com 7
http://vkedco.blogspot.nl 6
http://www.vkedco.blogspot.co.uk 6
http://www.vkedco.blogspot.fr 6
http://vkedco.blogspot.pt 5
http://vkedco.blogspot.com.au 5
http://vkedco.blogspot.sk 5
http://vkedco.blogspot.com.es 4
http://vkedco.blogspot.com.ar 4
http://www.vkedco.blogspot.tw 4
http://vkedco.blogspot.gr 3
http://www.vkedco.blogspot.nl 3
http://vkedco.blogspot.be 3
http://vkedco.blogspot.co.nz 3
http://www.vkedco.blogspot.ca 3
http://www.vkedco.blogspot.ro 2
http://vkedco.blogspot.cz 2
http://www.vkedco.blogspot.kr 2
http://vkedco.blogspot.ch 2
http://vkedco.blogspot.jp 2
http://vkedco.blogspot.ro 2
http://www.vkedco.blogspot.de 2
http://vkedco.blogspot.ie 1
http://vkedco.blogspot.hk 1
http://www.vkedco.blogspot.mx 1
http://www.vkedco.blogspot.com.au 1
http://www.vkedco.blogspot.jp 1
http://www.vkedco.blogspot.co.il 1
http://www.vkedco.blogspot.gr 1
http://www.vkedco.blogspot.it 1
http://vkedco.blogspot.fi 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

MobiCom on Android: Word Segmentation & Matching in Vision-Based Nutrition Information Extraction MobiCom on Android: Word Segmentation & Matching in Vision-Based Nutrition Information Extraction Presentation Transcript

  • Word Segmentation & Matching in Vision-Based Nutrition Information Extraction Vladimir Kulyukinwww.youtube.com/vkedco www.vkedco.blogspot.com
  • Outline ● Character & Word Segmentation ● Midline & Baseline Detection ● Image Filtering with Average & Gaussian Filters ● Intra- & Inter-Word Gaps ● Word Blob Representation & Matchingwww.youtube.com/vkedco www.vkedco.blogspot.com
  • Back to the Big Picturewww.youtube.com/vkedco www.vkedco.blogspot.com
  • Character & Word Segmentation ● Character segmentation is the decomposition of an image of a sequence of characters into individual character symbols ● Word segmentation is the decomposition of an image of a sequence of words into word blobs ● Character segmentation is more generic than word segmentation because it potentially leads to more words recognized: applicable for unlimited or very large lexicons ● Word segmentation is less generic but simpler: applicable for limited lexiconswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Topline, Midline, Baseline, Beardlinewww.youtube.com/vkedco www.vkedco.blogspot.com
  • Horizontal Projection of Segmented Lines Red line is the horizontal projection of black pixelswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Midline & Baseline Detection Midline & Baseline are Detected by Detecting HP Peakswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Vertical Projection & Gaps 1) Let VP(I) be the vertical project of the middle zone 2) VP(I) = 0 for intra-word & inter-word gaps Question: How do we distinguish intra-word from inter- word?www.youtube.com/vkedco www.vkedco.blogspot.com
  • Filtering ● Frequency domain analysis decomposes an image into its frequency content ● Low frequency means that slow variation of image intensities ● High frequency means rapid variation of image intensities ● A filter is an operation that amplifies a certain band of frequencies and reduces other frequency bandswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Average Filter ● Replace each pixel by the average value of pixels around it ● Most common masks are 3 x 3 and 5 x 5 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1www.youtube.com/vkedco www.vkedco.blogspot.com
  • 5 x 5 Average Filterwww.youtube.com/vkedco www.vkedco.blogspot.com
  • Gaussian Filter ● Gaussian filters are designed to weigh each pixel by its distance from the center pixel ● A sample 3 x 3 Gaussian .00761 .036075 .10959 .21345 .2666 .21345 .10959 .03608 .00761www.youtube.com/vkedco www.vkedco.blogspot.com
  • 5 x 5 Gaussian Filterwww.youtube.com/vkedco www.vkedco.blogspot.com
  • Back to the Gaps Question 1) Let VP(I) be the vertical project of the middle zone 2) VP(I) = 0 for intra-word & inter-word gaps Question: How do we distinguish intra-word from inter- word?www.youtube.com/vkedco www.vkedco.blogspot.com
  • Back to the Gaps Question 1) Take a blurring filter (average 3 x 3) and blur the middle zone 2) Invert black and white pixels 3) Computer VP of black pixels 4) Threshold to determine inter-word gapswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Word Segmentation Algorithm ● Take a text chunk (assume that it is a line) ● Determine the middle and base lines ● Use a blur filter (average or Gaussian) to blur the middle zone (the zone b/w middle and base lines) ● Determine inter-word gaps and use them to segment word blobswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Word Blob Representation ● Word blobs can be represented as [R, I], where R is the height-to-width ratio and I are grayscale image pixels ● Each blob can be scaled down so that the height of each template image is X pixels ● Example: Blobs can be scaled down using bilinear interpolation, e.g., the output pixel in a scaled- down image is the weighted average of the neighboring 2 x 2 pixelswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Word Blob Template Matching ● Let a word blob obtained in an image be represented as Bw = [Rw, Iw] ● Let TLib be a template library that consists of pre-computed template vectors: {B1, B2, ..., Bn} ● To match a word blob against the TLib images is a 2-stage process: – Compare ratios – If ratios are comparable, compare imageswww.youtube.com/vkedco www.vkedco.blogspot.com
  • Template Librarywww.youtube.com/vkedco www.vkedco.blogspot.com
  • Can OCR Engines Be Used? ● Yes, they can ● But, be prepared to deal with recognition errors ● There are two ways of dealing with these errors: – spelling corrections – improving image qualitywww.youtube.com/vkedco www.vkedco.blogspot.com
  • Tesseract Experiments Image Text Tesseract Output Nutrition Facts Nutrition Facts Serving size ¾ cup (32g) Sewmg SIZE 3X4 um 132g) Servings Per Container about 13 Semngs Ferfinnmmevahnm I3 Calories 160 calories from fat 40 lialmtss 160 Caiones 1mm Fa1 4U Total Fat 4.5g 7% TMil Fill 5g 7% Monounsaturated Fat 1g Finmunsaturaled Far 1n vitamin A 0% . Vitamin C 0% lvrramin A 0% I Vrtannn l) 0% Amount Per Serving " Blutllt Cereal trawl Cereal with ½ cup Fat Free Milk unnrusmrq Bani Fntmellwww.youtube.com/vkedco www.vkedco.blogspot.com
  • References ● https://code.google.com/p/tesseract-ocr/www.youtube.com/vkedco www.vkedco.blogspot.com
  • Can OCR Engines Be Used? ● Yes, they can ● But, be prepared to deal with recognition errors ● There are two ways of dealing with these errors: – spelling corrections – improving image qualitywww.youtube.com/vkedco www.vkedco.blogspot.com