Your SlideShare is downloading. ×
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity

110

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
110
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels that Maximizes Specificity Vladimir Kulyukin Department of Computer Science Utah State University Logan, UT, USA Christopher Blay YouTube, Inc Palo Alto, CA, USA vkedco.blogspot.com
  • 2. Introduction ● Many nutritionists consider proactive nutrition management to be a key factor in reducing and controlling cancer and diabetes ● According to the U.S. Department of Agriculture, U.S. residents have increased their caloric intake by 523 calories per day since 1970 ● Enabling consumers to use computer vision on smartphones to extract nutritional information from nutrition labels (NLs) will likely result in improved nutritional decisions vkedco.blogspot.com
  • 3. Outline ● Background ● Skewed NL Localization Algorithm ● Experiments & Results vkedco.blogspot.com
  • 4. Background vkedco.blogspot.com
  • 5. Relaxation of Alignment Constraints ● In our previous work (Kulyukin et al., IPCV 2013), we developed a vision-based algorithm for horizontally or vertically aligned NLs on smartphones (pdf) ● This algorithm improves our previous algorithm in that it handles not only aligned NLs but also the NLs that are skewed up to 35-40 degrees from the vertical axis of the captured frame ● This algorithm is designed to improve specificity, i.e., percentage of true negative matches out of all possible negative matches vkedco.blogspot.com
  • 6. Nutritional Data Analysis Automation ● Modern nutrition management system designers and developers assume that users understand how to collect nutritional data and can be triggered into data collection with digital prompts ● Many users find it difficult to integrate nutritional data collection into their daily activities due to lack of time, motivation, or training ● The current algorithm is a step in the direction of automating nutritional data collection and analysis vkedco.blogspot.com
  • 7. Why Localize NLs? ● Because localized NLs are easier to textchunk ● Text chunks tend to OCR better than complete NLs (Kulyukin, Vanka, and Wang, 2013) vkedco.blogspot.com
  • 8. Skewed NL Localization vkedco.blogspot.com
  • 9. Detection of Edges, Lines, Corners ● The algorithm uses three image processing methods: edge detection, line detection, and corner detection ● The algorithm uses the Canny edge detector (CED) to detect edges ● After the edges are detected, the Hough Transform (HT) is applied to detect lines ● Corner detection is done for text spotting because image segments with higher concentrations of corners are likely to contain text vkedco.blogspot.com
  • 10. Detection of Edges & Lines Edge Detection Line Detection vkedco.blogspot.com
  • 11. Rotation Correction ● NLs contain higher numbers of lines with the same skew angle ● All detected lines horizontal within 35 to 40 degrees in either direction (up or down) are used to compute the average skew angle ● After the average skew angle is computed, the image is rotated to align it horizontally ● Corner detection is done after the image rotation vkedco.blogspot.com
  • 12. Rotation Correction ● NLs contain higher numbers of lines with the same skew angle ● All detected lines horizontal within 35 to 40 degrees in either direction (up or down) are used to compute the average skew angle ● After the average skew angle is computed, the image is rotated to align it horizontally ● Corner detection is done after the image rotation vkedco.blogspot.com
  • 13. Corner Projections ● Horizontal & vertical projections of corner pixels are computed ● These projections determine the top, bottom, left, and right boundaries of the region in which most corners lie ● Projection values are averaged and a projection threshold is arbitrarily set to twice the average ● The first and last indexes of each projection greater than a threshold are selected as boundaries vkedco.blogspot.com
  • 14. Detection of Corners vkedco.blogspot.com
  • 15. Boundary Selection from Corner Projections vkedco.blogspot.com
  • 16. Experiments & Results online video vkedco.blogspot.com
  • 17. Experimental Design ● 378 images were assembled from a Google Nexus 7 Android 4.3 smartphone during a shopping session in a local supermarket ● Of these, 266 contained an NL and 112 did not ● Results were manually categorized into five categories: complete true positives, partial true positives, true negatives, false positives, and false negatives vkedco.blogspot.com
  • 18. Complete & Partial True Positives Complete (left) vs Partial (right) True Positives vkedco.blogspot.com
  • 19. NL Localization Results PR TR CR PR SP ACC 0.76 0.42 0.36 0.15 1.0 0.59 ● PR – precision ● TR – total recall ● CR – complete recall ● PR – partial recall ● SP – specificity ● ACC - accuracy vkedco.blogspot.com
  • 20. NL Localization Results ● Most false negative matches were caused by blurry images ● Bottles, bags, cans, and jars have a large showing in the false negative category due to HT line detection difficulties ● NLs with irregular layouts were also difficult to detect vkedco.blogspot.com
  • 21. NL with Curved Lines & Irregular Layouts NL with Curved Lines (left); NLs with Irregular Layouts (Middle & Right) vkedco.blogspot.com

×