Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Assessing product image quality for
online shopping
Anjan Goswami, Sung H. Chung,
Naren Chittar and Atiq Islam
eBay Confid...
Outline
• Motivation
• Definition
• Features
• Experiments
• Image quality models.
• What’s next?
eBay Confidential
Motivation
eBay Confidential
Motivation
• Better Aesthetics.
• Better customer experience?
• Better conversion?
• Better branding?
• Better looking mer...
Applications
• Ranking.
• Improve browsing experience.
• Guidelines for sellers.
• Merchandising.
• Many more.
eBay Confid...
What is product image quality?
• We care mainly about product images.
• Product images have specific characteristics.
eBay...
Computing image quality
• Machine learning problem.
• Factors from images to construct feature
vectors.
• Label the data p...
What factors? How do we compute?
• Size factors such as area, aspect ratio.
• Image attributes such as brightness,
saturat...
Brightness
Average grayscale intensity of the image.
0.299*red+0.587*green+0.114*blue;
eBay Confidential
0.18
0.58
Colorfulness
• Difference between a color against gray.
• Many empirical notions.
• We use an empirical expression designe...
Example
Global FG
20 8
60 64
eBay Confidential
Saturation
• Many notions and empirical formula.
• Average of (max – min) in RGB space. (Used
mainly in photography)
eBay ...
Example of Saturation
Score
86
255
eBay Confidential
Dynamic range
• Variants of expressions in photography
literature and in computer vision.
• We are using a simpler definit...
Example of Dynamic Range
Score
32
100
eBay Confidential
Contrast
• Michelson contrast : Range /(max + min) in a
color space.
• RMS contrast : stdev of spatial intensities.
eBay C...
Example of Michelson Contrast
Score
25
99
eBay Confidential
Segmentation
• Grabcut with a heuristic for automated
segmentation.
eBay Confidential
Segmentation
eBay Confidential
Background and foreground area ratio
• Use segmented image.
• An approximation is used by using ratio of
pixels in the for...
Example
Score
12
83
eBay Confidential
background and foreground
differences.
• Brightness L1 distance.
• Contrast L2 distance.
eBay Confidential
Properties of background
• stdev of lightness (distance from white in
RGB.)
• Mean of lightness. (RGB)
• A score on unifor...
An example
Image Col
orf
uln
ess
RMS
Contras
t
Bright
ness
diff
Area
ratio
Stdev of
lightness
of
backgrou
nd
Uniformity Wh...
Data Collection
• Crowdsourcing (52K images)
• Professional judgments. (6K images)
eBay Confidential
Crowdsourcing
• Has its own challenges.
• Require thoughts for framing questions.
• Require thoughts for conducting the
ex...
Image Quality Classes
• Good
• Fair
• Poor
eBay Confidential
Professional Quality Images
• Mostly white, light or uniform background.
• Image is free from compression artifact such
as...
Professional Quality
eBay Confidential
Poor images
• Poor or dark background
• Can have incomprehensible texture.
• Subject small.
• Subject unclear.
• Bad aspec...
Poor Images
eBay Confidential
Fair images
• These are images that are not poor. However,
they are not as clean as professional looking
photos. (Add exam...
How do we develop the model
• Multi-class classification with two data
sources.
• Direct Regression with crowd-sourced dat...
Factor importance in Quality Classifier
(Classifier)
• Background lightness
• Brightness
• Aspect ratio
• Dynamic range
• ...
Error Rates
• Poor misclassification 10%
• Fair misclassification 50% [BAD]
• Good misclassification 7%
• However, our tra...
Quality Score for Classification
• The quality score is the expected average of
the class weights using the class probabil...
Quality Score
201
244
102
119
eBay Confidential
Quality Score
177
137
153
eBay Confidential
Quality Score
122
102
105
eBay Confidential
Factor Importance in Regression
• Background lightness
• Saturation
• Aspect ratio
• RMS contrast
• Colorfulness
eBay Conf...
Comparison of Models
eBay Confidential
Image Classifier Regression
98 86
100 57
231 235
133 96
What’s Next (Semantics)
• Object recognition/relevance.
• Text/watermark detection.
• Human model detection.
>
eBay Confid...
Q & A
Thanks!
Ideas & questions?
eBay Confidential
Upcoming SlideShare
Loading in …5
×

Assessing product image quality for online shopping

901 views

Published on

Assessing product-image quality is important in the context of online shopping. A high quality image that
conveys more information about a product can boost the buyer’s confidence and can get more attention.
However, the notion of image quality for product-images is not the same as that in other domains. The
perception of quality of product-images depends not only on various photographic quality features but also
on various high level features such as clarity of the foreground or goodness of the background etc. In this
paper, we define a notion of product-image quality based on various such features. We conduct a crowdsourced
experiment to collect user judgments on thousands of eBay’s images. We formulate a multi-class
classification problem for modeling image quality by classifying images into good, fair and poor quality based
on the guided perceptual notions from the judges. We also conduct experiments with regression using average
crowd-sourced human judgments as target. We compute a pseudo-regression score with expected average of
predicted classes and also compute a score from the regression technique. We design many experiments with
various sampling and voting schemes with crowd-sourced data and construct various experimental image
quality models. Most of our models have reasonable accuracies (greater or equal to 70%) on test data set.
We observe that our computed image quality score has a high (0.66) rank correlation with average votes
from the crowd sourced human judgments.

Published in: Data & Analytics
  • Be the first to comment

Assessing product image quality for online shopping

  1. 1. Assessing product image quality for online shopping Anjan Goswami, Sung H. Chung, Naren Chittar and Atiq Islam eBay Confidential
  2. 2. Outline • Motivation • Definition • Features • Experiments • Image quality models. • What’s next? eBay Confidential
  3. 3. Motivation eBay Confidential
  4. 4. Motivation • Better Aesthetics. • Better customer experience? • Better conversion? • Better branding? • Better looking merchandising pages. eBay Confidential
  5. 5. Applications • Ranking. • Improve browsing experience. • Guidelines for sellers. • Merchandising. • Many more. eBay Confidential
  6. 6. What is product image quality? • We care mainly about product images. • Product images have specific characteristics. eBay Confidential
  7. 7. Computing image quality • Machine learning problem. • Factors from images to construct feature vectors. • Label the data points as one of the classes. • Build a classifier to get the class probabilities. • Alternatively, make a regression model from human judgment data. eBay Confidential
  8. 8. What factors? How do we compute? • Size factors such as area, aspect ratio. • Image attributes such as brightness, saturation, colorfulness, contrast, dynamic range. • Factors based on background and foreground segmentation. eBay Confidential
  9. 9. Brightness Average grayscale intensity of the image. 0.299*red+0.587*green+0.114*blue; eBay Confidential 0.18 0.58
  10. 10. Colorfulness • Difference between a color against gray. • Many empirical notions. • We use an empirical expression designed in Natural Color Space. (NCS) • This space has a concept of rg and yb coordinates. • stdev (rg,yb)+ 0.3 mean (rg,yb) • We compute this globally and for foreground. eBay Confidential
  11. 11. Example Global FG 20 8 60 64 eBay Confidential
  12. 12. Saturation • Many notions and empirical formula. • Average of (max – min) in RGB space. (Used mainly in photography) eBay Confidential
  13. 13. Example of Saturation Score 86 255 eBay Confidential
  14. 14. Dynamic range • Variants of expressions in photography literature and in computer vision. • We are using a simpler definition used in photography based on range of gray scale intensity. eBay Confidential
  15. 15. Example of Dynamic Range Score 32 100 eBay Confidential
  16. 16. Contrast • Michelson contrast : Range /(max + min) in a color space. • RMS contrast : stdev of spatial intensities. eBay Confidential
  17. 17. Example of Michelson Contrast Score 25 99 eBay Confidential
  18. 18. Segmentation • Grabcut with a heuristic for automated segmentation. eBay Confidential
  19. 19. Segmentation eBay Confidential
  20. 20. Background and foreground area ratio • Use segmented image. • An approximation is used by using ratio of pixels in the foreground and in the background. eBay Confidential
  21. 21. Example Score 12 83 eBay Confidential
  22. 22. background and foreground differences. • Brightness L1 distance. • Contrast L2 distance. eBay Confidential
  23. 23. Properties of background • stdev of lightness (distance from white in RGB.) • Mean of lightness. (RGB) • A score on uniformity of background intensity that approximates texture properties. eBay Confidential
  24. 24. An example Image Col orf uln ess RMS Contras t Bright ness diff Area ratio Stdev of lightness of backgrou nd Uniformity Whitene ss of backgrou nd Colorfulness of foreground 38. 85 0.30 142.3 3 0.75 4.22 2.63 100 35.76 44. 77 0.086 28.68 08 0.66 27.27 16.37 0 39.73 eBay Confidential
  25. 25. Data Collection • Crowdsourcing (52K images) • Professional judgments. (6K images) eBay Confidential
  26. 26. Crowdsourcing • Has its own challenges. • Require thoughts for framing questions. • Require thoughts for conducting the experiment. • Cheap labelers can attempt cheating. • Classifier result can be different based on voting techniques used to find the label. • More judgments are better. eBay Confidential
  27. 27. Image Quality Classes • Good • Fair • Poor eBay Confidential
  28. 28. Professional Quality Images • Mostly white, light or uniform background. • Image is free from compression artifact such as blurring. • Professionally photographed in proper lighting condition . • Subject has a reasonable size and is in focus. • Example of such images can be seen in branded retail websites. eBay Confidential
  29. 29. Professional Quality eBay Confidential
  30. 30. Poor images • Poor or dark background • Can have incomprehensible texture. • Subject small. • Subject unclear. • Bad aspect ratio. • Poor resolution and photography. eBay Confidential
  31. 31. Poor Images eBay Confidential
  32. 32. Fair images • These are images that are not poor. However, they are not as clean as professional looking photos. (Add examples) eBay Confidential
  33. 33. How do we develop the model • Multi-class classification with two data sources. • Direct Regression with crowd-sourced data. • Used Gradient boosted tree. eBay Confidential
  34. 34. Factor importance in Quality Classifier (Classifier) • Background lightness • Brightness • Aspect ratio • Dynamic range • Background foreground area ratio • Michelson contrast eBay Confidential
  35. 35. Error Rates • Poor misclassification 10% • Fair misclassification 50% [BAD] • Good misclassification 7% • However, our training data is so far not perfect. eBay Confidential
  36. 36. Quality Score for Classification • The quality score is the expected average of the class weights using the class probabilities. • Currently class weights are simple linear function that maps poor, fair and good to 1,2,3. eBay Confidential
  37. 37. Quality Score 201 244 102 119 eBay Confidential
  38. 38. Quality Score 177 137 153 eBay Confidential
  39. 39. Quality Score 122 102 105 eBay Confidential
  40. 40. Factor Importance in Regression • Background lightness • Saturation • Aspect ratio • RMS contrast • Colorfulness eBay Confidential
  41. 41. Comparison of Models eBay Confidential Image Classifier Regression 98 86 100 57 231 235 133 96
  42. 42. What’s Next (Semantics) • Object recognition/relevance. • Text/watermark detection. • Human model detection. > eBay Confidential
  43. 43. Q & A Thanks! Ideas & questions? eBay Confidential

×