2. “Lively Alejandra G peep-toe sandals cut
from neon-striped, snake-embossed leather.
An exposed zip secures the wide ankle cuff,
and stripes accent the leather sole. Covered
heel.”
For each shoe I scraped 2 images and a short text description
2200 pairs of shoes from shopbop.com
3. Hierarchical layers of non linear operations using wavelets at different scales
and orientations
Features
Image
Text
NLP: Tokenize, remove stop words, count word and bigram frequencies for each
shoe description and down weight words occurring in many shoe descriptions.
Hi my name is Amelia White and I am going to tell you about my app, shoely that aims to improve your online shopping experience by learning what you like. Online shopping can be very convenient if you know exactly what you are looking for but very frustrating and irritating if you want to browse items.
To develop shoely I scraped approximately 2000 images and text descriptions of shoes from the web site shopbop.com
And to earn a users preference I measure similarity between shoes by extracting image and text features.
I get text features by calculating word and bigram frequencies in the shoe descriptions, after removing stop words and down weighting very frequent words
I get image features by using hierarchical layers of non linear operations using wavelets at different scales and orientations
****
Show app here
I can then measure the difference between shoes by simply taking the least squares distance between the feature vectors
Here I am showing the shoes closest to this shoe both by image and text
The image features tend to find very visually similar shoes as closest and the text features often find shoes from the same brand or made of the same material etc. Here the text features are finding other kate spade pumps as similar and the image features find similar shaped pumps in similar colors as closest
I can then cluster the features using K means in order to quickly bring up shoes similar to ones the user selects
To validate the clusters based on image features I looked for enrichment in shoe categories within each cluster, here the blue bars show that most clusters contain only a few categories compared to randomly permuted cluster labels
Thanks!
I did my phd at NYU and Rutgers, where I built an object recognition machine used in htp screening of the nematode c elegans.
