More Related Content
Viewers also liked
Similar to Aka examples
Similar to Aka examples (20)
Recently uploaded
Recently uploaded (20)
Aka examples
- 1. AKA identifies unique coupons given different names in the SnipSnap coupon database using a combination of k-means clustering and "smoking gun" feature based rule inference. Github: https://github.com/snipsnap/aka-service/ Email: luke.otterblad@gmail.com
- 2. Step 1: Matches – same value, description text and activity dates
- 3. Matches – pairs are shown ,but many more than 2 items are matched into groups
- 4. More Examples…Different Barcodes – Same Coupon The above two were matched into a group. The coupon below was also in the same set of American Eagle but NOT put into the same group even though it has some similarity….
- 5. How does it work? • https://github.com/snipsnap/aka-service • run via the command line • $ python aka.py -db_pswd your_password store McDonald’s id face_value offer_details start_date expiriation_date 988767 Free With the purchase of an Egg McMuffin 2013-09-03 2013-10-31 989829 FREE Egg McMuffin with the purchase of an Egg McMuffin 2013-09-03 2013-10-31 997447 Free Egg McMuffin with the purchase of an Egg McMuffin 2013-09-03 2013-10-31
- 6. Active Coupons for a Store as a Graph • When the aka-service is started, for a particular store each active coupon is converted to dictionary format and face value and details based features are converted to the python version of a graph and normalized with some language processing. • Item - > Features {"CouponA": {[‘free’, ‘with’, ‘the’, ‘purchase’, ‘of’, ‘an’, ‘egg’, ‘mcmuffin’] {"CouponB": {[‘free’, ‘with’, ‘the’, ‘purchase’, ‘of’, ‘an’, ‘egg’, ‘mcmuffin’] • Features -> Item {“egg":["CouponA","CouponB"], “mcmuffin": ["CouponA","CouponB"], “free": ["CouponA","CouponB"], “with":["CouponA","CouponB"], “the": ["CouponA","CouponB"]} "purchase": ["CouponA","CouponB"] “of": ["CouponA","CouponB"]} “an": ["CouponA","CouponB"]}
- 7. Despite different text AKA identifies all of these as the same item id face_value 988767 Free 989829 FREE Egg McMuffin 997447 Free Egg McMuffin offer_details With the purchase of an Egg McMuffin with the purchase of an Egg McMuffin with the purchase of an Egg McMuffin start_date 2013-09-03 2013-09-03 2013-09-03 expiriation_date aka_guid 2013-10-31 de5086f035bc-11e38da3005056c0000 8 2013-10-31 de5086f035bc-11e38da3005056c0000 8 2013-10-31 de5086f035bc-11e38da3005056c0000 8
- 8. Free is treated as a value keyword (along with % and $ descriptions)
- 9. But, words and value alone don’t create the match. Expiry date also matters
- 10. Coupon with No Barcode connected to the same offer with a barcode Same offer value (free mini candle) and same data range (September 9October 6, 2013)
- 11. Matching picture and computer images
- 12. A change in degree…but the same coupon
- 13. Smoking Gun Features • A Smoking gun feature for a coupon is a piece of information that identifies it as being the same real world item as another coupon (with near certainty). • There are two sources of such identification in the database. The first is a barcode_id. Multiple coupons that have the same barcode_id are indeed the same physical coupon. The second is a promo_code. • Two coupons that have the same promo_code are the same coupon 95% + of the time. (Some stores like Dunkin Donuts don’t use unique codes…but more on that later)
- 14. More Matches Above two coupons are matched, and are also NOT matched with the below coupon despite having an extremely similar description and validity: The code in the upper right hand corner (9152 versus 9992 –the smoking gun) helps significantly in separating them into a different identification.
- 15. Two coupons Not matched, even though they have the same description and similar text (they are valid at different times)
- 16. Finding smoother images I experimented with using the number of recorded features as an indicator of picture quality – but that didn’t have much correlation. What did work was using the picture with the highest number of redemptions within an aka group
- 17. Better images
- 18. The Dollar Store $1 Off coupon problem – likely to be many of those These four were originally matched. But I had to introduce the notion of a confidence percentage. This is largely because AKA weights the value of an item more heavily than the details words describing the offer (for most stores they have few items that are the same price)
- 19. More equal prices, but with high confidence set
- 20. Trouble Spots: AKA identifies same offer due to assumed smoking gun, but while there is the same barcode there is a different expiry. Ignoring PLU for Dunkin Donuts (and other publishers that duplicate promocodes) and going with 99% confidence does the trick.
- 21. There’s Exceptions to every rule • Coupons are no different • In the settings.yaml (pictured above) you can define exceptions to global rules. • What pop_smoking_gun tells aka is that for Dunkin’ Donuts the global rules of promo_code and barcode_id does not apply– for Dunkin Donuts’ they don’t create PLU codes as unique to an offer.
- 22. Another example Ignoring PLU for Dunkin Donuts (and other publishers that duplicate promocodes) and going with 99% confidence does the trick.
- 23. But knowing the store “rules” also helps correct errors (if they stick to unique codes) Mechanical Turk expiry: 10/17/2012 Mechanical Turk expiry: 10/7/2012 http://c346897.r97.cf1.rackcdn.c http://c346897.r97.cf1.rackcdn.com/d32b578eom/cd0faf92-f85e-11e2-9f66fd2a-11e2-9be6-40406c9e1e47.jpg 40406c9e1e47.jpg Since Bed Bath & Beyond id’s and promocodes indicate the same item aka can reconcile the mistake
- 24. AKA- never misinterpret a store's coupon rules again ids sharable descrption_text Aka_guid 987120 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 987271 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 988484 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cdf139439 005056c00008 989519 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd9522 005056c00008 989774 1 save 20% on your entire purchase bath body works 990040 0 990943 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 992970 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 992998 0 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 994314 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 75926f4f-328f-11e3-a3cd005056c00008 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cdf139492 005056c00008 10 coupons all identified as the same item with some marked sharable and some not. Suppose a publisher had submitted coupon 990040 to not be shareable……
- 25. AKA- never misinterpret a store's coupon rules again sharabl descrption_text e ids Aka_guid aka_sharable 987120 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 987271 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 988484 1 save 20% on your entire purchase bath body works f139439 75926f4f-328f-11e3-a3cd005056c00008 0 989519 1 save 20% on your entire purchase bath body works 9522 75926f4f-328f-11e3-a3cd005056c00008 0 989774 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 990040 0 save 20% on your entire purchase bath body works f139492 75926f4f-328f-11e3-a3cd005056c00008 0 990943 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 992970 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 992998 0 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 994314 1 save 20% on your entire purchase bath body works 75926f4f-328f-11e3-a3cd005056c00008 0 An easy feature could be to treat a single not sharable within an aka group as a “presidential” vote and switch all to not sharable. This can also work for items tagged as manufacturer coupons. You’d basically only need 1 tag from Mechanical Turk (or a from classifier).
- 27. Kroger’s matches Kroger’s requires the highest confidence of any store, as many of their coupons are different only by a single word. These will match (incorrectly) without a high confidence set. Listed below is a sample false match made by AKA:
- 28. Same item in the database twice for Macy’s http://c346897.r97.cf1.rackcdn.com/59667340-1588-11e3-a8e340406c9e1e47-thumb.jpg http://c346897.r97.cf1.rackcdn.com/ac4dc266-1588-11e3-a7d040406c9e1e47-thumb.jpg
- 30. Rougher Image Connected with a better version at McDonalds
- 31. Does a big mac by any other name, still taste like a big mac?
- 32. Digital and print match
- 33. More Matches
- 35. Better coupon picture identification
- 36. Occasional data entry errors can lead to bad reconciliation aka_guid id barcode_id alt_barcode face_ _id value offer_details $5.00 Off Save $5.00 On Your Purchase 0 $25.0 Of $25.00 Or More 0 2719bf74-40b611e3-86dd22000a91806d 421909 138859 2719bf74-40b611e3-86dd22000a91806d 539197 46299 0 Save On Any Aveeno Product $1.00 2719bf74-40b611e3-86dd22000a91806d 560927 138859 0 Save On any $1.00 2719bf74-40b611e3-86dd22000a91806d 595323 138859 0 20% Off 1 Regular Priced Item Here the 99% reliable barcode_id is idenified with 3 different items (for Toys R Us)
- 37. These three items were matched via barcode which I can only assume is some type of data entry error. The difference is that for every other toys”r”us coupon the smoking gun rules are valid. These items barcodes are recorded incorrectly
- 38. But it is an isolated error
- 39. Background for entity resolution (aka collective reconciliation, de-duping) • Chapter 20 of Beautiful Data “Connecting Data” by Toby Segaran (who I think likely wrote the chapter while working on the YouTube reconciliation). • Indrajit Bhattacharya’s PhD dissertation, which you can find at: http://www.lib.umd.edu/drum/handle/1903/4241 • About me: Father of 2 lovely daughters with my wife Emma. Programmer, Statistician, Pot Limit Omaha and Mixed Game poker semi-professional (though I don’t get much time for poker nowadays). I'm located in historic Northfield, MN where I share an office with my Jack Russell Terrier, Kirby. • Email: luke.otterblad@gmail.com.
- 40. Questions
Editor's Notes
- http://c346897.r97.cf1.rackcdn.com/980ac2e4-1494-11e3-998b-40406c9e1e47-thumb.jpg