Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
What are the 100 best restaurants on Yelp
A difficult answer to a simple question
Travis Brooks
PM - Search and Data Scien...
Yelp’s Mission:
Connecting people with great
local businesses.
Yelp Stats:
As of Q2 2015
83M 3268%83M
What types of things are on Yelp?
But we eat three times a day
Question
What are the 100 best
restaurants on Yelp?
Problem
What are the 100 best
restaurants on Yelp?
Best?
Problem
What are the 100 best
restaurants on Yelp?
Restaurant?
Problem
What are the 100 best restaurants on Yelp?
Many words in the question are ill-defined
Best? For what? For whom?
Re...
Possible Answer
Possible Answer
But that’s cheating
Users give us a lot of information in their queries,
locations, times, etc
Search Experiments
Multiple...
How to start?
SELECT name, average_rating FROM business
WHERE
business_category = “restaurant”
ORDER BY average_rating DES...
SELECT name, average_rating FROM business
WHERE
business_category = “restaurant”
ORDER BY average_rating DESC LIMIT 100;
H...
Next Try?
SELECT name, average_rating FROM business WHERE
business_category = “restaurant”
AND
review_count > 50
ORDER BY ...
Popularity ~ Quality
Next Try?
SELECT name, average_rating + review_count as score FROM
business WHERE
business_category = “restaurant”
AND
rev...
Next Try?
score = average_rating + b * review_count
Choose b such that you are happy with the list!
What does b mean?
How ...
Next Try?
score = average_rating + b * review_count
Choose b such that you are happy with the list!
What does b mean?
How ...
Other Ideas
Wilson Score ~ estimate of confidence interval in the rating
Rank by the lower bound
Other Ideas
Smoothing - used by IMDB, among others
Add some number of “pseudoratings” at the global average rating.
Choosi...
Other Ideas
Dirichlet and more sophisticated modelling
2014 List - Smoothed - All time faves
2015 List - Dirichlet - Weight 2014 Reviews
Product Perspective
Analysis -> Insights -> Actionable Insights -> Impact
- Paraphrased from Ken Rudin
Goals?
Encourage exploration?
Probably have better ways to do that that are more targeted
Reward good restaurants?
I don’t...
Takeaways
Data driven products are subtle - even if the feature is trivial (a list)
Reasoning from first principles is gre...
Thanks
Thanks:
Dan Frank - Dirichlet work and 2015 List
Will Seltzer - Dirichlet work
Yelp Users - Ratings, Reviews and Ph...
@YelpEngineering
YelpEngineers
engineeringblog.yelp.com
github.com/yelpyelp.com/careers
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
What are the 100 best restaurants on Yelp? A difficult answer to a simple question
Upcoming SlideShare
Loading in …5
×

What are the 100 best restaurants on Yelp? A difficult answer to a simple question

1,768 views

Published on

This choosing the best restaurants in the country is an easy task? Think again. In this talk, I'll talk about the data and product thinking that went into assembling the list of the 100 best restaurants on Yelp.

Published in: Data & Analytics
  • ..............ACCESS that WEBSITE Over for All Ebooks ................ ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

What are the 100 best restaurants on Yelp? A difficult answer to a simple question

  1. 1. What are the 100 best restaurants on Yelp A difficult answer to a simple question Travis Brooks PM - Search and Data Science tbrooks@yelp @traviscb1998
  2. 2. Yelp’s Mission: Connecting people with great local businesses.
  3. 3. Yelp Stats: As of Q2 2015 83M 3268%83M
  4. 4. What types of things are on Yelp?
  5. 5. But we eat three times a day
  6. 6. Question What are the 100 best restaurants on Yelp?
  7. 7. Problem What are the 100 best restaurants on Yelp? Best?
  8. 8. Problem What are the 100 best restaurants on Yelp? Restaurant?
  9. 9. Problem What are the 100 best restaurants on Yelp? Many words in the question are ill-defined Best? For what? For whom? Restaurant? Coffee? Tacos? On Yelp? Tourists? Locals? We won’t answer all of these, but we can play with a few
  10. 10. Possible Answer
  11. 11. Possible Answer
  12. 12. But that’s cheating Users give us a lot of information in their queries, locations, times, etc Search Experiments Multiple Geographies I want to be able to talk about methodologies… ...so let’s do it from scratch
  13. 13. How to start? SELECT name, average_rating FROM business WHERE business_category = “restaurant” ORDER BY average_rating DESC LIMIT 100;
  14. 14. SELECT name, average_rating FROM business WHERE business_category = “restaurant” ORDER BY average_rating DESC LIMIT 100; How to start?
  15. 15. Next Try? SELECT name, average_rating FROM business WHERE business_category = “restaurant” AND review_count > 50 ORDER BY average_rating DESC LIMIT 100;
  16. 16. Popularity ~ Quality
  17. 17. Next Try? SELECT name, average_rating + review_count as score FROM business WHERE business_category = “restaurant” AND review_count > 50 ORDER BY score DESC LIMIT 100;
  18. 18. Next Try? score = average_rating + b * review_count Choose b such that you are happy with the list! What does b mean? How much difference, relative to rating, does an extra review make?
  19. 19. Next Try? score = average_rating + b * review_count Choose b such that you are happy with the list! What does b mean? How much difference, relative to rating, does 900 reviews make?
  20. 20. Other Ideas Wilson Score ~ estimate of confidence interval in the rating Rank by the lower bound
  21. 21. Other Ideas Smoothing - used by IMDB, among others Add some number of “pseudoratings” at the global average rating. Choosing how many to add is an art...
  22. 22. Other Ideas Dirichlet and more sophisticated modelling
  23. 23. 2014 List - Smoothed - All time faves
  24. 24. 2015 List - Dirichlet - Weight 2014 Reviews
  25. 25. Product Perspective Analysis -> Insights -> Actionable Insights -> Impact - Paraphrased from Ken Rudin
  26. 26. Goals? Encourage exploration? Probably have better ways to do that that are more targeted Reward good restaurants? I don’t object, but not sure it’s that effective Inspire trust in Yelp? Maybe, but how many of a top 100 list is any person likely to have been to? Generate buzz and discussion? Well - if you are interested in trying some of them, and itching to tell us we are wrong - perhaps we succeeded...
  27. 27. Takeaways Data driven products are subtle - even if the feature is trivial (a list) Reasoning from first principles is great - but getting dirty and playing with the data is a better way to see what is likely to work Keep the product goals in mind - the distractions here can be scientific and statistical puzzles and challenges, rather than traditional engineering and design rabbit holes
  28. 28. Thanks Thanks: Dan Frank - Dirichlet work and 2015 List Will Seltzer - Dirichlet work Yelp Users - Ratings, Reviews and Photos Further Reading: Bayesian methods for Hackers WIlson Score and How not to sort by rating Nate Silver’s Burrito Bracket (This intro highly recommended for interesting methodology)
  29. 29. @YelpEngineering YelpEngineers engineeringblog.yelp.com github.com/yelpyelp.com/careers

×