Starting Recommendations
Scott Triglia
Goals

● Show our thought process
● Expose some useful questions
● Practical solutions for new teams
Disclaimer

This is a case study, not a prescription
Yelp 101
Our Topic

Our Goal: Interesting businesses relevant to
you right now
Before
Decision Time

Brainstorming what matters with your ace
team of devs
Context matters
Context matters
Don’t be shy

Interesting reasons are half the point!
Know your Team

The organizational context matters too:
● We have very little infrastructure to support large-scale ML
● M...
So what do we build?
Guiding Principles

1) We know we need to solve a retrieval problem
Guiding Principles

2) Build for what you have, plan for expansion
Guiding Principles

3) We need to build a good product, not beat a benchmark
The Big Picture

API
Request

Experts
Experts
Experts

Elastic
Search

Final
Results

General flow:
1. Gather sufficient c...
Building the request

API
Request

Experts
Experts
Experts

Final
Results
●

From client: location, user_id

●

Derived co...
Expert Opinions

API
Request

Experts
Experts
Experts

Elastic
Search

Final
Results
Each expert handles a single reason a...
Expert Opinions

API
Request

Experts
Experts
Experts

Final
Results
Liked By Friends Expert:
General Requirements:
Open N...
Expert Opinions

API
Request

Experts
Experts
Experts

Elastic
Search

Final
Results
Why an expert-based system?
●

Think ...
Efficient Search

API
Request

Experts
Experts
Experts

Final
Results
What do we need from our datastore?
●

Fast geograph...
Final Decisions

API
Request

Experts
Experts
Experts

Elastic
Search

Final
Results

How to combine expert results? We ne...
Get to the point already!
Get to the point already!
Get to the point already!
Get to the point already!
Get to the point already!
Final Version
Extension

Now that we’re iterating, what are our future plans?
● Richer context (user, location, etc.)
● Infrastructure s...
Summary

So what are the takeaways for building a first
recommender system?
● Solve your problem, not someone else’s
● Bei...
Questions?
striglia@yelp.com
Upcoming SlideShare
Loading in...5
×

Scott Triglia, MLconf 2013

1,519

Published on

Scott Triglia, Search and Data Mining Engineer at Yelp

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,519
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
13
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Scott Triglia, MLconf 2013"

  1. 1. Starting Recommendations Scott Triglia
  2. 2. Goals ● Show our thought process ● Expose some useful questions ● Practical solutions for new teams
  3. 3. Disclaimer This is a case study, not a prescription
  4. 4. Yelp 101
  5. 5. Our Topic Our Goal: Interesting businesses relevant to you right now
  6. 6. Before
  7. 7. Decision Time Brainstorming what matters with your ace team of devs
  8. 8. Context matters
  9. 9. Context matters
  10. 10. Don’t be shy Interesting reasons are half the point!
  11. 11. Know your Team The organizational context matters too: ● We have very little infrastructure to support large-scale ML ● Must scale to all Yelp data+users on day 1. ● Our team is small, and will be so for a while ● This is a first version of a (hopefully!) long lived product
  12. 12. So what do we build?
  13. 13. Guiding Principles 1) We know we need to solve a retrieval problem
  14. 14. Guiding Principles 2) Build for what you have, plan for expansion
  15. 15. Guiding Principles 3) We need to build a good product, not beat a benchmark
  16. 16. The Big Picture API Request Experts Experts Experts Elastic Search Final Results General flow: 1. Gather sufficient contextual information 2. Consult each expert for their top candidates 3. Wisely combine suggestions from each expert
  17. 17. Building the request API Request Experts Experts Experts Final Results ● From client: location, user_id ● Derived context ● Neighborhood preferences ● User preferences ● Time preferences Elastic Search
  18. 18. Expert Opinions API Request Experts Experts Experts Elastic Search Final Results Each expert handles a single reason and knows its own requirements. For example, a LikedByFriends expert would only return candidate businesses which one of the user’s friends had rated highly.
  19. 19. Expert Opinions API Request Experts Experts Experts Final Results Liked By Friends Expert: General Requirements: Open Now Sufficiently Nearby Expert Requirements: At least one friend gave it 5 stars Elastic Search
  20. 20. Expert Opinions API Request Experts Experts Experts Elastic Search Final Results Why an expert-based system? ● Think in terms of small, isolated components ● Implementation agnostic ● Adding, removing experts is trivial
  21. 21. Efficient Search API Request Experts Experts Experts Final Results What do we need from our datastore? ● Fast geographic filtering ● Simple but efficient sorting ● All of this happening in 100ms Elastic Search
  22. 22. Final Decisions API Request Experts Experts Experts Elastic Search Final Results How to combine expert results? We need to factor in: ● Must balance preferences (distance, rating, category) ● Should prefer better reasons when possible ● Sufficiently high quality candidates makes this very safe
  23. 23. Get to the point already!
  24. 24. Get to the point already!
  25. 25. Get to the point already!
  26. 26. Get to the point already!
  27. 27. Get to the point already!
  28. 28. Final Version
  29. 29. Extension Now that we’re iterating, what are our future plans? ● Richer context (user, location, etc.) ● Infrastructure support for faster ML prototyping ● Better personalized ranking ● Training data!
  30. 30. Summary So what are the takeaways for building a first recommender system? ● Solve your problem, not someone else’s ● Being cutting edge may not be the top priority ● Build for the tools you have, plan for what will come ● Good software engineering enables quality ML
  31. 31. Questions? striglia@yelp.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×