Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Real-Time Recommendations and the Future of Search

8,602 views

Published on

From user preferences and location to time of day and weather, complex context representations have been the key to delivering personalised content. Graph databases excel at dealing with large amounts of complex data and therefore, they have been at the core of many modern real-time recommendation systems. In the near future, graph databases will play an equally important role in search personalisation.

In this talk, Michal will introduce real-world content personalization requirements and show how Neo4j has been used to implement them. He will show how much recommendation and search engines have in common and give an overview of the latest innovations in the field of search personalisation.

Published in: Software

Real-Time Recommendations and the Future of Search

  1. 1. and the Future of Search GraphAware® graphaware.com @graph_aware Real-Time Recommendations Michal Bachman
  2. 2. News you should read Books you should buy People you may know People you should date People you should market a product to … Recommendation Engines GraphAware®
  3. 3. Clinical Trial Matching Law Enforcement Event Planning Career Planning … Recommendation Engines GraphAware®
  4. 4. Content-based (features) Collaborative filtering (user <—> item relationships) Recommendation Engines GraphAware®
  5. 5. Features as well as relationships can be naturally represented as a graph. Good News GraphAware®
  6. 6. Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED GraphAware®
  7. 7. Traditional Approach GraphAware®
  8. 8. Global view Pre-computed (analytical) Traditional Approach GraphAware®
  9. 9. Bad Recommendation GraphAware®
  10. 10. Bad Recommendation GraphAware®
  11. 11. Global view Pre-computed (analytical) Lacks transparency Inflexible Lacks context Traditional Approach GraphAware®
  12. 12. Local view Real-time (query) Graph Approach GraphAware®
  13. 13. InfoJobs GraphAware®
  14. 14. Recommendation logic can be expressed as graph traversals. Graph Approach GraphAware®
  15. 15. Local view Real-time (query) Flexible Transparent Contextual Graph Approach GraphAware®
  16. 16. Cypher Query Example MATCH (u:User {id:1})-[:LIKED]->(m:Movie), (m)<-[:LIKED]-(another:User), (another)-[:LIKED]->(reco:Movie) WHERE NOT (u)-[:LIKED|DISLIKED]->(reco) RETURN reco; GraphAware®
  17. 17. Requirements of real-world recommendation engines are often much more complex. The Reality GraphAware®
  18. 18. Imagine you’re building the ”people you may know” feature on LinkedIn. Example GraphAware®
  19. 19. After a brainstorming session, your team came up with the following ways of finding people one may know: Example GraphAware®
  20. 20. Common contacts Facebook friends in common Email / mobile contacts in common Each others email / mobile contact Worked for the same company Studied at the same school Share the same interest Live in the same city People You May Know GraphAware®
  21. 21. But that’s just the beginning! Let’s go back and re-visit. Example GraphAware®
  22. 22. More contacts in common = better chance? Same city / school / company = does size matter? What about emails that don’t represent a person? What about people already connected? And pending… And rejected… And repeatedly ignored… People You May Know GraphAware®
  23. 23. Recommendation Engines GraphAware®
  24. 24. Discover Score Remove irrelevant Measure quality Recommendation Engines GraphAware®
  25. 25. Discover Score Remove irrelevant Measure quality Recommendation Engines GraphAware®
  26. 26. Discover GraphAware® IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  27. 27. Discover Score Remove irrelevant Measure quality Recommendation Engines GraphAware®
  28. 28. Score GraphAware® IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  29. 29. Score GraphAware®
  30. 30. Score GraphAware® LIVES_INMichal age:31 London Daniela age:25 LIVES_IN
  31. 31. Discover Score Remove irrelevant Measure quality Recommendation Engines GraphAware®
  32. 32. Blacklist Filter Remove Irrelevant GraphAware®
  33. 33. Remove Irrelevant GraphAware® IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  34. 34. Discover Score Remove irrelevant Measure quality Recommendation Engines GraphAware®
  35. 35. Measure Quality GraphAware® Recommendation: 33812631-079d-11e5 Input: 323423 Output: 99338 FriendsInCommon: 0.75 (raw value: 4) SkillsInCommon: 0.50 SameLocation: 1.00 AgeDifference: -0.25 Total: 2.00 Outcome: 1
  36. 36. GraphAware Reco is an open-source recommendation engine skeleton. You Can Do It Too GraphAware®
  37. 37. public class PeopleYouMayKnow extends Neo4jTopLevelDelegatingRecommendationEngine {
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return Arrays.<RecommendationEngine<Node, Node>>asList(
 new SharedEmployment(),
 new ConnectionsInCommon(),
 new MobileContacts(),
 new ReverseMobileContacts(),
 new MobileContactsInCommon(),
 new SharedStudies()
 );
 }
 
 @Override
 protected List<PostProcessor<Node, Node>> postProcessors() {
 return Arrays.<PostProcessor<Node, Node>>asList(
 new RewardSameCity(),
 new RewardSameState(),
 new PenalizeAlreadySeen()
 );
 }
 
 @Override
 protected List<BlacklistBuilder<Node, Node>> blacklistBuilders() {
 return Arrays.asList(
 new ExcludeSelf(),
 new ExistingRelationshipBlacklistBuilder(CONNECTED, BOTH),
 new ExistingRelationshipBlacklistBuilder(INVITED, OUTGOING),
 new ExistingRelationshipBlacklistBuilder(REJECTED, BOTH)
 );
 }
 
 @Override
 protected List<Filter<Node, Node>> filters() {
 return Arrays.<Filter<Node, Node>>asList(
 new PrivacySettingTooHigh()
 );
 }
 } GraphAware®
  38. 38. plugin to Neo4j (uses GraphAware Framework) you can use Cypher or a JVM-language opinionated architecture very fast very flexible handles all the plumbing GraphAware Reco GraphAware®
  39. 39. GraphAware® What’s Next?
  40. 40. LinkedIn GraphAware®
  41. 41. Has indexing It’s getting better There are more powerful tools Neo4j GraphAware®
  42. 42. Closer users on top Products likely to buy on top … Requirements GraphAware® => Recommendation Engine with a constrained set of possible recommendations
  43. 43. Discover Score Remove irrelevant Measure quality Recommendation Engines GraphAware®
  44. 44. Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED GraphAware®
  45. 45. Graph-Aided Search GraphAware®
  46. 46. Process of finding the most relevant results of a search query by combining the relevancy scores produced by a search engine with those produced by a graph-based recommendation engine. Graph-Aided Search GraphAware®
  47. 47. Graph-Aided Search GraphAware® Application Write Replicate (async) OK
  48. 48. Graph-Aided Search GraphAware® Application Results Boost Scores Search
  49. 49. Graph-Aided Search GraphAware® Application Write Replicate (async) Results BoostOK Scores Search Recommendation Engine Neo4j to Elasticsearch Elasticsearch to Neo4j
  50. 50. GraphAware®
  51. 51. GraphUnit
 & RestTest UUID WarmUp Auditing Importer GraphAware Framework GraphAware® Reco Engine Elasticsearch Integration TimeTree Algorithms NodeRank
  52. 52. Open Source (GPL) Active Production Ready Github: github.com/graphaware Our Web: graphaware.com Maven Central GraphAware Framework GraphAware®
  53. 53. Try it Give us feedback Contribute Build your own modules Get in touch for support / consultancy GraphAware Framework GraphAware®
  54. 54. GraphAware® www.graphaware.com @graph_aware GraphAware the world’s #1 Neo4j consultancy

×