Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
GraphAware
TM
Michal Bachman
graphaware.com
@graph_aware
Recommendations
with Neo4j
Building a high-performance recommenda...
GraphAware
TM
Quick Intro
Why Graphs?
Business and Technical Challenges
GraphAware Recommendation Engine
About This Talk
GraphAware
TM
News you should read
Books you should buy
People you may know
People you should date
People you should marke...
GraphAware
TM
Content-based (features)
Collaborative filtering (user <-> item relationships)
Recommendation Engines
GraphAware
TM
Features as well as relationships can be naturally
represented as a graph.
Good News
GraphAware
TM
Example
IS_OF_GENRE
title:
“Love Actually”
Movie
name: “Bob”
User
name:
“Comedy”
Genre
RATED
rating: 5
name:...
GraphAware
TM
Easy to understand
Natural to model
Flexible (schema-free)
Fast to query
Graphs (Neo4j)
GraphAware
TM
Great for a quick PoC
Great for smaller data sets
Great for relatively simple logic
Cypher
GraphAware
TM
Cypher
MATCH
(u:User)-[:LIKED]->(m:Movie),
(m)<-[:LIKED]-(another:User),
(another)-[:LIKED]->(reco:Movie)
WH...
GraphAware
TM
Requirements of real-world recommendation
engines are often much more complex.
The Reality
GraphAware
TM
Imagine you’re building the ”people you may
know” feature on LinkedIn.
Example
GraphAware
TM
After a brainstorming session, your team came up
with the following ways of finding people one may
know:
Exam...
GraphAware
TM
Common contacts
Facebook friends in common
Email / mobile contacts in common
Each others email / mobile cont...
GraphAware
TM
But that’s just the beginning! Let’s go back and
re-visit.
Example
GraphAware
TM
More contacts in common = better chance?
Same city / school / company = does size matter?
What about emails ...
GraphAware
TM
Finding things to recommend
Serving the most relevant recommendations
Measuring the quality of recommendatio...
GraphAware
TM
Performance (real-time!)
Simplicity
Flexibility
Technical Challenges
GraphAware
TM
So we came up with an open-source
recommendation engine skeleton that will help you
solve all the challenges...
GraphAware
TM
plugin to Neo4j (uses GraphAware Framework)
you have to use a JVM-language
opinionated architecture
very fas...
GraphAware
TM
Engine per recommendation “reason” (core logic)
Engine executes a graph traversal to find items
Engines are a...
GraphAware
TM
Example
IS_OF_GENRE
title:
“Love Actually”
Movie
name: “Bob”
User
name:
“Comedy”
Genre
RATED
rating: 5
name:...
GraphAware
TM
Engine per recommendation “reason” (core logic)
Engine executes a graph traversal to find items
Engines are a...
GraphAware
TM
Example
IS_OF_GENRE
title:
“Love Actually”
Movie
name: “Bob”
User
name:
“Comedy”
Genre
RATED
rating: 5
name:...
GraphAware
TM
Engine per recommendation “reason” (core logic)
Engine executes a graph traversal to find items
Engines are a...
GraphAware
TM
Example
IS_OF_GENRE
title:
“Love Actually”
Movie
name: “Bob”
User
name:
“Comedy”
Genre
RATED
rating: 5
name:...
GraphAware
TM
Input -> Engine -> Recommendations
Scores and Score Transformers
Blacklists
Filters
Post-processors
Context ...
GraphAware
TM
In 5 minutes, we’ll build a simple engine that
recommends who you should be friends with.
Let’s Build Someth...
GraphAware
TM
0) Model
GraphAware
TM
1) Discover
GraphAware
TM
public class FriendsInCommon extends SomethingInCommon {



@Override

public String name() {

return "frien...
GraphAware
TM
2) Score
GraphAware
TM
public class FriendsInCommon extends SomethingInCommon {

@Override

protected ScoreTransformer scoreTransfo...
GraphAware
TM
3) Post-Process
GraphAware
TM
public class RewardSameLocation extends RewardSomethingShared {



@Override

protected RelationshipType typ...
GraphAware
TM
public class RewardSameLabels implements PostProcessor<Node, Node> {



@Override

public void postProcess(R...
GraphAware
TM
4) Filter
GraphAware
TM
public final class FriendsContextFactory extends Neo4jContextFactory {



@Override

protected List<Blacklis...
GraphAware
TM
5) Assemble
GraphAware
TM
public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine {



public FriendsComputing...
GraphAware
TM
?) Precompute
GraphAware
TM
public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine {



public FriendsComputing...
GraphAware
TM
public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine {



public FriendsRecommendation...
GraphAware
TM
6) Log
GraphAware
TM
public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine {



public FriendsRecommendation...
GraphAware
TM
7) Test
GraphAware
TM
List<Recommendation<Node>> reco =

recommendationEngine.recommend(getPersonByName("Adam"), Mode.REAL_TIME, 2...
GraphAware
TM
List<Recommendation<Node>> reco =
recommendationEngine.recommend(getPersonByName("Luanne"), REAL_TIME, 4);

...
GraphAware
TM
Finding things to recommend
Serving the most relevant recommendations
Measuring the quality of recommendatio...
GraphAware
TM
Performance (real-time!)
Simplicity
Flexibility
Technical Challenges
GraphAware
TM
Getting Started
<dependencies>
...
<dependency>

<groupId>com.graphaware.neo4j</groupId>

<artifactId>recomm...
GraphAware
TM
Built-in ability to pre-compute recommendations
Other built-in base-classes
But we need your help!
https://g...
GraphAware
TM
Built-in algorithms
Time-based ParticipationPolicy
Integration with compute engines
Machine learning
Future
GraphAware
TM
GraphAware Framework makes it easy to build,
test, and deploy generic as well as domain-
specific functionali...
GraphAware
TM
GraphUnit

& RestTest
RelCount WarmUp Schema (wip)
Recommendation
Engine
GraphAware Framework
ChangeFeed UUI...
GraphAware
TM
Open Source (GPL)
Active
Production Ready
Github: github.com/graphaware
Our Web: graphaware.com
Maven Centra...
GraphAware
TM
Try it
Give us feedback
Contribute
Build your own modules
Get in touch for support / consultancy
GraphAware ...
GraphAware
TM
GraphAware Events
31

Jan
Recommendation
Engines in Brussels
(FOSDEM)
31

Jan
GraphGen in Brussels
(FOSDEM)
...
GraphAware
TM
GraphConnect Europe 2015
When:
Where:
Tickets:
Call for Papers:
Sponsors:
Thursday, 7th May, 2015 - main Con...
graphaware.com
@graph_aware
Thank You!
GraphAware
TM
Recommendations with Neo4j (FOSDEM 2015)
Recommendations with Neo4j (FOSDEM 2015)
Recommendations with Neo4j (FOSDEM 2015)
Upcoming SlideShare
Loading in …5
×

Recommendations with Neo4j (FOSDEM 2015)

Building a high performance Neo4j Recommendation Engine using the GraphAware Framework

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

Recommendations with Neo4j (FOSDEM 2015)

  1. 1. GraphAware TM Michal Bachman graphaware.com @graph_aware Recommendations with Neo4j Building a high-performance recommendation engine
  2. 2. GraphAware TM Quick Intro Why Graphs? Business and Technical Challenges GraphAware Recommendation Engine About This Talk
  3. 3. GraphAware TM News you should read Books you should buy People you may know People you should date People you should market a product to … Recommendation Engines
  4. 4. GraphAware TM Content-based (features) Collaborative filtering (user <-> item relationships) Recommendation Engines
  5. 5. GraphAware TM Features as well as relationships can be naturally represented as a graph. Good News
  6. 6. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  7. 7. GraphAware TM Easy to understand Natural to model Flexible (schema-free) Fast to query Graphs (Neo4j)
  8. 8. GraphAware TM Great for a quick PoC Great for smaller data sets Great for relatively simple logic Cypher
  9. 9. GraphAware TM Cypher MATCH (u:User)-[:LIKED]->(m:Movie), (m)<-[:LIKED]-(another:User), (another)-[:LIKED]->(reco:Movie) WHERE NOT (u)-[:LIKED|DISLIKED]->(reco) RETURN reco;
  10. 10. GraphAware TM Requirements of real-world recommendation engines are often much more complex. The Reality
  11. 11. GraphAware TM Imagine you’re building the ”people you may know” feature on LinkedIn. Example
  12. 12. GraphAware TM After a brainstorming session, your team came up with the following ways of finding people one may know: Example
  13. 13. GraphAware TM Common contacts Facebook friends in common Email / mobile contacts in common Each others email / mobile contact Worked for the same company Studied at the same school Share the same interest Live in the same city People You May Know
  14. 14. GraphAware TM But that’s just the beginning! Let’s go back and re-visit. Example
  15. 15. GraphAware TM More contacts in common = better chance? Same city / school / company = does size matter? What about emails that don’t represent a person? What about people already connected? And pending… And rejected… And repeatedly ignored… People You May Know
  16. 16. GraphAware TM Finding things to recommend Serving the most relevant recommendations Measuring the quality of recommendations Time to market / cost of development Business Challenges
  17. 17. GraphAware TM Performance (real-time!) Simplicity Flexibility Technical Challenges
  18. 18. GraphAware TM So we came up with an open-source recommendation engine skeleton that will help you solve all the challenges. We’ve done it before
  19. 19. GraphAware TM plugin to Neo4j (uses GraphAware Framework) you have to use a JVM-language opinionated architecture very fast very flexible handles all the plumbing Recommendation Engine
  20. 20. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled into higher-level engines Design Decisions
  21. 21. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  22. 22. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled to higher-level engines Items discovered multiple times are more relevant Relevance depends on how was item discovered Design Decision
  23. 23. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  24. 24. GraphAware TM Engine per recommendation “reason” (core logic) Engine executes a graph traversal to find items Engines are assembled to higher-level engines Items discovered multiple times are more relevant Relevance depends on how was item discovered Items not to be recommended: “cross-cutting” concern Design Decisions
  25. 25. GraphAware TM Example IS_OF_GENRE title: “Love Actually” Movie name: “Bob” User name: “Comedy” Genre RATED rating: 5 name: “Alice” User name: “Romance” Genre title: “American Pie” Movie IS_OF_GENRE IS_OF_GENRE RATEDrating: 5 INTERESTED_IN rating: 5 RATED
  26. 26. GraphAware TM Input -> Engine -> Recommendations Scores and Score Transformers Blacklists Filters Post-processors Context (how many, how fast,…?) Loggers Architecture
  27. 27. GraphAware TM In 5 minutes, we’ll build a simple engine that recommends who you should be friends with. Let’s Build Something
  28. 28. GraphAware TM 0) Model
  29. 29. GraphAware TM 1) Discover
  30. 30. GraphAware TM public class FriendsInCommon extends SomethingInCommon {
 
 @Override
 public String name() {
 return "friendsInCommon";
 }
 
 @Override
 protected RelationshipType getType() {
 return FRIEND_OF;
 }
 
 @Override
 protected Direction getDirection() {
 return BOTH;
 }
 }
  31. 31. GraphAware TM 2) Score
  32. 32. GraphAware TM public class FriendsInCommon extends SomethingInCommon {
 @Override
 protected ScoreTransformer scoreTransformer() {
 return new ParetoScoreTransformer(100, 10);
 }
 
 @Override
 public String name() {
 return "friendsInCommon";
 }
 
 @Override
 protected RelationshipType getType() {
 return FRIEND_OF;
 }
 
 @Override
 protected Direction getDirection() {
 return BOTH;
 }
 }
  33. 33. GraphAware TM 3) Post-Process
  34. 34. GraphAware TM public class RewardSameLocation extends RewardSomethingShared {
 
 @Override
 protected RelationshipType type() {
 return LIVES_IN;
 }
 
 @Override
 protected Direction direction() {
 return OUTGOING;
 }
 
 @Override
 protected float scoreValue(Node reco, Node in, Node shared) {
 return 10;
 }
 
 @Override
 protected String scoreName() {
 return "sameLocation";
 }
 }
  35. 35. GraphAware TM public class RewardSameLabels implements PostProcessor<Node, Node> {
 
 @Override
 public void postProcess(Recommendations<Node> out, Node in) {
 Label[] inLabels = toArray(in.getLabels());
 
 for (Recommendation<Node> reco : out.get()) {
 if (Arrays.equals(inLabels, toArray(reco.getItem().getLabels()))) {
 reco.add("sameGender", 10);
 }
 }
 }
 }
  36. 36. GraphAware TM 4) Filter
  37. 37. GraphAware TM public final class FriendsContextFactory extends Neo4jContextFactory {
 
 @Override
 protected List<BlacklistBuilder<Node, Node>> blacklistBuilders() {
 return asList(
 new ExcludeSelf(),
 new ExistingRelationshipBlacklistBuilder(FRIEND_OF, BOTH)
 );
 }
 
 @Override
 protected List<Filter<Node, Node>> filters() {
 return asList(
 new ExcludeSelf()
 );
 }
 }
  38. 38. GraphAware TM 5) Assemble
  39. 39. GraphAware TM public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsComputingEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new FriendsInCommon(),
 new RandomPeople()
 );
 }
 
 @Override
 protected List<PostProcessor<Node, Node>> postProcessors() {
 return asList(
 new RewardSameLabels(),
 new RewardSameLocation(),
 new PenalizeAgeDifference()
 );
 }
 }
  40. 40. GraphAware TM ?) Precompute
  41. 41. GraphAware TM public final class FriendsComputingEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsComputingEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new FriendsInCommon(),
 new RandomPeople()
 );
 }
 
 @Override
 protected List<PostProcessor<Node, Node>> postProcessors() {
 return asList(
 new RewardSameLabels(),
 new RewardSameLocation(),
 new PenalizeAgeDifference()
 );
 }
 
 @Override
 public ParticipationPolicy<Node, Node> participationPolicy(Context<Node, Node> context) {
 return ParticipationPolicy.IF_MORE_RESULTS_NEEDED;
 }
 }
  42. 42. GraphAware TM public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsRecommendationEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new Neo4jPrecomputedEngine(),
 new FriendsComputingEngine()
 );
 }
 }
  43. 43. GraphAware TM 6) Log
  44. 44. GraphAware TM public final class FriendsRecoEngine extends Neo4jTopLevelDelegatingEngine {
 
 public FriendsRecommendationEngine() {
 super(new FriendsContextFactory());
 }
 
 @Override
 protected List<RecommendationEngine<Node, Node>> engines() {
 return asList(
 new Neo4jPrecomputedEngine(),
 new FriendsComputingEngine()
 );
 }
 
 @Override
 protected List<Logger<Node, Node>> loggers() {
 return asList( 
 new Slf4jRecommendationLogger<Node, Node>(),
 new Slf4jStatisticsLogger<Node, Node>()
 );
 }
 }
  45. 45. GraphAware TM 7) Test
  46. 46. GraphAware TM List<Recommendation<Node>> reco =
 recommendationEngine.recommend(getPersonByName("Adam"), Mode.REAL_TIME, 2);
 
 String expected = 
 "(Vince {total:19.338144," +
 "ageDifference:-5.527864," +
 "friendsInCommon:14.866008," +
 "sameGender:10.0})," +
 
 "(Luanne {total:11.553411," +
 "ageDifference:-3.312597," +
 "friendsInCommon:14.866008})";
 
 assertEquals(expected, toString(reco));
  47. 47. GraphAware TM List<Recommendation<Node>> reco = recommendationEngine.recommend(getPersonByName("Luanne"), REAL_TIME, 4);
 
 assertEquals("Daniela", reco.get(0).getItem().getProperty("name"));
 assertEquals(22, reco.get(0).getScore().getTotalScore(), 0.5);
 
 assertEquals("Adam", reco.get(1).getItem().getProperty("name"));
 assertEquals(12, reco.get(1).getScore().getTotalScore(), 0.5);
 
 assertEquals("Vince", reco.get(2).getItem().getProperty("name"));
 assertEquals(8, reco.get(2).getScore().getTotalScore(), 0.5);
 
 assertEquals("Bob", reco.get(3).getItem().getProperty("name"));
 assertEquals(-9, reco.get(3).getScore().getTotalScore(), 0.5);
  48. 48. GraphAware TM Finding things to recommend Serving the most relevant recommendations Measuring the quality of recommendations Time to market / cost of development Business Challenges
  49. 49. GraphAware TM Performance (real-time!) Simplicity Flexibility Technical Challenges
  50. 50. GraphAware TM Getting Started <dependencies> ... <dependency>
 <groupId>com.graphaware.neo4j</groupId>
 <artifactId>recommendation-engine</artifactId> <version>2.1.6.27.2</version>
 </dependency> ... <dependencies>
  51. 51. GraphAware TM Built-in ability to pre-compute recommendations Other built-in base-classes But we need your help! https://github.com/graphaware/neo4j-reco There’s More!
  52. 52. GraphAware TM Built-in algorithms Time-based ParticipationPolicy Integration with compute engines Machine learning Future
  53. 53. GraphAware TM GraphAware Framework makes it easy to build, test, and deploy generic as well as domain- specific functionality for Neo4j. GraphAware Framework
  54. 54. GraphAware TM GraphUnit
 & RestTest RelCount WarmUp Schema (wip) Recommendation Engine GraphAware Framework ChangeFeed UUID TimeTree Algorithms NodeRank
  55. 55. GraphAware TM Open Source (GPL) Active Production Ready Github: github.com/graphaware Our Web: graphaware.com Maven Central GraphAware Framework
  56. 56. GraphAware TM Try it Give us feedback Contribute Build your own modules Get in touch for support / consultancy GraphAware Framework
  57. 57. GraphAware TM GraphAware Events 31
 Jan Recommendation Engines in Brussels (FOSDEM) 31
 Jan GraphGen in Brussels (FOSDEM) 5
 Feb Recommendation Engines Webinar 5
 Feb Meetup at GraphAware (build your own Recommendation Engine) 10
 Feb Neo4j Fundamentals in Manchester 10
 Feb Neo4j Meetup in Manchester 17
 Feb Neo4j Fundamentals in Edinburgh 17
 Feb Neo4j Meetup in Edinburgh
  58. 58. GraphAware TM GraphConnect Europe 2015 When: Where: Tickets: Call for Papers: Sponsors: Thursday, 7th May, 2015 - main Conference Day Wednesday, 6th May 2015 - Training Day Etc venues, 155 Bishopsgate, London (next to Liverpool Street Station) now available on www.graphconnect.com 199$ early bird plus 100$ for training 499$ full price plus 100$ for training open now till 29th January all Neo4j community members, customers or general graph enthusiasts are invited to submit their talk open now till 29th January, email: gceurope@neotechnology.com
  59. 59. graphaware.com @graph_aware Thank You! GraphAware TM

×