A Presentation from Neo4j GraphDay Chicago

2. github.com/maxdemarzi
About 200 public repositories
Max De Marzi
Neo4j Field Engineer
About
Me !
01
02
03
04
maxdemarzi.com
@maxdemarzi

3. Experience Technical Doesn’t
Matter
75 % 50% 95%

4. You go home, thinking about graphs
All that matters
You

5. Property
Graph
It’s super simple.
All you get is:
Property Graph Model
Properties
Nodes
Relationships

6. What you (probably) already know:

8. Joins are executed every time
you query the relationship
Executing a Join means to
search for a key
B-Tree Index: O(log(n))
Your data grows by 10x, your speed
slows down by half
More Data = More Searches
Slower Performance
The Problem
1
2
3
4

9. Same Data, Different Layout
No more Tables, no more Foreign Keys, no more Joins

10. Relational Databases can’t handle Relationships
Degraded Performance
Speed plummets as data grows
and as the number of joins grows
Wrong Language
SQL was built with Set Theory in
mind, not Graph Theory
Not Flexible
New types of data and relationships
require schema redesign
Wrong Model
They cannot model or store
relationships without complexity1
2
3
4

11. NoSQL Databases can’t handle Relationships
Degraded Performance
Speed plummets as you try to join
data together in the application
Wrong Languages
Lots of wacky “almost sql”
languages terrible at “joins”
Not ACID
Eventually Consistent means
Eventually Corrupt
Wrong Model
They cannot model or store
relationships without complexity1
2
3
4

12. What’s Our
Secret
Sauce?

15. Fixed Sized Records
“Joins” on Creation
Spin Spin Spin through
this data structure
Pointers instead of
Lookups
1
2
3
4
Neo4j Secret Sauce

16. Remains steady as database grows
Real Time Query Performance
Connectedness and Size of Data Set
Response Time
0 to 2 hops
0 to 3 degrees
Thousands of connections
Tens to hundreds of hops
Thousands of degrees
Billions of connections
Relational and
Other NoSQL
Databases
Neo4j
Neo4j is
1000x faster
Reduces minutes
to milliseconds

17. I don’t know the average height of all hollywood actors, but I do know the Six Degrees of Kevin Bacon
But not for every query

18. Reimagine your Data as a Graph
Better Performance
Query relationships in real time
Right Language
Cypher was purpose built for
Graphs
Flexible and Consistent
Evolve your schema seamlessly
while keeping transactions
Right Model
Graphs simplify how you think
1
2
3
4
Agile, High Performance
and Scalable without Sacrifice

19. Just draw stuff and “walla” there is your data model
Graphs are Whiteboard Friendly

21. Movie Property Graph
Some Models are Easy

23. Should Roles be their own Node?
Some Models are Easy but not for all Questions

24. How do you model Flight Data?

25. Airports Nodes with Flying To Relationships
How do you model Flight Data?

26. Maybe Flight should be its own Node?
How do you model Flight Data?

27. Don’t we care about Flights only on particular Days?
How do you model Flight Data?

28. What is this trick with the date in the relationship type?
How do you model Flight Data?

29. We don’t need Airports if we model this way!
How do you model Flight Data?

30. Lets get Creative

31. Group Destinations together!
How do you model Flight Data?

32. OMG WAT!
How do you model Flight Data?

33. Do not try and bend the data. That’s im possible.

34. If they can do it, you can do it!
How do you model Comic Books?

35. Cloning Twitter
Building a News Feed
9:00 am
@hipster
This is what I had for breakfast! <Insert Image of squirrel food>
8:30 am
@neo4j
Automated tweet telling me about Graph Connect 2017 in NYC on Oct 23-24
8:12 am
@ex-coworker
Stuff I no longer care about.
8:03 am
@someguy
Inspirational Quote of the Day

37. How do others do it?
Cloning Twitter

38. How do others do it?
Cloning Twitter

39. The Wrong Way
Modeling a Twitter Feed

40. A Better Way
Modeling a Twitter Feed

41. Bigger Model
Modeling a Twitter Feed

42. getDegree is your Friend

43. This is Java. What happened to Cypher?

44. Java Core API

45. Easy to Learn (no really)
Java Core API
• Step by Step from GraphDatabaseService
• Start a transaction (reads and writes)
• findNode(Label, Property, Value)
• findNodes(Label, Property, Value)
• findNodes(Label)
• getNodeById(Long)
• getRelationships(Direction, Type)
• getProperty(Property, (optional) Default Value)

46. Get friends of a User
Java Core API

47. Traversal API

48. Interesting to Learn
Traversal API
• Start with the Simple Defaults (order, relationships, depth,
uniqueness, etc)
• Custom Expanders
• Where should I go next
• Custom Evaluators
• I’ve gone there… should I accept this path?

49. Example
Traversal API

50. Cypher

51. Cypher: Powerful and Expressive Query Language
MATCH (:Person { name:“Dan”} ) -[:LOVES]-> (:Person { name:“Ann”} )
LOVES
Dan Ann
Label Property Label Property
Node Node

52. MATCH (boss)-[:MANAGES*0..3]->(sub),
(sub)-[:MANAGES*1..3]->(report)
WHERE boss.name = “John Doe”
RETURN sub.name AS Subordinate,
count(report) AS Total
Express Complex Queries Easily with Cypher
Find all direct reports and
how many people they manage,
up to 3 levels down
Cypher QuerySQL Query

53. Understanding User Behavior
EventsMetrics
TargetingSearching
Purchase
History

54. Learn from the Experts
• Alex Beutel, CMU
• Leman Akoglu, Stony Brook
• Christos Faloutsos, CMU
• Graph-Based User Behavior Modeling: From Prediction to
Fraud Detection
• http://www.cs.cmu.edu/~abeutel/kdd2015_tutorial/

55. User Behavior Challenges
• How can we understand
normal user behavior?

56. User Behavior Challenges
• How can we understand
normal user behavior?
• How can we find
suspicious behavior?

57. User Behavior Challenges
• How can we understand
normal user behavior?
• How can we find
suspicious behavior?
• How can we distinguish
the two?

58. Does your little girl like Rambo?

59. Demographics: Age

60. Demographics: Gender

61. Do Little Girls like Movies other Little Girls Like?

62. Yes! Little Girls like Movies other Little Girls Like

63. What do Little Girls Like?
MATCH (u:User)-[r:RATED]->(m:Movie)
WHERE u.age = 1 AND u.gender = "F" AND r.stars > 3
RETURN m.title, COUNT(r) AS cnt
ORDER BY cnt DESC
LIMIT 10

64. What do Little Girls Like?

65. What do Men 25-34 Like?
MATCH (u:User)-[r:RATED]->(m:Movie)
WHERE u.age = 25 AND u.gender = "M" AND r.stars > 3
RETURN m.title, COUNT(r) AS cnt
ORDER BY cnt DESC
LIMIT 10

66. What do Men 25-34 Like?

67. Modeling “Normal” Behavior
• Predict Edges
(Similar Users)

68. Modeling “Normal” Behavior
• Predict Edges
(Movies I should Watch)

69. What Rating should I give 101 Dalmatians?
MATCH (me:User {id:1})-[r1:RATED]->(m:Movie)
<-[r2:RATED]-(:User)-[r3:RATED]->
(m2:Movie {title:”101 Dalmatians”})
WHERE ABS(r1.stars-r2.stars) <=1
RETURN AVG(r3.stars)

70. Modeling “Normal” Behavior
• Predict Edges
• Predict Node Attributes
• Predict Edge Attributes
• Clustering and
Community Detection

71. Predict a Star Rating purely on Demographics
MATCH (u:User)-[r:RATED]->(m:Movie {title:”Toy Story”})
WHERE u.age = 1 AND u.gender = "F"
RETURN AVG(r.stars)

72. Modeling “Normal” Behavior
• Predict Edges
• Predict Node Attributes
• Predict Edge Attributes
• Clustering and
Community Detection
• Fraud Detection

73. Two Sides of the Same Coin
Recommendations
• Add the relationship
that does not exist
Fraud Detection
• Find the relationships
that should not exist

74. Modeling User Behavior
• Modeling normal users
and detecting anomalies
are two sides of
understanding user
behavior

75. Recommendation
Engines

76. Hello World Recommendation

77. Hello World Recommendation

78. Movie Data Model

79. Cypher Query: Movie Recommendation
MATCH (watched:Movie {title:"Toy Story”}) <-[r1:RATED]- () -[r2:RATED]-> (unseen:Movie)
WHERE r1.rating > 7 AND r2.rating > 7
AND watched.genres = unseen.genres
AND NOT( (:Person {username:”maxdemarzi"}) -[:RATED|WATCHED]-> (unseen) )
RETURN unseen.title, COUNT(*)
ORDER BY COUNT(*) DESC
LIMIT 25
What are the Top 25 Movies
• that I haven't seen
• with the same genres as Toy Story
• given high ratings
• by people who liked Toy Story

80. Movie Data Model

81. Cypher Query: k-NN Recommendation
MATCH (m:Movie) <-[r:RATED]- (b:Person) -[s:SIMILARITY]- (p:Person {name:'Zoltan Varju'})
WHERE NOT( (p) -[:RATED|WATCHED]-> (m) )
WITH m, s.similarity AS similarity, r.rating AS rating
ORDER BY m.name, similarity DESC
WITH m.name AS movie, COLLECT(rating)[0..3] AS ratings
WITH movie, REDUCE(s = 0, i IN ratings | s + i)*1.0 / LENGTH(ratings) AS recommendation
ORDER BY recommendation DESC
RETURN movie, recommendation
LIMIT 25
What are the Top 25 Movies
• that Zoltan Varju has not seen
• using the average rating
• by my top 3 neighbors

82. Cypher
Stored
Procedures

83. Combine any APIs
Cypher Stored Procedures

84. Don’t use SOLR Facets for this!
Multiple Dimensions
AgeSize
FeaturesProperty
Cost

85. Multiple Dimensions
Java
Audio Book!
What about Publisher?
What about Author?
What about Publication Year?
What about Java Version?
What About….
Left parentheses, n, right
parentheses, semi-colon!

86. Bucket or Group Values if you have to
Discrete Values for Each Dimension

87. Nodes for Discrete Dimensional Values
Dimensional Model
*Use Named Relationship Types instead of HAS

88. Stupid Glasses
Loud Pants
Skate Boards
Neon Colors
1
2
3
4
Who remembers this?

89. Look at how thick they were, even back in 1902!
It’s a Sears Catalogue!

90. Ares Predator
Street Samurai Catalog

91. With free two day shipping!
Cypher Version of the Catalog

92. A tree is a simple graph
A Tree of Data

93. So fast, it’s not even funny.
Promotions
About 2-4M Traversals per second per core
Traversing a 50 level Tree UP costs practically nothing.

94. Connecting unconnected Things indirectly

95. What are the Top 10 Jobs for me
• that are in the same location I’m in
• for which I have the necessary qualifications

96. Partial Subgraph Search

97. Data Cleansing
Look for Shared Features
Calculate Similarity
Connect
Extract Features
1
2
3
4

98. Using an Anchor

99. and many
more use
cases!

100. Thank You!