Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
09:00-09:30
09:30-10:15
10:15-11:00
11:00-11:30
11:30-12:30
12:30-13:30
13:30-17:00
Breakfast and Registration

The Connec...
Transform Your Data
Neo4j Graph Day - Santa Clara 2017
William Lyon
@lyonwj
A Worked Example
William Lyon
Developer Relations Engineer @neo4j
will@neo4j.com
@lyonwj
lyonwj.com
Agenda
• Graph database - developer perspective
• (relational)-[:TO]->(graph)
• Use cases
• Fraud Detection
• Recommendati...
Neo4j
Graph Database
• Property graph data model
• (open)Cypher query language
• Native graph processing
• Language driver...
Graph Data Model
Labeled Property Graph Model
Labeled Property Graph Model
The Graph
https://github.com/johnymontana/neo4j-datasets/tree/master/yelp
The Graph
https://www.youtube.com/watch?v=kSZHFlBDIfM
The Graph
https://github.com/johnymontana/mattermark-graphql-neo4j
The Graph
https://github.com/neo4j-meetups/modeling-worked-example
The Graph
https://neo4j.com/sandbox-v2/
openCypher
(queryLanguage)-[:FOR]->(graphs)
http://www.opencypher.org/
Cypher
https://neo4j.com/sandbox-v2/
Use Case:
Fraud Detection
Organized in groups Synthetic Identities Stolen Identities Hijacked Devices
Who Are Today’s Fraudsters?
Fraud Rings and Synthetic Identities
ACCOUNT
HOLDER 2
Modeling a fraud ring as a graph
ACCOUNT
HOLDER 1
ACCOUNT
HOLDER 3
ACCOUNT
HOLDER 2
ACCOUNT
HOLDER 1
ACCOUNT
HOLDER 3
CREDIT
CARD
BANK
ACCOUNT
BANK
ACCOUNT
BANK
ACCOUNT
PHONE
NUMBER
UNSECUR...
ACCOUNT
HOLDER 2
ACCOUNT
HOLDER 1
ACCOUNT
HOLDER 3
CREDIT
CARD
BANK
ACCOUNT
BANK
ACCOUNT
BANK
ACCOUNT
ADDRESS
PHONE
NUMBER...
https://neo4j.com/graphgist/9d627127-003b-411a-b3ce-f8d3970c2afa?ref=solutions
https://neo4j.com/graphgists/
Credit Card Fraud
(relational)-[:TO]->(graph)
FROM RDBMS TO GRAPHS
Northwind
Northwind - the canonical RDBMS Example
( )-[:TO]->(Graph)
( )-[:IS_BETTER_AS]->(Graph)
Starting with the ER Diagram
Locate the Foreign Keys
Drop the Foreign Keys
Find the JOIN Tables
(Simple) JOIN Tables Become Relationships
Attributed JOIN Tables -> Relationships with Properties
As a Graph
https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/master/readme.adoc
Northwind
Northwind
Neo4j Language Drivers
Neo4j Drivers
https://neo4j.com/developer/language-guides/
POWERING AN APP
Simple App
Simple App
Simple Python Code
Simple Python Code
Simple Python Code
Simple Python Code
https://neo4j.com/developer/
Use Case:
Personalized Recommendations
Recommendations drive user engagement
“35 percent of what consumers purchase on
Amazon and 75 percent of what they watch on
Netflix come from product recommenda...
Product
Recommendations
Effective product recommendation
algorithms has become the new
standard in online retail — directl...
Powerful recommendation engines rely
on the connections between multiple
sources of data
Powerful recommendation engines rely
on the connections between multiple
sources of data
Dreamhouse
Series 15% off
The Store
Search
Hi, login
My Account
People who bought Side Table also bought:
Coffee Table
$23...
Dreamhouse
Series 15% off
The Store
Search
Hi, login
My Account
People who bought Side Table also bought:
Coffee Table
$23...
The
Store
People who bought Side Table also bought: Similar product in from Home Office Series:
Hi, login
My AccountSearch
D...
People who bought Side Table also bought: Similar product in from Home Office Series:
Wood Side Table
$110
Green Side Table
...
Category Price ConfigurationsLocation
Silos & Polyglot Persistence
Purchase ViewReviewReturn In-store PurchasesInventory
Pr...
Purchases
RELATIONAL DB WIDE COLUMN STORE
Views
DOCUMENT STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping Cart
K...
Purchases
RELATIONAL DB WIDE COLUMN STORE
Views
DOCUMENT STORE
User Review
RELATIONAL DB
In-Store
Purchase
Shopping Cart
K...
Data Lake
Purchases
RELATIONAL DB
Product
Catalogue
DOCUMENT STORE WIDE COLUMN STORE
Views
DOCUMENT STORE
User Review
RELA...
Purchases
RELATIONAL DB
Product
Catalogue
DOCUMENT STORE WIDE COLUMN STORE
Views
DOCUMENT STORE
User Review
RELATIONAL DB
...
Using Data Relationships for Recommendations
Content-based filtering
Recommend items based on what
users have liked in the...
Collaborative Filtering
Collaborative Filtering
Collaborative Filtering
Collaborative Filtering
Link prediction
In Cypher
In Cypher
Basic initial approach. Improvements:
- aggregate across all purchases
- scoring / normalize
- compute similarit...
Content Filtering
Content Filtering
Content Filtering w/ Cypher
Content Filtering - Concept Hierarchy
Content Filtering - Concept Hierarchy
Content Filtering - Concept Hierarchy
Content Filtering - Concept Hierarchy w/ Cypher
Content Filtering - Concept Hierarchy w/ Cypher
Basic initial approach. Improvements:
- aggregate across all purchases
- c...
Neo4j Sandbox
neo4jsandbox.com
Want to learn more?
graphdatabases.com neo4j.com/sandbox
(you)-[:HAVE]->(?)
(?)<-[:ANSWERS]-(will)
09:00-09:30
09:30-10:15
10:15-11:00
11:00-11:30
11:30-12:30
12:30-13:30
13:30-17:00
Breakfast and Registration

The Connec...
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Transforming Your Data: A Worked Example
Upcoming SlideShare
Loading in …5
×

Transforming Your Data: A Worked Example

441 views

Published on

This talk examines graph databases and Neo4j with a use-case driven approach. First, we look at some property graph model examples, taken from real-world datasets. Next we discuss converting a relational model to graph, using the canonical Northwind example. Finally, we dive into Fraud Detection and Personalized Recommendation examples, learning about Neo4j developer tooling as we explore these use cases.

Published in: Technology
  • Want to preview some of our plans? You can get 50 Woodworking Plans and a 440-Page "The Art of Woodworking" Book... Absolutely FREE ◆◆◆ http://tinyurl.com/yy9yh8fu
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Get access to 16,000 woodworking plans, Download 50 FREE Plans... ◆◆◆ http://tinyurl.com/yy9yh8fu
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Transforming Your Data: A Worked Example

  1. 1. 09:00-09:30 09:30-10:15 10:15-11:00 11:00-11:30 11:30-12:30 12:30-13:30 13:30-17:00 Breakfast and Registration The Connected Data Imperative: Why Graphs Transform Your Data: A Worked Example Break Enterprise Ready: A Look at 
 Neo4j in Production Lunch Hands-On Training Session Agenda APRIL 26, 2017 SANTA CLARA
  2. 2. Transform Your Data Neo4j Graph Day - Santa Clara 2017 William Lyon @lyonwj A Worked Example
  3. 3. William Lyon Developer Relations Engineer @neo4j will@neo4j.com @lyonwj lyonwj.com
  4. 4. Agenda • Graph database - developer perspective • (relational)-[:TO]->(graph) • Use cases • Fraud Detection • Recommendations • Developer tooling
  5. 5. Neo4j Graph Database • Property graph data model • (open)Cypher query language • Native graph processing • Language drivers • Open source neo4j.com
  6. 6. Graph Data Model
  7. 7. Labeled Property Graph Model
  8. 8. Labeled Property Graph Model
  9. 9. The Graph https://github.com/johnymontana/neo4j-datasets/tree/master/yelp
  10. 10. The Graph https://www.youtube.com/watch?v=kSZHFlBDIfM
  11. 11. The Graph https://github.com/johnymontana/mattermark-graphql-neo4j
  12. 12. The Graph https://github.com/neo4j-meetups/modeling-worked-example
  13. 13. The Graph https://neo4j.com/sandbox-v2/
  14. 14. openCypher (queryLanguage)-[:FOR]->(graphs) http://www.opencypher.org/
  15. 15. Cypher
  16. 16. https://neo4j.com/sandbox-v2/
  17. 17. Use Case: Fraud Detection
  18. 18. Organized in groups Synthetic Identities Stolen Identities Hijacked Devices Who Are Today’s Fraudsters?
  19. 19. Fraud Rings and Synthetic Identities
  20. 20. ACCOUNT HOLDER 2 Modeling a fraud ring as a graph ACCOUNT HOLDER 1 ACCOUNT HOLDER 3
  21. 21. ACCOUNT HOLDER 2 ACCOUNT HOLDER 1 ACCOUNT HOLDER 3 CREDIT CARD BANK ACCOUNT BANK ACCOUNT BANK ACCOUNT PHONE NUMBER UNSECURED LOAN SSN 2 UNSECURED LOAN Modeling a fraud ring as a graph
  22. 22. ACCOUNT HOLDER 2 ACCOUNT HOLDER 1 ACCOUNT HOLDER 3 CREDIT CARD BANK ACCOUNT BANK ACCOUNT BANK ACCOUNT ADDRESS PHONE NUMBER PHONE NUMBER SSN 2 UNSECURED LOAN SSN 2 UNSECURED LOAN Modeling a fraud ring as a graph
  23. 23. https://neo4j.com/graphgist/9d627127-003b-411a-b3ce-f8d3970c2afa?ref=solutions https://neo4j.com/graphgists/
  24. 24. Credit Card Fraud
  25. 25. (relational)-[:TO]->(graph)
  26. 26. FROM RDBMS TO GRAPHS
  27. 27. Northwind
  28. 28. Northwind - the canonical RDBMS Example
  29. 29. ( )-[:TO]->(Graph)
  30. 30. ( )-[:IS_BETTER_AS]->(Graph)
  31. 31. Starting with the ER Diagram
  32. 32. Locate the Foreign Keys
  33. 33. Drop the Foreign Keys
  34. 34. Find the JOIN Tables
  35. 35. (Simple) JOIN Tables Become Relationships
  36. 36. Attributed JOIN Tables -> Relationships with Properties
  37. 37. As a Graph
  38. 38. https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/master/readme.adoc
  39. 39. Northwind
  40. 40. Northwind
  41. 41. Neo4j Language Drivers
  42. 42. Neo4j Drivers https://neo4j.com/developer/language-guides/
  43. 43. POWERING AN APP
  44. 44. Simple App
  45. 45. Simple App
  46. 46. Simple Python Code
  47. 47. Simple Python Code
  48. 48. Simple Python Code
  49. 49. Simple Python Code
  50. 50. https://neo4j.com/developer/
  51. 51. Use Case: Personalized Recommendations
  52. 52. Recommendations drive user engagement
  53. 53. “35 percent of what consumers purchase on Amazon and 75 percent of what they watch on Netflix come from product recommendations” http://www.mckinsey.com/industries/retail/our-insights/how-retailers-can-keep-up-with-consumers
  54. 54. Product Recommendations Effective product recommendation algorithms has become the new standard in online retail — directly affecting revenue streams and the shopping experience. Logistics/Delivery Routing recommendations allows companies to save money on routing and delivery, and provide better and faster service. Promotion recommendations Building powerful personalized promotion engines is another area within retail that requires input from multiple data sources, and real-time, session based queries, which is an ideal task to solve with Neo4j. Today Recommendation Engines are At the Core of Digitization in Retail
  55. 55. Powerful recommendation engines rely on the connections between multiple sources of data
  56. 56. Powerful recommendation engines rely on the connections between multiple sources of data
  57. 57. Dreamhouse Series 15% off The Store Search Hi, login My Account People who bought Side Table also bought: Coffee Table $235 Low Book Shelf $150 Bed Side Table $90 Mobile Brick & Mortar Multi-Channel Web The Store People who bought Side Table also bought: Similar product in from Home Office Series: Hi, login My AccountSearch Dreamhouse Series 15% off All departments Living room | Kitchen | Hallway | Lightning | Bedroom | Garden | Home Office Space Track Orders |  Gift Cards |  Store finder | Credit Card | Grocery Pickup |  Help Wood Side Table $110 Green Side Table $135 Walnut Side Table $120 Coffee Table $235 Low Book Shelf $150 Bed Side Table $90 Product Recommendations
  58. 58. Dreamhouse Series 15% off The Store Search Hi, login My Account People who bought Side Table also bought: Coffee Table $235 Low Book Shelf $150 Bed Side Table $90 Mobile Brick & Mortar Web The Store People who bought Side Table also bought: Similar product in from Home Office Series: Hi, login My AccountSearch Dreamhouse Series 15% off All departments Living room | Kitchen | Hallway | Lightning | Bedroom | Garden | Home Office Space Track Orders |  Gift Cards |  Store finder | Credit Card | Grocery Pickup |  Help Wood Side Table $110 Green Side Table $135 Walnut Side Table $120 Coffee Table $235 Low Book Shelf $150 Bed Side Table $90
  59. 59. The Store People who bought Side Table also bought: Similar product in from Home Office Series: Hi, login My AccountSearch Dreamhouse Series 15% off All departments Living room | Kitchen | Hallway | Lightning | Bedroom | Garden | Home Office Space Tra c k O rd e r s |   G i f t C a rd s |   S t o re fi n d e r | C re d i t C a rd | G ro c e r y P i c k u p |   H e lp Wood Side Table $110 Green Side Table $135 Walnut Side Table $120 Coffee Table $235 Low Book Shelf $150 Bed Side Table $90 Personalized Promotions Personalized Real-Time Recommendations Personalized Real-Time Recommendations
  60. 60. People who bought Side Table also bought: Similar product in from Home Office Series: Wood Side Table $110 Green Side Table $135 Walnut Side Table $120 Coffee Table $235 Low Book Shelf $150 Bed Side Table $90 Data-Model (Expressed as a graph) Category Category Product Product Product Collaborative Filtering An algorithm that considers users interactions with products, with the assumption that other users will behave in similar ways. Algorithm Types Content Based An algorithm that considers similarities between products and categories of products. Customer Customer Product Product Product
  61. 61. Category Price ConfigurationsLocation Silos & Polyglot Persistence Purchase ViewReviewReturn In-store PurchasesInventory Products Customers / Users Location Purchases RELATIONAL DB WIDE COLUMN STORE Views DOCUMENT STORE User Review RELATIONAL DB In-Store Purchase Shopping Cart KEY VALUE STORE Product Catalogue DOCUMENT STORE
  62. 62. Purchases RELATIONAL DB WIDE COLUMN STORE Views DOCUMENT STORE User Review RELATIONAL DB In-Store Purchase Shopping Cart KEY VALUE STORE Product Catalogue DOCUMENT STORE Silos & Polyglot Persistence Category Price ConfigurationsLocation Purchase ViewReviewReturn In-store PurchasesInventory Products Customers / Users Location
  63. 63. Purchases RELATIONAL DB WIDE COLUMN STORE Views DOCUMENT STORE User Review RELATIONAL DB In-Store Purchase Shopping Cart KEY VALUE STORE Product Catalogue DOCUMENT STORE Category Price ConfigurationsLocation Polyglot Persistence Purchase ViewReviewReturn In-store PurchasesInventory LocationCategory Price ConfigurationsLocation Purchase ViewReviewReturn In-store PurchasesInventory Products Customers / Users Location
  64. 64. Data Lake Purchases RELATIONAL DB Product Catalogue DOCUMENT STORE WIDE COLUMN STORE Views DOCUMENT STORE User Review RELATIONAL DB In-Store Purchase Shopping Cart KEY VALUE STORE Recommendations require an operational workload — it’s in the moment, real-time! Good for Analytics, BI, Map Reduce Non-Operational, Slow Queries
  65. 65. Purchases RELATIONAL DB Product Catalogue DOCUMENT STORE WIDE COLUMN STORE Views DOCUMENT STORE User Review RELATIONAL DB In-Store Purchase Shopping Cart KEY VALUE STORE Connector Drivers: Java |  JavaScript |  Python |  .Net |  PHP |  Go |  Ruby Apps and Systems Real-Time Queries
  66. 66. Using Data Relationships for Recommendations Content-based filtering Recommend items based on what users have liked in the past Collaborative filtering Predict what users like based on the similarity of their behaviors, activities and preferences to others Movie Person Person RATED SIMILARITY rating: 7 value: .92
  67. 67. Collaborative Filtering
  68. 68. Collaborative Filtering
  69. 69. Collaborative Filtering
  70. 70. Collaborative Filtering Link prediction
  71. 71. In Cypher
  72. 72. In Cypher Basic initial approach. Improvements: - aggregate across all purchases - scoring / normalize - compute similarity metrics
  73. 73. Content Filtering
  74. 74. Content Filtering
  75. 75. Content Filtering w/ Cypher
  76. 76. Content Filtering - Concept Hierarchy
  77. 77. Content Filtering - Concept Hierarchy
  78. 78. Content Filtering - Concept Hierarchy
  79. 79. Content Filtering - Concept Hierarchy w/ Cypher
  80. 80. Content Filtering - Concept Hierarchy w/ Cypher Basic initial approach. Improvements: - aggregate across all purchases - cold start - variable length concept hierarchy - tag similarity / clusters
  81. 81. Neo4j Sandbox neo4jsandbox.com
  82. 82. Want to learn more? graphdatabases.com neo4j.com/sandbox
  83. 83. (you)-[:HAVE]->(?) (?)<-[:ANSWERS]-(will)
  84. 84. 09:00-09:30 09:30-10:15 10:15-11:00 11:00-11:30 11:30-12:30 12:30-13:30 13:30-17:00 Breakfast and Registration The Connected Data Imperative: Why Graphs Transform Your Data: A Worked Example Break Enterprise Ready: A Look at 
 Neo4j in Production Lunch Hands-On Training Session Agenda APRIL 26, 2017 SANTA CLARA

×