Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

GraphTour - Next generation solutions using Neo4j

478 views

Published on

GraphTour Tel Aviv
Stefan Kolmar, Neo4j

Published in: Technology
  • Be the first to comment

GraphTour - Next generation solutions using Neo4j

  1. 1. Next Generation Solutions built on Neo4j Stefan Kolmar, VP Field Engineering Feb 2018
  2. 2. TEL AVIV, 13 FEBRUARY, 2018 Agenda ● Solutions using Neo4j ● Fraud ● Recommendations ● Conclusions
  3. 3. TEL AVIV, 13 FEBRUARY, 2018 Solutions: new mindset required?
  4. 4. TEL AVIV, 13 FEBRUARY, 2018 Solutions: new mindset Yesterday: - Static Applications - Designed to fulfill current requirements - Performance Constraints - Domain experts versus IT experts Tomorrow: - Flexible Applications - Designed to fulfill tomorrows requirements - Performance is not limiting - Domain experts work hand in hand with IT experts
  5. 5. TEL AVIV, 13 FEBRUARY, 2018 Evolution using Neo4j Neo4j Platform Graph Transactions Graph Analytics Data Integration Development & Admin Analytics Tooling Drivers & APIs Discovery & Visualization Developers Admins Applications Business Users Data Analysts Data Scientists 3rd Party Tools “The Graph Advantage” Domain know-how Professional Services PS Packages Graph Based Solution
  6. 6. TEL AVIV, 13 FEBRUARY, 2018 Evolution using Neo4j Neo4j Platform Graph Transactions Graph Analytics Data Integration Development & Admin Analytics Tooling Drivers & APIs Discovery & Visualization Developers Admins Applications Business Users Data Analysts Data Scientists 3rd Party Tools “The Graph Advantage” Domain know-how Professional Services PS Packages Graph Based Solution Neo4j enables Graph Based Solutions with a need for: - Agility - Intuitiveness - High Performance to support connected data scenarios - Scalable on traversing through connected data
  7. 7. TEL AVIV, 13 FEBRUARY, 2018 Look at this data…
  8. 8. TEL AVIV, 13 FEBRUARY, 2018 Swap glasses…
  9. 9. TEL AVIV, 13 FEBRUARY, 2018 … now look at it again, this time as a graph
  10. 10. Speed: Real time query enabled Graph Based Solutions Enables Up-Sell / Cross-sell Key Features Added Value 360 degree view on data Using data Connections as a value Intuitive: Supports Business Needs Flexible: enabled for additional requirements Finding patterns within the data Detect anomalies Prevent rather than detect Enables conversation across Functions Comply to regulations What-if Analysis Telco OSS GDPR Fraud Telco BSS Recomm endations MDM Resource efficient
  11. 11. TEL AVIV, 13 FEBRUARY, 2018 Fraud Detection Building Powerful Solutions to prevent Fraud Based on Neo4j
  12. 12. TEL AVIV, 13 FEBRUARY, 2018 The Impact of Fraud The payment card fraud alone, constitutes for over 16 billion dollar in losses for the bank-sector in the US. $16Bpayment card fraud in 2014* Banking $32Byearly e-commerce fraud** Fraud in E-commerce is estimated to cost over 32 billion dollars annually is the US.. E-commerce The impact of fraud on the insurance industry is estimated to be $80 billion annually in the US. Insurance $80Bestimated yearly impact*** *) Business Wire: http://www.businesswire.com/news/home/20150804007054/en/Global-Card-Fraud-Losses-Reach-16.31-Billion#.VcJZlvlVhBc **) E-commerce expert Andreas Thim, Klarna, 2015 ***) Coalition against insurance fraud: http://www.insurancefraud.org/article.htm?RecID=3274#.UnWuZ5E7ROA
  13. 13. TEL AVIV, 13 FEBRUARY, 2018 Who Are Today’s Fraudsters?
  14. 14. TEL AVIV, 13 FEBRUARY, 2018 Organized in groups Synthetic Identities Stolen Identities Who Are Today’s Fraudsters? Hijacked Devices
  15. 15. TEL AVIV, 13 FEBRUARY, 2018 “Don’t consider traditional technology adequate to keep up with criminal trends” Market Guide for Online Fraud Detection, April 27, 2015
  16. 16. TEL AVIV, 13 FEBRUARY, 2018 Endpoint-Centric Analysis of users and their end-points 1. Navigation Centric Analysis of navigation behavior and suspect patterns 2. Account-Centric Analysis of anomaly behavior by channel 3. PC:s Mobile Phones IP-addresses User ID:s Comparing Transaction Identity Vetting Traditional Fraud Detection Methods
  17. 17. TEL AVIV, 13 FEBRUARY, 2018 Unable to detect • Fraud rings • Fake IP-adresses • Hijacked devices • Synthetic Identities • Stolen Identities • And more… Weaknesses DISCRETE ANALYSIS Endpoint-Centric Analysis of users and their end-points 1. Navigation Centric Analysis of navigation behavior and suspect patterns 2. Account-Centric Analysis of anomaly behavior by channel 3. Traditional Fraud Detection Methods
  18. 18. TEL AVIV, 13 FEBRUARY, 2018 INVESTIGATE Revolving Debt Number of Accounts INVESTIGATE Normal behavior Fraud Detection With Discrete Analysis
  19. 19. TEL AVIV, 13 FEBRUARY, 2018 Revolving Debt Number of Accounts Normal behavior Fraud Detection With Connected Analysis Fraudulent pattern
  20. 20. TEL AVIV, 13 FEBRUARY, 2018 CONNECTED ANALYSIS Augmented Fraud Detection Endpoint-Centric Analysis of users and their end-points Navigation Centric Analysis of navigation behavior and suspect patterns Account-Centric Analysis of anomaly behavior by channel DISCRETE ANALYSIS 1. 2. 3. Cross Channel Analysis of anomaly behavior correlated across channels 4. Entity Linking Analysis of relationships to detect organized crime and collusion 5.
  21. 21. Preventing Fraud Networks of People Processes and Transactions Ownership E.g. e-commerce Fraud, AML E.g. detecting fraud rings, finding connections and shortest paths E.g. AML, tax fraud, legal entities Data connections assist the business by identifying patterns
  22. 22. TEL AVIV, 13 FEBRUARY, 2018 The Power of Cypher Fraud Ring: MATCH ring = (suspect:AccountHolder)-[*]->(contactInformation)<-[*..5]-(:AccountHolder)-[*]->(suspect) RETURN ring
  23. 23. TEL AVIV, 13 FEBRUARY, 2018 Top Tier Electronic Payment Services Case studyApply to AML regulations Challenge • Needed to apply to AML regulation • Unability to provide reports out of RDBMS leading systems Transactions fragmented and transfered „from rings to rings“ • Neo4j is used to store and report on transaction over previous 24 months • Business Users / Fraud Analysts are enabled to investigate data and detect patters Use of Neo4j • Complies to Regulations • Neo4j also enabled the company to detect potential AML usage early and act against them “We have been unable to detect AML fraud patterns in the SQL based operational systems. Graphs and Graph visualisation is a key enabler technology.” – Top Tier Payment Service Result/Outcome
  24. 24. TEL AVIV, 13 FEBRUARY, 2018 What about Machine Learning?
  25. 25. TEL AVIV, 13 FEBRUARY, 2018 What about Machine Learning? Neo4j is an enabler technology: • Automized detection of Fraud patterns via Cypher • Detecting Paths • Graph Algorithms (eg Centrality, Community) • Algorithms as background tasks -> mark corresponding nodes • Automatically cancel Business Transactions • Score identified patterns and weigh • ….
  26. 26. TEL AVIV, 13 FEBRUARY, 2018 Why Graph is Superior for Fraud DetectionFraud Requirement Traditional Approaches Neo4j Approach Find connected data patterns over unlimited amount of „hops“ Complex queries with hundreds of join tables Simple single query traverses all enterprise systems Real-time acting on incoming events in ever changing formats for potential fraud Limitations inherited from SQL Database Schema Schema free database enables to connect any nodes with each other Effort required to add new data and systems Days to weeks to rewrite schema and queries Draw new data connections on the spot Time to deployment Months to years Weeks to months Response time to Fraud requests Minutes to hours per query Milliseconds per query Form of Fraud Incidents / Investigations Text reports that are not visual and prove very little Visuals patterns and the path to follow through your system Bottom line Long, ineffective and expensive Easy, fast and affordable
  27. 27. TEL AVIV, 13 FEBRUARY, 2018 How Neo4j fits into your environment
  28. 28. TEL AVIV, 13 FEBRUARY, 2018 Money Transferring Purchases Bank Services Relational database Develop Patterns Data Science-team + Good for Discrete Analysis – No Holistic View of Data-Relationships – Slow query speed for connections
  29. 29. TEL AVIV, 13 FEBRUARY, 2018 Money Transferring Purchases Bank Services Relational database Data Lake + Good for Map Reduce + Good for Analytical Workloads – No holistic view – Non-operational workloads – Weeks-to-months processes Develop Patterns Data Science-team Merchant Data Credit Score Data Other 3rd Party Data
  30. 30. TEL AVIV, 13 FEBRUARY, 2018 Money Transferring Purchases Bank Services Neo4j powers 360° view of transactions in real-time Neo4j Cluster SENSE Transaction stream RESPOND Alerts & notification LOAD RELEVANT DATA Relational database Data Lake Visualization UI Fine Tune Patterns Develop Patterns Data Science-team Merchant Data Credit Score Data Other 3rd Party Data
  31. 31. TEL AVIV, 13 FEBRUARY, 2018 Money Transferring Purchases Bank Services Neo4j powers 360° view of transactions in real-time Neo4j Cluster SENSE Transaction stream RESPOND Alerts & notification LOAD RELEVANT DATA Relational database Data Lake Visualization UI Fine Tune Patterns Develop Patterns Data Science-team Merchant Data Credit Score Data Other 3rd Party Data Data-set used to explore new insights
  32. 32. TEL AVIV, 13 FEBRUARY, 2018 Example Fraud Solution Architecture
  33. 33. Neo4j Database Cluster Data Visualization Neo4j APOC Fraud Detection Algorithms Management Dashboard Neo4j Bolt Driver Data Ingest Mgmt. … Customer Data Sources / Systems / Applications Legend: Neo4j Provided Components Custom built Neo4j/Customer Customer/SI Fraud Reports Real Time Alerts Batch Data Buffering (Queue) Real-Time Neo4j BrowserAdmin UI UI for Fraud Analysis System Specific Adapters / Scripts / Connecters Fraud Analysts Admin / SuperuserFraud Analysts Fraud Analysts
  34. 34. TEL AVIV, 13 FEBRUARY, 2018 Neo4j powered Fraud Solution Characteristic Benefit for Fraud Solution • Agility • Constant catch up with fraudster techniques supported • Enabled for Future Requirements • Solution can be built iteratively • Fast implementation cycles • Schema free DB supports “connect anything” • Intuitiveness • Enable Fraud Analysts to use Technology • Using visualization to detect pattern • Drilling into suspicious patterns • Speed • Unlimited number of traversals to detect complex connections within the data • Response time enables fraud prevention • Leverage Data Connections • 360 degree customer view enabled / provided • Scalability • Hardware efficiency with real-time patterns • TCO/ROI • Adding on top of existing infrastructure protects investments
  35. 35. TEL AVIV, 13 FEBRUARY, 2018 Recommendation EnginesBuilding Powerful Recommendation Engines With Neo4j
  36. 36. TEL AVIV, 13 FEBRUARY, 2018 “If you liked this, you might like that…” Powerful, real-time, recommendations and personalization engines have become fundamental for creating superior user experience and commercial success in retail
  37. 37. TEL AVIV, 13 FEBRUARY, 2018 Creating Relevance in an Ocean of Possibilities
  38. 38. TEL AVIV, 13 FEBRUARY, 2018 How Graph Based Recommendations Transformed the Consumer Web People Graph “People you may know” Disruptor: Facebook Industry: Media Ad-business Disruptor: Amazon Industry: Retail People & Products “Other people also bought” People & Content “You might also like” Disruptor: Netflix Industry: Broadcasting Media
  39. 39. TEL AVIV, 13 FEBRUARY, 2018 Today Recommendation Engines are At the Core of Digitization in Retail Product Recommendations Effective product recommendation algorithms has become the new standard in online retail — directly affecting revenue streams and the shopping experience. Logistics/Delivery Routing recommendations allows companies to save money on routing and delivery, and provide better and faster service. Promotion recommendations Building powerful personalized promotion engines is another area within retail that requires input from multiple data sources, and real-time, session based queries, which is an ideal task to solve with Neo4j.
  40. 40. TEL AVIV, 13 FEBRUARY, 2018 ... and Recommendation Engines are at the core of: Content Recommendations Content recommendation algorithms are the basis to use portals providing value added content — directly affecting the behaviour of the users and have them stay on the web page Fraud Taking timely action based on patterns / recommendations you find inside connected data . May require input from multiple data sources, and real-time, session based queries, which is an ideal task to solve with Neo4j. Social Networks Building powerful personalized engines to recommend new contacts, friends, based on patterns, preferences, status „friends-of-friends“ taking advantage of the value of connected data
  41. 41. TEL AVIV, 13 FEBRUARY, 2018 Why Graph Based Recommendation Engines? • Increase revenue • Create Higher Engagement • Mitigate RiskValue • Real-Time capabilities • Ability to use the most recent transaction data • Flexibility to incorporate new data sourcesPerformance
  42. 42. TEL AVIV, 13 FEBRUARY, 2018 The Impact of Bad Recommendations Characteristic Impact for Recommendations Examples • “Batch Oriented Recommendation” • Unable to react on real-time changes • Unable to fulfill real-time needs • Recommending “out-of-stock” products • Content recommendation, eg news: the latest news are the most important ones • Lack of Performance • Recommendation slow down the user interaction • Recommendation alternatives limited • Delayed response time lead to customer dissatisfaction • Recommend just the obvious (“similarities”) and inability to recommend more complex scenarios (Account specific and product specific and buying history and …) • Limited by Data Connections • Recommendations are limited by number of hops • Inability to recommend more complex correlations (eg product hierarchies and dependencies) • No complex recommendation algorithms supported • Missing Feedback Loop • Inability to react on Feedback • Customer never picks Top 3 recommendations • Recommendations are getting meaningless • No Graph Algorithm support • Limitations on Machine Learning approaches • “Centrality” for Products to be recommend can be essential
  43. 43. TEL AVIV, 13 FEBRUARY, 2018 Case Studies
  44. 44. TEL AVIV, 13 FEBRUARY, 2018 Case studySolving real-time recommendations for the World’s largest retailer. Challenge • In its drive to provide the best web experience for its customers, Walmart wanted to optimize its online recommendations. • Walmart recognized the challenge it faced in delivering recommendations with traditional relational database technology. • Walmart uses Neo4j to quickly query customers’ past purchases, as well as instantly capture any new interests shown in the customers’ current online visit – essential for making real-time recommendations. Use of Neo4j “As the current market leader in graph databases, and with enterprise features for scalability and availability, Neo4j is the right choice to meet our demands”. - Marcos Vada, Walmart • With Neo4j, Walmart could substitute a heavy batch process with a simple and real-time graph database. Result/Outcome
  45. 45. TEL AVIV, 13 FEBRUARY, 2018 Case studyeBay Now Tackles eCommerce Delivery Service Routing with Neo4j Challenge • The queries used to select the best courier for eBays routing system were simply taking too long and they needed a solution to maintain a competitive service. • The MySQL joins being used created a code base too slow and complex to maintain. • eBay is now using Neo4j’s graph database platform to redefine e-commerce, by making delivery of online and mobile orders quick and convenient. Use of Neo4j • With Neo4j eBay managed to eliminate the biggest roadblock between retailers and online shoppers: the option to have your item delivered the same day. • The schema-flexible nature of the database allowed easy extensibility, speeding up development. • Neo4j solution was more than 1000x faster than the prior MySQL Soltution. Our Neo4j solution is literally thousands of times faster than the prior MySQL solution, with queries that require 10-100 times less code. Result/Outcome – Volker Pacher, eBay
  46. 46. TEL AVIV, 13 FEBRUARY, 2018 Example Recommendation Solution Architecture
  47. 47. Neo4j Database Cluster Neo4j APOC Recommen dation Algorithms (Scheduled) Management Dashboard Neo4j Bolt Driver Data Ingest Mgmt. … Customer Data Sources / Systems / Applications Legend: Neo4j Provided Components Custom built Neo4j/Customer Customer/SI Batch Data Buffering (Queue) Real-Time Admin UI System Specific Adapters / Scripts / Connecters Admin / Superuser Apps Websites Affiliate Programs Points of sale User Interface Retail Web Shop functionality / Shipment / etc.
  48. 48. TEL AVIV, 13 FEBRUARY, 2018 Why Graph is Superior for Recommendation Engines Recommendation Requirement Traditional Approaches Neo4j Approach Usage of connected data over unlimited amount of „hops“ Complex queries with hundreds of join tables Simple single query traverses all enterprise systems Real-time 360 degree view on data within your System Performance limitations with increasing number of connections / hops Traversing over connections in near real-time provided Effort required to add additional data sources to support reco Days to weeks to rewrite schema and queries Minutes to draw new data connections Time to deployment Months to years Weeks to months Response time to Recommendations Minutes to hours per query Milliseconds per query Machine Learning Enablement Static Database scheme leads to static processes ML algorithms can use Graph algorithms and take advantage of connected data Bottom line Long, ineffective and expensive Easy, fast and affordable
  49. 49. TEL AVIV, 13 FEBRUARY, 2018 What about Machine Learning?
  50. 50. TEL AVIV, 13 FEBRUARY, 2018 What about Machine Learning? Neo4j is an enabler technology: • Detecting Recommendation patterns via Cypher queries • Recommendation Algorithms based on scores • Feedback loop • Learn from feedback (eg never used “friends recommendation”) and change scoring • Algorithm to automatically add relevant connections • ….
  51. 51. TEL AVIV, 13 FEBRUARY, 2018 Neo4j powered Recommendation Engine Characteristic Benefit for Recommendation Solution • Agility • Constant learning of recommendations given feedback enabled • Enabled for Future Requirements • Solution can be built iteratively • Fast implementation cycles • Schema free DB supports “connect anything” • Intuitiveness • Enable Business Analysts to use technology • All channels and data sources can be easily connected • Speed • Unlimited number of traversals to detect potential recommendations • Response time enables fraud prevention • Leverage Data Connections • 360 degree customer view enabled / provided • Scalability • Hardware efficiency with real-time patterns • TCO/ROI • Adding on top of existing infrastructure protects investments
  52. 52. TEL AVIV, 13 FEBRUARY, 2018 Conclusion (graphs)-[:ARE]-> (everywhere) and (Solutions)-[:NEED]-> (graphs)
  53. 53. TEL AVIV, 13 FEBRUARY, 2018 Who can help? Neo4j Platform Graph Transactions Graph Analytics Data Integration Development & Admin Analytics Tooling Drivers & APIs Discovery & Visualization Developers Admins Applications Business Users Data Analysts Data Scientists 3rd Party Tools “The Graph Advantage” Domain know-how Professional Services PS Packages Graph Based Solution Professional Services: - Extend and leverage Domain Expertise - Best Practices - Using Building Blocks - Don’t “re-invent the wheel” - Speed up development and deployment - Access to Neo4j infrastructure (Development, Support, Product management)
  54. 54. Next Generation Solutions built on Neo4j Stefan Kolmar, VP Field Engineering Feb 2018

×