Building Intelligent Solutions with Graphs, Stefan Kolmar, Neo4j
•
1 like•656 views
GraphTour Europe 2019: Building Intelligent Solutions with Graphs, Stefan Kolmar, VP Field Engineering, Neo4j
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Building Intelligent Solutions with Graphs, Stefan Kolmar, Neo4j
Similar to Building Intelligent Solutions with Graphs, Stefan Kolmar, Neo4j (20)
More from Neo4j
More from Neo4j (20)
Recently uploaded
Recently uploaded (20)
Building Intelligent Solutions with Graphs, Stefan Kolmar, Neo4j
- 1. Building Intelligent Solutions with Graphs Stefan Kolmar VP Field Engineering EMEA & APAC
- 2. • Neo4j services • Solutions and Managed Services • Adding AI/ML to Solutions • Real world examples and best practices 2 Agenda
- 4. 4 Neo4j Professional Services Training how to efficiently use Build a transport vehicle with (or for) you
- 5. 5 Neo4j PS Professional Services Offer Training & Enablement Solution Delivery and Management Packaged Services Typically 5-25 days Neo4j advises Customer builds 80% of engagements Custom Scoped 50+ days Neo4j delivers Customer supports 20% of engagements
- 6. PROFESSIONAL SERVICES GRAPH ACADEMY SOLUTIONS CUSTOMER SUPPORT ● Packaged Services ● Staff Augmentation ● Project/Solution Delivery ● Class room training ● Online/Virtual training ● Certification ● Innovation Labs ● Solution Workshops ● Solutions Development ●24x7x365 & KB ●Platinum support ●Cloud Managed Services ●DBaas (NEW) ●Agile Solution Support Training Enablement Solution Delivery and Mgmt Organisation and offerings
- 7. 7 Packaged Services Project Lifecyle Graph Awareness Technical Assessment Solution Implementation Roll-out / Production INNOVATION LAB BOOTCAMP SOLUTION DESIGN WORKSHOP SOLUTION AUDIT STAFF AUGMENTATION PRODUCT TRAINING
- 9. 9
- 10. „… an IT solution is an aggregation of products and services, as opposed to a single, discrete product ... which helps to solve a particular problem“ https://searchitchannel.techtarget.com/definition/solution 10 What is an IT Solution?
- 11. - Performance - Agility - Value connections - 360 degree views (customer, product etc) - Finding patterns / anomalies - Sysadmin friendly - Hardware efficiency - Intuitivity 11 Technical Requirements
- 12. 12 From use case to solution delivery Solution accelerators
- 14. 14 What is needed? Solution accelerators SOLUTION (FOUNDATION) FRAMEWORK DELIVERY METHODOLOGY SKILLS & RESOURCES
- 15. Solution (Foundation) Framework Neo4j Graph Platform Recom Framework Custom App Solution Foundation Framework Neo4j Data Orchestrator Framework Neo4j Deployment Framework Neo4j Managed Service Fraud Framework Network Mgmt Framework Custom App Custom App Custom App Custom App Custom App Neo4j Version Management Service
- 16. Solution (Foundation) Framework Neo4j Graph Platform Recom Telco App Solution Foundation Framework Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers App App
- 17. Solution (Foundation) Framework Neo4j Graph Platform Recom Telco App Solution Foundation Framework Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers App App API dev 3rd party graph viz Custom dev - graph viz libraries 3rd party analytics Python, Java ML, ... Kettle 3rd party DI/EAI Docker Kubernetes Git Lineage Kettle
- 18. UI /GRAND stack API Layer Neo4j Data Orchestrator APOC Trigge r Bloom Apollo Client ReactJS ( ReactGraphVis) VisJS Dashboards Business Logic (JS) Neo4j Graph Server Apollo Business Logic (Server) Cypher
- 20. Solution (Foundation) Framework Neo4j Graph Platform Recom Telco App Solution Foundation Framework App App Availability Demand Framing & Collateral Sales Demo Solution Beta Repeatable Solution Recommendation Framework HCM Framework Privacy Shield Framework Risk Mgmt Framework Fraud Framework C360 Framework CRM Framework Network Mgmt Bill Of Materials
- 22. Network Management (Telco) Graph Transactions Element Manager Geography Service definitions Kettle Customer data ... Dependency Analysis Spatial queries and path exploration Capacity Analysis Fulfillment Assurance > Impact analysis > Event correlation > Root cause analysis Predefined CYPHER queries (and API) metadata
- 23. 23 Managed Solutions Cloud Managed Services Agile Solution Support
- 24. Project definition Solution Design Workshop Deploy Agile Sprints Solution Delivery Methodology 24 Product backlog Backlog Product Increment Main building blocks ● Project definition: clarity about objectives and organisation ● Solution design workshop: requirements and high level design ● Solution Delivery ○ Agile/SCRUM ○ Traditional / Waterfall ● (Regular) Releases / ● Solution support Solution Support Waterfall
- 25. Main artefacts Small Medium Large Scoping SoW ● ● ● Project def Project definition doc ● ● Design Graph model ● ● ● UI design ● Technical Architecture ● ● ● Backlog / Project plan ● ● ● Sprints Sprint backlog ● ● User Stories ● Roll-out User guide ● Ops guide ● ● Support SLA ● Support document ●
- 26. The picture can't be displayed. Architecture and design Roll-out and deploy Project management Operations Management IntegrationAPILogicModel Skills & Methods 26 Database
- 28. 28
- 29. Where AI and ML fit in 29 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI
- 30. Differences between ML and Analytics 30 Machine learning: • Determine domain parameters • Historical-based discoveries • Learn and improve without explicit programming
- 31. Graph analytics: • Uses inherent graph structures • Uncover real-world networks through their connections • Forecast complex network behavior and identify action Differences between ML and Analytics
- 32. Today challenges with Machine Learning: • Doesn’t take multiple relationship hops into account • Takes time to iteratively train a model • Computational inefficiency of connecting data Benefits of Mixing Graph Analytics with ML Graphs bring: • Context to machine learning • Feature filtration • Connected feature extraction
- 33. Neo4j has an ‘out of the box’ Graph Algorithms plugin: • Path finding and Search • Centrality and Importance • Community Detection • Similarity and Machine Learning Workflow Many different ways to work with your ML algorithms in Neo4j: • Support for many languages (Python, .Net, Java, Go, Ruby, etc.) • Different data integration options • Triggers, event-driven architecture, user-defined functions 34 Working with Graph Analytics and ML
- 34. Knowledge graph example: • Using topic finding ML processes (e.g. Latent Dirichlet Allocation) • Feeding the output into a graph database • Search for topics, find related concepts, etc. 35 Graph and Machine Learning Examples Recommendation engine example: • Use ML processes such as collaborative filtering • Enrich graph with the output • Use graph as feedback for future iterations
- 35. IoT/Connected Home: • Master Data Management • Entity resolution using community detection and similarity Customer Experience Management: • Customer journey path analysis (path finding) 36 Graph Analytics and Algorithm Examples 36 CONNECTS CONNECTS C O N N EC T S RUNS_ON CONNECTS RUNS_ON CONNECTSC O N N EC TS CONNECTS RUNS_ON RUNS_ON RUNS_ONC O N N EC TS ZONE ZONE ZONE DEVICE DEVICE DEVICE DEVICE DEVICE DEVICE APP APP
- 36. Finding connections (pathfinding) • How are these two parties connected? 37 Graph Algorithm Examples Finding influencers (community detection) • Will this individual’s friends churn? Finding key components (centrality) • What components do we depend upon? What can’t fail?
- 37. Putting it all Together
- 39. Our Neo4j activity implementation has led to a great decrease in complexity, storage, and infrastructure costs. Our full dataset size is now around 40 GB, down from 50 TB of data that we had stored in Cassandra. We’re able to power our entire activity feed infrastructure using a cluster of 3 Neo4j instances, down from 48 Cassandra instances of pretty much equal specs. That has also led to reduced infrastructure costs. Most importantly, it’s been a breeze for our operations staff to manage since the architecture is simple and lean.” David Fox, Adobe, Oct 2018 40 Customer Quote How can Neo4j Services help you to get there?
- 40. Customer Use Case: • Leading online platform to showcase and discover creative work • More than 10 million members • Allows creatives to share their work with millions of daily visitors • Highlights Adobe software used in the creation process • Drives people to the Adobe Creative Cloud • Social platform for discovery, learning, and more 41 Adobe – Project Behance Activity feed: • Mongo DB (2011) - 125 nodes, dataset size of about 20tb (terabytes) • Cassandra (2015) - 48 nodes, dataset size of about 50tb (terabytes) • Neo4j (2018) - 3 nodes, dataset size of 33gb (gigabytes) 5 day BOOTCAMP
- 41. 42 Large Chemical Producer and Supplier Customer Journey 5 day Bootcamp Educate Customer on Graph technology Demonstrate that BOM use cases are a good fit for Neo4j Build a small graph prototype
- 42. 43 Large Chemical Producer and Supplier Customer Journey 5 day Bootcamp 30 day POC Load subset of real life data (C4) Build a GUI prototype to demonstrate value to end-users. Initial roll-out to power users
- 43. 44 Large Chemical Producer and Supplier Customer Journey 5 day BOOTCAMP 30 day POC FULL PROJECT OWNERSHIP (200md) Data for multiple production chains Improved GUI Add profit margin calculation across value chain
- 44. 45 Large Commercial Bank Customer Journey INNOVATION LAB STAFF AUGMENTATION CAMPAIGN MGMT INNOVATION LAB FRAUD PROJECT 80 mandays TBD YET ANOTHER INNOVATION LAB
- 45. 46 Conclusion • Neo4j PS makes customer projects successful • through enablement • through project / solution delivery • Graph Based Solutions as accelerators • Neo4j is the foundation for AI and ML • Customer are using Neo4j for their success
- 46. 47 Thank you