Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What's New in Neo4j

68 views

Published on

Stefan Kolmar, Neo4j

Published in: Software
  • Be the first to comment

  • Be the first to like this

What's New in Neo4j

  1. 1. What’s New? Neo4j Graph Platform Overview Stefan Kolmar VP Field Engineering EMEA&APAC
  2. 2. What Makes Neo4j Different? 2 TRADITIONAL DATABASES BIG DATA TECHNOLOGY Store and retrieve data Aggregate and filter data Connections in data Real time storage & retrieval Real-Time Connected Insights Long running queries Aggregation & filtering “Our Neo4j solution is literally thousands of times faster than the prior MySQL solution, with queries that require 10-100 times less code” Volker Pacher, Senior Developer Up to 3 Max # of hops 1 Million s
  3. 3. What Makes Neo4j Different? Index-Free Adjacency 3
  4. 4. Connectedness and Size of Data Set! ResponseTime! Relational and Other NoSQL Databases 0 to 2 hops 0 to 3 degrees Thousands of connections 1000x Advantage Tens to hundreds of hops Thousands of degrees Billions of connections Neo4j “Minutes to milliseconds” What Makes Neo4j Different : “Minutes to Milliseconds” Real-Time Query Performance
  5. 5. ACID Consistency Non ‘Graph-ACID’ DBMSs 5 Maintains Integrity Over Time Guaranteed Graph Consistency Becomes Corrupt Over Time Not ‘Good Enough’ for Graphs What Is Different In Neo4j? ACID Graph Writes : A Requirement for Graph Transactions
  6. 6. What Is Different In Neo4j? Cypher Query Language 6 MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report) WHERE boss.name = “John Doe” RETURN sub.name AS Subordinate, 
 count(report) AS Total Project Impact Less time writing queries •  More time understanding the answers •  Leaving time to ask the next question Less time debugging queries: •  More time writing the next piece of code •  Improved quality of overall code base Code that’s easier to read: •  Faster ramp-up for new project members •  Improved maintainability & troubleshooting
  7. 7. 7 Neo4j Graph Advantage: Foundational Components 1 2 3 4 5 6 Index-Free Adjacency In memory and on flash/disk vs ACID Foundation Required for safe writes Full-Stack Clustering Causal consistency Language, Drivers, Tooling Developer Experience, Graph Efficiency, Type Safety Graph Engine Cost-Based Optimizer, Graph Statistics, Cypher Runtime Hardware Optimizations For next-gen infrastructure
  8. 8. Neo4j Enterprise Maturity & Robustness 8 Neo4j Security Foundation Multi-Clustering Support for Global Internet Apps Rolling Upgrades Schema Constraints Concurrent/Transactional Write Performance Auto Cache Reheating For Restarts, Restores and Cluster Expansion Neo4j 3.4 now supports rolling upgrades 3.4 3.5 Upgrade older instances while keeping other members stable and without requiring a restart of the environment 3.5
  9. 9. DBMS Market Evolution: Connections as Imperative 9 Column Family Document Data Storage & Retrieval for all new use cases 1990 Relational Key-Value Data Storage & Retrieval for specific use cases Codd’s Paper 1979 Oracle IPO 1986 Graph Connected Data for specific use cases 2010 Neo4j v1.0 2019 openCypher Launch Connected Data for all new use cases 2025 GQL 1.0
  10. 10. Neo4j: Enabling the Connected Enterprise Consumers of Connected Data 10 AI & Graph Analytics •  Sentiment analysis •  Customer segmentation •  Machine learning •  Cognitive computing •  Community detection Transactional Graphs •  Fraud detection •  Real-time recommendations •  Network and IT operations management •  Knowledge Graphs •  Master Data Management Discovery & Visualization •  Fraud detection •  Network and IT operations •  Product information management •  Risk and portfolio analysis Data Scientists Business Users Applications
  11. 11. 11 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Neo4j Graph Platform
  12. 12. 12 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Improved Admin Experience - Rolling upgrades - Brute force attack prevention - Fast, resumable backups - Cache Warming on startup - Improved diagnostics Neo4j Graph Platform in 2019
  13. 13. 13 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Improved Admin Experience - Rolling upgrades - Brute force attack prevention - Fast, resumable backups - Cache Warming on startup - Improved diagnostics Multi-Cluster routing built into Bolt drivers Seabolt & Go Driver - Other v1.7 Supported Drivers: Java, JavaScript, Python, .NET - Community Drivers: Perl, PhP, Ruby, Erlang, R, Haskell, Clojure, JDBC and many others Neo4j Graph Platform in 2019
  14. 14. 14 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Improved Admin Experience - Rolling upgrades - Brute force attack prevention - Fast, resumable backups - Cache Warming on startup - Improved diagnostics Multi-Cluster routing built into Bolt drivers Seabolt & Go Driver - Other v1.7 Supported Drivers: Java, JavaScript, Python, .NET - Community Drivers: Perl, PhP, Ruby, Erlang, R, Haskell, Clojure, JDBC and many others Neo4j Bloom - New graph illustration and communication tool for non-technical users - Explore and edit graph - Search-based - Create storyboards - Foundation for graph data discovery - Integrated with graph platform Neo4j Graph Platform in 2019
  15. 15. 15 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Improved Admin Experience - Rolling upgrades - Brute force attack prevention - Fast, resumable backups - Cache Warming on startup - Improved diagnostics Multi-Cluster routing built into Bolt drivers Seabolt & Go Driver - Other v1.7 Supported Drivers: Java, JavaScript, Python, .NET - Community Drivers: Perl, PhP, Ruby, Erlang, R, Haskell, Clojure, JDBC and many others Neo4j Bloom - New graph illustration and communication tool for non-technical users - Explore and edit graph - Search-based - Create storyboards - Foundation for graph data discovery - Integrated with graph platform Graph Data Science High speed graph algorithms Neo4j Graph Platform in 2019
  16. 16. 16 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Improved Admin Experience - Rolling upgrades - Brute force attack prevention - Fast, resumable backups - Cache Warming on startup - Improved diagnostics Multi-Cluster routing built into Bolt drivers Seabolt & Go Driver - Other v1.7 Supported Drivers: Java, JavaScript, Python, .NET - Community Drivers: Perl, PhP, Ruby, Erlang, R, Haskell, Clojure, JDBC and many others Neo4j Bloom - New graph illustration and communication tool for non-technical users - Explore and edit graph - Search-based - Create storyboards - Foundation for graph data discovery - Integrated with graph platform Graph Data Science High speed graph algorithms Neo4j Database 3.4 & 3.5 - 70% faster Cypher - Native GraphB+Tree Indexes (up to 5x faster writes) - Full-text search - Index-Backed Optimisation - 100B+ bulk importer - Date/Time data type - 3-D Geospatial search - Secure, Horizontal Multi-Clustering - Property Blacklisting - Causal Cluster with Raft v2 Protocol - Hostname verification, Intra-cluster discovery encryption Neo4j Graph Platform in 2019
  17. 17. Neo4j Graph Platform: Where We Are Today 17 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI Improved Admin Experience - Rolling upgrades - Brute force attack prevention - Fast, resumable backups - Cache Warming on startup - Improved diagnostics Multi-Cluster routing built into Bolt drivers Seabolt & Go Driver - Other v1.7 Supported Drivers: Java, JavaScript, Python, .NET - Community Drivers: Perl, PhP, Ruby, Erlang, R, Haskell, Clojure, JDBC and many others SparkCypher/Morpheus (pre-EAP) Spark community has voted to include Cypher in Spark 3.0 Neo4j Bloom - New graph illustration and communication tool for non-technical users - Explore and edit graph - Search-based - Create storyboards - Foundation for graph data discovery - Integrated with graph platform Graph Data Science High speed graph algorithms Neo4j Database 3.4 & 3.5 - 70% faster Cypher - Native GraphB+Tree Indexes (up to 5x faster writes) - Full-text search - Index-Backed Optimisation - 100B+ bulk importer - Date/Time data type - 3-D Geospatial search - Secure, Horizontal Multi-Clustering - Property Blacklisting - Causal Cluster with Raft v2 Protocol - Hostname verification, Intra-cluster discovery encryption
  18. 18. 18 Graph Visualization Options for Neo4j Neo4j Bloom Provided by Neo4j Exclusively optimized for Neo4j graphs Deploys easily in Neo4j Desktop Focused on graph exploration thru a code-free UI Near natural language search Currently caters to data analysts and graph SMEs Currently for individual or small team use Viz Toolkits 3rd party e.g. vis.js, d3.js, Keylines Some offer data hooks into Neo4j, others may require custom integration Offer robust APIs for flexible control of the viz output Cater to developers who will create a custom solution, usually with limited interactivity Departmental, enterprise or public use BI Tools 3rd party e.g. Tableau, Qlik Not optimized for graph data, may require a special connector UI for dashboard and report creation with many kinds of viz, in addition to graph viz Cater to business users and data analysts Departmental, cross- department or enterprise use Graph Viz Solutions 3rd party e.g. Linkurious, Tom Sawyer Have to support multiple graph models and sources Feature UI for exploration or APIs for customizing output and embedding/publishing Solutions may cater to business users, analysts or developers Small team, departmental or cross-department use Little technical expertise Most technically involved Exploration focused Publishing / Consumption focused Smaller deployments Larger deployments
  19. 19. Perspective Visualization Exploration Inspection Editing Search 19 Business view of the graph Departmental views • Hiding PII • Styling GPU Accelerated Visualization High performance physics & rendering Direct graph interactions Select, expand, dismiss, find paths Node + Relationship details Browse from neighbor to neighbor Create, Connect, Update Code-free graph changes Near-natural Language Search Full-text search • Graph patterns • Custom Search Phrases Neo4j Bloom Features
  20. 20. Confidential - Neo4j, Inc. A Journey through Neo4j 3.X 20 3.0 3.1 3.2 3.3 3.4 3.5 Causal Clustering N/A - Causal Clustering released - Multi-datacenter support - Tiered replicas - Least-connected load balancing - ID re-use - Multi-clustering - Improved large Txn handling Cypher & Performance - User-Defined Procedures - Increased relationship type limits - Native Label index - Node Keys - Composite Indexes - Depth query in DISTINCT function - Compiled runtime - Faster and less memory intensive runtime - Native Numeric schema indexes - Local locks for schema changes - Datetime data types - Spatial data types - 70% faster Cypher reads (average) - Native String schema index - Full-text search - Index-based ORDER BY - Native index for all data types - Improved large Txn handling Security - Native users and roles - LDAP integration - Kerberos authentication plugin - Intra-cluster encryption - Property blacklisting - SNI / Hostname verification - Cluster discovery service encryption Developer - Bolt (binary) Protocol w/ Java, JS, .NET, Python - APOC v.1 - Bolt+routing - Schema viewer within Browser - Faster Neo4j Browser UI - Offline bulk import performance improvements - Neo4j Desktop - Graph Algorithms v.1 - Go Driver - New Graph Algorithms Operations - Official Debian packages - View and manage running queries - Execution guard for long running queries - Detailed query metrics - Official RPM packages - IPv6 Support - Dynamic config settings - Off-heap page cache metadata - Rolling Upgrades - Improved online backup performance (in CC)
  21. 21. Confidential - Neo4j, Inc. Causal Cluster APPLICATION SERVERS Replica Servers Query, View Core Servers Synced Cluster CORE SERVERS READ REPLICAS Async Replication Writes (transactions) Reads (Graph Queries)
  22. 22. Confidential - Neo4j, Inc. Causal Cluster - Multi-Data Center
  23. 23. 23 Machine Learning and Analytics
  24. 24. Where AI and ML fit in 24 Development & Administration Analytics Tooling Graph Analytics Graph Transactions Data Integration Discovery & VisualizationDrivers & APIs AI
  25. 25. Confidential - Neo4j, Inc. Differences between ML and Analytics 25 Machine learning: •  Determine domain parameters •  Historical-based discoveries •  Learn and improve without explicit programming
  26. 26. Confidential - Neo4j, Inc. Graph analytics: •  Uses inherent graph structures •  Uncover real-world networks through their connections •  Forecast complex network behavior and identify action Differences between ML and Analytics
  27. 27. Confidential - Neo4j, Inc. Today challenges with Machine Learning: •  Doesn’t take multiple relationship hops into account •  Takes time to iteratively train a model •  Computational inefficiency of connecting data Benefits of Mixing Graph Analytics with ML Graphs bring: •  Context to machine learning •  Feature filtration •  Connected feature extraction
  28. 28. Confidential - Neo4j, Inc. •  Support for many languages (Python, .Net, Java, Go, JavaScript, R, etc.) •  Different data integration options •  Triggers, event-driven architecture •  User-defined functions and procedures Working with your Machine Learning algorithms and Neo4j
  29. 29. 29 Neo4j Graph Algorithm Library Finds the optimal path or evaluates route availability and quality Pathfinding & Search Determines the importance of distinct nodes in the network Centrality Evaluates how a group is clustered or partitioned Community Detection
  30. 30. 30 Neo4j Graph Algorithm Library - Parallel Breadth First Search & DFS - Shortest Path - Single-Source Shortest Path - All Pairs Shortest Path - Minimum Spanning Tree - A* Shortest Path - Yen’s K Shortest Path - K-Spanning Tree (MST) - Degree Centrality - Closeness Centrality - Betweenness Centrality - PageRank - Wasserman & Faust Closeness Centrality - Harmonic Closeness Centrality - Dangalchev Closeness Centrality - Approx. Betweenness Centrality - Personalise PageRank - Triangle Count - Clustering Coefficients - Strongly Connected Components - Label Propagation - Louvian Modularity - Louvian (Multi-step) - Balanced Triad (identification) - Connected Components (Union Find) - Euclidean Distance - Cosine Similarity - Jaccard Similarity - Random Walk - One Hot Encoding
  31. 31. Thank You! Stefan.kolmar@neo4j.com 31
  32. 32. What’s New? Neo4j Graph Platform Overview Stefan Kolmar VP Field Engineering EMEA&APAC

×