SlideShare a Scribd company logo

Writing a Cypher Engine in Clojure

This talk shows how to run openCypher property graph queries with a search-based algorithm. Presented at the Budapest Clojure meetup.

1 of 35
Download to read offline
Writing a Cypher Engine in Clojure
Dávid Szakállas (BME)
Gábor Szárnyas (BME, MTA)
3rd Budapest Clojure meetup 2018
AGENDA
Gábor:
 Graph databases
 Cypher language
 Query evaluation
Dávid:
 Graph exploration strategies
 Optimization
 How to compile patterns
GRAPHS
𝐺 = 𝑉, 𝐸 𝐸 ⊆ 𝑉 × 𝑉
Textbook definition
Flavours:
 directed/undirected edges
 labels
 properties
GRAPH DATABASES
pattern
matching
NoSQL family
Data model: property graphs = vertices, edges, and properties
analytics
Writing a Cypher Engine in Clojure
CYPHER
„Cypher is a declarative, SQL-inspired language for describing
patterns in graphs visually using an ascii-art syntax.”
MATCH (pers:Person)-[:PRESENTERS]->
(:Presentation)<-[:TOPIC_OF]->(proj:Project)
WHERE proj.name = 'ingraph'
RETURN pers.name
pers.name
David
Gabor
Ad

Recommended

What Makes Graph Queries Difficult?
What Makes Graph Queries Difficult?What Makes Graph Queries Difficult?
What Makes Graph Queries Difficult?Gábor Szárnyas
 
Mapping Graph Queries to PostgreSQL
Mapping Graph Queries to PostgreSQLMapping Graph Queries to PostgreSQL
Mapping Graph Queries to PostgreSQLGábor Szárnyas
 
Graph x pregel
Graph x pregelGraph x pregel
Graph x pregelSigmoid
 
GraphX and Pregel - Apache Spark
GraphX and Pregel - Apache SparkGraphX and Pregel - Apache Spark
GraphX and Pregel - Apache SparkAshutosh Trivedi
 
GraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLGraphFrames: Graph Queries In Spark SQL
GraphFrames: Graph Queries In Spark SQLSpark Summit
 
GraphFrames: Graph Queries in Spark SQL by Ankur Dave
GraphFrames: Graph Queries in Spark SQL by Ankur DaveGraphFrames: Graph Queries in Spark SQL by Ankur Dave
GraphFrames: Graph Queries in Spark SQL by Ankur DaveSpark Summit
 
論文紹介:Graph Pattern Entity Ranking Model for Knowledge Graph Completion
論文紹介:Graph Pattern Entity Ranking Model for Knowledge Graph Completion論文紹介:Graph Pattern Entity Ranking Model for Knowledge Graph Completion
論文紹介:Graph Pattern Entity Ranking Model for Knowledge Graph CompletionNaomi Shiraishi
 
Machine Learning and GraphX
Machine Learning and GraphXMachine Learning and GraphX
Machine Learning and GraphXAndy Petrella
 

More Related Content

What's hot

GraphX: Graph analytics for insights about developer communities
GraphX: Graph analytics for insights about developer communitiesGraphX: Graph analytics for insights about developer communities
GraphX: Graph analytics for insights about developer communitiesPaco Nathan
 
Apache Spark GraphX highlights.
Apache Spark GraphX highlights. Apache Spark GraphX highlights.
Apache Spark GraphX highlights. Doug Needham
 
GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)
GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)
GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)Ankur Dave
 
GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014
GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014
GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014鉄平 土佐
 
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use CaseApache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use CaseMo Patel
 
R-programming-training-in-mumbai
R-programming-training-in-mumbaiR-programming-training-in-mumbai
R-programming-training-in-mumbaiUnmesh Baile
 
Graph analytic and machine learning
Graph analytic and machine learningGraph analytic and machine learning
Graph analytic and machine learningStanley Wang
 
The 2nd graph database in sv meetup
The 2nd graph database in sv meetupThe 2nd graph database in sv meetup
The 2nd graph database in sv meetupJoshua Bae
 
Graphs are everywhere! Distributed graph computing with Spark GraphX
Graphs are everywhere! Distributed graph computing with Spark GraphXGraphs are everywhere! Distributed graph computing with Spark GraphX
Graphs are everywhere! Distributed graph computing with Spark GraphXAndrea Iacono
 
Improve ML Predictions using Connected Feature Extraction
Improve ML Predictions using Connected Feature ExtractionImprove ML Predictions using Connected Feature Extraction
Improve ML Predictions using Connected Feature ExtractionDatabricks
 
Rethinking Geocoding In Utah
Rethinking Geocoding In UtahRethinking Geocoding In Utah
Rethinking Geocoding In UtahBert Granberg
 
Chengqi zhang graph processing and mining in the era of big data
Chengqi zhang graph processing and mining in the era of big dataChengqi zhang graph processing and mining in the era of big data
Chengqi zhang graph processing and mining in the era of big datajins0618
 
Graphalytics: A big data benchmark for graph-processing platforms
Graphalytics: A big data benchmark for graph-processing platformsGraphalytics: A big data benchmark for graph-processing platforms
Graphalytics: A big data benchmark for graph-processing platformsGraph-TA
 
Locally densest subgraph discovery
Locally densest subgraph discoveryLocally densest subgraph discovery
Locally densest subgraph discoveryaftab alam
 
Pg Conf - Implementing Graph Database based-on PostgreSQL
Pg Conf - Implementing Graph Database based-on PostgreSQLPg Conf - Implementing Graph Database based-on PostgreSQL
Pg Conf - Implementing Graph Database based-on PostgreSQLJoshua Bae
 
Data visualization using R
Data visualization using RData visualization using R
Data visualization using RUmmiya Mohammedi
 
SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"
SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"
SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"Boris Glavic
 
R and Visualization: A match made in Heaven
R and Visualization: A match made in HeavenR and Visualization: A match made in Heaven
R and Visualization: A match made in HeavenEdureka!
 
R programming language in spatial analysis
R programming language in spatial analysisR programming language in spatial analysis
R programming language in spatial analysisAbhiram Kanigolla
 

What's hot (20)

GraphX: Graph analytics for insights about developer communities
GraphX: Graph analytics for insights about developer communitiesGraphX: Graph analytics for insights about developer communities
GraphX: Graph analytics for insights about developer communities
 
Apache Spark GraphX highlights.
Apache Spark GraphX highlights. Apache Spark GraphX highlights.
Apache Spark GraphX highlights.
 
GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)
GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)
GraphX: Graph Analytics in Apache Spark (AMPCamp 5, 2014-11-20)
 
GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014
GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014
GraphX is the blue ocean for scala engineers @ Scala Matsuri 2014
 
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use CaseApache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
Apache Spark GraphX & GraphFrame Synthetic ID Fraud Use Case
 
R-programming-training-in-mumbai
R-programming-training-in-mumbaiR-programming-training-in-mumbai
R-programming-training-in-mumbai
 
Graph analytic and machine learning
Graph analytic and machine learningGraph analytic and machine learning
Graph analytic and machine learning
 
The 2nd graph database in sv meetup
The 2nd graph database in sv meetupThe 2nd graph database in sv meetup
The 2nd graph database in sv meetup
 
Graphs are everywhere! Distributed graph computing with Spark GraphX
Graphs are everywhere! Distributed graph computing with Spark GraphXGraphs are everywhere! Distributed graph computing with Spark GraphX
Graphs are everywhere! Distributed graph computing with Spark GraphX
 
Improve ML Predictions using Connected Feature Extraction
Improve ML Predictions using Connected Feature ExtractionImprove ML Predictions using Connected Feature Extraction
Improve ML Predictions using Connected Feature Extraction
 
Rethinking Geocoding In Utah
Rethinking Geocoding In UtahRethinking Geocoding In Utah
Rethinking Geocoding In Utah
 
Spark graphx
Spark graphxSpark graphx
Spark graphx
 
Chengqi zhang graph processing and mining in the era of big data
Chengqi zhang graph processing and mining in the era of big dataChengqi zhang graph processing and mining in the era of big data
Chengqi zhang graph processing and mining in the era of big data
 
Graphalytics: A big data benchmark for graph-processing platforms
Graphalytics: A big data benchmark for graph-processing platformsGraphalytics: A big data benchmark for graph-processing platforms
Graphalytics: A big data benchmark for graph-processing platforms
 
Locally densest subgraph discovery
Locally densest subgraph discoveryLocally densest subgraph discovery
Locally densest subgraph discovery
 
Pg Conf - Implementing Graph Database based-on PostgreSQL
Pg Conf - Implementing Graph Database based-on PostgreSQLPg Conf - Implementing Graph Database based-on PostgreSQL
Pg Conf - Implementing Graph Database based-on PostgreSQL
 
Data visualization using R
Data visualization using RData visualization using R
Data visualization using R
 
SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"
SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"
SIGMOD 2013 - Patricia's talk on "Value invention for Data Exchange"
 
R and Visualization: A match made in Heaven
R and Visualization: A match made in HeavenR and Visualization: A match made in Heaven
R and Visualization: A match made in Heaven
 
R programming language in spatial analysis
R programming language in spatial analysisR programming language in spatial analysis
R programming language in spatial analysis
 

Similar to Writing a Cypher Engine in Clojure

Cypher and apache spark multiple graphs and more in open cypher
Cypher and apache spark  multiple graphs and more in  open cypherCypher and apache spark  multiple graphs and more in  open cypher
Cypher and apache spark multiple graphs and more in open cypherNeo4j
 
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...Databricks
 
(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014
(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014
(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014Amazon Web Services
 
MongoDB.local DC 2018: Tutorial - Data Analytics with MongoDB
MongoDB.local DC 2018: Tutorial - Data Analytics with MongoDBMongoDB.local DC 2018: Tutorial - Data Analytics with MongoDB
MongoDB.local DC 2018: Tutorial - Data Analytics with MongoDBMongoDB
 
Graph db - Pramati Technologies [Meetup]
Graph db - Pramati Technologies [Meetup]Graph db - Pramati Technologies [Meetup]
Graph db - Pramati Technologies [Meetup]Pramati Technologies
 
The openCypher Project - An Open Graph Query Language
The openCypher Project - An Open Graph Query LanguageThe openCypher Project - An Open Graph Query Language
The openCypher Project - An Open Graph Query LanguageNeo4j
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4jNeo4j
 
PGQL: A Language for Graphs
PGQL: A Language for GraphsPGQL: A Language for Graphs
PGQL: A Language for GraphsJean Ihm
 
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...Databricks
 
Algorithms on Hadoop at Last.fm
Algorithms on Hadoop at Last.fmAlgorithms on Hadoop at Last.fm
Algorithms on Hadoop at Last.fmMark Levy
 
managing big data
managing big datamanaging big data
managing big dataSuveeksha
 
3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB
3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB
3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DBAthens Big Data
 
게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming
게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming
게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 GamingAmazon Web Services Korea
 
Hands on Training – Graph Database with Neo4j
Hands on Training – Graph Database with Neo4jHands on Training – Graph Database with Neo4j
Hands on Training – Graph Database with Neo4jSerendio Inc.
 
Graph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft EcosystemGraph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft EcosystemMarco Parenzan
 
Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™Databricks
 
Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...
Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...
Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...Databricks
 
What to do when one size does not fit all?!
What to do when one size does not fit all?!What to do when one size does not fit all?!
What to do when one size does not fit all?!Arjen de Vries
 

Similar to Writing a Cypher Engine in Clojure (20)

Cypher and apache spark multiple graphs and more in open cypher
Cypher and apache spark  multiple graphs and more in  open cypherCypher and apache spark  multiple graphs and more in  open cypher
Cypher and apache spark multiple graphs and more in open cypher
 
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...
 
(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014
(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014
(BDT203) From Zero to NoSQL Hero: Amazon DynamoDB Tutorial | AWS re:Invent 2014
 
MongoDB.local DC 2018: Tutorial - Data Analytics with MongoDB
MongoDB.local DC 2018: Tutorial - Data Analytics with MongoDBMongoDB.local DC 2018: Tutorial - Data Analytics with MongoDB
MongoDB.local DC 2018: Tutorial - Data Analytics with MongoDB
 
Graph db - Pramati Technologies [Meetup]
Graph db - Pramati Technologies [Meetup]Graph db - Pramati Technologies [Meetup]
Graph db - Pramati Technologies [Meetup]
 
The openCypher Project - An Open Graph Query Language
The openCypher Project - An Open Graph Query LanguageThe openCypher Project - An Open Graph Query Language
The openCypher Project - An Open Graph Query Language
 
Data Modeling with Neo4j
Data Modeling with Neo4jData Modeling with Neo4j
Data Modeling with Neo4j
 
3DRepo
3DRepo3DRepo
3DRepo
 
PGQL: A Language for Graphs
PGQL: A Language for GraphsPGQL: A Language for Graphs
PGQL: A Language for Graphs
 
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
Graph Features in Spark 3.0: Integrating Graph Querying and Algorithms in Spa...
 
Algorithms on Hadoop at Last.fm
Algorithms on Hadoop at Last.fmAlgorithms on Hadoop at Last.fm
Algorithms on Hadoop at Last.fm
 
managing big data
managing big datamanaging big data
managing big data
 
3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB
3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB
3rd Athens Big Data Meetup - 2nd Talk - Neo4j: The World's Leading Graph DB
 
게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming
게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming
게임을 위한 DynamoDB 사례 및 팁 - 김일호 솔루션즈 아키텍트:: AWS Cloud Track 3 Gaming
 
Hands on Training – Graph Database with Neo4j
Hands on Training – Graph Database with Neo4jHands on Training – Graph Database with Neo4j
Hands on Training – Graph Database with Neo4j
 
Graph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft EcosystemGraph Databases in the Microsoft Ecosystem
Graph Databases in the Microsoft Ecosystem
 
Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™Web-Scale Graph Analytics with Apache® Spark™
Web-Scale Graph Analytics with Apache® Spark™
 
Cypher
CypherCypher
Cypher
 
Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...
Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...
Neo4j Morpheus: Interweaving Documents, Tables and and Graph Data in Spark wi...
 
What to do when one size does not fit all?!
What to do when one size does not fit all?!What to do when one size does not fit all?!
What to do when one size does not fit all?!
 

More from Gábor Szárnyas

GraphBLAS: A linear algebraic approach for high-performance graph queries
GraphBLAS: A linear algebraic approach for high-performance graph queriesGraphBLAS: A linear algebraic approach for high-performance graph queries
GraphBLAS: A linear algebraic approach for high-performance graph queriesGábor Szárnyas
 
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...Gábor Szárnyas
 
Incremental View Maintenance for openCypher Queries
Incremental View Maintenance for openCypher QueriesIncremental View Maintenance for openCypher Queries
Incremental View Maintenance for openCypher QueriesGábor Szárnyas
 
Learning Timed Automata with Cypher
Learning Timed Automata with CypherLearning Timed Automata with Cypher
Learning Timed Automata with CypherGábor Szárnyas
 
Időzített automatatanulás Cypherrel
Időzített automatatanulás CypherrelIdőzített automatatanulás Cypherrel
Időzített automatatanulás CypherrelGábor Szárnyas
 
Compiling openCypher graph queries with Spark Catalyst
Compiling openCypher graph queries with Spark CatalystCompiling openCypher graph queries with Spark Catalyst
Compiling openCypher graph queries with Spark CatalystGábor Szárnyas
 
Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...
Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...
Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...Gábor Szárnyas
 
Sharded Joins for Scalable Incremental Graph Queries
Sharded Joins for Scalable Incremental Graph QueriesSharded Joins for Scalable Incremental Graph Queries
Sharded Joins for Scalable Incremental Graph QueriesGábor Szárnyas
 
Towards a Macrobenchmark Framework for Performance Analysis of Java Applications
Towards a Macrobenchmark Framework for Performance Analysis of Java ApplicationsTowards a Macrobenchmark Framework for Performance Analysis of Java Applications
Towards a Macrobenchmark Framework for Performance Analysis of Java ApplicationsGábor Szárnyas
 
IncQuery-D: Distributed Incremental Graph Queries
IncQuery-D: Distributed Incremental Graph QueriesIncQuery-D: Distributed Incremental Graph Queries
IncQuery-D: Distributed Incremental Graph QueriesGábor Szárnyas
 
IncQuery-D: Incremental Queries in the Cloud
IncQuery-D: Incremental Queries in the CloudIncQuery-D: Incremental Queries in the Cloud
IncQuery-D: Incremental Queries in the CloudGábor Szárnyas
 

More from Gábor Szárnyas (12)

GraphBLAS: A linear algebraic approach for high-performance graph queries
GraphBLAS: A linear algebraic approach for high-performance graph queriesGraphBLAS: A linear algebraic approach for high-performance graph queries
GraphBLAS: A linear algebraic approach for high-performance graph queries
 
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
An early look at the LDBC Social Network Benchmark's Business Intelligence wo...
 
Incremental View Maintenance for openCypher Queries
Incremental View Maintenance for openCypher QueriesIncremental View Maintenance for openCypher Queries
Incremental View Maintenance for openCypher Queries
 
Learning Timed Automata with Cypher
Learning Timed Automata with CypherLearning Timed Automata with Cypher
Learning Timed Automata with Cypher
 
Időzített automatatanulás Cypherrel
Időzített automatatanulás CypherrelIdőzített automatatanulás Cypherrel
Időzített automatatanulás Cypherrel
 
Parsing process
Parsing processParsing process
Parsing process
 
Compiling openCypher graph queries with Spark Catalyst
Compiling openCypher graph queries with Spark CatalystCompiling openCypher graph queries with Spark Catalyst
Compiling openCypher graph queries with Spark Catalyst
 
Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...
Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...
Towards the Characterization of Realistic Models: Evaluation of Multidiscipli...
 
Sharded Joins for Scalable Incremental Graph Queries
Sharded Joins for Scalable Incremental Graph QueriesSharded Joins for Scalable Incremental Graph Queries
Sharded Joins for Scalable Incremental Graph Queries
 
Towards a Macrobenchmark Framework for Performance Analysis of Java Applications
Towards a Macrobenchmark Framework for Performance Analysis of Java ApplicationsTowards a Macrobenchmark Framework for Performance Analysis of Java Applications
Towards a Macrobenchmark Framework for Performance Analysis of Java Applications
 
IncQuery-D: Distributed Incremental Graph Queries
IncQuery-D: Distributed Incremental Graph QueriesIncQuery-D: Distributed Incremental Graph Queries
IncQuery-D: Distributed Incremental Graph Queries
 
IncQuery-D: Incremental Queries in the Cloud
IncQuery-D: Incremental Queries in the CloudIncQuery-D: Incremental Queries in the Cloud
IncQuery-D: Incremental Queries in the Cloud
 

Recently uploaded

sahana sri D AD21046 SELF INTRODUCTION.pdf
sahana sri D AD21046 SELF INTRODUCTION.pdfsahana sri D AD21046 SELF INTRODUCTION.pdf
sahana sri D AD21046 SELF INTRODUCTION.pdfsahanaaids46
 
biofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquaculturebiofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquacultureVINETUBE2
 
Metrology Measurements and All units PPT
Metrology Measurements and  All units PPTMetrology Measurements and  All units PPT
Metrology Measurements and All units PPTdinesh babu
 
CDE_Sustainability Performance_20240214.pdf
CDE_Sustainability Performance_20240214.pdfCDE_Sustainability Performance_20240214.pdf
CDE_Sustainability Performance_20240214.pdf8-koi
 
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...GauravBhartie
 
Introduction to Binary Tree and Conersion of General tree to Binary Tree
Introduction to Binary Tree  and Conersion of General tree to Binary TreeIntroduction to Binary Tree  and Conersion of General tree to Binary Tree
Introduction to Binary Tree and Conersion of General tree to Binary TreeSwarupaDeshpande4
 
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdfROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdfRudraPratapSingh871925
 
S. Kim, NeurIPS 2023, MLILAB, KAISTAI
S. Kim,  NeurIPS 2023,  MLILAB,  KAISTAIS. Kim,  NeurIPS 2023,  MLILAB,  KAISTAI
S. Kim, NeurIPS 2023, MLILAB, KAISTAIMLILAB
 
Bresenham line-drawing-algorithm By S L Sonawane.pdf
Bresenham line-drawing-algorithm By S L Sonawane.pdfBresenham line-drawing-algorithm By S L Sonawane.pdf
Bresenham line-drawing-algorithm By S L Sonawane.pdfSujataSonawane11
 
21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf
21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf
21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdfDr. Shivashankar
 
Student Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCETStudent Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCETGDSCNKOCET
 
Introduction communication assignmen.pdf
Introduction communication assignmen.pdfIntroduction communication assignmen.pdf
Introduction communication assignmen.pdfKannigaSaraswathyM
 
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptxDeep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptxpmgdscunsri
 
SR Globals Profile - Building Vision, Exceeding Expectations.
SR Globals Profile -  Building Vision, Exceeding Expectations.SR Globals Profile -  Building Vision, Exceeding Expectations.
SR Globals Profile - Building Vision, Exceeding Expectations.srglobalsenterprises
 
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHIINTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHIKiranKandhro1
 
Module 2_ Divide and Conquer Approach.pptx
Module 2_ Divide and Conquer Approach.pptxModule 2_ Divide and Conquer Approach.pptx
Module 2_ Divide and Conquer Approach.pptxnikshaikh786
 
(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...
(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...
(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...محمد قصص فتوتة
 
BRINDHA G AD21012 SELF INTRODUCTION.pdf
BRINDHA G  AD21012 SELF INTRODUCTION.pdfBRINDHA G  AD21012 SELF INTRODUCTION.pdf
BRINDHA G AD21012 SELF INTRODUCTION.pdfbrindhaaids12
 
Deluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdfDeluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdfartpoa9
 
nervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesnervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesPhebeLois1
 

Recently uploaded (20)

sahana sri D AD21046 SELF INTRODUCTION.pdf
sahana sri D AD21046 SELF INTRODUCTION.pdfsahana sri D AD21046 SELF INTRODUCTION.pdf
sahana sri D AD21046 SELF INTRODUCTION.pdf
 
biofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquaculturebiofilm fouling of the membrane present in aquaculture
biofilm fouling of the membrane present in aquaculture
 
Metrology Measurements and All units PPT
Metrology Measurements and  All units PPTMetrology Measurements and  All units PPT
Metrology Measurements and All units PPT
 
CDE_Sustainability Performance_20240214.pdf
CDE_Sustainability Performance_20240214.pdfCDE_Sustainability Performance_20240214.pdf
CDE_Sustainability Performance_20240214.pdf
 
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
Microstrip Bandpass Filter Design using EDA Tolol such as keysight ADS and An...
 
Introduction to Binary Tree and Conersion of General tree to Binary Tree
Introduction to Binary Tree  and Conersion of General tree to Binary TreeIntroduction to Binary Tree  and Conersion of General tree to Binary Tree
Introduction to Binary Tree and Conersion of General tree to Binary Tree
 
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdfROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
ROBOT PERCEPTION FOR AGRICULTURE AND GOOD PRODUCTION1.1.pdf
 
S. Kim, NeurIPS 2023, MLILAB, KAISTAI
S. Kim,  NeurIPS 2023,  MLILAB,  KAISTAIS. Kim,  NeurIPS 2023,  MLILAB,  KAISTAI
S. Kim, NeurIPS 2023, MLILAB, KAISTAI
 
Bresenham line-drawing-algorithm By S L Sonawane.pdf
Bresenham line-drawing-algorithm By S L Sonawane.pdfBresenham line-drawing-algorithm By S L Sonawane.pdf
Bresenham line-drawing-algorithm By S L Sonawane.pdf
 
21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf
21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf
21 SCHEME_21EC53_VTU_MODULE-4_COMPUTER COMMUNCATION NETWORK.pdf
 
Student Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCETStudent Challange as Google Developers at NKOCET
Student Challange as Google Developers at NKOCET
 
Introduction communication assignmen.pdf
Introduction communication assignmen.pdfIntroduction communication assignmen.pdf
Introduction communication assignmen.pdf
 
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptxDeep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
Deep Learning For Computer Vision- Day 3 Study Jams GDSC Unsri.pptx
 
SR Globals Profile - Building Vision, Exceeding Expectations.
SR Globals Profile -  Building Vision, Exceeding Expectations.SR Globals Profile -  Building Vision, Exceeding Expectations.
SR Globals Profile - Building Vision, Exceeding Expectations.
 
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHIINTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
INTERACTIVE AQUATIC MUSEUM AT BAGH IBN QASIM CLIFTON KARACHI
 
Module 2_ Divide and Conquer Approach.pptx
Module 2_ Divide and Conquer Approach.pptxModule 2_ Divide and Conquer Approach.pptx
Module 2_ Divide and Conquer Approach.pptx
 
(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...
(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...
(تحليل أداء وإنبعاثات محرك ديزل يستخدم خليط وقود الديزل و زيت الجاتروفا )بحث ...
 
BRINDHA G AD21012 SELF INTRODUCTION.pdf
BRINDHA G  AD21012 SELF INTRODUCTION.pdfBRINDHA G  AD21012 SELF INTRODUCTION.pdf
BRINDHA G AD21012 SELF INTRODUCTION.pdf
 
Deluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdfDeluck Technical Works Company Profile.pdf
Deluck Technical Works Company Profile.pdf
 
nervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nervesnervous system ppt pptx anatomy system of nerves
nervous system ppt pptx anatomy system of nerves
 

Writing a Cypher Engine in Clojure

  • 1. Writing a Cypher Engine in Clojure Dávid Szakállas (BME) Gábor Szárnyas (BME, MTA) 3rd Budapest Clojure meetup 2018
  • 2. AGENDA Gábor:  Graph databases  Cypher language  Query evaluation Dávid:  Graph exploration strategies  Optimization  How to compile patterns
  • 3. GRAPHS 𝐺 = 𝑉, 𝐸 𝐸 ⊆ 𝑉 × 𝑉 Textbook definition Flavours:  directed/undirected edges  labels  properties
  • 4. GRAPH DATABASES pattern matching NoSQL family Data model: property graphs = vertices, edges, and properties analytics
  • 6. CYPHER „Cypher is a declarative, SQL-inspired language for describing patterns in graphs visually using an ascii-art syntax.” MATCH (pers:Person)-[:PRESENTERS]-> (:Presentation)<-[:TOPIC_OF]->(proj:Project) WHERE proj.name = 'ingraph' RETURN pers.name pers.name David Gabor
  • 7. DBMS POPULARITY BY DATA MODEL version 2.0 introduces the Cypher query language
  • 8. OPENCYPHER SYSTEMS  Goal: deliver a full and open specification of Cypher  Relational databases: o SAP HANA o AGENS Graph  Research prototypes: o Graphflow (Univesity of Waterloo) o ingraph (incremental graph engine) (Source: Keynote talk @ GraphConnect NYC 2017)
  • 9. USE CASES  Analytics o IT security o Investigative journalism  “Real-time” execution o Fraud detection o Recommendation systems
  • 10. INGRAPH  MTA-BME project  PoC query engine for openCypher  Primarily written in Scala  Goals: o Provide continuous evaluation (incremental view maintenance) o Cover standard openCypher constructs o Does not work efficiently for one-time query execution Szárnyas, G. et al., IncQuery-D: A distributed incremental model query framework in the cloud. MODELS, 2014, https://link.springer.com/chapter/10.1007/978-3-319-11653-2_40
  • 13. GRAPH QUERY PROCESSING APPROACHES  Relational: projection, selection, join o SAP HANA o Agens Graph (PostgreSQL + Cypher) o ingraph  Relational + expand o Neo4j  Constraint satisfaction o VIATRA (BME project since 2002): “local search” engine o Our proposed openCypher engine (this talk)
  • 14. RELATIONAL MODEL person parent Alice Sarah Alice Bob a b Bob Sarah Parent Married Person { name: Alice } Person { name: Bob } Person { name: Sarah } PARENT PARENT MARRIED PROPERTY GRAPH label properties edge type
  • 15. EXPLORATION BASED METHODS  given this graph  and query Person { name: Alice } Person { name: Bob } Person { name: Sarah } PARENT PARENT MARRIED MATCH (u:Person) -[:PARENT]-> (f:Person) -[m:MARRIED]-> (m:Person) RETURN m, f find an execution plan
  • 16. EXPLORATION BASED METHODS Person { name: Alice } Person { name: Bob } Person { name: Sarah } PARENT PARENT MARRIED  DFT -> small memory footprint  Small state-space -> less steps
  • 17. EXPLORATION BASED METHODS Goal: minimize state-space generated for a query.  Cost based optimizations  Constraint satisfaction problems  SAT is an NP-full problem  Greedy algorithm unsatisfactory
  • 18. EXPLORATION BASED METHODS Goal: minimize state-space generated for a query.  Iterative Dynamic Programming (IDP): oGeneralization of DP and greedy algorithms oPolynomial complexity oMultiple variants oWidely researched in the RDMS area -> use iterative dynamic programming
  • 19. FULL VS GREEDY VS IDP ⋮ :e _ -[_]-> _ MATCH (a)-[:e]->(b) RETURN a, b
  • 20. FULL VS GREEDY VS IDP ⋮ :e _-[_]->_ MATCH (a)-[:e]->(b) RETURN a, b a-[_]->_ _-[_]->b 1,0 100,0
  • 21. FULL VS GREEDY VS IDP ⋮ :e _-[_]->_ MATCH (a)-[:e]->(b) RETURN a, b a-[_]->_ _-[_]->b 1 100 a-[:e]->b a-[:e]->b 2 0.02
  • 22. FULL VS GREEDY VS IDP ⋮ :e _-[_]->_ MATCH (a)-[:e]->(b) RETURN a, b a-[_]->_ _-[_]->b 1 100 _-[_]->_ a-[_]->_ _-[_]->b 1 100 a-[:e]->b a-[:e]->b 2 0.02
  • 23. FULL VS GREEDY VS IDP ⋮ :e _-[_]->_ MATCH (a)-[:e]->(b) RETURN a, b a-[_]->_ _-[_]->b 1 100 _-[_]->_ a-[_]->_ _-[_]->b 1 100 a-[:e]->b a-[:e]->b 2 0.02
  • 24. FULL VS GREEDY VS IDP ⋮ :e _-[_]->_ MATCH (a)-[:e]->(b) RETURN a, b a-[_]->_ _-[_]->b 1 100 a-[:e]->b 2 _-[_]->_ a-[_]->_ _-[_]->b 1 100 a-[:e]->b a-[:e]->b 2 0.02
  • 25. FULL VS GREEDY VS IDP ⋮ :e _-[_]->_ MATCH (a)-[:e]->(b) RETURN a, b a-[_]->_ _-[_]->b 1 100 a-[:e]->b 2 _-[_]->_ a-[_]->_ _-[_]->b 1 100 a-[:e]->b a-[:e]->b 2 0.02 IDP: Something in between...
  • 26. MODEL-SENSITIVE APPROACH G. Varró, F. Deckwerth, M. Wieber, A. Schürr, An algorithm for generating model-sensitive search plans for pattern matching on EMF models, Software and Systems Modeling, 2013  Derives cost based on the graph elements  Object-oriented approach  Different goal: model validation Idea: adapt to property graphs.
  • 27. ADAPTING IT Requirements  Variables for edges and vertices  Estimate based on the graph  Map to specialized indexer operations Difficulties  No type information  Original algorithm oblivious to edges with properties
  • 28. CONSTRAINT DEFINITIONS  Constraints and operations are separate  Constraints can imply other constraints
  • 31. OPERATIONS – COSTS  indexer support (context sensitive)  Based on variables referenced in the constraints  Given a (HasLabels a t) and a (Vertex a) constraints we can get the number of vertices for that label
  • 33. NEGATIVE PATTERNS Compiles to a single constraint with inner constraints. Negation needs at least one bound variable
  • 35. CONCLUSION  Difficult to adapt this OO algorithm to property graphs  Clojure is well fitted to the problem  Subpar performance from the initial implementation  Looking for maintainers  Interested? Contact us: @szarnyasg @szdavid92 FTSRG/ingraph Dávid Szakállas: Evaluation of openCypher Graph Queries with a Local Search-Based Algorithm. Master’s thesis, Budapest University of Technology and Economics, 2017.