Sparsity Technologies, profile picture
Uploaded bySparsity Technologies
8,182 views

Sparksee Technology overview

The document provides an overview of the Sparksee graph database, a high-performance system designed for large-scale labeled and attributed multigraphs. It discusses Sparksee's architecture, performance metrics, and innovative storage techniques using bitmap structures, highlighting its scalability, efficiency, and support for complex graph queries. Additionally, it reviews comparison benchmarks against other databases, showcasing Sparksee's capabilities in handling extensive data with subsecond query responses.

Embed presentation

Sparksee Graph Database! Technology overview! April 2014 º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Graph Databases! Introduction to Sparksee! Sparksee Internals! Performance analysis! High scalability! HPC-SGAB Benchmark !
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Graph Databases!
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases Graphs are everywhere! ! — Increasing number of huge networks such as the Web, Social Networks, Biological Systems, GPS…! ! — Very large graphs! ! — Interest for analyzing the ! interrelation between the entities ! in theses networks! !
Classical graph representation! ! — Adjacency matrix! ! Very large NxN sparse matrix, no labels, no multigraph, ! no attributes! — Adjacency list! ! No labels, no attributes, still sparse consuming! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
Classical graph storage! — Relational database! ! Prefixed schema or very large table for nodes and edges, not ! ! suitable for path traversals and graph exploration! — XML! ! XML data is stored in the form of trees! ! Much work done on finding exact or approximate patterns ! ! (subtrees)! ! Not thought for complex graph queries! — RDF! ! Widely adopted standard for manipulating graph-like data! ! Large support from large vendors! ! SPARQL has become a de facto standard º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
New approaches to graph analysis! ! — Complex analysis computations on very large distributed graphs ! ! Map-reduce (Pegasus)! ! Vertex-centric computation model (Pregel) ! — Graph Databases: database functionalities to store and query graph-like data ! ! Graph storage in a file system of a computer node with buffer ! ! pool (Neo4j, Hypergraph, OrientDB, Infinitegraph! ! Multiple servers accessible through a load balancer (Neo4j HA) º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
Requirements for graph databases! ! — Data and schema represented as a graph! — Data operations based on graph operations! — Graph-based integrity restrictions! — Multigraphs! — Attributes attached to both vertices and edges! — Graph queries combining edge traversals with attribute ! accesses! — Diversity of workloads! — Efficient secondary memory management! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Introduction to Sparksee!
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee Sparksee! ! IS a high-performance and out-of-core ! graph database management system ! FOR large scale labeled and attributed multigraphs! ! BASED ON vertical partitioning and collections of objects identifiers stored as bitmaps
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee ! Sparksee — Characteristics! ! — Graph split into small structures ! Move to main memory just significant parts (caching) — Object identifiers (oids) instead of complex objects ! Reduce memory requirements — Specific structures to improve traversals ! Index the edges and the neighbors of each node — Attribute indices ! Improve queries based on value filters — Implemented in C++ ! Different APIs (Java, .NET, etc.) through wrappers
! ! Sparksee — Capabilities! ! Efficiency ! very compact representation using bitmaps. Highly compressible data ! ! structures. Capacity ! more than 100 billion vertices and edges in a single multicore computer. Performance ! subsecond response in recommendation queries. Scalability ! high throughput for concurrent queries. Consistency ! partial transactional support with recovery. Multiplatform ! Linux, Windows, MacOSX, Mobile º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee
! Logical graph model! ! Labeled ! a label (type) for each vertex and edge ! Directed ! edges can have a fixed direction, from tail to head ! Attributed ! variable list of attributes for each! ! vertex and edge ! Multigraph ! multiple edges between two ! ! vertices ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee
! ! ! Sparksee — Architecture! ! ! ! ! ! ! ! ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee GDB GRAPH DATA STRUCTURES PLATFORM DEXCORE SparkseeCpp – Graph Algorithms SWIG SparkseeJava SparkseeNet .NET App JAVA App C++ App BUFFERPOOL Python App Mobile App SparkseePhyton
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Sparksee internals
! ! ! Graph representation! ! We define a graph G = (V,E,L,T,H,A1,…,Ap) as: ! LABELS L = {(o, l ) | o ∈ (V ∪ E ) ∧ l ∈ string} TAILS T = {(e, t ) | e ∈ E ∧ t ∈ V } HEADS H = {(e, h) | e ∈ E ∧ h ∈ V } ATTRIBUTES Ai = {(o, c ) | o ∈ (V ∪ E ) ∧ c ∈ {int, string, ...}} ! With this representation: — the graph is split into multiple lists of pairs! — the first element of each pair is always a vertex or an edge! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
Graph representation! ! ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals L v1, ARTICLE), (v2, ARTICLE),T (e1, v1), (e2, v2), (e3, v4), (e , v ), (e ,H (e1, v3), (e2, v3), (e3, v3), (e , v ), (e ,Aid (v1, 1), (v2, 2), (v3, 3), (v4, 4), (v , 1), (v , 2)Atitle (v1, Europa), (v2, Europe), (v , Europe),Anlc (v1, ca), (v2, fr), (v3, en), (v , en), (e ,Afilename (v5, europe.png), (v , bcn.jpg)Atag (e4, continent)
! ! Value sets! ! Groups all pairs of the ! original set with the ! same value as a pair ! between the value and ! the set of objects with ! such value. ! ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals L v1, ARTICLE), (v2, ARTICLE), (v3, ARTICLE), (v4, ARTICLE), (v5, IMAGE), (v6, IMAGE), (e1, BABEL), (e2, BABEL), (e3, REF), (e4, REF), (e5, CONTAINS), (e6, CONTAINS), (e7, CONTAINS) (ARTICLE, {v1, v2, v3, v4}), (BABEL, {e1, e2}), (CONTAINS, {e5, e6, e7}), (IMAGE, {v5, v6}), (REF, {e3, e4}) T (e1, v1), (e2, v2), (e3, v4), (e4, v4), (e5, v3), (e6, v3), (e7, v4) (v1, {e1}), (v2, {e2}), (v3, {e5, e6}), (v4, {e3, e4, e7}) H (e1, v3), (e2, v3), (e3, v3), (e4, v3), (e5, v5), (e6, v6), (e7, v6) (v3, {e1, e2, e3, e4}), (v5, {e5}), (v6, {e6, e7}) Aid (v1, 1), (v2, 2), (v3, 3), (v4, 4), (v5, 1), (v6, 2) (1, {v1, v5}), (2, {v2, v6}), (3, {v3}), (4, {v4}) Atitle (v1, Europa), (v2, Europe), (v3, Europe), (v4, Barcelona) (Barcelona, {v4}), (Europa, {v1}), (Europe, {v2, v3}) Anlc (v1, ca), (v2, fr), (v3, en), (v4, en), (e1, en),(e2, en) (ca, {v1}), (en, {v3, v4, e1, e2}), (fr, {v2}) Afilena me (v5, europe.png), (v6, bcn.jpg) (bcn.jpg, {v6}), (europe.png, {v5}) Atag (e4, continent) (continent, {e4})
! Bitmap representation! ! — Each vertex and edge is identified by a unique and immutable ! oid (object identifier) ! — Each vertex or edge set is stored in a bitmap structure: ! Each position in a bitmap corresponds to the oid of an object! ! Reduced amount of space (compression techniques) ! Very efficient binary logic operations º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
! Value set representation! ! — A value set is represented as two maps! ! One maps each different value to a vertex or edge set! ! The other maps each vertex or edge to a value oid ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals ! Example of a bitmap based representation! ! ! ! ! ! ! ! !
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals ! Integrity rules! ! ! ! ! ! ! ! !
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals ! ! ! Value set operations! ! domain returns the set of distinct values objects returns the set of vertices or edges associated to a value! lookup returns the set of values ! associated to a set of objects! insert adds a vertex or edge to the ! collection of objects of a value! remove removes a vertex or edge ! from the collection of objects of a value
Graph query examples — Number of articles! ! |objects (LABELS, ‘ARTICLE’)| — Out-degree of English article ‘Europe’! ! |objects (TAILS, objects( TITLE, ‘Europe’) ∩ objects (NLC, ‘en’) ∩ objects ! (LABELS, ‘ARTICLE’))| — Articles with references to the image with filename ‘bcn.jpg’ ! ! {lookup(TAILS, x ) |x ∈ objects (HEAD, objects (FILENAME, ′ bcn.jpg′ ) ! ! ! ∩ objects (LABELS, ′ IMAGE′ ))} ! — Count the articles of each language {(x , y ) | x ∈ domain(NLC) ∧ y = |(objects (NLC, x ) ∩ objects (LABELS, ! ! ! ′ ARTICLE′ ))|} º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
! Implementation details — Bitmaps are compressed by grouping the bits into clusters of 32 consecutive bits (up to 137 billion objects per graph)! — Locality is improved by generating consecutive oids for each distinct vertex or edge labels! — Sorted tree structure of bitmap clusters to speedup the insert, remove, and binary logic operations! — Maps are implemented using B+ trees — The tail, head and attribute value sets have been split into specific value sets for each label º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Performance analysis
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Queries! ! ! ! ! ! ! ! ! ! Q1: Find the article with the largest outdegree and traverse its shortest path tree Q2: Recommend articles related to the most popular one Q3: Find new images for articles from translations in other languages Q4: Find, for each different language, the number of articles and images referenced Q5: For each article with images, materialize the count of images Q6: Remove all articles without images Q1 Q2 Q3 Q4 Q5 Q6 k-hops and path traversals + + graph pattern matching + aggregations and edge connectivity + graph transformation + +
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Performance Out-of-core! ! Wikipedia Benchmark out-of-core, 1GB buffer pool. ! ! ! ! ! ! ! ! (⋆) Java VM with 45 GB MonetDb MySQL Neo4J* SPARKSEE Graph Size (GB) 12.00 15.72 42.00 16.98 Load (h) Error 1.36 8.99 2.89 Q1 (s) 4,801.6 > 12 h. > 12 h. 120.5 Q2 (s) 3,788.4 13,841.6 > 12 h. 205.4 Q3 (s) 458.9 33.0 481.0 10.8 Q4 (s) 279,3 45.0 > 12 h. 144.9 Q5 (s) 267.4 930.3 > 12 h. 140.9 Q6 (s) Error 10707.0 > 12 h. 25791.6
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Query statistics! ! ! ! ! ! ! ! ! ! ! Query results edge trav. edge trav./sec mem MB bitmaps Q1 624,525 236,387,207 1,987,616.30 832.19 42.97% Q2 5 261,735,954 1,270,747.94 2,974.50 48.59% Q3 51,780 1,536,698 143,885.58 320.81 48.00% Q4 254 4,987,879 33,984.32 245.13 77.67% Q5 2,401,597 5,934,724 42,072.39 319.00 80.64% Q6 52,380,949 281,433,106 37,434.27 11,583.88 67.76% !
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Bitmap memory usage! ! ! ! ! ! ! ! ! ! ! Size (MB) Q1 Q2 Q3 Q4 Q5 Q6 LABELS 13.56 11.60 11.60 11.60 11.60 11.60 1.51 TAILS 1,272.32 1,030.90 857.09 229.67 164.79 164.79 90.18 HEADS 633.98 506.98 47.09 Attr. ID 122.77 0.85 Attr. TITLE 835.92 10.87 Attr. NLC 3,618.49 791.29 833.64 617.15 Attr. FILENAME 769.79 Attr. TAG 31.94 2.29 TOTAL 7,298.77 1,042.50 1,375.67 1,032.56 1,010.03 176.39 769.94
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Analysis of bitmap usage! ! ! ! ! ! ! ! ! ! !
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Bitmap size distribution! ! ! ! ! ! ! ! ! ! !
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis Out of core stress test! ! ! ! ! ! ! ! ! ! !
º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee technology ! RMAT/Query 1 Scalability Test! ! ! ! ! ! ! ! ! ! ! 228 is out-of-core (2 billion edges)
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph Database Sparksee technology SNA Benchmark — Q1, Q6, Q9 and Q12 ! ! ! ! ! ! ! ! !
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex High scalability
! ! ! High Scalability test — Mirror Servers in Amazon Elastic with a Load Balancer ! ! ! ! ! ! ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee technology
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex HPC-SGAB Benchmark
! Definition — HPC-SGAB: Badet et al. 2009! ! Measured in TEPS: traversed edges per second! — Graph! ! Synthetic (R-Mat)! ! Power law distribution! ! Average: 8 edges/node — Operations! ! ! Kernel 1: load graph and create indexes! ! ! Kernel 2: find the edge(s) with maximum weight! ! ! Kernel 3: k-hops! ! ! Kernel 4: betweenness centrality (Brandes algorithm) º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
! ! Experimental setup — Systems Tested! ! Sparksee (former DEX)! ! Neo4j! ! HypergraphDB! ! Jena (RDF) — Platform! ! Single computer with 2 quad core Xeon E5410! ! 11GB RAM! ! LFF 2.25 TB disk! ! Single threaded — Default benchmark configuration º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
Summary of results ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
Kernel 1 - Load time ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
Kernel 4 - Betweenness centrality ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
! ! Bibliography! ! R. Angles, A. Prat, D. Dominguez, J.L. Larriba, Benchmarking database systems for social network applications (GRADES 2013) ! N. Martínez, V. Muntés, S. Gómez, M.A. Águila, D. Dominguez, J.L. Larriba, Efficient Graph Management Based On Bitmap Indices (IDEAS 2012) ! N. Martínez, S. Gómez, F. Escalé, DEX: a High-Performance Graph Database Management System (GDM 2011) ! D. Dominguez, P. Urbón, A. Giménez, S. Gómez, N. Martínez, and J. L. Larriba, Survey of Graph Database Performance on the HPC Scalable Graph Analysis Benchmark (IWDG 2010) ! N. Martínez, V. Muntés, S. Gómez, J. Nin, M. A. Sánchez, and J. Larriba, Dex: High-performance Exploration on Large Graphs for Information Retrieval (CIKM 2007) ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee technology
º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Thanks! Q&A!

More Related Content

PDF
Sparksee overview
bySparsity Technologies
 
PDF
USF Seminar Series: Apache Spark, Machine Learning, Recommendations Feb 05 2016
byChris Fregly
 
PDF
Scotland Data Science Meetup Oct 13, 2015: Spark SQL, DataFrames, Catalyst, ...
byChris Fregly
 
PDF
Advanced Apache Spark Meetup Data Sources API Cassandra Spark Connector Spark...
byChris Fregly
 
PDF
Madrid Spark Big Data Bluemix Meetup - Spark Versus Hadoop @ 100 TB Daytona G...
byChris Fregly
 
PDF
Large Scale Geospatial Indexing and Analysis on Apache Spark
byDatabricks
 
PDF
Istanbul Spark Meetup Nov 28 2015
byChris Fregly
 
PDF
Advanced Analytics and Recommendations with Apache Spark - Spark Maryland/DC ...
byChris Fregly
 
Sparksee overview
bySparsity Technologies
 
USF Seminar Series: Apache Spark, Machine Learning, Recommendations Feb 05 2016
byChris Fregly
 
Scotland Data Science Meetup Oct 13, 2015: Spark SQL, DataFrames, Catalyst, ...
byChris Fregly
 
Advanced Apache Spark Meetup Data Sources API Cassandra Spark Connector Spark...
byChris Fregly
 
Madrid Spark Big Data Bluemix Meetup - Spark Versus Hadoop @ 100 TB Daytona G...
byChris Fregly
 
Large Scale Geospatial Indexing and Analysis on Apache Spark
byDatabricks
 
Istanbul Spark Meetup Nov 28 2015
byChris Fregly
 
Advanced Analytics and Recommendations with Apache Spark - Spark Maryland/DC ...
byChris Fregly
 

What's hot

PDF
5th Athens Big Data Meetup - PipelineIO Workshop - Real-Time Training and Dep...
byAthens Big Data
 
PDF
Spark, Similarity, Approximations, NLP, Recommendations - Boulder Denver Spar...
byChris Fregly
 
PDF
Advanced Apache Spark Meetup Approximations and Probabilistic Data Structures...
byChris Fregly
 
PDF
Barcelona Spain Apache Spark Meetup Oct 20, 2015: Spark Streaming, Kafka, MLl...
byChris Fregly
 
PDF
Chicago Spark Meetup 03 01 2016 - Spark and Recommendations
byChris Fregly
 
PDF
Toronto Spark Meetup Dec 14 2015
byChris Fregly
 
PDF
Spark Summit East NYC Meetup 02-16-2016
byChris Fregly
 
PDF
Advanced Apache Spark Meetup Spark and Elasticsearch 02-15-2016
byChris Fregly
 
PDF
Paris Spark Meetup Oct 26, 2015 - Spark After Dark v1.5 - Best of Advanced Ap...
byChris Fregly
 
PDF
DC Spark Users Group March 15 2016 - Spark and Netflix Recommendations
byChris Fregly
 
PDF
SystemML - Declarative Machine Learning
byLuciano Resende
 
PDF
Dallas DFW Data Science Meetup Jan 21 2016
byChris Fregly
 
PDF
Sydney Spark Meetup Dec 08, 2015
byChris Fregly
 
PDF
Melbourne Spark Meetup Dec 09 2015
byChris Fregly
 
PDF
HUG Italy meet-up with Tugdual Grall, MapR Technical Evangelist
bySpagoWorld
 
PDF
Budapest Big Data Meetup Nov 26 2015
byChris Fregly
 
PDF
Luciano Resende's keynote at Apache big data conference
byLuciano Resende
 
PDF
Strata 2015 Data Preview: Spark, Data Visualization, YARN, and More
byPaco Nathan
 
PDF
Copenhagen Spark Meetup Nov 25, 2015
byChris Fregly
 
5th Athens Big Data Meetup - PipelineIO Workshop - Real-Time Training and Dep...
byAthens Big Data
 
Spark, Similarity, Approximations, NLP, Recommendations - Boulder Denver Spar...
byChris Fregly
 
Advanced Apache Spark Meetup Approximations and Probabilistic Data Structures...
byChris Fregly
 
Barcelona Spain Apache Spark Meetup Oct 20, 2015: Spark Streaming, Kafka, MLl...
byChris Fregly
 
Chicago Spark Meetup 03 01 2016 - Spark and Recommendations
byChris Fregly
 
Toronto Spark Meetup Dec 14 2015
byChris Fregly
 
Spark Summit East NYC Meetup 02-16-2016
byChris Fregly
 
Advanced Apache Spark Meetup Spark and Elasticsearch 02-15-2016
byChris Fregly
 
Paris Spark Meetup Oct 26, 2015 - Spark After Dark v1.5 - Best of Advanced Ap...
byChris Fregly
 
DC Spark Users Group March 15 2016 - Spark and Netflix Recommendations
byChris Fregly
 
SystemML - Declarative Machine Learning
byLuciano Resende
 
Dallas DFW Data Science Meetup Jan 21 2016
byChris Fregly
 
Sydney Spark Meetup Dec 08, 2015
byChris Fregly
 
Melbourne Spark Meetup Dec 09 2015
byChris Fregly
 
HUG Italy meet-up with Tugdual Grall, MapR Technical Evangelist
bySpagoWorld
 
Budapest Big Data Meetup Nov 26 2015
byChris Fregly
 
Luciano Resende's keynote at Apache big data conference
byLuciano Resende
 
Strata 2015 Data Preview: Spark, Data Visualization, YARN, and More
byPaco Nathan
 
Copenhagen Spark Meetup Nov 25, 2015
byChris Fregly
 

Similar to Sparksee Technology overview

PDF
Dex: Introduction
bySparsity Technologies
 
PDF
Distributed graph processing
byBartosz Konieczny
 
PDF
Ted Willke, Senior Principal Engineer & GM, Datacenter Group, Intel at MLconf SF
byMLconf
 
PDF
GraphX: Graph analytics for insights about developer communities
byPaco Nathan
 
PDF
Graph Analytics in Spark
byPaco Nathan
 
PDF
Microservices, containers, and machine learning
byPaco Nathan
 
PDF
Introduction to the graph technologies landscape
byLinkurious
 
PDF
Introduction to the graph technologies landscape
byLinkurious
 
PDF
How Graph Databases used in Police Department?
bySamet KILICTAS
 
PPTX
Dexjava Technical Seminar Dec 2011
bySergio Gomez Villamor
 
PDF
Graph store
byInder Singh
 
PDF
Multiplaform Solution for Graph Datasources
byStratio
 
PPTX
Graphs in data structures are non-linear data structures made up of a finite ...
bybhargavi804095
 
PPTX
Large Scale Graph Analytics with JanusGraph
byDataWorks Summit
 
PPTX
Large Scale Graph Analytics with JanusGraph
byP. Taylor Goetz
 
PDF
DEX: Seminar Tutorial
bySparsity Technologies
 
PDF
Graph Theory and Databases
byPere Urbón-Bayes
 
PDF
A Survey on Graph Database Management Techniques for Huge Unstructured Data
byIJECEIAES
 
PDF
Graph analytic and machine learning
byStanley Wang
 
PDF
Leverage graph technologies to discover hidden insights in your EHS & Sustain...
byiPoint BiS GmbH
 
Dex: Introduction
bySparsity Technologies
 
Distributed graph processing
byBartosz Konieczny
 
Ted Willke, Senior Principal Engineer & GM, Datacenter Group, Intel at MLconf SF
byMLconf
 
GraphX: Graph analytics for insights about developer communities
byPaco Nathan
 
Graph Analytics in Spark
byPaco Nathan
 
Microservices, containers, and machine learning
byPaco Nathan
 
Introduction to the graph technologies landscape
byLinkurious
 
Introduction to the graph technologies landscape
byLinkurious
 
How Graph Databases used in Police Department?
bySamet KILICTAS
 
Dexjava Technical Seminar Dec 2011
bySergio Gomez Villamor
 
Graph store
byInder Singh
 
Multiplaform Solution for Graph Datasources
byStratio
 
Graphs in data structures are non-linear data structures made up of a finite ...
bybhargavi804095
 
Large Scale Graph Analytics with JanusGraph
byDataWorks Summit
 
Large Scale Graph Analytics with JanusGraph
byP. Taylor Goetz
 
DEX: Seminar Tutorial
bySparsity Technologies
 
Graph Theory and Databases
byPere Urbón-Bayes
 
A Survey on Graph Database Management Techniques for Huge Unstructured Data
byIJECEIAES
 
Graph analytic and machine learning
byStanley Wang
 
Leverage graph technologies to discover hidden insights in your EHS & Sustain...
byiPoint BiS GmbH
 

Recently uploaded

PDF
TOS in History.pdf ENGLISH macro.pdf mnbv
bykristinegorre7
 
PDF
Data Science vs Machine Learning vs AI | Industry Skills Explained
bymuhsinam9074
 
PDF
"Master Excel with CA Suvidha Chaplot: 40 Must-Know Tips for Efficiency"
byCA Suvidha Chaplot
 
PDF
Content Strategy Presentation.pdfjjjjhhhhh
byCerineAmmarkhodja1
 
PDF
SF User Group - Account-Contact Relationship Feb 2026
byLukas Lunow
 
PPTX
Internship ppt about multiple disease prediction
byGOUTHAMTr2
 
PDF
제 23회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [MBOAX] : ABSA를 활용한 소비자 반응 분석 기반 운영 효율화 대시보드 설계
byBOAZ Bigdata
 
PPTX
Merge Sort Algorithm: Analysis of Time and Space Complexity.pptx
byrigurahu
 
PDF
Baltic Startup Funding Report 2025 by FIRSTPICK and Practica Capital
byFIRSTPICK VC
 
PDF
"Top Excel Tips and Tricks: Conditional Formatting, Sorting, and More - Learn...
byCA Suvidha Chaplot
 
PPTX
SQL FOR DATA ANALYSTS - MSSQLServer for Analysts
bysirtwumasi77
 
PDF
Whitepaper Optymyze / Expand further (2026)
byOptymyze
 
PPTX
7.__Developing_a_Research_Proposal[1].pptx
bynahomeshetu340
 
PPTX
Smart Crop prediction presentation for final year
bySurya386647
 
PDF
T1 DPB20042 Management Information System.pdf
by68gr55d8f9
 
PDF
"Essential Excel Tips for Beginners & Professionals – 50+ Techniques to Maste...
byCA Suvidha Chaplot
 
PPTX
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
 
PDF
Plainte française dans le cadre de l'affaire Epstein
bySociété Tripalio
 
PDF
What is Context Engineering for Agentforce Developer and Architect
byParis Salesforce Developer Group
 
PDF
Whitepaper / No code end-user app development (2026)
byOptymyze
 
TOS in History.pdf ENGLISH macro.pdf mnbv
bykristinegorre7
 
Data Science vs Machine Learning vs AI | Industry Skills Explained
bymuhsinam9074
 
"Master Excel with CA Suvidha Chaplot: 40 Must-Know Tips for Efficiency"
byCA Suvidha Chaplot
 
Content Strategy Presentation.pdfjjjjhhhhh
byCerineAmmarkhodja1
 
SF User Group - Account-Contact Relationship Feb 2026
byLukas Lunow
 
Internship ppt about multiple disease prediction
byGOUTHAMTr2
 
제 23회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [MBOAX] : ABSA를 활용한 소비자 반응 분석 기반 운영 효율화 대시보드 설계
byBOAZ Bigdata
 
Merge Sort Algorithm: Analysis of Time and Space Complexity.pptx
byrigurahu
 
Baltic Startup Funding Report 2025 by FIRSTPICK and Practica Capital
byFIRSTPICK VC
 
"Top Excel Tips and Tricks: Conditional Formatting, Sorting, and More - Learn...
byCA Suvidha Chaplot
 
SQL FOR DATA ANALYSTS - MSSQLServer for Analysts
bysirtwumasi77
 
Whitepaper Optymyze / Expand further (2026)
byOptymyze
 
7.__Developing_a_Research_Proposal[1].pptx
bynahomeshetu340
 
Smart Crop prediction presentation for final year
bySurya386647
 
T1 DPB20042 Management Information System.pdf
by68gr55d8f9
 
"Essential Excel Tips for Beginners & Professionals – 50+ Techniques to Maste...
byCA Suvidha Chaplot
 
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
 
Plainte française dans le cadre de l'affaire Epstein
bySociété Tripalio
 
What is Context Engineering for Agentforce Developer and Architect
byParis Salesforce Developer Group
 
Whitepaper / No code end-user app development (2026)
byOptymyze
 

Sparksee Technology overview

  • 1.
    Sparksee Graph Database! Technologyoverview! April 2014 º *Sparsity Technologies — Powering Extreme Data sparsity–technologies.com
  • 2.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Graph Databases! Introduction to Sparksee! Sparksee Internals! Performance analysis! High scalability! HPC-SGAB Benchmark !
  • 3.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Graph Databases!
  • 4.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases Graphs are everywhere! ! — Increasing number of huge networks such as the Web, Social Networks, Biological Systems, GPS…! ! — Very large graphs! ! — Interest for analyzing the ! interrelation between the entities ! in theses networks! !
  • 5.
    Classical graph representation! ! —Adjacency matrix! ! Very large NxN sparse matrix, no labels, no multigraph, ! no attributes! — Adjacency list! ! No labels, no attributes, still sparse consuming! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
  • 6.
    Classical graph storage! —Relational database! ! Prefixed schema or very large table for nodes and edges, not ! ! suitable for path traversals and graph exploration! — XML! ! XML data is stored in the form of trees! ! Much work done on finding exact or approximate patterns ! ! (subtrees)! ! Not thought for complex graph queries! — RDF! ! Widely adopted standard for manipulating graph-like data! ! Large support from large vendors! ! SPARQL has become a de facto standard º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
  • 7.
    New approaches tograph analysis! ! — Complex analysis computations on very large distributed graphs ! ! Map-reduce (Pegasus)! ! Vertex-centric computation model (Pregel) ! — Graph Databases: database functionalities to store and query graph-like data ! ! Graph storage in a file system of a computer node with buffer ! ! pool (Neo4j, Hypergraph, OrientDB, Infinitegraph! ! Multiple servers accessible through a load balancer (Neo4j HA) º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
  • 8.
    Requirements for graphdatabases! ! — Data and schema represented as a graph! — Data operations based on graph operations! — Graph-based integrity restrictions! — Multigraphs! — Attributes attached to both vertices and edges! — Graph queries combining edge traversals with attribute ! accesses! — Diversity of workloads! — Efficient secondary memory management! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseGraph Databases
  • 9.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Introduction to Sparksee!
  • 10.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee Sparksee! ! IS a high-performance and out-of-core ! graph database management system ! FOR large scale labeled and attributed multigraphs! ! BASED ON vertical partitioning and collections of objects identifiers stored as bitmaps
  • 11.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee ! Sparksee — Characteristics! ! — Graph split into small structures ! Move to main memory just significant parts (caching) — Object identifiers (oids) instead of complex objects ! Reduce memory requirements — Specific structures to improve traversals ! Index the edges and the neighbors of each node — Attribute indices ! Improve queries based on value filters — Implemented in C++ ! Different APIs (Java, .NET, etc.) through wrappers
  • 12.
    ! ! Sparksee — Capabilities! ! Efficiency !very compact representation using bitmaps. Highly compressible data ! ! structures. Capacity ! more than 100 billion vertices and edges in a single multicore computer. Performance ! subsecond response in recommendation queries. Scalability ! high throughput for concurrent queries. Consistency ! partial transactional support with recovery. Multiplatform ! Linux, Windows, MacOSX, Mobile º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee
  • 13.
    ! Logical graph model! ! Labeled !a label (type) for each vertex and edge ! Directed ! edges can have a fixed direction, from tail to head ! Attributed ! variable list of attributes for each! ! vertex and edge ! Multigraph ! multiple edges between two ! ! vertices ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee
  • 14.
    ! ! ! Sparksee — Architecture! ! ! ! ! ! ! ! ! ! ! ! ! º*SparsityTechnologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseIntroduction to Sparksee GDB GRAPH DATA STRUCTURES PLATFORM DEXCORE SparkseeCpp – Graph Algorithms SWIG SparkseeJava SparkseeNet .NET App JAVA App C++ App BUFFERPOOL Python App Mobile App SparkseePhyton
  • 15.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Sparksee internals
  • 16.
    ! ! ! Graph representation! ! We definea graph G = (V,E,L,T,H,A1,…,Ap) as: ! LABELS L = {(o, l ) | o ∈ (V ∪ E ) ∧ l ∈ string} TAILS T = {(e, t ) | e ∈ E ∧ t ∈ V } HEADS H = {(e, h) | e ∈ E ∧ h ∈ V } ATTRIBUTES Ai = {(o, c ) | o ∈ (V ∪ E ) ∧ c ∈ {int, string, ...}} ! With this representation: — the graph is split into multiple lists of pairs! — the first element of each pair is always a vertex or an edge! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 17.
    Graph representation! ! ! ! ! ! ! º*Sparsity Technologies— Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals L v1, ARTICLE), (v2, ARTICLE),T (e1, v1), (e2, v2), (e3, v4), (e , v ), (e ,H (e1, v3), (e2, v3), (e3, v3), (e , v ), (e ,Aid (v1, 1), (v2, 2), (v3, 3), (v4, 4), (v , 1), (v , 2)Atitle (v1, Europa), (v2, Europe), (v , Europe),Anlc (v1, ca), (v2, fr), (v3, en), (v , en), (e ,Afilename (v5, europe.png), (v , bcn.jpg)Atag (e4, continent)
  • 18.
    ! ! Value sets! ! Groups allpairs of the ! original set with the ! same value as a pair ! between the value and ! the set of objects with ! such value. ! ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals L v1, ARTICLE), (v2, ARTICLE), (v3, ARTICLE), (v4, ARTICLE), (v5, IMAGE), (v6, IMAGE), (e1, BABEL), (e2, BABEL), (e3, REF), (e4, REF), (e5, CONTAINS), (e6, CONTAINS), (e7, CONTAINS) (ARTICLE, {v1, v2, v3, v4}), (BABEL, {e1, e2}), (CONTAINS, {e5, e6, e7}), (IMAGE, {v5, v6}), (REF, {e3, e4}) T (e1, v1), (e2, v2), (e3, v4), (e4, v4), (e5, v3), (e6, v3), (e7, v4) (v1, {e1}), (v2, {e2}), (v3, {e5, e6}), (v4, {e3, e4, e7}) H (e1, v3), (e2, v3), (e3, v3), (e4, v3), (e5, v5), (e6, v6), (e7, v6) (v3, {e1, e2, e3, e4}), (v5, {e5}), (v6, {e6, e7}) Aid (v1, 1), (v2, 2), (v3, 3), (v4, 4), (v5, 1), (v6, 2) (1, {v1, v5}), (2, {v2, v6}), (3, {v3}), (4, {v4}) Atitle (v1, Europa), (v2, Europe), (v3, Europe), (v4, Barcelona) (Barcelona, {v4}), (Europa, {v1}), (Europe, {v2, v3}) Anlc (v1, ca), (v2, fr), (v3, en), (v4, en), (e1, en),(e2, en) (ca, {v1}), (en, {v3, v4, e1, e2}), (fr, {v2}) Afilena me (v5, europe.png), (v6, bcn.jpg) (bcn.jpg, {v6}), (europe.png, {v5}) Atag (e4, continent) (continent, {e4})
  • 19.
    ! Bitmap representation! ! — Eachvertex and edge is identified by a unique and immutable ! oid (object identifier) ! — Each vertex or edge set is stored in a bitmap structure: ! Each position in a bitmap corresponds to the oid of an object! ! Reduced amount of space (compression techniques) ! Very efficient binary logic operations º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 20.
    ! Value set representation! ! —A value set is represented as two maps! ! One maps each different value to a vertex or edge set! ! The other maps each vertex or edge to a value oid ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 21.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals ! Example of a bitmap based representation! ! ! ! ! ! ! ! !
  • 22.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals ! Integrity rules! ! ! ! ! ! ! ! !
  • 23.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals ! ! ! Value set operations! ! domain returns the set of distinct values objects returns the set of vertices or edges associated to a value! lookup returns the set of values ! associated to a set of objects! insert adds a vertex or edge to the ! collection of objects of a value! remove removes a vertex or edge ! from the collection of objects of a value
  • 24.
    Graph query examples —Number of articles! ! |objects (LABELS, ‘ARTICLE’)| — Out-degree of English article ‘Europe’! ! |objects (TAILS, objects( TITLE, ‘Europe’) ∩ objects (NLC, ‘en’) ∩ objects ! (LABELS, ‘ARTICLE’))| — Articles with references to the image with filename ‘bcn.jpg’ ! ! {lookup(TAILS, x ) |x ∈ objects (HEAD, objects (FILENAME, ′ bcn.jpg′ ) ! ! ! ∩ objects (LABELS, ′ IMAGE′ ))} ! — Count the articles of each language {(x , y ) | x ∈ domain(NLC) ∧ y = |(objects (NLC, x ) ∩ objects (LABELS, ! ! ! ′ ARTICLE′ ))|} º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 25.
    ! Implementation details — Bitmapsare compressed by grouping the bits into clusters of 32 consecutive bits (up to 137 billion objects per graph)! — Locality is improved by generating consecutive oids for each distinct vertex or edge labels! — Sorted tree structure of bitmap clusters to speedup the insert, remove, and binary logic operations! — Maps are implemented using B+ trees — The tail, head and attribute value sets have been split into specific value sets for each label º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 26.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Performance analysis
  • 27.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Queries! ! ! ! ! ! ! ! ! ! Q1: Find the article with the largest outdegree and traverse its shortest path tree Q2: Recommend articles related to the most popular one Q3: Find new images for articles from translations in other languages Q4: Find, for each different language, the number of articles and images referenced Q5: For each article with images, materialize the count of images Q6: Remove all articles without images Q1 Q2 Q3 Q4 Q5 Q6 k-hops and path traversals + + graph pattern matching + aggregations and edge connectivity + graph transformation + +
  • 28.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Performance Out-of-core! ! Wikipedia Benchmark out-of-core, 1GB buffer pool. ! ! ! ! ! ! ! ! (⋆) Java VM with 45 GB MonetDb MySQL Neo4J* SPARKSEE Graph Size (GB) 12.00 15.72 42.00 16.98 Load (h) Error 1.36 8.99 2.89 Q1 (s) 4,801.6 > 12 h. > 12 h. 120.5 Q2 (s) 3,788.4 13,841.6 > 12 h. 205.4 Q3 (s) 458.9 33.0 481.0 10.8 Q4 (s) 279,3 45.0 > 12 h. 144.9 Q5 (s) 267.4 930.3 > 12 h. 140.9 Q6 (s) Error 10707.0 > 12 h. 25791.6
  • 29.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Query statistics! ! ! ! ! ! ! ! ! ! ! Query results edge trav. edge trav./sec mem MB bitmaps Q1 624,525 236,387,207 1,987,616.30 832.19 42.97% Q2 5 261,735,954 1,270,747.94 2,974.50 48.59% Q3 51,780 1,536,698 143,885.58 320.81 48.00% Q4 254 4,987,879 33,984.32 245.13 77.67% Q5 2,401,597 5,934,724 42,072.39 319.00 80.64% Q6 52,380,949 281,433,106 37,434.27 11,583.88 67.76% !
  • 30.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Bitmap memory usage! ! ! ! ! ! ! ! ! ! ! Size (MB) Q1 Q2 Q3 Q4 Q5 Q6 LABELS 13.56 11.60 11.60 11.60 11.60 11.60 1.51 TAILS 1,272.32 1,030.90 857.09 229.67 164.79 164.79 90.18 HEADS 633.98 506.98 47.09 Attr. ID 122.77 0.85 Attr. TITLE 835.92 10.87 Attr. NLC 3,618.49 791.29 833.64 617.15 Attr. FILENAME 769.79 Attr. TAG 31.94 2.29 TOTAL 7,298.77 1,042.50 1,375.67 1,032.56 1,010.03 176.39 769.94
  • 31.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Analysis of bitmap usage! ! ! ! ! ! ! ! ! ! !
  • 32.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis ! Bitmap size distribution! ! ! ! ! ! ! ! ! ! !
  • 33.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabasePerformance analysis Out of core stress test! ! ! ! ! ! ! ! ! ! !
  • 34.
    º*Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee technology ! RMAT/Query 1 Scalability Test! ! ! ! ! ! ! ! ! ! ! 228 is out-of-core (2 billion edges)
  • 35.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com º Sparksee Graph Database Sparksee technology SNA Benchmark — Q1, Q6, Q9 and Q12 ! ! ! ! ! ! ! ! !
  • 36.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex High scalability
  • 37.
    ! ! ! High Scalability test— Mirror Servers in Amazon Elastic with a Load Balancer ! ! ! ! ! ! ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee technology
  • 38.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex HPC-SGAB Benchmark
  • 39.
    ! Definition — HPC-SGAB: Badetet al. 2009! ! Measured in TEPS: traversed edges per second! — Graph! ! Synthetic (R-Mat)! ! Power law distribution! ! Average: 8 edges/node — Operations! ! ! Kernel 1: load graph and create indexes! ! ! Kernel 2: find the edge(s) with maximum weight! ! ! Kernel 3: k-hops! ! ! Kernel 4: betweenness centrality (Brandes algorithm) º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 40.
    ! ! Experimental setup — SystemsTested! ! Sparksee (former DEX)! ! Neo4j! ! HypergraphDB! ! Jena (RDF) — Platform! ! Single computer with 2 quad core Xeon E5410! ! 11GB RAM! ! LFF 2.25 TB disk! ! Single threaded — Default benchmark configuration º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 41.
    Summary of results ! ! ! ! ! º*SparsityTechnologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 42.
    Kernel 1 -Load time ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 43.
    Kernel 4 -Betweenness centrality ! ! ! ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee internals
  • 44.
    ! ! Bibliography! ! R. Angles, A.Prat, D. Dominguez, J.L. Larriba, Benchmarking database systems for social network applications (GRADES 2013) ! N. Martínez, V. Muntés, S. Gómez, M.A. Águila, D. Dominguez, J.L. Larriba, Efficient Graph Management Based On Bitmap Indices (IDEAS 2012) ! N. Martínez, S. Gómez, F. Escalé, DEX: a High-Performance Graph Database Management System (GDM 2011) ! D. Dominguez, P. Urbón, A. Giménez, S. Gómez, N. Martínez, and J. L. Larriba, Survey of Graph Database Performance on the HPC Scalable Graph Analysis Benchmark (IWDG 2010) ! N. Martínez, V. Muntés, S. Gómez, J. Nin, M. A. Sánchez, and J. Larriba, Dex: High-performance Exploration on Large Graphs for Information Retrieval (CIKM 2007) ! ! º*Sparsity Technologies — Powering Extreme Data sparsity–technologies.com º Sparksee Graph DatabaseSparksee technology
  • 45.
    º *Sparsity Technologies —Powering Extreme Data sparsity–technologies.com Sparksee Graph DatabaseIndex Thanks! Q&A!