ICOODB 2010 - Frankfurt, Deutschland




              The Synergy Between the Object
             Database, Graph Databas...
AGENDA
        • Historical Overview
        • Inherent Advantages of ODBMSs
        • Technology Evolution
        • Leve...
Historical Overview




3   Leon Guzenda at ICOODB 2010     © Objectivity, Inc. 2010
The ODBMS Players
     1986       1988      1990         1992      1994      1996      1998         2000       2002   2004...
ODBMS Evolution
• 1980s
     – “Performance, Performance, Performance!”
                                                  ...
Inherent ODBMS Advantages




6   Leon Guzenda at ICOODB 2010   © Objectivity, Inc. 2010
Faster Navigation
    PROBLEM:
    Find all of the Suspects
    linked to a chosen Incident
                              ...
Lower Query Latency
    Relational
                                    Receive &
     O/R              Send               ...
Technology Evolution




9   Leon Guzenda at ICOODB 2010   © Objectivity, Inc. 2010
Grids and Clouds
• 1996 - CERN started looking for a DBMS for the LHC.


• RD45 team verified that a distributed ODBMS coul...
The “NoSQL” Movement
• Some web application developers found RDBMSs too
  restrictive.
• They were dealing with:
     – Hu...
Sharding
Splits large tables into groups of related rows and puts them onto separate servers. They each have a schema.



...
Hadoop’s HDFS
• Hadoop Distributed File System (HDFS) is the primary storage system used by Hadoop
     applications.
• HD...
Leveraging Technologies




14   Leon Guzenda at ICOODB 2010   © Objectivity, Inc. 2010
Objectivity/DB
• Federated object-oriented database platform
• Single Logical View across distributed persistent objects
•...
Distributed Architecture
                     Client                            Simple, Distributed Servers

          App...
Parallel Query Engine

 Application                                                                Lock Server
           ...
Objectivity/DB Advantages
       Fully Distributed with Client-Side Smart Caching

       Highly efficient storage and navi...
Objectivity/DB 10.1

     • User-replaceable Parallel Query Engine
       search agents
     • Page level and partial back...
Graph Databases




20   Leon Guzenda at ICOODB 2010        © Objectivity, Inc. 2010
The Link Hunter




21   Leon Guzenda at ICOODB 2010        © Objectivity, Inc. 2010
Graph Databases

     • Nodes are represented as Vertices
                                                                ...
InfiniteGraph...

     •Dedicated Graph API
     •Easy to use and deploy
     •Java now, C# soon...
     •Built on Objecti...
...InfiniteGraph...
• Create and update graph structures
• Traversal with constraints:
     – Return only designated Edge ...
...InfiniteGraph
• Search and Query Processing
     – Supports Objectivity/DB and Lucene indexing
     – Key and range que...
InfiniteGraph Licensing
• 60-day free trial
• Free GoGrid cloud development and
 deployment environment for qualified
 star...
Case Studies




27   Leon Guzenda at ICOODB 2010       © Objectivity, Inc. 2010
Objectivity/DB & Hadoop...
 HDFS Weaknesses

 • HDFS cannot be directly mounted by an existing operating system
      • Ge...
...Objectivity/DB & Hadoop...

                                                                                           ...
...Objectivity/DB & Hadoop

                                                                                 Name Node FD’...
InfiniteGraph and Cassandra
• Apache Cassandra provides a structured key-value
  store with eventual consistency.
• It is ...
Summary




32   Leon Guzenda at ICOODB 2010    © Objectivity, Inc. 2010
NoSQL?
•All of these features could have been obtained
 from “Commercial Off The Shelf” ODBMSs:

        • Unique object/d...
Summary
     • We need to re-examine the reasons that the
       NoSQL movement didn’t just use ODBMSs.
     • The NoSQL m...
Questions?

             • objectivity.com - White papers, downloads etc.
             • EMAIL: info @ objectivity.com


 ...
Upcoming SlideShare
Loading in …5
×

The Synergy Between the Object Database, Graph Database, Cloud Computing and NoSQL Paradigms

2,103 views

Published on

Leon Guzenda's presentation at the International Conference on Objects and Databases (ICOODB) 2010

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,103
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

The Synergy Between the Object Database, Graph Database, Cloud Computing and NoSQL Paradigms

  1. 1. ICOODB 2010 - Frankfurt, Deutschland The Synergy Between the Object Database, Graph Database, Cloud Computing and NoSQL Paradigms Leon Guzenda - Objectivity, Inc. 1 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  2. 2. AGENDA • Historical Overview • Inherent Advantages of ODBMSs • Technology Evolution • Leveraging Technologies • Graph Databases • Summary 2 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  3. 3. Historical Overview 3 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  4. 4. The ODBMS Players 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2009 2010 Matisse VBase (Ontologic)  Ontos GemStone (Servio Logic  Servio  GemStone) Objectivity/DB InfiniteGraph ObjectStore (Object Design  Excelon  Progress) GBase (Graphael) Versant Object Database (Versant) O2 ( Ardent  Ascential) UniSQL Poet  FastObjects db4objects Note that many of these companies were founded earlier, e.g. Objectivity, Inc. Berkeley DB (Sleepycat  Oracle) was founded in June 1988. 4 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  5. 5. ODBMS Evolution • 1980s – “Performance, Performance, Performance!” DATA – Primarily scientific and engineering applications MANIPULATION • 1990s Applications – Reliability and Scalability tended to generate data and – New languages and Operating Systems relationships – Large deployments in the scientific domain • 2000s RELATIONSHIP – Ease of use and instrumentation ANALYTICS – Query languages Applications – Performance and scalability ingest and correlate data and – Grids and Clouds relationships – Embedded systems, government and more... 5 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  6. 6. Inherent ODBMS Advantages 6 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  7. 7. Faster Navigation PROBLEM: Find all of the Suspects linked to a chosen Incident Incident_Table Join_Table Suspect_Table Relational solution: N * 2 B-Tree lookups N * 2 logical reads B-Trees Incident Suspects Objectivity/DB solution: 1 B-Tree lookup 1 + N logical reads Significantly faster navigation of relationships 7 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  8. 8. Lower Query Latency Relational Receive & O/R Send Access Read Create Return Interpret Optimize Qualify Loop Mapping Request Indices Data View Result Request Objectivity/DB Qualified objects are Initialize Start Access Qualify Open Loop returned as soon as Iterator Loop Indices Object they are found. Time 8 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  9. 9. Technology Evolution 9 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  10. 10. Grids and Clouds • 1996 - CERN started looking for a DBMS for the LHC. • RD45 team verified that a distributed ODBMS could handle complexity, performance and Petabyte+ scale. • Lead to grid deployments at SLAC and Brookhaven • Meanwhile, developers migrated from CORBA to SOA. • Then, our grid and SOA experience made the migration to cloud environments very easy. 10 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  11. 11. The “NoSQL” Movement • Some web application developers found RDBMSs too restrictive. • They were dealing with: – Huge parallel ingest streams – Applications that scan or navigate rather than query – Unconventional transaction models. • So... they re-invented the wheel! – Sharding (Hadoop and Big Table) – Key-Value tables (Big Table) – Dynamo... 11 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  12. 12. Sharding Splits large tables into groups of related rows and puts them onto separate servers. They each have a schema. Schema 1 Client Split/Combine Server Africa & Antarctica Product Catalog • Big tables can be split Schema 1 Server Americas • Small “hot” tables may be replicated Product Catalog Schema 1 • The client has to figure out Server Australasia and Asia which server to use. Product Catalog • Data has to be compatible with the Server ETC. client hardware, OS and language. 12 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  13. 13. Hadoop’s HDFS • Hadoop Distributed File System (HDFS) is the primary storage system used by Hadoop applications. • HDFS creates multiple replicas of 64+ Megabyte data blocks and distributes them on compute nodes throughout a cluster to enable reliable, extremely rapid computations. • Each data block is replicated 3 times - twice on the same rack and once on another rack. Client HDFS client Name Node Directory Data Node Data block 1a Data block 1b • The client needs to know Data block 1c the name of the block to be Data block 2 operated on. Data block 3 • The block is copied to the Data Node Data block 4 client for processing. 13 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  14. 14. Leveraging Technologies 14 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  15. 15. Objectivity/DB • Federated object-oriented database platform • Single Logical View across distributed persistent objects • Eliminates the OO language to database mapping layer • Ultra-fast object navigation • Customizable, distributed query engine • Dynamic schema changes and low administration overheads • Interoperability across multiple languages and platforms • High Availability • Grid and cloud environment enabled 15 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  16. 16. Distributed Architecture Client Simple, Distributed Servers Application Lock Server Lock Server Smart Lock Server Cache Query Server Objectivity/DB Lock Server Data Server Enhances scalability and availability 16 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  17. 17. Parallel Query Engine Application Lock Server Lock Server Smart Lock Server Filter/ Cache Query Server Query Server Gateway PQE Objectivity/DB Task Lock Server Splitter Data Server The Task Splitter aims queries at specific Filters can run complex qualification methods. databases and containers Gateways can access other databases or search engines. Replaceable components for smarter optimization 17 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  18. 18. Objectivity/DB Advantages Fully Distributed with Client-Side Smart Caching Highly efficient storage and navigation of relationships Flexible Clustering Scalable Collections Customizable Parallel Query Engine Quorum-based Replication and High Availability Flexible, Multi-mode Indexing Fully Interoperable Across Platforms and Languages 18 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  19. 19. Objectivity/DB 10.1 • User-replaceable Parallel Query Engine search agents • Page level and partial backups • Eclipse RCP • Visual Studio 2010 support • Mac OS X support 19 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  20. 20. Graph Databases 20 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  21. 21. The Link Hunter 21 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  22. 22. Graph Databases • Nodes are represented as Vertices Vertex • Relationships are represented as Edges • Edges may be weighted Edge • Both are regular objects – Properties – Methods – Inheritance 22 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  23. 23. InfiniteGraph... •Dedicated Graph API •Easy to use and deploy •Java now, C# soon... •Built on Objectivity/DB •Fully interoperable 23 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  24. 24. ...InfiniteGraph... • Create and update graph structures • Traversal with constraints: – Return only designated Edge types – Return if not an excluded Edge type – Direction matches intent – Properties do not match a provided predicate – Maximum path depth has been reached... • Optional Event notification – Target vertex was found – Path is being abandoned 24 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  25. 25. ...InfiniteGraph • Search and Query Processing – Supports Objectivity/DB and Lucene indexing – Key and range queries – Full text searching – Regular expression search of string-based keys • Path Finding – Start and end vertices – Maximum depth – Vertex type inclusion and exclusion lists – Edge type list 25 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  26. 26. InfiniteGraph Licensing • 60-day free trial • Free GoGrid cloud development and deployment environment for qualified startups and non-profits –Licenses must be procured after a pre-agreed annual revenue level is reached • Open Source framework licenses • Standard commercial licenses 26 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  27. 27. Case Studies 27 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  28. 28. Objectivity/DB & Hadoop... HDFS Weaknesses • HDFS cannot be directly mounted by an existing operating system • Getting data into and out of the HDFS file system, an action that often needs to be performed before and after executing a job, can be inconvenient. • A filesystem in userspace has been developed to address this problem, at least for Linux and some other Unix systems. • It moves 64+ MB blocks and doesnʼt support POSIX file operations • Replicating data three times is costly. • However, there is a version that uses a parity block, decreasing the physical storage requirements from 3x to around 2.2x. • The Name Node is a single point of failure • If the name node goes down, the filesystem is offline. When it comes back up the name node must replay all outstanding operations. This replay process can take over half an hour for a big cluster. 28 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  29. 29. ...Objectivity/DB & Hadoop... 64+ MB Container file/blocks PAGE OOFS Objectivity/DB + HDFS Client SERVER Security (HDFS client) “AMS PAGE SERVER” Cache • Problem: HDFS works best when transferring 64+ MB blocks of data. Objectivity/DB works best with 16-64KB blocks. • Solution: Implement a memory cache for the HDFS blocks. 29 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  30. 30. ...Objectivity/DB & Hadoop Name Node FD’ Client HDFS client Objectivity/DB Name Node FD Data Node Data block 1a Data block 1b • Removes the single point Data block 1c of failure by using an Data block 2 Objectivity/DB Name Node Data block 3 Federation (replicated). Data Node Data block 4 • Could provide other services too, such as content tagging and connectivity. 30 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  31. 31. InfiniteGraph and Cassandra • Apache Cassandra provides a structured key-value store with eventual consistency. • It is a resilient, distributed DBMS. • The prototype uses social network data. • It extracts data from Cassandra, then finds the shortest paths between people. Application InfiniteGraph Cassandra 31 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  32. 32. Summary 32 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  33. 33. NoSQL? •All of these features could have been obtained from “Commercial Off The Shelf” ODBMSs: • Unique object/document IDs • Flexible object clustering • Sharding • Effectivity (data in a relationship) • Shared-Nothing • Geospatial and multi-dimensional indexing • Fully distributed • Hash table (key-value) lookups • No lock and novel transaction modes • Hyperspace = = single logical view of a federation • Iterators and fast, predictable traversals • Multi-way replication • Fast scans • High Availability • Optimization for random access • Text searching • In Memory Database configurability 33 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  34. 34. Summary • We need to re-examine the reasons that the NoSQL movement didn’t just use ODBMSs. • The NoSQL movement helps strengthen our argument that RDBMSs aren’t always the best choice. • Graph DBMSs can supplement RDBMS, NoSQL and ODBMS technologies. • The best Graph DBMSs are built on ODBMSs. • ODBMSs are here to stay. 34 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010
  35. 35. Questions? • objectivity.com - White papers, downloads etc. • EMAIL: info @ objectivity.com • infinitegraph.com - WPs, downloads etc. • EMAIL: info @ infinitegraph.com • Presenter: leon @ objectivity.com If they give you paper with lines on, write sideways! 35 Leon Guzenda at ICOODB 2010 © Objectivity, Inc. 2010

×