Forget the ORM!
  Persistent data with
Non-traditional Databases
              By Randal L. Schwartz,
      Stonehenge Con...
What we are covering
What we are covering


• What is an Object-Relational-Mapper?
What we are covering


• What is an Object-Relational-Mapper?
• Why is it evil?
What we are covering


• What is an Object-Relational-Mapper?
• Why is it evil?
• What are the alternatives?
Brief review of ORMs
Brief review of ORMs

• Objects in memory
Brief review of ORMs

• Objects in memory
• Tables in RDBMS
Brief review of ORMs

• Objects in memory
• Tables in RDBMS
• Load objects from tables
Brief review of ORMs

• Objects in memory
• Tables in RDBMS
• Load objects from tables
• Store objects (and changes) back ...
Brief review of ORMs

• Objects in memory
• Tables in RDBMS
• Load objects from tables
• Store objects (and changes) back ...
Brief review of ORMs

• Objects in memory
• Tables in RDBMS
• Load objects from tables
• Store objects (and changes) back ...
But things change
But things change

• Object attributes get updated
But things change

• Object attributes get updated
• New objects get created
But things change

• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
But things change

• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
• Updates ...
But things change

• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
• Updates ...
But things change

• Object attributes get updated
• New objects get created
• Need to map that back to the RDB
• Updates ...
One-to-many messes
One-to-many messes

• Messy when an attribute is a reference
One-to-many messes

• Messy when an attribute is a reference
 • Or a non-DB data type, like a set
One-to-many messes

• Messy when an attribute is a reference
 • Or a non-DB data type, like a set
• Mapping involves joins
One-to-many messes

• Messy when an attribute is a reference
 • Or a non-DB data type, like a set
• Mapping involves joins...
One-to-many messes

• Messy when an attribute is a reference
 • Or a non-DB data type, like a set
• Mapping involves joins...
One-to-many messes

• Messy when an attribute is a reference
 • Or a non-DB data type, like a set
• Mapping involves joins...
Why this is bad
Why this is bad
• If your car were an ORM...
Why this is bad
• If your car were an ORM...
 • Reduce it to parts each night in garage
Why this is bad
• If your car were an ORM...
 • Reduce it to parts each night in garage
 • Rebuild it each morning
Why this is bad
• If your car were an ORM...
 • Reduce it to parts each night in garage
 • Rebuild it each morning
• Is th...
Why this is bad
• If your car were an ORM...
 • Reduce it to parts each night in garage
 • Rebuild it each morning
• Is th...
Why this is bad
• If your car were an ORM...
 • Reduce it to parts each night in garage
 • Rebuild it each morning
• Is th...
Why this is bad
• If your car were an ORM...
 • Reduce it to parts each night in garage
 • Rebuild it each morning
• Is th...
How to solve it
How to solve it
• Rectangles don’t fit today’s data
How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• ...
How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• ...
How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• ...
How to solve it
• Rectangles don’t fit today’s data
• Solution: Don’t store rectangles
• Also known as “NoSQL” solutions
• ...
Dropping ACID
Dropping ACID

• Automicity, Consistency, Isolation,
  Durability
Dropping ACID

• Automicity, Consistency, Isolation,
  Durability
• Often expressed through transactions
Dropping ACID

• Automicity, Consistency, Isolation,
  Durability
• Often expressed through transactions
• Most NoSQL offe...
Dropping ACID

• Automicity, Consistency, Isolation,
  Durability
• Often expressed through transactions
• Most NoSQL offe...
Dropping ACID

• Automicity, Consistency, Isolation,
  Durability
• Often expressed through transactions
• Most NoSQL offe...
Dropping ACID

• Automicity, Consistency, Isolation,
  Durability
• Often expressed through transactions
• Most NoSQL offe...
Eric Brewer’s CAP
Eric Brewer’s CAP
• Consistency
Eric Brewer’s CAP
• Consistency
 • All working, or not working
Eric Brewer’s CAP
• Consistency
 • All working, or not working
 • Yes, this is the “A” in ACID
Eric Brewer’s CAP
• Consistency
 • All working, or not working
 • Yes, this is the “A” in ACID
• Availability
Eric Brewer’s CAP
• Consistency
 • All working, or not working
 • Yes, this is the “A” in ACID
• Availability
 • Is the se...
Eric Brewer’s CAP
• Consistency
 • All working, or not working
 • Yes, this is the “A” in ACID
• Availability
 • Is the se...
Eric Brewer’s CAP
• Consistency
 • All working, or not working
 • Yes, this is the “A” in ACID
• Availability
 • Is the se...
Eric Brewer’s CAP
• Consistency
 • All working, or not working
 • Yes, this is the “A” in ACID
• Availability
 • Is the se...
Current solutions
Current solutions

• Document store
Current solutions

• Document store
• Graph
Current solutions

• Document store
• Graph
• Key/Value in RAM/Disk
Current solutions

• Document store
• Graph
• Key/Value in RAM/Disk
• Key/Value on Disk
Current solutions

• Document store
• Graph
• Key/Value in RAM/Disk
• Key/Value on Disk
• Tabular
Current solutions

• Document store
• Graph
• Key/Value in RAM/Disk
• Key/Value on Disk
• Tabular
• Object Database
Document Store
Document Store

• CouchDB
Document Store

• CouchDB
• MongoDB
Document Store

• CouchDB
• MongoDB
• Apache Jackrabbit
Document Store

• CouchDB
• MongoDB
• Apache Jackrabbit
• XML Databases
Document Store

• CouchDB
• MongoDB
• Apache Jackrabbit
• XML Databases
 • MarkLogic Server
Document Store

• CouchDB
• MongoDB
• Apache Jackrabbit
• XML Databases
 • MarkLogic Server
 • GemFire
Document Store

• CouchDB
• MongoDB
• Apache Jackrabbit
• XML Databases
 • MarkLogic Server
 • GemFire
 • eXist
CouchDB
CouchDB
• Open source (Erlang, Apache 2.0)
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
 • Uses SpiderMonkey for processing
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
 • Uses SpiderMonkey for processing
• Provides “views”
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
 • Uses SpiderMonkey for processing
• Provides “views”
...
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
 • Uses SpiderMonkey for processing
• Provides “views”
...
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
 • Uses SpiderMonkey for processing
• Provides “views”
...
CouchDB
• Open source (Erlang, Apache 2.0)
• Stores JSON documents
 • Uses SpiderMonkey for processing
• Provides “views”
...
MongoDB
MongoDB
• Open source (C++, GNU AGPL3)
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive ...
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive ...
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive ...
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive ...
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive ...
MongoDB
• Open source (C++, GNU AGPL3)
• Wire/Storage protocol is BSON
• Embedded JavaScript for map/reduce
• Interactive ...
Apache Jackrabbit
Apache Jackrabbit
• Open source (Java, Apache 2.0)
Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
 • Implements Java Content Repository API
Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
 • Implements Java Content Repository API
• Full ...
Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
 • Implements Java Content Repository API
• Full ...
Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
 • Implements Java Content Repository API
• Full ...
Apache Jackrabbit
• Open source (Java, Apache 2.0)
• XML content on disk
 • Implements Java Content Repository API
• Full ...
MarkLogic Server
MarkLogic Server
• Not open-source
MarkLogic Server
• Not open-source
• XML database
MarkLogic Server
• Not open-source
• XML database
 • Implements XQuery
MarkLogic Server
• Not open-source
• XML database
 • Implements XQuery
• Full-text and structured search
MarkLogic Server
• Not open-source
• XML database
 • Implements XQuery
• Full-text and structured search
• Native geospati...
MarkLogic Server
• Not open-source
• XML database
 • Implements XQuery
• Full-text and structured search
• Native geospati...
MarkLogic Server
• Not open-source
• XML database
 • Implements XQuery
• Full-text and structured search
• Native geospati...
MarkLogic Server
• Not open-source
• XML database
 • Implements XQuery
• Full-text and structured search
• Native geospati...
GemFire
GemFire

• Not open-source
GemFire

• Not open-source
• Mostly-in-memory distributed object cache
GemFire

• Not open-source
• Mostly-in-memory distributed object cache
• Change notification
GemFire

• Not open-source
• Mostly-in-memory distributed object cache
• Change notification
• Highly available
GemFire

• Not open-source
• Mostly-in-memory distributed object cache
• Change notification
• Highly available
• Object Qu...
GemFire

• Not open-source
• Mostly-in-memory distributed object cache
• Change notification
• Highly available
• Object Qu...
GemFire

• Not open-source
• Mostly-in-memory distributed object cache
• Change notification
• Highly available
• Object Qu...
eXist
eXist
• Open source (Java, LGPL)
eXist
• Open source (Java, LGPL)
• Native XML queries
eXist
• Open source (Java, LGPL)
• Native XML queries
 • XQuery, XPath, XSLT, XUpdate
eXist
• Open source (Java, LGPL)
• Native XML queries
 • XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
eXist
• Open source (Java, LGPL)
• Native XML queries
 • XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Ca...
eXist
• Open source (Java, LGPL)
• Native XML queries
 • XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Ca...
eXist
• Open source (Java, LGPL)
• Native XML queries
 • XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Ca...
eXist
• Open source (Java, LGPL)
• Native XML queries
 • XQuery, XPath, XSLT, XUpdate
• Rich APIs: REST, WebDAV, SOAP
• Ca...
Graph
Graph


• InfoGrid
Graph


• InfoGrid
• Neo4J
Graph


• InfoGrid
• Neo4J
• AllegroGraph
InfoGrid
InfoGrid

• Open source (Java, AGPL3)
InfoGrid

• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
InfoGrid

• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
• MeshBase - self contained
InfoGrid

• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
• MeshBase - self contained
• NetMeshBase - d...
InfoGrid

• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
• MeshBase - self contained
• NetMeshBase - d...
InfoGrid

• Open source (Java, AGPL3)
• GraphDatabase stores nodes and edges
• MeshBase - self contained
• NetMeshBase - d...
Neo4j
Neo4j
• Open source (Java, AGPL3)
Neo4j
• Open source (Java, AGPL3)
 • Commercial support/version available
Neo4j
• Open source (Java, AGPL3)
 • Commercial support/version available
• Java objects on disk
Neo4j
• Open source (Java, AGPL3)
 • Commercial support/version available
• Java objects on disk
• Transactional
Neo4j
• Open source (Java, AGPL3)
 • Commercial support/version available
• Java objects on disk
• Transactional
• Scalabl...
Neo4j
• Open source (Java, AGPL3)
 • Commercial support/version available
• Java objects on disk
• Transactional
• Scalabl...
Neo4j
• Open source (Java, AGPL3)
 • Commercial support/version available
• Java objects on disk
• Transactional
• Scalabl...
AllegroGraph
AllegroGraph
• Free versions (not open source)
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)...
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)...
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)...
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)...
AllegroGraph
• Free versions (not open source)
• Holds Resource Description Frameworks
• From Franz, Inc (Franz Lisp, etc)...
Key/Value in RAM/Disk
Key/Value in RAM/Disk



• memcached
Key/Value in RAM/Disk



• memcached
• Redis
Memcached
Memcached
• Open source (C, BSD)
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
 • used by many large sites
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
 • used by many large sites
• ...
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
 • used by many large sites
• ...
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
 • used by many large sites
• ...
Memcached
• Open source (C, BSD)
• LRU cache (data may be lost)
• Developed for LiveJournal
 • used by many large sites
• ...
Redis
Redis
• Open source (C, BSD)
Redis
• Open source (C, BSD)
 • Development funded by VMWare
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists...
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists...
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists...
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists...
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists...
Redis
• Open source (C, BSD)
 • Development funded by VMWare
• Similar to Memcached (key/value pair)
• Values can be lists...
Key/Value on Disk
Key/Value on Disk

• Amazon SimpleDB
Key/Value on Disk

• Amazon SimpleDB
• Berkeley DB
Key/Value on Disk

• Amazon SimpleDB
• Berkeley DB
• Tokyo Cabinet
Key/Value on Disk

• Amazon SimpleDB
• Berkeley DB
• Tokyo Cabinet
• Cassandra
Key/Value on Disk

• Amazon SimpleDB
• Berkeley DB
• Tokyo Cabinet
• Cassandra
• GT.M
Amazon SimpleDB
Amazon SimpleDB

• Built by Amazon in Erlang
Amazon SimpleDB

• Built by Amazon in Erlang
• Supports EC2 applications
Amazon SimpleDB

• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
Amazon SimpleDB

• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
Amazon SimpleDB

• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
• Item...
Amazon SimpleDB

• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
• Item...
Amazon SimpleDB

• Built by Amazon in Erlang
• Supports EC2 applications
• Highly scalable
• “Eventual” Consistency
• Item...
Berkeley DB
Berkeley DB

• Open source (C, Sleepycat)
Berkeley DB

• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
Berkeley DB

• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
• Key/value pairs in various storage for...
Berkeley DB

• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
• Key/value pairs in various storage for...
Berkeley DB

• Open source (C, Sleepycat)
• Enhancement of original DBM from AT&T
• Key/value pairs in various storage for...
Tokyo Cabinet
Tokyo Cabinet
• Open source (C, LGPL)
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
 • Very efficient in both space and speed
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
 • Very efficient in both space and speed
• Simple key/v...
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
 • Very efficient in both space and speed
• Simple key/v...
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
 • Very efficient in both space and speed
• Simple key/v...
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
 • Very efficient in both space and speed
• Simple key/v...
Tokyo Cabinet
• Open source (C, LGPL)
• Successor to GDBM and QDBM
 • Very efficient in both space and speed
• Simple key/v...
Cassandra
Cassandra

• Open source (Java, Apache 2)
Cassandra

• Open source (Java, Apache 2)
• Originally developed at Facebook
Cassandra

• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
Cassandra

• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availa...
Cassandra

• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availa...
Cassandra

• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availa...
Cassandra

• Open source (Java, Apache 2)
• Originally developed at Facebook
• Uses key-column-value storage
• High availa...
GT.M
GT.M

• Open Source (Mumps, GPL)
GT.M

• Open Source (Mumps, GPL)
 • Commercial support from Fidelity
GT.M

• Open Source (Mumps, GPL)
 • Commercial support from Fidelity
• Distributed persistence of key/value pairs
GT.M

• Open Source (Mumps, GPL)
 • Commercial support from Fidelity
• Distributed persistence of key/value pairs
• Transa...
GT.M

• Open Source (Mumps, GPL)
 • Commercial support from Fidelity
• Distributed persistence of key/value pairs
• Transa...
GT.M

• Open Source (Mumps, GPL)
 • Commercial support from Fidelity
• Distributed persistence of key/value pairs
• Transa...
Tabular
Tabular


• Mnesia
Tabular


• Mnesia
• Hbase
Tabular


• Mnesia
• Hbase
• Hypertable
Mnesia
Mnesia

• Open source (Erlang, Erlang license)
Mnesia

• Open source (Erlang, Erlang license)
• Key/value pairs
Mnesia

• Open source (Erlang, Erlang license)
• Key/value pairs
 • Value is any Erlang datatype
Mnesia

• Open source (Erlang, Erlang license)
• Key/value pairs
 • Value is any Erlang datatype
• Live reconfiguration
Mnesia

• Open source (Erlang, Erlang license)
• Key/value pairs
 • Value is any Erlang datatype
• Live reconfiguration
• S...
Hbase
Hbase

• Open source (Java, Apache 2)
Hbase

• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
Hbase

• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
 • Push map/reduce down to shards
Hbase

• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
 • Push map/reduce down to shards
• Runs on Hadoop Dist...
Hbase

• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
 • Push map/reduce down to shards
• Runs on Hadoop Dist...
Hbase

• Open source (Java, Apache 2)
• Mimics Google’s “BigTable”
 • Push map/reduce down to shards
• Runs on Hadoop Dist...
Hypertable
Hypertable


• Open source (C++, GPL 2)
Hypertable


• Open source (C++, GPL 2)
• Yet another BigTable clone
Hypertable


• Open source (C++, GPL 2)
• Yet another BigTable clone
• Designed for highly scalable data storage
Object database
Object database

• Db4o
Object database

• Db4o
• InterSystems Cache
Object database

• Db4o
• InterSystems Cache
• ZODB
Object database

• Db4o
• InterSystems Cache
• ZODB
• GemStone/S
Object database

• Db4o
• InterSystems Cache
• ZODB
• GemStone/S
• Magma
Db4o
Db4o
• Open source (Java/C#, GPL)
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
• Supports object persistence
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
• Supports object persistence
 • both Java and d...
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
• Supports object persistence
 • both Java and d...
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
• Supports object persistence
 • both Java and d...
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
• Supports object persistence
 • both Java and d...
Db4o
• Open source (Java/C#, GPL)
 • Commercial license/support available
• Supports object persistence
 • both Java and d...
InterSystems Cache
InterSystems Cache

• Commercial
InterSystems Cache

• Commercial
• “World’s fastest object database”
InterSystems Cache

• Commercial
• “World’s fastest object database”
• Cross-platform
InterSystems Cache

• Commercial
• “World’s fastest object database”
• Cross-platform
• Persistence of multidimensional ar...
InterSystems Cache

• Commercial
• “World’s fastest object database”
• Cross-platform
• Persistence of multidimensional ar...
InterSystems Cache

• Commercial
• “World’s fastest object database”
• Cross-platform
• Persistence of multidimensional ar...
ZODB
ZODB

• Open source (Python, Zope license)
ZODB

• Open source (Python, Zope license)
• “Seamless object persistence”
ZODB

• Open source (Python, Zope license)
• “Seamless object persistence”
• Transactions, MVCC
ZODB

• Open source (Python, Zope license)
• “Seamless object persistence”
• Transactions, MVCC
• Scaling across a network
ZODB

• Open source (Python, Zope license)
• “Seamless object persistence”
• Transactions, MVCC
• Scaling across a network...
ZODB

• Open source (Python, Zope license)
• “Seamless object persistence”
• Transactions, MVCC
• Scaling across a network...
GemStone/S
GemStone/S
• Commercial (free license for small apps)
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
• Auto...
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
• Auto...
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
• Auto...
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
• Auto...
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
• Auto...
GemStone/S
• Commercial (free license for small apps)
• Smalltalk objects just “persist”
 • Includes Java interface
• Auto...
Magma
Magma

• Open source (Squeak Smalltalk, MIT)
Magma

• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
Magma

• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
Magma

• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-serve...
Magma

• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-serve...
Magma

• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-serve...
Magma

• Open source (Squeak Smalltalk, MIT)
• Transparent Smalltalk object persistence
• “Free GemStone/S”
• Client-serve...
In summary
In summary


• Lots of ways to not use SQL
In summary


• Lots of ways to not use SQL
• Many open source
In summary


• Lots of ways to not use SQL
• Many open source
 • Many commercially supported
In summary


• Lots of ways to not use SQL
• Many open source
 • Many commercially supported
• Don’t assume you need an OR...
How to contact me
How to contact me


• Email: merlyn@stonehenge.com
How to contact me


• Email: merlyn@stonehenge.com
• Twitter: @merlyn
How to contact me


• Email: merlyn@stonehenge.com
• Twitter: @merlyn
• Website: www.stonehenge.com
How to contact me


• Email: merlyn@stonehenge.com
• Twitter: @merlyn
• Website: www.stonehenge.com
• Phone: +1 503 777 00...
Upcoming SlideShare
Loading in …5
×

Forget The ORM!

6,467 views

Published on

Reaching for that SQL database for your next project? Think again! Maybe a non-SQL database is for you!

Published in: Technology

Forget The ORM!

  1. 1. Forget the ORM! Persistent data with Non-traditional Databases By Randal L. Schwartz, Stonehenge Consulting Services, Inc. <merlyn@stonehenge.com> http://www.stonehenge.com/merlyn/ version 1.2 at 13 April 2010
  2. 2. What we are covering
  3. 3. What we are covering • What is an Object-Relational-Mapper?
  4. 4. What we are covering • What is an Object-Relational-Mapper? • Why is it evil?
  5. 5. What we are covering • What is an Object-Relational-Mapper? • Why is it evil? • What are the alternatives?
  6. 6. Brief review of ORMs
  7. 7. Brief review of ORMs • Objects in memory
  8. 8. Brief review of ORMs • Objects in memory • Tables in RDBMS
  9. 9. Brief review of ORMs • Objects in memory • Tables in RDBMS • Load objects from tables
  10. 10. Brief review of ORMs • Objects in memory • Tables in RDBMS • Load objects from tables • Store objects (and changes) back to tables
  11. 11. Brief review of ORMs • Objects in memory • Tables in RDBMS • Load objects from tables • Store objects (and changes) back to tables • Typically mapped one attribute per column
  12. 12. Brief review of ORMs • Objects in memory • Tables in RDBMS • Load objects from tables • Store objects (and changes) back to tables • Typically mapped one attribute per column • Occasionally more complex mappings
  13. 13. But things change
  14. 14. But things change • Object attributes get updated
  15. 15. But things change • Object attributes get updated • New objects get created
  16. 16. But things change • Object attributes get updated • New objects get created • Need to map that back to the RDB
  17. 17. But things change • Object attributes get updated • New objects get created • Need to map that back to the RDB • Updates often require custom SQL
  18. 18. But things change • Object attributes get updated • New objects get created • Need to map that back to the RDB • Updates often require custom SQL • Or live with updating more than needed
  19. 19. But things change • Object attributes get updated • New objects get created • Need to map that back to the RDB • Updates often require custom SQL • Or live with updating more than needed • Detecting what has changed can be hard
  20. 20. One-to-many messes
  21. 21. One-to-many messes • Messy when an attribute is a reference
  22. 22. One-to-many messes • Messy when an attribute is a reference • Or a non-DB data type, like a set
  23. 23. One-to-many messes • Messy when an attribute is a reference • Or a non-DB data type, like a set • Mapping involves joins
  24. 24. One-to-many messes • Messy when an attribute is a reference • Or a non-DB data type, like a set • Mapping involves joins • Left joins to get child rows
  25. 25. One-to-many messes • Messy when an attribute is a reference • Or a non-DB data type, like a set • Mapping involves joins • Left joins to get child rows • Or multiple trips to get joined data
  26. 26. One-to-many messes • Messy when an attribute is a reference • Or a non-DB data type, like a set • Mapping involves joins • Left joins to get child rows • Or multiple trips to get joined data • Either way, expensive
  27. 27. Why this is bad
  28. 28. Why this is bad • If your car were an ORM...
  29. 29. Why this is bad • If your car were an ORM... • Reduce it to parts each night in garage
  30. 30. Why this is bad • If your car were an ORM... • Reduce it to parts each night in garage • Rebuild it each morning
  31. 31. Why this is bad • If your car were an ORM... • Reduce it to parts each night in garage • Rebuild it each morning • Is this sane in the 21st century?
  32. 32. Why this is bad • If your car were an ORM... • Reduce it to parts each night in garage • Rebuild it each morning • Is this sane in the 21st century? • Most ORMs generate the SQL on the fly
  33. 33. Why this is bad • If your car were an ORM... • Reduce it to parts each night in garage • Rebuild it each morning • Is this sane in the 21st century? • Most ORMs generate the SQL on the fly • Recompiling text on each hit?
  34. 34. Why this is bad • If your car were an ORM... • Reduce it to parts each night in garage • Rebuild it each morning • Is this sane in the 21st century? • Most ORMs generate the SQL on the fly • Recompiling text on each hit? • Often called the “object relational impedence mismatch”
  35. 35. How to solve it
  36. 36. How to solve it • Rectangles don’t fit today’s data
  37. 37. How to solve it • Rectangles don’t fit today’s data • Solution: Don’t store rectangles
  38. 38. How to solve it • Rectangles don’t fit today’s data • Solution: Don’t store rectangles • Also known as “NoSQL” solutions
  39. 39. How to solve it • Rectangles don’t fit today’s data • Solution: Don’t store rectangles • Also known as “NoSQL” solutions • Two main options:
  40. 40. How to solve it • Rectangles don’t fit today’s data • Solution: Don’t store rectangles • Also known as “NoSQL” solutions • Two main options: • Document storage (typically JSON)
  41. 41. How to solve it • Rectangles don’t fit today’s data • Solution: Don’t store rectangles • Also known as “NoSQL” solutions • Two main options: • Document storage (typically JSON) • Object storage
  42. 42. How to solve it • Rectangles don’t fit today’s data • Solution: Don’t store rectangles • Also known as “NoSQL” solutions • Two main options: • Document storage (typically JSON) • Object storage • (Generally) not possible if other apps still need the data as RDB
  43. 43. Dropping ACID
  44. 44. Dropping ACID • Automicity, Consistency, Isolation, Durability
  45. 45. Dropping ACID • Automicity, Consistency, Isolation, Durability • Often expressed through transactions
  46. 46. Dropping ACID • Automicity, Consistency, Isolation, Durability • Often expressed through transactions • Most NoSQL offer no multi-update ACID
  47. 47. Dropping ACID • Automicity, Consistency, Isolation, Durability • Often expressed through transactions • Most NoSQL offer no multi-update ACID • But atomic within a single doc update
  48. 48. Dropping ACID • Automicity, Consistency, Isolation, Durability • Often expressed through transactions • Most NoSQL offer no multi-update ACID • But atomic within a single doc update • Thus, think about your “schema” carefully
  49. 49. Dropping ACID • Automicity, Consistency, Isolation, Durability • Often expressed through transactions • Most NoSQL offer no multi-update ACID • But atomic within a single doc update • Thus, think about your “schema” carefully • Ensure single doc update suffices
  50. 50. Eric Brewer’s CAP
  51. 51. Eric Brewer’s CAP • Consistency
  52. 52. Eric Brewer’s CAP • Consistency • All working, or not working
  53. 53. Eric Brewer’s CAP • Consistency • All working, or not working • Yes, this is the “A” in ACID
  54. 54. Eric Brewer’s CAP • Consistency • All working, or not working • Yes, this is the “A” in ACID • Availability
  55. 55. Eric Brewer’s CAP • Consistency • All working, or not working • Yes, this is the “A” in ACID • Availability • Is the service up and running?
  56. 56. Eric Brewer’s CAP • Consistency • All working, or not working • Yes, this is the “A” in ACID • Availability • Is the service up and running? • Partition Tolerance
  57. 57. Eric Brewer’s CAP • Consistency • All working, or not working • Yes, this is the “A” in ACID • Availability • Is the service up and running? • Partition Tolerance • Can parts of it go offline safely?
  58. 58. Eric Brewer’s CAP • Consistency • All working, or not working • Yes, this is the “A” in ACID • Availability • Is the service up and running? • Partition Tolerance • Can parts of it go offline safely? • .... Pick any two
  59. 59. Current solutions
  60. 60. Current solutions • Document store
  61. 61. Current solutions • Document store • Graph
  62. 62. Current solutions • Document store • Graph • Key/Value in RAM/Disk
  63. 63. Current solutions • Document store • Graph • Key/Value in RAM/Disk • Key/Value on Disk
  64. 64. Current solutions • Document store • Graph • Key/Value in RAM/Disk • Key/Value on Disk • Tabular
  65. 65. Current solutions • Document store • Graph • Key/Value in RAM/Disk • Key/Value on Disk • Tabular • Object Database
  66. 66. Document Store
  67. 67. Document Store • CouchDB
  68. 68. Document Store • CouchDB • MongoDB
  69. 69. Document Store • CouchDB • MongoDB • Apache Jackrabbit
  70. 70. Document Store • CouchDB • MongoDB • Apache Jackrabbit • XML Databases
  71. 71. Document Store • CouchDB • MongoDB • Apache Jackrabbit • XML Databases • MarkLogic Server
  72. 72. Document Store • CouchDB • MongoDB • Apache Jackrabbit • XML Databases • MarkLogic Server • GemFire
  73. 73. Document Store • CouchDB • MongoDB • Apache Jackrabbit • XML Databases • MarkLogic Server • GemFire • eXist
  74. 74. CouchDB
  75. 75. CouchDB • Open source (Erlang, Apache 2.0)
  76. 76. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents
  77. 77. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents • Uses SpiderMonkey for processing
  78. 78. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents • Uses SpiderMonkey for processing • Provides “views”
  79. 79. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents • Uses SpiderMonkey for processing • Provides “views” • Aggregate functions
  80. 80. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents • Uses SpiderMonkey for processing • Provides “views” • Aggregate functions • Scales with map/reduce
  81. 81. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents • Uses SpiderMonkey for processing • Provides “views” • Aggregate functions • Scales with map/reduce • RESTful API
  82. 82. CouchDB • Open source (Erlang, Apache 2.0) • Stores JSON documents • Uses SpiderMonkey for processing • Provides “views” • Aggregate functions • Scales with map/reduce • RESTful API • FLOSS Weekly #36
  83. 83. MongoDB
  84. 84. MongoDB • Open source (C++, GNU AGPL3)
  85. 85. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON
  86. 86. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce
  87. 87. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce • Interactive JavaScript shell
  88. 88. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce • Interactive JavaScript shell • In-place updates
  89. 89. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce • Interactive JavaScript shell • In-place updates • Replication
  90. 90. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce • Interactive JavaScript shell • In-place updates • Replication • Auto-sharding
  91. 91. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce • Interactive JavaScript shell • In-place updates • Replication • Auto-sharding • Bindings for many languages
  92. 92. MongoDB • Open source (C++, GNU AGPL3) • Wire/Storage protocol is BSON • Embedded JavaScript for map/reduce • Interactive JavaScript shell • In-place updates • Replication • Auto-sharding • Bindings for many languages • FLOSS Weekly #105
  93. 93. Apache Jackrabbit
  94. 94. Apache Jackrabbit • Open source (Java, Apache 2.0)
  95. 95. Apache Jackrabbit • Open source (Java, Apache 2.0) • XML content on disk
  96. 96. Apache Jackrabbit • Open source (Java, Apache 2.0) • XML content on disk • Implements Java Content Repository API
  97. 97. Apache Jackrabbit • Open source (Java, Apache 2.0) • XML content on disk • Implements Java Content Repository API • Full text and XPath search
  98. 98. Apache Jackrabbit • Open source (Java, Apache 2.0) • XML content on disk • Implements Java Content Repository API • Full text and XPath search • Versioning, transactions, observation
  99. 99. Apache Jackrabbit • Open source (Java, Apache 2.0) • XML content on disk • Implements Java Content Repository API • Full text and XPath search • Versioning, transactions, observation • Authentication
  100. 100. Apache Jackrabbit • Open source (Java, Apache 2.0) • XML content on disk • Implements Java Content Repository API • Full text and XPath search • Versioning, transactions, observation • Authentication • Object persistence using Object Content Manager
  101. 101. MarkLogic Server
  102. 102. MarkLogic Server • Not open-source
  103. 103. MarkLogic Server • Not open-source • XML database
  104. 104. MarkLogic Server • Not open-source • XML database • Implements XQuery
  105. 105. MarkLogic Server • Not open-source • XML database • Implements XQuery • Full-text and structured search
  106. 106. MarkLogic Server • Not open-source • XML database • Implements XQuery • Full-text and structured search • Native geospatial searches
  107. 107. MarkLogic Server • Not open-source • XML database • Implements XQuery • Full-text and structured search • Native geospatial searches • In heavy use by their customers
  108. 108. MarkLogic Server • Not open-source • XML database • Implements XQuery • Full-text and structured search • Native geospatial searches • In heavy use by their customers • Scalable to “hundreds of terabytes”
  109. 109. MarkLogic Server • Not open-source • XML database • Implements XQuery • Full-text and structured search • Native geospatial searches • In heavy use by their customers • Scalable to “hundreds of terabytes” • Both native and RESTful APIs
  110. 110. GemFire
  111. 111. GemFire • Not open-source
  112. 112. GemFire • Not open-source • Mostly-in-memory distributed object cache
  113. 113. GemFire • Not open-source • Mostly-in-memory distributed object cache • Change notification
  114. 114. GemFire • Not open-source • Mostly-in-memory distributed object cache • Change notification • Highly available
  115. 115. GemFire • Not open-source • Mostly-in-memory distributed object cache • Change notification • Highly available • Object Query Language compatible
  116. 116. GemFire • Not open-source • Mostly-in-memory distributed object cache • Change notification • Highly available • Object Query Language compatible • Native Java/C++/C# objects
  117. 117. GemFire • Not open-source • Mostly-in-memory distributed object cache • Change notification • Highly available • Object Query Language compatible • Native Java/C++/C# objects • Persistence to file system, databases, etc
  118. 118. eXist
  119. 119. eXist • Open source (Java, LGPL)
  120. 120. eXist • Open source (Java, LGPL) • Native XML queries
  121. 121. eXist • Open source (Java, LGPL) • Native XML queries • XQuery, XPath, XSLT, XUpdate
  122. 122. eXist • Open source (Java, LGPL) • Native XML queries • XQuery, XPath, XSLT, XUpdate • Rich APIs: REST, WebDAV, SOAP
  123. 123. eXist • Open source (Java, LGPL) • Native XML queries • XQuery, XPath, XSLT, XUpdate • Rich APIs: REST, WebDAV, SOAP • Can serve entire webapps!
  124. 124. eXist • Open source (Java, LGPL) • Native XML queries • XQuery, XPath, XSLT, XUpdate • Rich APIs: REST, WebDAV, SOAP • Can serve entire webapps! • Used by large installations
  125. 125. eXist • Open source (Java, LGPL) • Native XML queries • XQuery, XPath, XSLT, XUpdate • Rich APIs: REST, WebDAV, SOAP • Can serve entire webapps! • Used by large installations • US State Department (history.state.gov)
  126. 126. eXist • Open source (Java, LGPL) • Native XML queries • XQuery, XPath, XSLT, XUpdate • Rich APIs: REST, WebDAV, SOAP • Can serve entire webapps! • Used by large installations • US State Department (history.state.gov) • FLOSS Weekly #97
  127. 127. Graph
  128. 128. Graph • InfoGrid
  129. 129. Graph • InfoGrid • Neo4J
  130. 130. Graph • InfoGrid • Neo4J • AllegroGraph
  131. 131. InfoGrid
  132. 132. InfoGrid • Open source (Java, AGPL3)
  133. 133. InfoGrid • Open source (Java, AGPL3) • GraphDatabase stores nodes and edges
  134. 134. InfoGrid • Open source (Java, AGPL3) • GraphDatabase stores nodes and edges • MeshBase - self contained
  135. 135. InfoGrid • Open source (Java, AGPL3) • GraphDatabase stores nodes and edges • MeshBase - self contained • NetMeshBase - distributed knowledge
  136. 136. InfoGrid • Open source (Java, AGPL3) • GraphDatabase stores nodes and edges • MeshBase - self contained • NetMeshBase - distributed knowledge • API in front of other SQL and NoSQL DBs
  137. 137. InfoGrid • Open source (Java, AGPL3) • GraphDatabase stores nodes and edges • MeshBase - self contained • NetMeshBase - distributed knowledge • API in front of other SQL and NoSQL DBs • Includes components for authentication
  138. 138. Neo4j
  139. 139. Neo4j • Open source (Java, AGPL3)
  140. 140. Neo4j • Open source (Java, AGPL3) • Commercial support/version available
  141. 141. Neo4j • Open source (Java, AGPL3) • Commercial support/version available • Java objects on disk
  142. 142. Neo4j • Open source (Java, AGPL3) • Commercial support/version available • Java objects on disk • Transactional
  143. 143. Neo4j • Open source (Java, AGPL3) • Commercial support/version available • Java objects on disk • Transactional • Scalable (several billion nodes on single machine)
  144. 144. Neo4j • Open source (Java, AGPL3) • Commercial support/version available • Java objects on disk • Transactional • Scalable (several billion nodes on single machine) • Small footprint (JAR under 500KB)
  145. 145. Neo4j • Open source (Java, AGPL3) • Commercial support/version available • Java objects on disk • Transactional • Scalable (several billion nodes on single machine) • Small footprint (JAR under 500KB) • RDF mappings
  146. 146. AllegroGraph
  147. 147. AllegroGraph • Free versions (not open source)
  148. 148. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks
  149. 149. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks • From Franz, Inc (Franz Lisp, etc)
  150. 150. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks • From Franz, Inc (Franz Lisp, etc) • Interfaces in many common languages
  151. 151. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks • From Franz, Inc (Franz Lisp, etc) • Interfaces in many common languages • Active development
  152. 152. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks • From Franz, Inc (Franz Lisp, etc) • Interfaces in many common languages • Active development • In use by FLOSS/commercial/government
  153. 153. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks • From Franz, Inc (Franz Lisp, etc) • Interfaces in many common languages • Active development • In use by FLOSS/commercial/government • Queries with SPARQL
  154. 154. AllegroGraph • Free versions (not open source) • Holds Resource Description Frameworks • From Franz, Inc (Franz Lisp, etc) • Interfaces in many common languages • Active development • In use by FLOSS/commercial/government • Queries with SPARQL • ... Protocol and RDF Query Language
  155. 155. Key/Value in RAM/Disk
  156. 156. Key/Value in RAM/Disk • memcached
  157. 157. Key/Value in RAM/Disk • memcached • Redis
  158. 158. Memcached
  159. 159. Memcached • Open source (C, BSD)
  160. 160. Memcached • Open source (C, BSD) • LRU cache (data may be lost)
  161. 161. Memcached • Open source (C, BSD) • LRU cache (data may be lost) • Developed for LiveJournal
  162. 162. Memcached • Open source (C, BSD) • LRU cache (data may be lost) • Developed for LiveJournal • used by many large sites
  163. 163. Memcached • Open source (C, BSD) • LRU cache (data may be lost) • Developed for LiveJournal • used by many large sites • Simple key/value storage
  164. 164. Memcached • Open source (C, BSD) • LRU cache (data may be lost) • Developed for LiveJournal • used by many large sites • Simple key/value storage • Keys up to 120 bytes, values 1 MB
  165. 165. Memcached • Open source (C, BSD) • LRU cache (data may be lost) • Developed for LiveJournal • used by many large sites • Simple key/value storage • Keys up to 120 bytes, values 1 MB • Clientside horizontal scaling
  166. 166. Memcached • Open source (C, BSD) • LRU cache (data may be lost) • Developed for LiveJournal • used by many large sites • Simple key/value storage • Keys up to 120 bytes, values 1 MB • Clientside horizontal scaling • Can also store on disk (memcachedb)
  167. 167. Redis
  168. 168. Redis • Open source (C, BSD)
  169. 169. Redis • Open source (C, BSD) • Development funded by VMWare
  170. 170. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair)
  171. 171. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair) • Values can be lists, sets, hashes
  172. 172. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair) • Values can be lists, sets, hashes • Master/slave replication supported
  173. 173. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair) • Values can be lists, sets, hashes • Master/slave replication supported • Many client languages available
  174. 174. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair) • Values can be lists, sets, hashes • Master/slave replication supported • Many client languages available • Used by many big sites
  175. 175. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair) • Values can be lists, sets, hashes • Master/slave replication supported • Many client languages available • Used by many big sites • Github, Craigslist, Engine Yard, Guardian
  176. 176. Redis • Open source (C, BSD) • Development funded by VMWare • Similar to Memcached (key/value pair) • Values can be lists, sets, hashes • Master/slave replication supported • Many client languages available • Used by many big sites • Github, Craigslist, Engine Yard, Guardian • Can also store on disk
  177. 177. Key/Value on Disk
  178. 178. Key/Value on Disk • Amazon SimpleDB
  179. 179. Key/Value on Disk • Amazon SimpleDB • Berkeley DB
  180. 180. Key/Value on Disk • Amazon SimpleDB • Berkeley DB • Tokyo Cabinet
  181. 181. Key/Value on Disk • Amazon SimpleDB • Berkeley DB • Tokyo Cabinet • Cassandra
  182. 182. Key/Value on Disk • Amazon SimpleDB • Berkeley DB • Tokyo Cabinet • Cassandra • GT.M
  183. 183. Amazon SimpleDB
  184. 184. Amazon SimpleDB • Built by Amazon in Erlang
  185. 185. Amazon SimpleDB • Built by Amazon in Erlang • Supports EC2 applications
  186. 186. Amazon SimpleDB • Built by Amazon in Erlang • Supports EC2 applications • Highly scalable
  187. 187. Amazon SimpleDB • Built by Amazon in Erlang • Supports EC2 applications • Highly scalable • “Eventual” Consistency
  188. 188. Amazon SimpleDB • Built by Amazon in Erlang • Supports EC2 applications • Highly scalable • “Eventual” Consistency • Items have query-able attributes
  189. 189. Amazon SimpleDB • Built by Amazon in Erlang • Supports EC2 applications • Highly scalable • “Eventual” Consistency • Items have query-able attributes • No text search
  190. 190. Amazon SimpleDB • Built by Amazon in Erlang • Supports EC2 applications • Highly scalable • “Eventual” Consistency • Items have query-able attributes • No text search • Build your own indexes
  191. 191. Berkeley DB
  192. 192. Berkeley DB • Open source (C, Sleepycat)
  193. 193. Berkeley DB • Open source (C, Sleepycat) • Enhancement of original DBM from AT&T
  194. 194. Berkeley DB • Open source (C, Sleepycat) • Enhancement of original DBM from AT&T • Key/value pairs in various storage formats
  195. 195. Berkeley DB • Open source (C, Sleepycat) • Enhancement of original DBM from AT&T • Key/value pairs in various storage formats • Transactional locking, HA features
  196. 196. Berkeley DB • Open source (C, Sleepycat) • Enhancement of original DBM from AT&T • Key/value pairs in various storage formats • Transactional locking, HA features • Widely used
  197. 197. Tokyo Cabinet
  198. 198. Tokyo Cabinet • Open source (C, LGPL)
  199. 199. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM
  200. 200. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM • Very efficient in both space and speed
  201. 201. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM • Very efficient in both space and speed • Simple key/value pairs
  202. 202. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM • Very efficient in both space and speed • Simple key/value pairs • Multiple storage strategies
  203. 203. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM • Very efficient in both space and speed • Simple key/value pairs • Multiple storage strategies • Multiple client language interfaces
  204. 204. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM • Very efficient in both space and speed • Simple key/value pairs • Multiple storage strategies • Multiple client language interfaces • Local storage only: no network interface
  205. 205. Tokyo Cabinet • Open source (C, LGPL) • Successor to GDBM and QDBM • Very efficient in both space and speed • Simple key/value pairs • Multiple storage strategies • Multiple client language interfaces • Local storage only: no network interface • But see Tokyo Tyrant
  206. 206. Cassandra
  207. 207. Cassandra • Open source (Java, Apache 2)
  208. 208. Cassandra • Open source (Java, Apache 2) • Originally developed at Facebook
  209. 209. Cassandra • Open source (Java, Apache 2) • Originally developed at Facebook • Uses key-column-value storage
  210. 210. Cassandra • Open source (Java, Apache 2) • Originally developed at Facebook • Uses key-column-value storage • High availability/elastic through replication
  211. 211. Cassandra • Open source (Java, Apache 2) • Originally developed at Facebook • Uses key-column-value storage • High availability/elastic through replication • Used by Facebook, Digg, Twitter, Rackspace
  212. 212. Cassandra • Open source (Java, Apache 2) • Originally developed at Facebook • Uses key-column-value storage • High availability/elastic through replication • Used by Facebook, Digg, Twitter, Rackspace • Eventually consistent
  213. 213. Cassandra • Open source (Java, Apache 2) • Originally developed at Facebook • Uses key-column-value storage • High availability/elastic through replication • Used by Facebook, Digg, Twitter, Rackspace • Eventually consistent • Queries can ask “majority” or “all”
  214. 214. GT.M
  215. 215. GT.M • Open Source (Mumps, GPL)
  216. 216. GT.M • Open Source (Mumps, GPL) • Commercial support from Fidelity
  217. 217. GT.M • Open Source (Mumps, GPL) • Commercial support from Fidelity • Distributed persistence of key/value pairs
  218. 218. GT.M • Open Source (Mumps, GPL) • Commercial support from Fidelity • Distributed persistence of key/value pairs • Transactions via optimistic concurrency
  219. 219. GT.M • Open Source (Mumps, GPL) • Commercial support from Fidelity • Distributed persistence of key/value pairs • Transactions via optimistic concurrency • Everything you read must be unchanged
  220. 220. GT.M • Open Source (Mumps, GPL) • Commercial support from Fidelity • Distributed persistence of key/value pairs • Transactions via optimistic concurrency • Everything you read must be unchanged • Mumps-to-C-to-Mumps APIs
  221. 221. Tabular
  222. 222. Tabular • Mnesia
  223. 223. Tabular • Mnesia • Hbase
  224. 224. Tabular • Mnesia • Hbase • Hypertable
  225. 225. Mnesia
  226. 226. Mnesia • Open source (Erlang, Erlang license)
  227. 227. Mnesia • Open source (Erlang, Erlang license) • Key/value pairs
  228. 228. Mnesia • Open source (Erlang, Erlang license) • Key/value pairs • Value is any Erlang datatype
  229. 229. Mnesia • Open source (Erlang, Erlang license) • Key/value pairs • Value is any Erlang datatype • Live reconfiguration
  230. 230. Mnesia • Open source (Erlang, Erlang license) • Key/value pairs • Value is any Erlang datatype • Live reconfiguration • Supports transactions and distribution
  231. 231. Hbase
  232. 232. Hbase • Open source (Java, Apache 2)
  233. 233. Hbase • Open source (Java, Apache 2) • Mimics Google’s “BigTable”
  234. 234. Hbase • Open source (Java, Apache 2) • Mimics Google’s “BigTable” • Push map/reduce down to shards
  235. 235. Hbase • Open source (Java, Apache 2) • Mimics Google’s “BigTable” • Push map/reduce down to shards • Runs on Hadoop Distributed File System
  236. 236. Hbase • Open source (Java, Apache 2) • Mimics Google’s “BigTable” • Push map/reduce down to shards • Runs on Hadoop Distributed File System • Java, REST, Thrift APIs
  237. 237. Hbase • Open source (Java, Apache 2) • Mimics Google’s “BigTable” • Push map/reduce down to shards • Runs on Hadoop Distributed File System • Java, REST, Thrift APIs • Might be the DB behind Bing
  238. 238. Hypertable
  239. 239. Hypertable • Open source (C++, GPL 2)
  240. 240. Hypertable • Open source (C++, GPL 2) • Yet another BigTable clone
  241. 241. Hypertable • Open source (C++, GPL 2) • Yet another BigTable clone • Designed for highly scalable data storage
  242. 242. Object database
  243. 243. Object database • Db4o
  244. 244. Object database • Db4o • InterSystems Cache
  245. 245. Object database • Db4o • InterSystems Cache • ZODB
  246. 246. Object database • Db4o • InterSystems Cache • ZODB • GemStone/S
  247. 247. Object database • Db4o • InterSystems Cache • ZODB • GemStone/S • Magma
  248. 248. Db4o
  249. 249. Db4o • Open source (Java/C#, GPL)
  250. 250. Db4o • Open source (Java/C#, GPL) • Commercial license/support available
  251. 251. Db4o • Open source (Java/C#, GPL) • Commercial license/support available • Supports object persistence
  252. 252. Db4o • Open source (Java/C#, GPL) • Commercial license/support available • Supports object persistence • both Java and dot-net
  253. 253. Db4o • Open source (Java/C#, GPL) • Commercial license/support available • Supports object persistence • both Java and dot-net • Can replicate to traditional RDBMS
  254. 254. Db4o • Open source (Java/C#, GPL) • Commercial license/support available • Supports object persistence • both Java and dot-net • Can replicate to traditional RDBMS • Supports class migration
  255. 255. Db4o • Open source (Java/C#, GPL) • Commercial license/support available • Supports object persistence • both Java and dot-net • Can replicate to traditional RDBMS • Supports class migration • ... if you provide the code
  256. 256. Db4o • Open source (Java/C#, GPL) • Commercial license/support available • Supports object persistence • both Java and dot-net • Can replicate to traditional RDBMS • Supports class migration • ... if you provide the code • Large community
  257. 257. InterSystems Cache
  258. 258. InterSystems Cache • Commercial
  259. 259. InterSystems Cache • Commercial • “World’s fastest object database”
  260. 260. InterSystems Cache • Commercial • “World’s fastest object database” • Cross-platform
  261. 261. InterSystems Cache • Commercial • “World’s fastest object database” • Cross-platform • Persistence of multidimensional arrays
  262. 262. InterSystems Cache • Commercial • “World’s fastest object database” • Cross-platform • Persistence of multidimensional arrays • Similar to MUMPS/Pick data
  263. 263. InterSystems Cache • Commercial • “World’s fastest object database” • Cross-platform • Persistence of multidimensional arrays • Similar to MUMPS/Pick data • Can be embedded in web pages
  264. 264. ZODB
  265. 265. ZODB • Open source (Python, Zope license)
  266. 266. ZODB • Open source (Python, Zope license) • “Seamless object persistence”
  267. 267. ZODB • Open source (Python, Zope license) • “Seamless object persistence” • Transactions, MVCC
  268. 268. ZODB • Open source (Python, Zope license) • “Seamless object persistence” • Transactions, MVCC • Scaling across a network
  269. 269. ZODB • Open source (Python, Zope license) • “Seamless object persistence” • Transactions, MVCC • Scaling across a network • Widely used
  270. 270. ZODB • Open source (Python, Zope license) • “Seamless object persistence” • Transactions, MVCC • Scaling across a network • Widely used • Zope Replication Services (commercial)
  271. 271. GemStone/S
  272. 272. GemStone/S • Commercial (free license for small apps)
  273. 273. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist”
  274. 274. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface
  275. 275. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface • Automatic and/or guided class upgrades
  276. 276. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface • Automatic and/or guided class upgrades • Horizontally scaleable
  277. 277. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface • Automatic and/or guided class upgrades • Horizontally scaleable • OOCL manages 40% of overseas traffic
  278. 278. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface • Automatic and/or guided class upgrades • Horizontally scaleable • OOCL manages 40% of overseas traffic • JPM’s Kapital created the financial crisis :)
  279. 279. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface • Automatic and/or guided class upgrades • Horizontally scaleable • OOCL manages 40% of overseas traffic • JPM’s Kapital created the financial crisis :) • Integrates with Seaside
  280. 280. GemStone/S • Commercial (free license for small apps) • Smalltalk objects just “persist” • Includes Java interface • Automatic and/or guided class upgrades • Horizontally scaleable • OOCL manages 40% of overseas traffic • JPM’s Kapital created the financial crisis :) • Integrates with Seaside • Web apps with transparent persistence
  281. 281. Magma
  282. 282. Magma • Open source (Squeak Smalltalk, MIT)
  283. 283. Magma • Open source (Squeak Smalltalk, MIT) • Transparent Smalltalk object persistence
  284. 284. Magma • Open source (Squeak Smalltalk, MIT) • Transparent Smalltalk object persistence • “Free GemStone/S”
  285. 285. Magma • Open source (Squeak Smalltalk, MIT) • Transparent Smalltalk object persistence • “Free GemStone/S” • Client-server model
  286. 286. Magma • Open source (Squeak Smalltalk, MIT) • Transparent Smalltalk object persistence • “Free GemStone/S” • Client-server model • HA mode (multiple slaves ready for master)
  287. 287. Magma • Open source (Squeak Smalltalk, MIT) • Transparent Smalltalk object persistence • “Free GemStone/S” • Client-server model • HA mode (multiple slaves ready for master) • Transaction-based (commit/rollback)
  288. 288. Magma • Open source (Squeak Smalltalk, MIT) • Transparent Smalltalk object persistence • “Free GemStone/S” • Client-server model • HA mode (multiple slaves ready for master) • Transaction-based (commit/rollback) • Works nicely with Seaside
  289. 289. In summary
  290. 290. In summary • Lots of ways to not use SQL
  291. 291. In summary • Lots of ways to not use SQL • Many open source
  292. 292. In summary • Lots of ways to not use SQL • Many open source • Many commercially supported
  293. 293. In summary • Lots of ways to not use SQL • Many open source • Many commercially supported • Don’t assume you need an ORM!
  294. 294. How to contact me
  295. 295. How to contact me • Email: merlyn@stonehenge.com
  296. 296. How to contact me • Email: merlyn@stonehenge.com • Twitter: @merlyn
  297. 297. How to contact me • Email: merlyn@stonehenge.com • Twitter: @merlyn • Website: www.stonehenge.com
  298. 298. How to contact me • Email: merlyn@stonehenge.com • Twitter: @merlyn • Website: www.stonehenge.com • Phone: +1 503 777 0095

×