The Elastically Scalable     Database™                           1	  
20th Century Database                                                        Powerful Query                            9% ...
21st Century Problem       Commodity Datacenters                               ✗                   Big Data ✗             ...
Database Crisis           Amazon           Flickr    WikipediaFacebook                                      Google        ...
Jim Starkey“Elastically Scalable Transactions represent the biggest breakthroughin database technology in 25 years”       ...
Emergent Database                     Architecture"    “An emergent     behavior can     appear when a     number of     s...
NuoDB Plus One"  Second machinetypically doubles TPS"  Second machine isadded to live databasewhile it is running at1,000’...
Adding a Third Machine"  Third machinetypically triples singlemachine TPS"  Third machine isadded to live databasewhile it...
More Machines? Bring ‘em On          Nodes   TPSMySQL      1     3,000NuoDB      1     4,500                   27,00NuoDB ...
Or Scale-out on IAAS‣  Nuodb scales linearly  on EC2‣  Per-node  performance on  m1.large nodes           TPS	    approx 5...
Trek Palmer
Building An Elastically Scalable   Database The Easy Way
Tonight’s Agenda"   A Bit About Me"   Introduction to NuoDB"   Architecture Overview"   Some Unnatural Acts     "    Quick...
A Bit About Me"    A Refugee from Academia     "     -Researched Programming Languages     "     -Transactional Memory imp...
What is NuoDB?"    Elastically scalable"    Multi-tenant"    Transactionally Consistent"    Easy to Manage                ...
娜 graceful, like a cloud                           1                           6	  
Architecture"    Three tiers     "    Each is independent    Management	       "    Single model for all          environm...
Agents"    Management tier"    Provision hosts for use"    Expose XML messaging for     management"    Make scripting and ...
Brokers"    Agent with additional special     knowledge"    At least one per domain"    Redirects clients to TE     "     ...
Transaction Engines"    Peer-to-peer"    In-memory"    Multi-Version Concurrency Control"    Asynchronous messaging (repli...
Storage Managers"    Persistence points for atoms"    Key-value backing stores"     -Local FS, S3, HDFS"    Independent ar...
And Now, Some Unnatural Acts                               2                               2	  
Sharding, an Unnatural Act"    The ideal DB application                                   Scales	  up	  to	  the	  capacit...
Sharding"    Shard the DB among several nodes     New	  Client	  Layer	                Now	  you	  need	  to	  implement	 ...
Other Sharding Bugbears"    Global operations (searches, scans)"     -Doing joins in the application"     -Implementing Cu...
Scaling Shards"    A recipe for changing the number of     shards"     1) Ask boss for permission"     2) Provision hardwa...
NuoDB SolutionManagement	  Client	  Domain	      Host A            Host B           Host C        	   Broker	     	   Agen...
NuoDB on a single node Management	                        Client	   Client	   Domain 	      Host A                    Host...
NuoDB Scaling outManagement	                       Client	              Client	         Client	  Client	  Domain	      Hos...
NuoDB ‘Adding a Shard’ Management	                        Client	              Client	         Client	   Client	   Domain ...
NuoDB ‘Sharding’"    Literally as simple as just adding     nodes"    No client code had to be harmed in     the making of...
Eventual Consistency"    Eventual consistency is latent     inconsistency"      -Not transactionally consistent"      -App...
NuoDB Consistency"    NuoDB is transactionally consistent"    All the time, everywhere"    When a transaction is committed...
Multi-Tenancy"    Traditional databases monopolize a     node"    NuoDB supports many databases in a     single pool of ma...
Multi-Tenancy ExampleManagement	                              JDBC	  Client	  Client	  Domain	      Host A              Ho...
Multi-Tenancy ExampleManagement	                JDBC	  Client	  Client	                              JDBC	  Client	  Domai...
Multi-Tenancy ExampleManagement	                JDBC	  Client	  Client	                              JDBC	  Client	  Domai...
Multi-Tenancy ExampleManagement	                JDBC	  Client	                                                   SQL	  Cli...
And now, a demo…
The Elastically Scalable     Database™                           40	  
Upcoming SlideShare
Loading in …5
×

NYC Meetup November 15, 2012

480 views

Published on

Wiqar Chaudry and Trek Palmer present NuoDB and how to overcome administrative acrobatics with your database.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
480
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
21
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

NYC Meetup November 15, 2012

  1. 1. The Elastically Scalable Database™ 1  
  2. 2. 20th Century Database Powerful Query 9% 3% 4% Language 44% 19% Industry StandardsORACLEIBM Data GuaranteesMicrosoftSybase 21% ToolsTeradataOthers Employee Skills Existing Data 2  
  3. 3. 21st Century Problem Commodity Datacenters ✗ Big Data ✗ Powerful Query Language Modern Workloads ✗ Industry Standards 24x7 Operation ✗ Data Guarantees Geo-distribution ✗ ToolsDeveloperEmpowerment ✗ Employee Skills Existing Data 3  
  4. 4. Database Crisis Amazon Flickr WikipediaFacebook Google Source: Marc Bojoly 4  
  5. 5. Jim Starkey“Elastically Scalable Transactions represent the biggest breakthroughin database technology in 25 years” ‣  DEC RDB/ELN ‣  InterBase ‣  Firebird ‣  Falcon ‣  BLOBS ‣  MVCC 5  
  6. 6. Emergent Database Architecture"  “An emergent behavior can appear when a number of simple entities operate in an environment, forming more complex behaviors as a collective.” "   - Wikipedia 6  
  7. 7. NuoDB Plus One"  Second machinetypically doubles TPS"  Second machine isadded to live databasewhile it is running at1,000’s of TPS"  Performance increase isimmediate"  BTW - you can takeeither machine away and Second  Machine    the database keeps Instant  Performance  Increase  running without data loss 7  
  8. 8. Adding a Third Machine"  Third machinetypically triples singlemachine TPS"  Third machine isadded to live databasewhile it is running at1,000’s of TPS"  Performance increase Second  &  Third  Machine    is immediate Instant  Performance  Increase  "  BTW - you can takeany machine away andthe database keepsrunning without dataloss 8  
  9. 9. More Machines? Bring ‘em On Nodes TPSMySQL 1 3,000NuoDB 1 4,500 27,00NuoDB 9 0"  Technical Details: TPS  ‣  2-9 Tx engines‣  1 storage manager‣  Best sustained TPS and # clients Number  of  Nodes   combination‣  50% updates NuoDB running on 9 nodes was approx. 9x faster than MySQL running on 1 node. 9  
  10. 10. Or Scale-out on IAAS‣  Nuodb scales linearly on EC2‣  Per-node performance on m1.large nodes TPS   approx 50% of our commodity servers‣  Just started on optimizing‣  RDS runs on 1 node, and gets overloaded Number  of  EC2  Nodes   with 10+ connections 10  
  11. 11. Trek Palmer
  12. 12. Building An Elastically Scalable Database The Easy Way
  13. 13. Tonight’s Agenda"  A Bit About Me"   Introduction to NuoDB"   Architecture Overview"   Some Unnatural Acts "  Quick Demo"  Beer 1 3  
  14. 14. A Bit About Me"  A Refugee from Academia "  -Researched Programming Languages "  -Transactional Memory impl. and semantics"  Worked on distributed metadata database for HDS HCP "  -Clustered appliance 1 4  
  15. 15. What is NuoDB?"  Elastically scalable"  Multi-tenant"  Transactionally Consistent"  Easy to Manage 1 5  
  16. 16. 娜 graceful, like a cloud 1 6  
  17. 17. Architecture"  Three tiers "  Each is independent Management   "  Single model for all environments "  Extensible at TransacEon   various points Handling   Storage  
  18. 18. Agents"  Management tier"  Provision hosts for use"  Expose XML messaging for management"  Make scripting and automation easy 1 8  
  19. 19. Brokers"  Agent with additional special knowledge"  At least one per domain"  Redirects clients to TE "  -Clients need no knowledge of topology "  -Brokers are responsible for any load- balancing 1 9  
  20. 20. Transaction Engines"  Peer-to-peer"  In-memory"  Multi-Version Concurrency Control"  Asynchronous messaging (replication)"  Atoms 2 0  
  21. 21. Storage Managers"  Persistence points for atoms"  Key-value backing stores"  -Local FS, S3, HDFS"  Independent archives 2 1  
  22. 22. And Now, Some Unnatural Acts 2 2  
  23. 23. Sharding, an Unnatural Act"  The ideal DB application Scales  up  to  the  capacity     of  a  single  node   client   DB   What  if  you  need  more  read     client   and/or  write  throughput?   2 3  
  24. 24. Sharding"  Shard the DB among several nodes New  Client  Layer   Now  you  need  to  implement   consistency  in  your  applicaEon   Client1   Client2   TransacEonal  consistency  is   DB0   DB1   very  very  hard  to  get  right   2 4  
  25. 25. Other Sharding Bugbears"  Global operations (searches, scans)"  -Doing joins in the application"  -Implementing Cursors"  -Chunking and memory management"  And, of course, adding or removing shards 2 5  
  26. 26. Scaling Shards"  A recipe for changing the number of shards"  1) Ask boss for permission"  2) Provision hardware"  3) Rewrite the app over 6 months"  4) Hope / Pray 2 6  
  27. 27. NuoDB SolutionManagement  Client  Domain   Host A Host B Host C   Broker     Agent     Agent   2 7  
  28. 28. NuoDB on a single node Management   Client   Client   Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Database A Storage   Manager   2 8  
  29. 29. NuoDB Scaling outManagement   Client   Client   Client  Client  Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Txn  Engine   Database A Storage   Storage   Manager   Manager   2 9  
  30. 30. NuoDB ‘Adding a Shard’ Management   Client   Client   Client   Client   Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Txn  Engine   Txn  Engine   Database A Storage   Storage   Storage   Manager   Manager   Manager   3 0  
  31. 31. NuoDB ‘Sharding’"  Literally as simple as just adding nodes"  No client code had to be harmed in the making of this distributed database"  -Brokers hide topology changes"  -NuoDB is transactionally consistent 3 1  
  32. 32. Eventual Consistency"  Eventual consistency is latent inconsistency"  -Not transactionally consistent"  -Application porting is non-trivial"  -Performance/Correctness tradeoff icky 3 2  
  33. 33. NuoDB Consistency"  NuoDB is transactionally consistent"  All the time, everywhere"  When a transaction is committed, it’s guaranteed consistent"  Tradeoff is between Performance and Availability 3 3  
  34. 34. Multi-Tenancy"  Traditional databases monopolize a node"  NuoDB supports many databases in a single pool of machines (a domain)"  Each DB can be scaled as needed, independantly 3 4  
  35. 35. Multi-Tenancy ExampleManagement   JDBC  Client  Client  Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Storage   Manager   Database A 3 5  
  36. 36. Multi-Tenancy ExampleManagement   JDBC  Client  Client   JDBC  Client  Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Storage   Txn  Engine   Manager   Database A 3 6  
  37. 37. Multi-Tenancy ExampleManagement   JDBC  Client  Client   JDBC  Client  Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Storage   Txn  Engine   Manager   Database A Storage   Txn  Engine   Manager   Database 1 3 7  
  38. 38. Multi-Tenancy ExampleManagement   JDBC  Client   SQL  Client   ...  Client   JDBC  Client  Domain   Host A Host B Host C   Broker     Agent     Agent   Txn  Engine   Storage   Txn  Engine   Manager   Database A Storage   Txn  Engine   Manager   Database 1 3 8  
  39. 39. And now, a demo…
  40. 40. The Elastically Scalable Database™ 40  

×