Wiqar Chaudry and Trek Palmer present NuoDB and how to overcome administrative acrobatics with your database.

1. The Elastically Scalable
Database™
1

2. 20th Century Database
Powerful Query
9%
3%
4%
Language
44%
19% Industry Standards
ORACLE
IBM
Data Guarantees
Microsoft
Sybase 21%
Tools
Teradata
Others
Employee Skills
Existing Data
2

3. 21st Century Problem
Commodity Datacenters
✗
Big Data ✗
Powerful Query
Language
Modern Workloads ✗
Industry Standards
24x7 Operation ✗
Data Guarantees
Geo-distribution
✗
Tools
Developer
Empowerment ✗
Employee Skills
Existing Data
3

4. Database Crisis
Amazon Flickr Wikipedia
Facebook
Google
Source: Marc Bojoly 4

5. Jim Starkey
“Elastically Scalable Transactions represent the biggest breakthrough
in database technology in 25 years”
‣ DEC RDB/ELN
‣ InterBase
‣ Firebird
‣ Falcon
‣ BLOBS
‣ MVCC
5

6. Emergent Database
Architecture
" “An emergent
behavior can
appear when a
number of
simple entities
operate in an
environment,
forming more
complex
behaviors as a
collective.”
" - Wikipedia
6

7. NuoDB Plus One
" Second machine
typically doubles TPS
" Second machine is
added to live database
while it is running at
1,000’s of TPS
" Performance increase is
immediate
" BTW - you can take
either machine away and
Second
Machine
the database keeps
Instant
Performance
Increase
running without data loss
7

8. Adding a Third Machine
" Third machine
typically triples single
machine TPS
" Third machine is
added to live database
while it is running at
1,000’s of TPS
" Performance increase Second
&
Third
Machine
is immediate Instant
Performance
Increase
" BTW - you can take
any machine away and
the database keeps
running without data
loss
8

9. More Machines? Bring ‘em On
Nodes
TPS
MySQL
1
3,000
NuoDB
1
4,500
27,00
NuoDB
9
0
" Technical Details:
TPS
‣ 2-9 Tx engines
‣ 1 storage
manager
‣ Best sustained TPS
and # clients
Number
of
Nodes
combination
‣ 50% updates
NuoDB running on 9 nodes was approx. 9x faster than MySQL running
on 1 node.
9

10. Or Scale-out on IAAS
‣ Nuodb scales linearly
on EC2
‣ Per-node
performance on
m1.large nodes TPS
approx 50% of our
commodity servers
‣ Just started on
optimizing
‣ RDS runs on 1 node,
and gets overloaded Number
of
EC2
Nodes
with 10+ connections
10

11. Trek Palmer

12. Building An Elastically Scalable
Database The Easy Way

13. Tonight’s Agenda
" A Bit About Me
" Introduction to NuoDB
" Architecture Overview
" Some Unnatural Acts
" Quick Demo
" Beer
1
3

14. A Bit About Me
" A Refugee from Academia
" -Researched Programming Languages
" -Transactional Memory impl. and
semantics
" Worked on distributed metadata
database for HDS HCP
" -Clustered appliance
1
4

15. What is NuoDB?
" Elastically scalable
" Multi-tenant
" Transactionally Consistent
" Easy to Manage
1
5

16. 娜 graceful, like a cloud
1
6

17. Architecture
" Three tiers
" Each is independent Management
" Single model for all
environments
" Extensible at TransacEon
various
points Handling
Storage

18. Agents
" Management tier
" Provision hosts for use
" Expose XML messaging for
management
" Make scripting and automation easy
1
8

19. Brokers
" Agent with additional special
knowledge
" At least one per domain
" Redirects clients to TE
" -Clients need no knowledge of
topology
" -Brokers are responsible for any load-
balancing
1
9

20. Transaction Engines
" Peer-to-peer
" In-memory
" Multi-Version Concurrency Control
" Asynchronous messaging (replication)
" Atoms
2
0

21. Storage Managers
" Persistence points for atoms
" Key-value backing stores
" -Local FS, S3, HDFS
" Independent archives
2
1

22. And Now, Some Unnatural Acts
2
2

23. Sharding, an Unnatural Act
" The ideal DB application
Scales
up
to
the
capacity
of
a
single
node
client
DB
What
if
you
need
more
read
client
and/or
write
throughput?
2
3

24. Sharding
" Shard the DB among several nodes
New
Client
Layer
Now
you
need
to
implement
consistency
in
your
applicaEon
Client1
Client2
TransacEonal
consistency
is
DB0
DB1
very
very
hard
to
get
right
2
4

25. Other Sharding Bugbears
" Global operations (searches, scans)
" -Doing joins in the application
" -Implementing Cursors
" -Chunking and memory
management
" And, of course, adding or removing
shards
2
5

26. Scaling Shards
" A recipe for changing the number of
shards
" 1) Ask boss for permission
" 2) Provision hardware
" 3) Rewrite the app over 6 months
" 4) Hope / Pray
2
6

27. NuoDB Solution
Management
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
2
7

28. NuoDB on a single node
Management
Client
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Database A
Storage
Manager
2
8

29. NuoDB Scaling out
Management
Client
Client
Client
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Txn
Engine
Database A
Storage
Storage
Manager
Manager
2
9

30. NuoDB ‘Adding a Shard’
Management
Client
Client
Client
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Txn
Engine
Txn
Engine
Database A
Storage
Storage
Storage
Manager
Manager
Manager
3
0

31. NuoDB ‘Sharding’
" Literally as simple as just adding
nodes
" No client code had to be harmed in
the making of this distributed
database
" -Brokers hide topology changes
" -NuoDB is transactionally consistent
3
1

32. Eventual Consistency
" Eventual consistency is latent
inconsistency
" -Not transactionally consistent
" -Application porting is non-trivial
" -Performance/Correctness tradeoff
icky
3
2

33. NuoDB Consistency
" NuoDB is transactionally consistent
" All the time, everywhere
" When a transaction is committed,
it’s guaranteed consistent
" Tradeoff is between Performance and
Availability
3
3

34. Multi-Tenancy
" Traditional databases monopolize a
node
" NuoDB supports many databases in a
single pool of machines (a domain)
" Each DB can be scaled as needed,
independantly
3
4

35. Multi-Tenancy Example
Management
JDBC
Client
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Storage
Manager
Database A
3
5

36. Multi-Tenancy Example
Management
JDBC
Client
Client
JDBC
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Storage
Txn
Engine
Manager
Database A
3
6

37. Multi-Tenancy Example
Management
JDBC
Client
Client
JDBC
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Storage
Txn
Engine
Manager
Database A
Storage
Txn
Engine
Manager
Database 1
3
7

38. Multi-Tenancy Example
Management
JDBC
Client
SQL
Client
...
Client
JDBC
Client
Domain
Host A Host B Host C
Broker
Agent
Agent
Txn
Engine
Storage
Txn
Engine
Manager
Database A
Storage
Txn
Engine
Manager
Database 1
3
8

39. And now, a demo…

40. The Elastically Scalable
Database™
40