Databases and the Cloud

Databases and the cloud Henrik Ingo TTKK 2012-02-27 Please share and re-use this presentation, licensed under Creative Commons Attribution license.2012-02-27 TTKK 1

2011-10-25 2

History ...things that used to be difficult at different points in time.2011-10-25 3

19942012-02-27 TTKK 4

1994USSR fell 3 years ago Finland yet to win first gold medal in hockey Windows 3.0 replacing MS-DOS Windows 95 did not yet exist I learn how to write .bat scripts from my dad Word 6.0 replacing WordPerfect 5.1 Excel 5.0 replacing Lotus I had not yet used Internet2012-02-27 TTKK 5

1994 SQL Standard InterpretedC++ & Visual Basic Flexible and expressive command line env LAN "SQL for secretaries" Printer Database Good for English speakers 2000: Bad for IDE w IntelliSense Client - Server architecture Stored procedures rule DBA is king of business logic2012-02-27 TTKK 6

19972012-02-27 TTKK 7

19972012-02-27 TTKK 8

Learning SQLNoSQL advocates say: Henrik says: SQL is hard to learn MS Access easy to learn To really scale, you must Darn, I always got de-normalize normalization Most people dont get I know, teaching n:n normalization right relations was always fun Impedance mismatch INSERT INTO t ... between oo and sql serialize($obj) I just need a simple key- SELECT value FROM t value store WHERE key=?;2012-02-27 TTKK 9

2000 MS Access not supported in Linux PostgreSQL and MySQL Learn SQL for real phpMyAdmin to ease the pain2012-02-27 TTKK 10

2005 Graduate & do websites for a living. Spend 3-6 days creating and re-creating properly normalized DB schema In MS Access I just clicked next, next, next and ok. Time is money Invent NoSQL: PHP serialize() MySQL BLOB2012-02-27 TTKK 11

2008 Join MySQL AB Confused about synchronous vs asynchronous replication Learn a lot about MySQL NDB Cluster2012-02-27 TTKK 12

High Availability ...and why it is more difficult for databases.2011-10-25 13

Performance DurabilityTransactions / second (throughput) Speaking of databases Response time (latency) Committed data is not lost Percentiles (95% - 99%) D in ACID Get any response at all (tps > 0) Replicas, snapshots Measured as percentile (99.999%) point in time, backups High Availability Clustering Replication Monitoring Redundancy Failover 2011-10-25 14

Uptime Percentile target Max downtime per year 90% 36 days 99% 3.65 days 99.5% 1.83 days 99.9% 8.76 hours 99.99% 52.56 minutes 99.999% 5.26 minutes 99.9999% 31.5 seconds Beyond system availability: Average downtime per user.2011-10-25 15

Clustering frameworks - general Heartbeat Corosync VM of choice Red Hat Cluster Solaris Cluster ... Failover2011-10-25 16

Clustering frameworks - DB Heartbeat Corosync MMM VM of choice MHA Tungsten Enterprise Solaris Cluster ... Failover2011-10-25 17

Sounds simple. What could possibly go wrong? Old Master must stop service (VIP, os, DB). But it is not responding, so how do you make it stop? Polling from the outside. Interval = 1 sec, 10 sec, 60 sec! What if replication fails first and client transactions dont? Polling connectivity of DB nodes but not client p.o.v. Failover can be expensive (SAN, DRBD) -> false positives costly2011-10-25 18

Active-Active Shared disk clustering. (State of the art?) Oracle RAC (ScaleDB?) Failover Disk Disk2011-10-25 19

Sounds simple. What could possibly go wrong? Well, actually its pretty good. Data integrity protection is good. But... SAN is considered the biggest SPOF of all. Recovery time on single node failure is +60 sec Recovery time? Because internally each node will lock some pages and process them locally. (Bloody expensive) Disk2011-10-25 20

Synchronous multi-master NDB Galera Failover2011-10-25 21

Sounds simple. What could possibly go wrong? Synchronous Multi Master replication rocks :-) Failure detection inherent in replication protocol. Instant failovers. Bonus: Both Galera and NDB provision new nodes automatically. Problem is solved. Time for new problems...2011-10-25 22

Performance SAN has "some" overhead compared to local disk DRBD = 50% performance penalty Replication, when implemented correctly, has 0 performance penalty. Galera and NDB = more performance2011-10-25 23

Is a clustering solution part of the solution or the part of the problem? "Causes of Downtime in Production MySQL Servers" by Baron Schwartz: #1: Human error #2: SAN Complex clustering framework + SAN = More problems, not less! Galera (and NDB) = Replication based, no SAN or DRBD No "failover moment", no false positives No clustering framework needed No load balancer needed (JDBC loadbalance) Simple and elegant!2011-10-25 24

Scale-out and elasticity ...and why it is more difficult for databases.2011-10-25 25

Scale-out Invented by MySQL / LAMP stack. Laughed at by other RDBMSes Now everyone does it. Because the Internet is too big. Originally with read-only replicas. Then sharding. Easy for http, inconvenient for databases. NoSQL systems do it really well. MySQL NDB does it really well and Galera pretty well.2012-02-27 TTKK 26

DBAs life is more interesting!HTTP RDBMS Stateless Where everyone else stores their state Usually can partition/shard Needs expertise to partition/shard Scale-out = boot more servers Scale-out = Boot more servers. Backup DB. Restore DB. Setup replication. Tweak application code... Writes = Write to the database Write = Which partition/node? Beware of read-only slaves. Beware of eventual consistency...2012-02-27 TTKK 27

But it can be done Automating DB deployments is MySQL more complex. But not Amazon RDS, others. impossible. Severalnines NDB and Galera handle data provisioning really well. But Also supports Galera and deploying the empty nodes still NDB manual labor. Scalr Scale-out happens because you PostgreSQL must. Scale-down will never happen if its too much work. Heroku EnterpriseDB NoSQL Usually do this relatively well.2012-02-27 TTKK 28

NoSQL ...and why it is more difficult for relational databases.2011-10-25 29

It used to be2012-02-27 TTKK 30

The future is2012-02-27 TTKK 31

The future is All Open Source2012-02-27 TTKK 32

Things NoSQL guys do really well Reading tip: Reading tip: No SQL (parsing) Original Amazon Dynamo Original Amazon Dynamo Schemaless = Win! for agile development paper paper HA with quorum consistency: R + W > N Transparent sharding Graph databases N:N relationships, whats the big deal? Actually makes sense to give up on SQL! Map Reduce Bypass ETL, get clean data Implemented in Java or Python (or Erlang)2012-02-27 TTKK 33

Best of both worldsNoSQL MySQL No SQL HandlerSocket . Memcache API, NDB API Simple key-value store BLOB . SELECT v FROM ... WHERE k=? ...and secondary indexes Functional indexes Virtual columns, etc... Synchronous replication Quorum consistency Galera, NDB . We have it too Transparent sharding NDB, Spider + proprietary solutions Graph databases Damn N:N relations! Map Reduce against text files Map Reduce against RDBMS Java and Python C++ can be done right . Drizzle2012-02-27 TTKK 34

Cloud ...and why it is more difficult for databases.2011-10-25 35

4 different DB deployments Server HW VM DB process User (schema)2011-10-25 36

Consider memory utilization "All of computation is just different layers of caching." Dedicated HW No virtualization overhead Great performance €€€€ Memory allocation per DB Not cloud instance is fixed (without restart) -> cache of idle DB instances is wasted. Virtualization overhead No virtualization overhead. "Safety margin" of unallocated memory per VM. Busy schema can use more cache and idle schema evicted Memory allocation per VM is from cache. fixed (without reboot) -> cache of idle DB instances is wasted.2011-10-25 37

Multi-tenancy Web hosting w MySQL Drizzle Cpanel = hack True multi-tenancy, "virtualization for databases" Not "cloud". Users expect "dedicated" database Ready but experimental instance.2011-10-25 38

Thank you2012-02-27 TTKK 39

http://openlife.cc	295
http://asyncionews.com	201
http://www.openlife.cc	3
http://127.0.0.1	2
http://translate.googleusercontent.com	1

Databases and the Cloud

by Henrik Ingo on Mar 06, 2012

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 502

Statistics

Databases and the Cloud Presentation Transcript