Databases and the cloud                                 Henrik Ingo                                   TTKK                ...
2011-10-25   2
History                   ...things that used to be             difficult at different points in time.2011-10-25          ...
19942012-02-27   TTKK   4
1994USSR fell 3 years ago    Finland yet to win first gold medal in hockey         Windows 3.0 replacing MS-DOS           ...
1994                                                 SQL                                         Standard InterpretedC++ &...
19972012-02-27   TTKK   7
19972012-02-27   TTKK   8
Learning SQLNoSQL advocates say:              Henrik says:    SQL is hard to learn               MS Access easy to learn  ...
2000                    MS Access not supported                    in Linux                    PostgreSQL and MySQL       ...
2005                    Graduate & do websites for                    a living.                    Spend 3-6 days creating...
2008                    Join MySQL AB                    Confused about                    synchronous vs                 ...
High Availability             ...and why it is more difficult                    for databases.2011-10-25                 ...
Performance                               DurabilityTransactions / second (throughput)                                 Spe...
Uptime                      Percentile target Max downtime per year                                  90% 36 days          ...
Clustering frameworks - general                        Heartbeat                        Corosync                       VM ...
Clustering frameworks - DB                      Heartbeat                      Corosync                        MMM        ...
Sounds simple. What could possibly go wrong?                              Old Master must stop service                    ...
Active-Active Shared disk clustering. (State of the art?)                                Oracle RAC                       ...
Sounds simple. What could possibly go wrong?    Well, actually its pretty good.    Data integrity protection is good.    B...
Synchronous multi-master                       NDB                      Galera                       Failover2011-10-25   ...
Sounds simple. What could possibly go wrong?                              Synchronous Multi Master                        ...
Performance    SAN has "some" overhead compared to local    disk    DRBD = 50% performance penalty    Replication, when im...
Is a clustering solution part of the solution or the part of the problem?    "Causes of Downtime in Production MySQL Serve...
Scale-out and elasticity             ...and why it is more difficult                    for databases.2011-10-25          ...
Scale-out                         Invented by MySQL / LAMP                         stack.                         Laughed ...
DBAs life is more interesting!HTTP                                   RDBMS    Stateless                               Wher...
But it can be done    Automating DB deployments is        MySQL    more complex. But not                                  ...
NoSQL             ...and why it is more difficult                for relational databases.2011-10-25                      ...
It used to be2012-02-27       TTKK        30
The future is2012-02-27       TTKK        31
The future is             All Open Source2012-02-27            TTKK        32
Things NoSQL guys do really well                                                                           Reading tip:   ...
Best of both worldsNoSQL                                    MySQL    No SQL                                   HandlerSocke...
Cloud             ...and why it is more difficult                    for databases.2011-10-25                             ...
4 different DB deployments                                          Server HW                                          VM ...
Consider memory utilization             "All of computation is just different layers of caching."    Dedicated HW         ...
Multi-tenancy    Web hosting w MySQL                 Drizzle         Cpanel = hack                        True multi-tenan...
Thank you2012-02-27    TTKK       39
Upcoming SlideShare
Loading in …5
×

Databases and the Cloud

2,240 views
2,141 views

Published on

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,240
On SlideShare
0
From Embeds
0
Number of Embeds
571
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Databases and the Cloud

  1. 1. Databases and the cloud Henrik Ingo TTKK 2012-02-27 Please share and re-use this presentation, licensed under Creative Commons Attribution license.2012-02-27 TTKK 1
  2. 2. 2011-10-25 2
  3. 3. History ...things that used to be difficult at different points in time.2011-10-25 3
  4. 4. 19942012-02-27 TTKK 4
  5. 5. 1994USSR fell 3 years ago Finland yet to win first gold medal in hockey Windows 3.0 replacing MS-DOS Windows 95 did not yet exist I learn how to write .bat scripts from my dad Word 6.0 replacing WordPerfect 5.1 Excel 5.0 replacing Lotus I had not yet used Internet2012-02-27 TTKK 5
  6. 6. 1994 SQL Standard InterpretedC++ & Visual Basic Flexible and expressive command line env LAN "SQL for secretaries" Printer Database Good for English speakers 2000: Bad for IDE w IntelliSense Client - Server architecture Stored procedures rule DBA is king of business logic2012-02-27 TTKK 6
  7. 7. 19972012-02-27 TTKK 7
  8. 8. 19972012-02-27 TTKK 8
  9. 9. Learning SQLNoSQL advocates say: Henrik says: SQL is hard to learn MS Access easy to learn To really scale, you must Darn, I always got de-normalize normalization Most people dont get I know, teaching n:n normalization right relations was always fun Impedance mismatch INSERT INTO t ... between oo and sql serialize($obj) I just need a simple key- SELECT value FROM t value store WHERE key=?;2012-02-27 TTKK 9
  10. 10. 2000 MS Access not supported in Linux PostgreSQL and MySQL Learn SQL for real phpMyAdmin to ease the pain2012-02-27 TTKK 10
  11. 11. 2005 Graduate & do websites for a living. Spend 3-6 days creating and re-creating properly normalized DB schema In MS Access I just clicked next, next, next and ok. Time is money Invent NoSQL: PHP serialize() MySQL BLOB2012-02-27 TTKK 11
  12. 12. 2008 Join MySQL AB Confused about synchronous vs asynchronous replication Learn a lot about MySQL NDB Cluster2012-02-27 TTKK 12
  13. 13. High Availability ...and why it is more difficult for databases.2011-10-25 13
  14. 14. Performance DurabilityTransactions / second (throughput) Speaking of databases Response time (latency) Committed data is not lost Percentiles (95% - 99%) D in ACID Get any response at all (tps > 0) Replicas, snapshots Measured as percentile (99.999%) point in time, backups High Availability Clustering Replication Monitoring Redundancy Failover 2011-10-25 14
  15. 15. Uptime Percentile target Max downtime per year 90% 36 days 99% 3.65 days 99.5% 1.83 days 99.9% 8.76 hours 99.99% 52.56 minutes 99.999% 5.26 minutes 99.9999% 31.5 seconds Beyond system availability: Average downtime per user.2011-10-25 15
  16. 16. Clustering frameworks - general Heartbeat Corosync VM of choice Red Hat Cluster Solaris Cluster ... Failover2011-10-25 16
  17. 17. Clustering frameworks - DB Heartbeat Corosync MMM VM of choice MHA Tungsten Enterprise Solaris Cluster ... Failover2011-10-25 17
  18. 18. Sounds simple. What could possibly go wrong? Old Master must stop service (VIP, os, DB). But it is not responding, so how do you make it stop? Polling from the outside. Interval = 1 sec, 10 sec, 60 sec! What if replication fails first and client transactions dont? Polling connectivity of DB nodes but not client p.o.v. Failover can be expensive (SAN, DRBD) -> false positives costly2011-10-25 18
  19. 19. Active-Active Shared disk clustering. (State of the art?) Oracle RAC (ScaleDB?) Failover Disk Disk2011-10-25 19
  20. 20. Sounds simple. What could possibly go wrong? Well, actually its pretty good. Data integrity protection is good. But... SAN is considered the biggest SPOF of all. Recovery time on single node failure is +60 sec Recovery time? Because internally each node will lock some pages and process them locally. (Bloody expensive) Disk2011-10-25 20
  21. 21. Synchronous multi-master NDB Galera Failover2011-10-25 21
  22. 22. Sounds simple. What could possibly go wrong? Synchronous Multi Master replication rocks :-) Failure detection inherent in replication protocol. Instant failovers. Bonus: Both Galera and NDB provision new nodes automatically. Problem is solved. Time for new problems...2011-10-25 22
  23. 23. Performance SAN has "some" overhead compared to local disk DRBD = 50% performance penalty Replication, when implemented correctly, has 0 performance penalty. Galera and NDB = more performance2011-10-25 23
  24. 24. Is a clustering solution part of the solution or the part of the problem? "Causes of Downtime in Production MySQL Servers" by Baron Schwartz: #1: Human error #2: SAN Complex clustering framework + SAN = More problems, not less! Galera (and NDB) = Replication based, no SAN or DRBD No "failover moment", no false positives No clustering framework needed No load balancer needed (JDBC loadbalance) Simple and elegant!2011-10-25 24
  25. 25. Scale-out and elasticity ...and why it is more difficult for databases.2011-10-25 25
  26. 26. Scale-out Invented by MySQL / LAMP stack. Laughed at by other RDBMSes Now everyone does it. Because the Internet is too big. Originally with read-only replicas. Then sharding. Easy for http, inconvenient for databases. NoSQL systems do it really well. MySQL NDB does it really well and Galera pretty well.2012-02-27 TTKK 26
  27. 27. DBAs life is more interesting!HTTP RDBMS Stateless Where everyone else stores their state Usually can partition/shard Needs expertise to partition/shard Scale-out = boot more servers Scale-out = Boot more servers. Backup DB. Restore DB. Setup replication. Tweak application code... Writes = Write to the database Write = Which partition/node? Beware of read-only slaves. Beware of eventual consistency...2012-02-27 TTKK 27
  28. 28. But it can be done Automating DB deployments is MySQL more complex. But not Amazon RDS, others. impossible. Severalnines NDB and Galera handle data provisioning really well. But Also supports Galera and deploying the empty nodes still NDB manual labor. Scalr Scale-out happens because you PostgreSQL must. Scale-down will never happen if its too much work. Heroku EnterpriseDB NoSQL Usually do this relatively well.2012-02-27 TTKK 28
  29. 29. NoSQL ...and why it is more difficult for relational databases.2011-10-25 29
  30. 30. It used to be2012-02-27 TTKK 30
  31. 31. The future is2012-02-27 TTKK 31
  32. 32. The future is All Open Source2012-02-27 TTKK 32
  33. 33. Things NoSQL guys do really well Reading tip: Reading tip: No SQL (parsing) Original Amazon Dynamo Original Amazon Dynamo Schemaless = Win! for agile development paper paper HA with quorum consistency: R + W > N Transparent sharding Graph databases N:N relationships, whats the big deal? Actually makes sense to give up on SQL! Map Reduce Bypass ETL, get clean data Implemented in Java or Python (or Erlang)2012-02-27 TTKK 33
  34. 34. Best of both worldsNoSQL MySQL No SQL HandlerSocket . Memcache API, NDB API Simple key-value store BLOB . SELECT v FROM ... WHERE k=? ...and secondary indexes Functional indexes Virtual columns, etc... Synchronous replication Quorum consistency Galera, NDB . We have it too Transparent sharding NDB, Spider + proprietary solutions Graph databases Damn N:N relations! Map Reduce against text files Map Reduce against RDBMS Java and Python C++ can be done right . Drizzle2012-02-27 TTKK 34
  35. 35. Cloud ...and why it is more difficult for databases.2011-10-25 35
  36. 36. 4 different DB deployments Server HW VM DB process User (schema)2011-10-25 36
  37. 37. Consider memory utilization "All of computation is just different layers of caching." Dedicated HW No virtualization overhead Great performance €€€€ Memory allocation per DB Not cloud instance is fixed (without restart) -> cache of idle DB instances is wasted. Virtualization overhead No virtualization overhead. "Safety margin" of unallocated memory per VM. Busy schema can use more cache and idle schema evicted Memory allocation per VM is from cache. fixed (without reboot) -> cache of idle DB instances is wasted.2011-10-25 37
  38. 38. Multi-tenancy Web hosting w MySQL Drizzle Cpanel = hack True multi-tenancy, "virtualization for databases" Not "cloud". Users expect "dedicated" database Ready but experimental instance.2011-10-25 38
  39. 39. Thank you2012-02-27 TTKK 39

×