Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Shootout at the PAAS Corral

1,786 views

Published on

Talk from pgConf NYC and other conferences.

Published in: Data & Analytics
  • Be the first to comment

Shootout at the PAAS Corral

  1. 1. Shootout at the PAAS Corral EC2EC2RDS Heroku Josh BerkusJosh Berkus PostgreSQL Experts Inc.PostgreSQL Experts Inc. pgConf NYC 2015pgConf NYC 2015 DODO RackspaceRackspace GCEGCE
  2. 2. https://github.com/ manageacloud/ cloud-benchmark-postgres
  3. 3. Thanks! Ruben Rubio Rey
  4. 4. Thanks!
  5. 5. What is The Cloud?
  6. 6. Magic? Image by Sperlingsmaedchen. Free for non-profit use only.
  7. 7. a bunch of servers
  8. 8. with virtualization
  9. 9. shared storage
  10. 10. … and an API
  11. 11. The Good ● Fast deployment – new servers in minutes, with a script ● Easy scale-out – add replicas in minutes ● Minimize ops staff – no HW wranglers
  12. 12. ¢heap (at the low end)
  13. 13. ex$pensive (at the high end)
  14. 14. The Bad ● Low system resources – VMs are small/slow ● Security – more attack vectors
  15. 15. The Ugly ● Everything is shared – Network – IO / Storage – CPU (partly) Your performance depends on somebody else's peak load.
  16. 16. Sharing is Not Caring
  17. 17. ephemeral cloud ● DR is not optional ● your virtual DB server will go away ● you need replicas & backup
  18. 18. why Postgres?
  19. 19. transactional DB workout ● works CPU ● works RAM ● works IO ● works network IO at the same time, in parallel
  20. 20. A Six-Shooter of Clouds EC2EC2RDS Heroku DODORackspaceRackspace GCEGCE ??????
  21. 21. Three from AWS The Gunslinger Roll-Your-Own The Rancher RDS The Dandy Heroku
  22. 22. AWS commons ● comprehensive API – covering all Amazon services – do everything in code ● global distribution – multiple regions/data centers – US, Asia, South America, Europe
  23. 23. The Gunslinger Roll Your Own on EC2
  24. 24. Become a Gunslinger I. Create an EC2 instance II. Install PostgreSQL on it III. Configure PostgreSQL
  25. 25. The Gunslinger ● Cloud Type: platform ● Administration: DIY ● HA: DIY ● Versions Available: anything ● Extensions: all ● Extra Features: none ● Price: cheap
  26. 26. Instance Types ● m3.* general-purpose ● c3.* small, CPU-bound DBs $$ ● r3.* maximize caching $$$ ● i2.* Data warehousing $$$$
  27. 27. instance tips ● Use m3.* if you don't know what to use ● Get enough RAM to cache your database if you can
  28. 28. Storage Types ● EBS + Provisioned IOPS – large size, latency issues – reliable, snapshots – choose “EBS optimized” ● General SSD – better for bursts (about 20% better) – high variability
  29. 29. Storage Types ● Instance Storage + SSD option – low-latency, limited size – risky: data loss, data corruption
  30. 30. PrIOPS != througput
  31. 31. PrIOPS fallacy ● not a guarantee, a limit – but mostly pretty consistent ● each IOP is no more than 8K ● random access → each page is an IOP – no real prefetch ● PrIOPS ~ rows/second
  32. 32. Stuff to Set Up ● Backup: WAL-E to S3 ● Replication: not optional – in another Availbility Zone ● Monitoring for instance failure ● Secure your instance – SSL – pg_hba.conf
  33. 33. Junior Gunslinger ● “small” “economical” ● m3.medium ● 1 core 3.75GB RAM ● 40GB + 1000 PrIOPS
  34. 34. Senior Gunslinger ● “large” “performance” ● r3.2xlarge ● 8 cores 61GB RAM ● 200GB + 4000 PrIOPS
  35. 35. Junior Gunslinger Cost instance: $36.50/month EBS PrIOPS: $105/month S3 Archive: $5/month X2 for replica == $288.00 a month (+ misc charges) PricesSubject toChange
  36. 36. Cheaper Gunslinger instance: $36.50/month EBS PrIOPS: $105/month S3 Archive: $5/month no replica == $146.50 a month (+ misc charges) PricesSubject toChange
  37. 37. Senior Gunslinger archive-only: $760.70/month with replica: $1509.40/month (+ misc charges) PricesSubject toChange
  38. 38. The Rancher Relational Database Service
  39. 39. “I like AWS but I don't want to do all this instance management”
  40. 40. DB-a-a-S ● “Database as a Service” ● No OS ● No system config ● Just Port 5432
  41. 41. DB-a-a-S++ ● SEDBA: Somebody Else Is The DBA – uptime – backups – configuration – updates ● Let your DevOps manage the DBs
  42. 42. DB-a-a-S - - ● Limited Options – versions – extensions – configuration – security setups ● Cost
  43. 43. The Rancher ● Cloud Type: database ● Administration: mostly automatic ● HA: replication, M.A.Z ● Versions Available: 9.3, 9.4 ● Extensions: 2 dozen ● Extra Features: none ● Price: moderate
  44. 44. M.A.Z. ● “Multi-Availability Zone” – syncronous replication to another node – automated failover – uptime guarantee ● major performance cost
  45. 45. The Dandy Heroku
  46. 46. “I just want to develop; the cloud should handle all administration.”
  47. 47. The Dandy ● Cloud Type: database ● Administration: automatic ● HA: auto replication + PITR ● Versions Available: 9.3, 9.4, betas ● Extensions: 2 dozen ● Extra Features: several ● Price: expensive
  48. 48. Dandy Bling ● git-based instance management – works really well with Rails/Django ● Dataclips – web-sharable matviews! ● Followers == replicas
  49. 49. Heroku options ● 5 database “sizes” ● 3 levels of HA/uptime ● that's it
  50. 50. Heroku Sizing ● Small Standard 2: 3.5 GB RAM shared hosting ● Large Standard 6: 60GB RAM dedicated instance
  51. 51. The Businessman Rackspace
  52. 52. “I have Rackspace servers and want to expand via cloud.”
  53. 53. The Businessman ● Cloud Type: platform ● Administration: DIY (+ support) ● HA: DIY ● Versions Available: all ● Extensions: all ● Extras: hybrid cloud ● Price: moderate
  54. 54. Rackspace support ● “managed infrastucture” support required – cost added to instance costs – can bump up to higher levels of support – includes some uptime support ● No help with DBA tasks though
  55. 55. Rackspace storage ● primarily instance storage – could be risky ● block store not available on small instances
  56. 56. Rackspace Sizing ● Small general1-4 4 cores, 4GB RAM ● Large memory1-60 8 cores, 60GB RAM
  57. 57. The Drifter Google Compute Engine
  58. 58. “I want an EC2-like platform much cheaper.”
  59. 59. The Drifter ● Cloud Type: platform ● Administration: DIY ● HA: DIY ● Versions Available: all ● Extensions: all ● Extras: Docker integration ● Price: cheap
  60. 60. GCE Sizing ● Small n1-standard-1 1 core, 4GB RAM ● Large n1-highmem-8 8 cores, 52GB RAM
  61. 61. The Kid Digital Ocean
  62. 62. “I want cheap, simple and fast, and nothing else.”
  63. 63. The Kid ● Cloud Type: platform ● Administration: DIY ● HA: DIY ● Versions Available: all ● Extensions: all ● Extras: nada ● Price: OMG cheap!
  64. 64. Kid vulnerability ● no block store – all storage is ephemeral instance storage – nodes completely non-durable ● no storage cloud for backups ● no HA features
  65. 65. D.O. Sizing ● Small 4GB 2 cores, 4GB RAM ● Large 48GB 16 cores, 48GB RAM
  66. 66. More Clouds
  67. 67. More Clouds ● EDB Postgres Plus Cloud ● OpenShift ● Joyent + Manatee ● Azure ● ???
  68. 68. the shootout
  69. 69. pgbench
  70. 70. pgbench++ ● ships with Postgres ● microbenchmark – very simple “bank trade” workload ● fast to set up and run
  71. 71. pgbench-- ● doesn't do complex queries ● pure random data / access ● unrealistic balance of work – too reliant on single-row write speed ● not very tunable
  72. 72. pgbench sizing 1. memory read-write (RW): – 50% of RAM, write transactions 2. memory read-only (RO): – 50% of RAM, read-only queries 3. disk read-write (RW): – 200% of RAM, write transactions
  73. 73. pgbench small ● memory RW ● pgbench -i -s 100 --foreign-keys ● pgbench -c 4 -T 900 ● memory RO ● pgbench -i -s 100 --foreign-keys ● pgbench -c 4 -T 900 -S ● disk RW ● pgbench -i -s 400 --foreign-keys ● pgbench -c 4 -T 900
  74. 74. pgbench large ● memory RW ● pgbench -i -s 1000 --foreign-keys ● pgbench -c 16 -T 900 ● memory RO ● pgbench -i -s 1000 --foreign-keys ● pgbench -c 16 -T 900 -S ● disk RW ● pgbench -i -s 7000 --foreign-keys ● pgbench -c 16 -T 900
  75. 75. metrics ● TPS: transactions-per-second – measures multiple things ● Load Time: time to build the database – measures INSERT time
  76. 76. other conditions ● all Postgres 9.3 – 9.3.5 and 9.3.6 ● some CentOS 7, some Ubuntu 14.04 ● pgbench on same instance size as Postgres
  77. 77. run many many timesrun many many times
  78. 78. Box PlotBox Plot
  79. 79. 495 TPS 587 TPS 1685 TPS 2537 TPS 6156 TPS 50 90 10 Min Max
  80. 80. 0.3X 0.4X Median 1.7X 4X 50 90 10 Min Max
  81. 81. 0.3X 0.4X Median 1.7X 4X 50 90 10 Min Max
  82. 82. 0.3X 0.4X Median 1.7X 4X 50 90 10 Min Max
  83. 83. when the smoke clears ...
  84. 84. comparability problems I. Rackspace, DO not as durable II. Only DBaaS HA is automated III. Instance sizes not identical IV. Instance OSes are not identical
  85. 85. work in progress
  86. 86. digitalocean ec2 gce heroku rackspace rds 0 100 200 300 400 500 Cost Per Small Node (lower is better) HA Non-HA dollars/month
  87. 87. digitalocean ec2 gce heroku rackspace rds 0 500 1000 1500 2000 2500 3000 3500 Cost Per Large Node (lower is better) HA Non-HA dollars/month
  88. 88. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 1 2 3 4 5 6 Load Time: Small In-Memory (smaller is better) median 90.00% Minutes
  89. 89. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 1 2 3 4 5 6 Load Time: Small In-Memory (smaller is better) median 90.00% Minutes Instance Storage
  90. 90. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 1 2 3 4 5 6 Load Time: Small In-Memory (smaller is better) median 90.00% Minutes archiving overhead
  91. 91. digitalocean ec2 gce heroku rackspace rds rds HA 0 5 10 15 20 25 30 Load Time: Small On-Disk (smaller is better) median 90.00% Minutes
  92. 92. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 5 10 15 20 25 30 35 Load Time: Large In-Memory (smaller is better) median 90.00% Minutes
  93. 93. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 0.2 0.4 0.6 0.8 1 1.2 $ Per LoadMBPS: Small In-Memory (smaller is better) median 90 Dollars digitalocean ec2 gce heroku rackspace rds rds MAZ 0 0.2 0.4 0.6 0.8 1 1.2 $ Per LoadMBPS: Small In-Memory (smaller is better) median 90 Dollars
  94. 94. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 0.5 1 1.5 2 2.5 3 3.5 4 $ Per LoadMBPS: Large In-Memory (smaller is better) median 90 Dollars
  95. 95. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 100 200 300 400 500 600 700 800 900 TPS: Small In-Memory RW (larger is better) 10.00% median TPS
  96. 96. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 TPS: Small In Memory RO (larger is better) 10.00% median TPS
  97. 97. digitalocean ec2 gce heroku rackspace rds rds HA 0 100 200 300 400 500 600 700 800 TPS: Small On-Disk RW (larger is better) 10.00% median TPS
  98. 98. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 500 1000 1500 2000 2500 TPS: Large In-Memory RW (larger is better) 10.00% median TPS
  99. 99. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 5000 10000 15000 20000 25000 30000 35000 40000 TPS: Large In-Memory RO (larger is better) 10.00% median TPS
  100. 100. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 200 400 600 800 1000 1200 1400 1600 TPS: Large On-Disk (larger is better) 10.00% median TPS
  101. 101. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 $ Per TPS: Small In Memory RW (smaller is better) 10.00% median Dollars
  102. 102. digitalocean ec2 gce heroku rackspace rds rds MAZ 0 0.5 1 1.5 2 2.5 3 3.5 $ Per TPS: Large In-Memory RW (smaller is better) 10.00% median Dollars
  103. 103. Who is this hombre?
  104. 104. The Gambler running with scissors mode
  105. 105. GamblerDB      bgwriter_lru_maxpages = 0      wal_level = minimal      fsync = off      synchronous_commit = off      full_page_writes = off      wal_log_hints = off      wal_buffers = 64MB
  106. 106. digitalocean ec2 gce heroku rackspace rds rds MAZ scis-rep scis-ulg 0 100 200 300 400 500 600 700 800 900 TPS: Small In-Memory RW (larger is better) 10.00% median TPS
  107. 107. the trail ahead
  108. 108. More Benchmarks ● OLTPBench? – Wikipedia, Auctionmark, Epinions ● DVDStore? ● New benchmark? – really need something more “webby” ● NoSQLish benchmark?
  109. 109. Better Visualizations ● better graphs ● automated graph generation ● detailed response times and time graphs
  110. 110. more shooting ● Josh Berkus: josh@pgexperts.com – www.pgexperts.com ● More Shootouts – www.databasesoup.com – https://github.com/manageacloud/cloud- benchmark-postgres/ – LFNW, pgCon Ottawa Copyright 2015 PostgreSQL Experts Inc. Released under the Creative Commons Share-Alike 3.0 License. All images and trademarks are the property of their respective owners. Maginificent Seven stills are property of MGM and used here under principles of fair use.

×