Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Replication - Nick Carboni - ManageIQ Design Summit 2016

519 views

Published on

Database Replication in ManageIQ by Nick Carboni at ManageIQ Design Summit 2016

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Replication - Nick Carboni - ManageIQ Design Summit 2016

  1. 1. Database Replication in ManageIQ
  2. 2. Introduction Nick Carboni Software Engineer at Red Hat Appliance and database for ManageIQ (Platform Team)
  3. 3. Replication in ManageIQ Multiple databases can be used for far away environments Database replication allows for easy reporting and visibility Database 1 Database 2 Global Database Replication!
  4. 4. rubyrep http://www.rubyrep.org https://github.com/rubyrep/rubyrep Last commit in 2011
  5. 5. rubyrep Ruby + ActiveRecord Triggers = Developer headaches SQL Configuration is … hardelsif commands.include? command status = commands[command][:command].run(args.slice(1, 1_000_000))
  6. 6. rubyrep in ManageIQ We use our own fork https://github.com/ManageIQ/rubyrep Mainly support for new versions of ActiveRecord
  7. 7. rubyrep in ManageIQ PR to add support for Rails 4.2 March 27, 2015 “Yup. It's basically dead. Our changes are also SOOO crazy different, that this is like a Frankenstein project now. I wonder if we should just write something different from scratch given our experience...or perhaps we should rethink replication altogether and come up with a plan.” - @Fryguy
  8. 8. rubyrep in ManageIQ Worker process Database Synchronization role Worker will run on the server with the role Passive global database
  9. 9. pglogical http://2ndquadrant.com/en-us/resources/pglogical/ https://github.com/2ndQuadrant/pglogical/ Inclusion into PostgreSQL core in progress 9.7+??
  10. 10. pglogical PostgreSQL extension Process runs on database server Managed by PostgreSQL service SQL stored procedure API postgresql.conf configuration
  11. 11. pglogical PostgreSQL logical decoding https://www.postgresql.org/docs/9.4/static/logicaldecoding.html Output plugin pglogical_output Consumer extension pglogical Replication slots https://www.postgresql.org/docs/9.4/static/warm-
  12. 12. pglogical in ManageIQ AR connection adapter extension Exposes a ruby method per pglogical stored procedure Configure “subscriptions” to remote regions through global region UI Exclude table configuration unchanged “Advanced Settings” tab in remote region UI
  13. 13. Global Region Configuration
  14. 14. Performance testing How long does it take to replicate a “backlog” of data? Vary amount of data and artificial network latency Measure time until “backlog” is zero rubyrep - number of rows in rr_pending_changes table pglogical - transaction log location compared to flush location SELECT pg_xlog_location_diff(pg_current_xlog_location(), flush_location) AS lag_bytes FROM pg_stat_replication;
  15. 15. Insert Performance Rows pglogical rubyrep 1,000 0.158 (σ = 0.006) 0.189 (σ = 0.011) 10,000 1.499 (σ = 0.095) 1.84 (σ = 0.093) 100,000 16.105 (σ = 0.296) 18.042 (σ = 0.320) 1,000,000 147.508 (σ = 3.557) 195.399 (σ = 18.317) Sample size = 10
  16. 16. No Latency
  17. 17. 25ms Latency (50ms ping time)
  18. 18. 100ms Latency (200ms ping time)
  19. 19. Q&A

×