Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PLAM 2015 - Evolving Backups Strategy, Devploying pyxbackup

1,086 views

Published on

Not all backup requirements are the same, there are simple backups and then there are complex backup and recovery strategies. After working with hundreds of users laying out the ground work, we've decided that although not all installations are the same, all of them run through the same procedure so we have decided to automate most of this things with pyxbackup. This talk will highlight the evolution of the project from the most basic backup implementation to extending capabilities for different scenarios like compression, remote storage, encryption and being a portable, comprehensible backup utility for the novice and experts alike.

The pyxbackup project can be found at:

https://github.com/dotmanila/pyxbackup

Published in: Engineering
  • Be the first to comment

PLAM 2015 - Evolving Backups Strategy, Devploying pyxbackup

  1. 1. Evolving Backups Strategy, Devploying pyxbackup Jervin Real Percona Live 2015 - Amsterdam
  2. 2. Managing Backups
  3. 3. Percona ♥ Backups
  4. 4. Why? Why not?
  5. 5. Why?
  6. 6. Everybody ♥ Backups s9s_backup
  7. 7. Everybody ♥ Backups s9s_backup holland backup manager
  8. 8. Everybody ♥ Backups xtrabackup manager s9s_backup holland backup manager
  9. 9. Everybody ♥ Backups xbackup.sh xtrabackup manager s9s_backup holland backup manager
  10. 10. Everybody ♥ Backups xbackup.sh xtrabackup manager s9s_backup holland backup manager fromdual_brman
  11. 11. Everybody ♥ Backups xbackup.sh xtrabackup manager s9s_backup holland backup manager fromdual_brman twindb
  12. 12. Everybody ♥ Backups xbackup.sh xtrabackup manager s9s_backup holland backup manager fromdual_brman twindb … and more
  13. 13. Where things fall short ● dependencies ● portability ● for non-dbas ● babysitting ● validation ● remote streaming ● restore help ● encryption ● expanded retention sets ● full + incremental (+ PS bitmaps)
  14. 14. So we began with xbackup.sh ● bash shell ● had most of the initial required features ● difficult to maintain and debug ● rewritten to pyxbackup
  15. 15. pyxbackup https://github.com/dotmanila/pyxbackup
  16. 16. Simple backups xtrabackup pyxbackup full|incr
  17. 17. Saving disk space xtrabackup compress pyxbackup --compress --compress-with=gzip full|incr
  18. 18. Validation xtrabackup apply-log compress pyxbackup --apply-log full|incr
  19. 19. Optional encryption xtrabackup apply-log compress encrypt pyxbackup --compress --compress-with=gzip full|incr
  20. 20. Sending copies to remote xtrabackup apply-log compress encrypt stream (scp/nc) pyxbackup --remote-host --remote-push-only full|incr
  21. 21. Validation from remote xtrabackup compress encrypt stream (scp/nc) apply-log pyxbackup apply-last
  22. 22. Validation can be run independently apply-last apply-last pyxbackup apply-last
  23. 23. Restoring a backup (full/incremental) restore-set restore-set pyxbackup restore-set
  24. 24. Binary log streaming to support PITR binary log stream pyxbackup pyxbackup binlog-stream
  25. 25. Binlog streaming from master binary log stream pyxbackup stream (nc/scp)S M pyxbackup --binlog-from-master binlog-stream
  26. 26. Limitations ● python <= 2.7
  27. 27. Limitations ● python <= 2.7 ● xtrabackup backup only (can be extended for logical dumps, per database/table restore)
  28. 28. Limitations ● python <= 2.7 ● xtrabackup backup only (can be extended for logical dumps, per database/table restore) ● local or remote Linux servers only (SSH + Python + xtrabackup)
  29. 29. Future ● xrabackup 2.3
  30. 30. Future ● xrabackup 2.3 ● python 3.x
  31. 31. Future ● xrabackup 2.3 ● python 3.x ● multiple storage locations (round-robin)
  32. 32. Future ● xrabackup 2.3 ● python 3.x ● multiple storage locations (round-robin) ● point in time recovery (some)
  33. 33. Future ● xrabackup 2.3 ● python 3.x ● multiple storage locations (round-robin) ● point in time recovery (some) ● restore/test database/table
  34. 34. Future ● xrabackup 2.3 ● python 3.x ● multiple storage locations (round-robin) ● point in time recovery (some) ● restore/test database/table ● logical dumps
  35. 35. Future ● xrabackup 2.3 ● python 3.x ● multiple storage locations (round-robin) ● point in time recovery (some) ● restore/test database/table ● logical dumps ● unit tests (ahem)
  36. 36. Questions? Yes, #jervinisreal
  37. 37. xtrabackup apply-log compress encrypt stream (scp/nc)

×