Backup UUASC 9/6/12

656 views

Published on

A presentation on rsync and Duplicity

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
656
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Backup UUASC 9/6/12

  1. 1. backupsusing rsync and duplicity
  2. 2. Who needs to backup?There are 2 kinds of people in this world:1. Those who keep backups.2. Those who lose their data.
  3. 3. What is a backup?The copying and archiving of computer data toassist in recovering from a data loss event.
  4. 4. What causes data loss?● Failure● Disaster● Crime● Intentional action● Unintentional action
  5. 5. Why keep a backup?● Keeps important files safe● Helps us recover from stupid mistakes● Helps recover from malicious attacks● CYA ( Cover Your Ass )
  6. 6. Types of backups● RAID!?
  7. 7. RAID?!RAID is not a backup!ಠ_ಠ
  8. 8. Types of backups● RAID!?● Unstructured
  9. 9. Types of backups● RAID!?● Unstructured● Differential● Incremental
  10. 10. Rsync● extremely efficient file transfer protocol for Unix● open source (GNU license)● written by Andrew Tridgell and Paul Mackerras at the Australian National University in 1996
  11. 11. RsyncWe can backup to either local or remote serverLOCALGo to external disk for local backup. Backing up to samehard drive is not sufficient for a comprehensive backupsolution!EXTERNALGoes to rsync daemon (can be standalone or started fromrsync client by inetd). Route through SSH shell for security.
  12. 12. RsyncLocal locations:~/docs/importantRemote locations:Remote Shell[user@]server.com:~/docs/importantRsync Daemon[user@]server.com::~/docs/importantrsync://[user@]server.com/~/docs/important
  13. 13. Rsynccommon flags: ● -v : verbose
  14. 14. Rsynccommon flags: ● -v : verbose ● -a : archive
  15. 15. Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress
  16. 16. Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress ● --rsh=/usr/bin/ssh
  17. 17. Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress ● --rsh=/usr/bin/ssh ● --delete : delete extraneous files
  18. 18. Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress ● --rsh=/usr/bin/ssh ● --delete : delete extraneous files ● --link-dest : create hard links (incremental backup)
  19. 19. Incremental backups with RsyncHard links allow us to create extra nodes forour files so we can make sure at least one copyexists without using extra space.
  20. 20. Advantages of Rsync over cp● Mirroring data● Transmitting lots of data● Only copy what is needed● Use with --link-dest to create hard links for incremental backups
  21. 21. Duplicity● First developed in 2002.● Current version is 0.6.19 released in May 2012● GNU licensed● Written in Python & C● Leverages Rsync algorithm● Available via package managers, or via RPM, binaries, or source downloadable from http://duplicity.nongnu.org/
  22. 22. DuplicityProduces tar-format volumes on local or remoteservers
  23. 23. Duplicity files● Backup● Manifest● Signature
  24. 24. Duplicity filesystemsLocal File Storage, scp/SSH, Rsync, FTP, HSI,WebDAV, Tahoe-LAFS, Amazon S3, withothers easily added
  25. 25. DuplicityRemote locations are specified using URIs:● ftp://[user@]archive.business.com/backups● rsync://[user@]newspeak.marketing.biz/outside_box/● s3+http://awesometown-socool
  26. 26. Using DuplicityBy default, Duplicity doesnt need anycommand line options to make encrypted,incremental backups to a local or remotelocation. However, for a sane backup strategywe should:● Perform periodic full (snapshot) backups● Delete old backups● Ignore certain files
  27. 27. Using Duplicity● --full-if-older-than
  28. 28. Using Duplicity● --full-if-older-than● remove-all-but-n-full
  29. 29. Using Duplicity● --full-if-older-than● remove-all-but-n-full● --include/exclude-globbing-filelist
  30. 30. Using Duplicity● --full-if-older-than● remove-all-but-n-full● --include/exclude-globbing-filelist● PASSPHRASE environment variable
  31. 31. Moar DuplicityAlso used to verify backups and restore backupsVerify:duplicity verify [backup] [source]can also be used to determine changes since backup
  32. 32. Moar DuplicityAlso used to verify backups and restore backupsVerify:duplicity verify [backup] [source]can also be used to determine changes since backupRestore:duplicity restore [backup] [destination]Can use --file-to-restore and --time flags to specify moreprecise restoration
  33. 33. Why use Duplicity over Rsync?● Encrypted and signed archives● Built in incremental backup● Ability to use a wide range of local and remote filesystems
  34. 34. Best Practices● Trust, but verify● Use strong passphrase● Backup to multiple locations● Automate
  35. 35. AutomationFor the best backup strategy, we take thehuman out of the equation.● write a Bash script and cron it● Use a GUI
  36. 36. Related LinksJamie Zawinski on backupshttp://www.jwz.org/doc/backups.htmlSnapshot Backups with Rsync by Mike Ruebelhttp://www.mikerubel.org/computers/rsync_snapshots/Epydocs for Duplicityhttp://duplicity.nongnu.org/epydoc/index.htmlDuplicity to Amazon S3 Bash Scripthttp://blog.damontimm.com/bash-script-incremental-encrypted-backups-duplicity-amazon-s3/RTFM!http://linux.die.net/man/1/rsynchttp://duplicity.nongnu.org/duplicity.1.htmlLots of Horror Stories about not Backing Uphttp://reddit.com/r/talesfromtechsupport

×