Backup UUASC 9/6/12
Upcoming SlideShare
Loading in...5

Backup UUASC 9/6/12



A presentation on rsync and Duplicity

A presentation on rsync and Duplicity



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Backup UUASC 9/6/12 Backup UUASC 9/6/12 Presentation Transcript

    • backupsusing rsync and duplicity
    • Who needs to backup?There are 2 kinds of people in this world:1. Those who keep backups.2. Those who lose their data.
    • What is a backup?The copying and archiving of computer data toassist in recovering from a data loss event.
    • What causes data loss?● Failure● Disaster● Crime● Intentional action● Unintentional action
    • Why keep a backup?● Keeps important files safe● Helps us recover from stupid mistakes● Helps recover from malicious attacks● CYA ( Cover Your Ass )
    • Types of backups● RAID!?
    • RAID?!RAID is not a backup!ಠ_ಠ
    • Types of backups● RAID!?● Unstructured
    • Types of backups● RAID!?● Unstructured● Differential● Incremental
    • Rsync● extremely efficient file transfer protocol for Unix● open source (GNU license)● written by Andrew Tridgell and Paul Mackerras at the Australian National University in 1996
    • RsyncWe can backup to either local or remote serverLOCALGo to external disk for local backup. Backing up to samehard drive is not sufficient for a comprehensive backupsolution!EXTERNALGoes to rsync daemon (can be standalone or started fromrsync client by inetd). Route through SSH shell for security.
    • RsyncLocal locations:~/docs/importantRemote locations:Remote Shell[user@] Daemon[user@][user@]
    • Rsynccommon flags: ● -v : verbose
    • Rsynccommon flags: ● -v : verbose ● -a : archive
    • Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress
    • Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress ● --rsh=/usr/bin/ssh
    • Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress ● --rsh=/usr/bin/ssh ● --delete : delete extraneous files
    • Rsynccommon flags: ● -v : verbose ● -a : archive ● -z : compress ● --rsh=/usr/bin/ssh ● --delete : delete extraneous files ● --link-dest : create hard links (incremental backup)
    • Incremental backups with RsyncHard links allow us to create extra nodes forour files so we can make sure at least one copyexists without using extra space.
    • Advantages of Rsync over cp● Mirroring data● Transmitting lots of data● Only copy what is needed● Use with --link-dest to create hard links for incremental backups
    • Duplicity● First developed in 2002.● Current version is 0.6.19 released in May 2012● GNU licensed● Written in Python & C● Leverages Rsync algorithm● Available via package managers, or via RPM, binaries, or source downloadable from
    • DuplicityProduces tar-format volumes on local or remoteservers
    • Duplicity files● Backup● Manifest● Signature
    • Duplicity filesystemsLocal File Storage, scp/SSH, Rsync, FTP, HSI,WebDAV, Tahoe-LAFS, Amazon S3, withothers easily added
    • DuplicityRemote locations are specified using URIs:● ftp://[user@]● rsync://[user@]● s3+http://awesometown-socool
    • Using DuplicityBy default, Duplicity doesnt need anycommand line options to make encrypted,incremental backups to a local or remotelocation. However, for a sane backup strategywe should:● Perform periodic full (snapshot) backups● Delete old backups● Ignore certain files
    • Using Duplicity● --full-if-older-than
    • Using Duplicity● --full-if-older-than● remove-all-but-n-full
    • Using Duplicity● --full-if-older-than● remove-all-but-n-full● --include/exclude-globbing-filelist
    • Using Duplicity● --full-if-older-than● remove-all-but-n-full● --include/exclude-globbing-filelist● PASSPHRASE environment variable
    • Moar DuplicityAlso used to verify backups and restore backupsVerify:duplicity verify [backup] [source]can also be used to determine changes since backup
    • Moar DuplicityAlso used to verify backups and restore backupsVerify:duplicity verify [backup] [source]can also be used to determine changes since backupRestore:duplicity restore [backup] [destination]Can use --file-to-restore and --time flags to specify moreprecise restoration
    • Why use Duplicity over Rsync?● Encrypted and signed archives● Built in incremental backup● Ability to use a wide range of local and remote filesystems
    • Best Practices● Trust, but verify● Use strong passphrase● Backup to multiple locations● Automate
    • AutomationFor the best backup strategy, we take thehuman out of the equation.● write a Bash script and cron it● Use a GUI
    • Related LinksJamie Zawinski on backups Backups with Rsync by Mike Ruebel for Duplicity to Amazon S3 Bash Script! of Horror Stories about not Backing Up