Backups,
esta vez de verdad
Svet Ivantchev, eFaber
UniEE, 27 de marzo de 2007

Las copias de seguridad
• Necesidad
• Ya, pero es un rollo
• Los dos tipos de personas

¿Cómo nos quedará?
X X

¿Cómo nos quedará?

¿Cómo nos quedará?

¿Cómo nos quedará?
• cd /backup/snapshots/
• ls -l
drwxr-xr-x 14 svet admin 476 Mar 20 23:44 snapshot-2007-03-20
drwxr-xr-x 14 svet admin 476 Mar 21 23:44 snapshot-2007-03-21
drwxr-xr-x 14 svet admin 476 Mar 22 23:44 snapshot-2007-03-22
drwxr-xr-x 14 svet admin 476 Mar 23 23:44 snapshot-2007-03-23
drwxr-xr-x 14 svet admin 476 Mar 24 23:44 snapshot-2007-03-24
drwxr-xr-x 14 svet admin 476 Mar 25 23:44 snapshot-2007-03-25

Foto

¿Cuanto ocupa?
• $ cd /backup/snapshots/
$ du -s -h snapshot-2007-03-20
470M snapshot-2007-03-20
$ du -s -h snapshot-2007-03-23
688M snapshot-2007-03-23
$ du -s -h snapshot-2007-03-25
689M snapshot-2007-03-25
$ du -s -h
753M .
• 470M + 688M + 689M + ⋯ = 753M ???

¿Cuanto ocupa?
• $ cd /backup/snapshots/
$ du -s -h snapshot-2007-03-20
470M snapshot-2007-03-20
$ du -s -h snapshot-2007-03-23
688M snapshot-2007-03-23
$ du -s -h snapshot-2007-03-25
689M snapshot-2007-03-25
$ du -s -h
753M .
• 470M + 688M + 689M + ⋯ = 753M ???
Nota: la programación necesaria son dos lineas (2 de verdad)

Componentes
• Software:
• rsync
• cp y hardlinks
• (opcionalmente) cron
• Hardware (opcional)
• Un NSLU2 (+ Unslung firmware)
• Disco duro externo USB2

rsync
• $ rsync -ae ssh mi_portatil:/home/datos_importantes /backup/2007-03-27

Día 1

Día 2

Día 3

Evolución

No todo cambia

Nuevo

Nuevo

Nuevo

Enlaces simbólicos
• $ ls -l
total 85052
-rw-r--r-- 1 svet svet 86999860 2007-03-26 22:27 fichero1.dat
$ ln -s fichero1.dat fichero2.dat
$ ls -l
total 85052
-rw-r--r-- 1 svet svet 86999860 2007-03-26 22:27 fichero1.dat
lrwxrwxrwx 1 svet svet 12 2007-03-26 22:27 fichero2.dat -> fichero1.dat
$ du -s
85056 .

Enlaces (hardlinks)
• $ ls -l
total 85052
-rw-r--r-- 1 svet svet 86999860 2007-03-26 22:27 fichero1.dat
$ ln fichero1.dat fichero2.dat
$ ls -l
total 170104
-rw-r--r-- 2 svet svet 86999860 2007-03-26 22:27 fichero1.dat
-rw-r--r-- 2 svet svet 86999860 2007-03-26 22:27 fichero2.dat
$ du -s
85056 .
$ ls -i fichero1.dat
3908701 fichero1.dat
$ ls -i fichero2.dat
3908701 fichero2.dat

cp -al origen destino

Todo junto
• rsync -v --delete -a servidor:/mis_datos /backup
cp -al /backup/mis_datos /snapshots/snapshot-`date -I`
X X

Harwdare
Cisco/Linksys NSLU2:
Intel XScale IXP420, 266 MHz,
8MB flash, 32MB SDRAM,
Ethernet,
2 puertos USB 2

Unslung
• $ ssh -l root 192.168.1.78
Welcome to Unslung V2.3R63-uNSLUng-6.8-beta
---------- NOTE: THIS SYSTEM IS CURRENTLY UNSLUNG ----------
BusyBox v0.60.4 (2005.03.22-06:52+0000) Built-in shell (ash)
Enter 'help' for a list of built-in commands.
# df
Filesystem 1k-blocks Used Available Use% Mounted on
rootfs 240125792 75010680 162675564 32% /
/dev/sdb1 6528 6332 196 97% /initrd
/dev/sdb1 240125792 75010680 162675564 32% /
/dev/sda1 307426936 223085068 81218592 73% /share/flash/data
/dev/sdb1 240125792 75010680 162675564 32% /share/hdd/data
/dev/sdb2 116661 4155 111302 4% /share/hdd/conf
/dev/sdb2 116661 4155 111302 4% /share/flash/conf

Unslung: ipkg
• # ipkg list
abook - 0.5.6-1 - Abook is a text-based addressbook program designed ...
adduser - 1.1.3-9 - a multi-call binary for login and user account administration
adns - 1.3-2 - Asynchronous resolver library and DNS resolver utilities.
alac-decoder - 0.1.0-2 - A decoder for the apple lossless file format
...
asterisk - 1.2.14-3 - Open Source VoIP PBX System
...
rsync - 2.6.9-2 - fast remote file copy program (like rcp)
...
zoo - 2.10-1 - zoo - File archiving utility with compression
zsh - 4.2.6-3 - Zsh is a shell designed for interactive use.

Unslung: ipkg
• # ipkg info rsync
Package: rsync
Version: 2.6.9-2
Status: install user installed
Section: net
Architecture: armeb
maintainer: NSLU2 Linux <nslu2-linux@yahoogroups.com>
MD5Sum: 4a7310cefe3d43ab5e6bd49577cdd057
Size: 200677
Filename: rsync_2.6.9-2_armeb.ipk
Source: http://www.samba.org/ftp/rsync/rsync-2.6.9.tar.gz
Description: fast remote file copy program (like rcp)
# ipkg install rsync
# ipkg install bash

Backups again

Backups again

Backups again

Backups again

Extra: Amazon S3

Ejemplo de uso
• import flickr
import BitBucket
import urllib
me = flickr.people_findByUsername(\"svet\")
bucket = BitBucket.BitBucket(\"svet-flickr\")
page = 1
total_f = found_f = 0
while 1:
try:
photos = flickr.people_getPublicPhotos(me.id, 1000, page)
for photo in photos:
total_f = total_f + 1
if bucket.has_key(\"%s-%s\" % (photo.title, photo.id)):
pass # ok, ya la tenemos
else:
data = urllib.urlretrieve(\"http://static.flickr.com/%s/%s_%s_o.jpg\" % \\
(photo.server, photo.id, photo.secret), \"flickr.jpg\")
bits = BitBucket.Bits(filename=\"flickr.jpg\")
bucket[\"%s-%s\" % (photo.title, photo.id)] = bits
print \"saving %s\" % photo.title
found_f = found_f + 1
page = page + 1
except AttributeError:
break # bucket vacío
print \"Encontradas %s fotos, guardadas %s nuevas\" % (total_f, found_f)

Refrencias
• http://www.nslu2-linux.org/
• http://samba.anu.edu.au/rsync/
• http://aws.amazon.com
• http://www.mikerubel.org/computers/rsync_snapshots/
• http://www.postneo.com/2006/03/22/