1.
Prodution Architecture and Deployment<br />with Fabric<br />- Andy McCurdy -<br />@andymccurdy<br />

2.
Whiskey Media<br />

3.
Whiskey Sites<br />

4.
Your First Django App<br />

5.
Basic Config (web)<br /><ul><li>Apache

6.
mod_wsgi

7.
use daemon mode

8.
threads more efficient

9.
processes if you're unsure of thread safety</li></ul>WSGIDaemonProcess my-site python-path=/home/code/<br /> processes=2 threads=150 maximum-requests=5000<br />WSGIProcessGroup my-site<br />WSGIScriptAlias / /home/code/my-site/deploy/wsgi/my-site.wsgi<br />

10.
Basic Config (media)<br /><ul><li>Nginx

11.
Use Nginx to proxy traffic to Apache

12.
Meanwhile Nginx serves media</li></ul>upstream my-site {<br /> server 127.0.0.1:8000;<br />}<br />server {<br /> listen 80;<br /> location ~ ^/media/ {<br /> root /home/code/my-site;<br /> expires 30d;<br /> }<br /> location / {<br /> proxy_pass http://my-site;<br /> proxy_set_header X-Real-IP $remote_addr;<br /> }<br />}<br />

13.
Basic Config (db)<br /><ul><li>Databases

14.
PostgreSQL

15.
Frank Wiles @ www.revsys.com

16.
MySQL

17.
Percona @ www.mysqlperformanceblog.com</li></li></ul><li>Basic Config (cache)<br /><ul><li>Use Memcached! Even 16mb will significantly help against a Digg or being Slashdot'ed

18.
It's incredibly easy...</li></ul># settings.py<br />MIDDLEWARE_CLASSES = (<br /> &apos;django.middleware.cache.UpdateCacheMiddleware&apos;,<br /> &apos;django.middleware.common.CommonMiddleware&apos;,<br /> &apos;django.middleware.cache.FetchFromCacheMiddleware&apos;)<br />CACHE_BACKEND = &apos;memcached://127.0.0.1:11211/&apos;<br />CACHE_MIDDLEWARE_SECONDS = 60*5 # 5 minutes<br />CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True <br />

19.
Basic Deployment<br /><ul><li>Copy Code & Media (rsync or scp)

20.
Run DB migrations / syncdb

21.
Bounce wsgi daemons</li></li></ul><li>Managing Growth (db)<br /><ul><li>Move DB to its own server

22.
As much RAM as possible

23.
Write-heavy (>10%)? Get fast disks

24.
Tune your config file</li></li></ul><li>Managing Growth (web)<br /><ul><li>Add more web servers

25.
Use a resource monitoring tool like Munin to understand if your app is CPU or memory bound</li></li></ul><li>Even More Growth<br /><ul><li>Replicated or sharded Databases

26.
Multiple load balancers for redundancy

27.
Message queues

28.
Crons

29.
Search daemons (Solr, Sphinx)

30.
etc...</li></li></ul><li>(not so) Basic Deployment<br />image credit:<br />Brad Fitzpatrick<br />

31.
Deployment Reqs<br /><ul><li>Copy media to CDN

32.
Maintenance splash page

33.
Run DB migrations

34.
Install/Upgrade Python dependencies

35.
Add a new web server to the cluster

36.
Execute arbritrary commands for sysadmin maintenance tasks</li></li></ul><li>Deployment Options<br />Capistrano<br />+ Out of box support for common use cases<br />+ Hooks to customize tasks<br />+ Source control integration<br />+ Threaded deployment to multiple hosts<br />- Ruby :(<br />

37.
Deployment Options<br />Fabric<br />+ Very simple, tasks are just Python functions<br />+ Easy to chain together tasks to create complex scripts out of bite size pieces<br />- No source control integration<br />- No out of box support<br />- Some bugs, although fairly easy to work around, and new maintainer is working on fixes<br />

38.
Fabric Basics<br /><ul><li>sudo easy_install fabric

39.
need a fabfile.py

40.
from fabric.api import *

41.
be mindful of tasks that may fail

42.
each remote command starts fresh

43.
changing directories</li></li></ul><li>Core Functionality<br /><ul><li>local() - Run a command locally

44.
run() - Run a command remotely

45.
sudo() - Run a command remotely as another user

46.
put() - Copy a file from local to remote

47.
get() - Copy a file from remote to local

48.
many more helper-ish commands</li></li></ul><li>Authentication<br /><ul><li>Relies on SSH model

49.
Use SSH keys

50.
Control access to root user via sudoers

51.
When you have to revoke access, you just turn off their SSH account</li></li></ul><li>Configuration<br /><ul><li>Fabric environment (env) -- it's just a dictionary

52.
Hosts and Roles

53.
Code Repositories

54.
Whatever you need

55.
~/fabricrc

56.
Global settings or all Fabric deployments

57.
SSH username</li></li></ul><li>Example Config<br /># fabfile.py<br />from fabric.api import *<br />env.roledefs = {<br /> &apos;web&apos; : [&apos;10.1.1.1&apos;, &apos;10.1.1.2&apos;],<br /> &apos;db&apos; : [&apos;10.1.1.3&apos;],<br /> &apos;lb&apos; : [&apos;10.1.1.4&apos;],<br /> }<br />env.repositories = {...}<br />

58.
Tasks<br /># fabfile.py<br />def uptime():<br /> run(&apos;uptime&apos;)<br />$&gt; fab uptime -H 10.1.1.3<br />[10.1.1.3] run: uptime<br />[10.1.1.3] out: 05:20:39 up 88 days, 12:00, 0 users, load average: 0.03, 0.03, 0.00 <br />

59.
Mapping Roles to Tasks<br /># fabfile.py<br />@roles(&apos;web&apos;)<br />def uptime():<br /> run(&apos;uptime&apos;)<br />$&gt; fab uptime<br />[10.1.1.1] run: uptime<br />[10.1.1.1] out: 05:20:39 up 88 days... <br />[10.1.1.2] run: uptime<br />[10.1.1.2] out: 05:20:39 up 88 days...<br />

60.
Decorator Problems<br /><ul><li>Some problems with Fabric's role management

61.
Can't override decorated tasks at command line as docs suggest</li></ul>def default_roles(*role_list):<br /> def selectively_attach(func):<br /> if not env.roles and not env.hosts:<br /> return roles(*role_list)(func)<br /> else:<br /> if env.hosts:<br /> func = hosts(*env.hosts)(func)<br /> if env.roles:<br /> func = roles(*env.roles)(func)<br /> return func<br /> return selectively_attach<br />

62.
All better now<br />#fabfile.py<br />@default_roles(&apos;web&apos;, &apos;db&apos;)<br />def uptime():<br /> run(&apos;uptime&apos;)<br />$&gt; fab uptime<br /># runs on all hosts in the &apos;web&apos; and &apos;db&apos; roles<br />$&gt; fab uptime --roles lb<br /># runs only on hosts in the &apos;lb&apos; role <br />

63.
Dealing with Failures<br /><ul><li>By default Fabric dies if a task fails

64.
Use a context manager when failures are anticipated</li></ul># fabfile.py<br />from __future__ import with_statement # py2.5<br />def symlink_me():<br /> with settings(warn_only=True):<br /> run(&apos;rm /path/to/symlink&apos;)<br /> run(&apos;ln -s /home/andy /path/to/symlink&apos;)<br />

65.
Easy sys-admin<br /><ul><li>Make an "invoke" command

66.
Great for sys-admin and one-off tasks</li></ul># fabfile.py<br />@default_roles(&apos;all&apos;)<br />def invoke(command):<br /> &quot;Invoke an arbritrary command&quot;<br /> sudo(command)<br /># install new packages on all hosts in one command<br />$&gt; fab invoke:&quot;apt-get install git-core&quot;<br />

67.
Real World Tasks<br />$&gt; fab --list<br />Available commands:<br />bounce_wsgi_procs Bounce the WSGI procs by touching the files<br />deploy Full deployment<br />deploy_media Push media to S3<br />invoke Invoke an arbritrary command<br />migrate Run any migrations via South<br />reload_nginx Update Nginx&apos;s running config<br />splash_off Configure Nginx to serve the site<br />splash_on Configure Nginx to serve a downed-site page<br />update_repositories Push code to servers<br />update_dependencies Update dependencies to third party libs<br />

68.
Whiskey&apos;s Deployment<br />def deploy(splash=&apos;no&apos;):<br /> &quot;Full deployment&quot;<br /> deploy_media()<br /> update_cached_repositories()<br /> update_dependencies()<br /> generate_releases()<br /> if splash == &apos;yes&apos;:<br /> splash_on()<br /> _symlink_code()<br /> migrate()<br /> bounce_wsgi_procs()<br /> if splash == &apos;yes&apos;:<br /> splash_off()<br />$&gt; fab deploy:splash=yes<br />

69.
Questions?<br />