This document discusses using Fabric for Python application deployment and configuration management. It provides an overview of Fabric basics like tasks, roles, and environments. It also describes using Fabric for common operations like code deployment, database migrations, and managing server growth. Key advantages of Fabric include its simple task-based interface and ability to control multiple servers simultaneously. The document provides an example of using Fabric for a full deployment process including pushing code, running migrations, and restarting processes.

1. Prodution Architecture and Deploymentwith Fabric- Andy McCurdy -@andymccurdy

2. Whiskey Media

3. Whiskey Sites

4. Your First Django App

5. Basic Config (web)Apache

6. mod_wsgi

7. use daemon mode

8. threads more efficient

9. processes if you're unsure of thread safetyWSGIDaemonProcess my-site python-path=/home/code/ processes=2 threads=150 maximum-requests=5000WSGIProcessGroup my-siteWSGIScriptAlias / /home/code/my-site/deploy/wsgi/my-site.wsgi

10. Basic Config (media)Nginx

11. Use Nginx to proxy traffic to Apache

12. Meanwhile Nginx serves mediaupstream my-site { server 127.0.0.1:8000;}server { listen 80; location ~ ^/media/ { root /home/code/my-site; expires 30d; } location / { proxy_pass http://my-site; proxy_set_header X-Real-IP $remote_addr; }}

13. Basic Config (db)Databases

14. PostgreSQL

15. Frank Wiles @ www.revsys.com

16. MySQL

17. Percona @ www.mysqlperformanceblog.comBasic Config (cache)Use Memcached! Even 16mb will significantly help against a Digg or being Slashdot'ed

18. It's incredibly easy...# settings.pyMIDDLEWARE_CLASSES = ( 'django.middleware.cache.UpdateCacheMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.cache.FetchFromCacheMiddleware')CACHE_BACKEND = 'memcached://127.0.0.1:11211/'CACHE_MIDDLEWARE_SECONDS = 60*5 # 5 minutesCACHE_MIDDLEWARE_ANONYMOUS_ONLY = True

19. Basic DeploymentCopy Code & Media (rsync or scp)

20. Run DB migrations / syncdb

21. Bounce wsgi daemonsManaging Growth (db)Move DB to its own server

22. As much RAM as possible

23. Write-heavy (>10%)? Get fast disks

24. Tune your config fileManaging Growth (web)Add more web servers

25. Use a resource monitoring tool like Munin to understand if your app is CPU or memory boundEven More GrowthReplicated or sharded Databases

26. Multiple load balancers for redundancy

27. Message queues

28. Crons

29. Search daemons (Solr, Sphinx)

30. etc...(not so) Basic Deploymentimage credit:Brad Fitzpatrick

31. Deployment ReqsCopy media to CDN

32. Maintenance splash page

33. Run DB migrations

34. Install/Upgrade Python dependencies

35. Add a new web server to the cluster

36. Execute arbritrary commands for sysadmin maintenance tasksDeployment OptionsCapistrano+ Out of box support for common use cases+ Hooks to customize tasks+ Source control integration+ Threaded deployment to multiple hosts- Ruby :(

37. Deployment OptionsFabric+ Very simple, tasks are just Python functions+ Easy to chain together tasks to create complex scripts out of bite size pieces- No source control integration- No out of box support- Some bugs, although fairly easy to work around, and new maintainer is working on fixes

38. Fabric Basicssudo easy_install fabric

39. need a fabfile.py

40. from fabric.api import *

41. be mindful of tasks that may fail

42. each remote command starts fresh

43. changing directoriesCore Functionalitylocal() - Run a command locally

44. run() - Run a command remotely

45. sudo() - Run a command remotely as another user

46. put() - Copy a file from local to remote

47. get() - Copy a file from remote to local

48. many more helper-ish commandsAuthenticationRelies on SSH model

49. Use SSH keys

50. Control access to root user via sudoers

51. When you have to revoke access, you just turn off their SSH accountConfigurationFabric environment (env) -- it's just a dictionary

52. Hosts and Roles

53. Code Repositories

54. Whatever you need

55. ~/fabricrc

56. Global settings or all Fabric deployments

57. SSH usernameExample Config# fabfile.pyfrom fabric.api import *env.roledefs = { 'web' : ['10.1.1.1', '10.1.1.2'], 'db' : ['10.1.1.3'], 'lb' : ['10.1.1.4'], }env.repositories = {...}

58. Tasks# fabfile.pydef uptime(): run('uptime')$> fab uptime -H 10.1.1.3[10.1.1.3] run: uptime[10.1.1.3] out: 05:20:39 up 88 days, 12:00, 0 users, load average: 0.03, 0.03, 0.00

59. Mapping Roles to Tasks# fabfile.py@roles('web')def uptime(): run('uptime')$> fab uptime[10.1.1.1] run: uptime[10.1.1.1] out: 05:20:39 up 88 days... [10.1.1.2] run: uptime[10.1.1.2] out: 05:20:39 up 88 days...

60. Decorator ProblemsSome problems with Fabric's role management

61. Can't override decorated tasks at command line as docs suggestdef default_roles(*role_list): def selectively_attach(func): if not env.roles and not env.hosts: return roles(*role_list)(func) else: if env.hosts: func = hosts(*env.hosts)(func) if env.roles: func = roles(*env.roles)(func) return func return selectively_attach

62. All better now#fabfile.py@default_roles('web', 'db')def uptime(): run('uptime')$> fab uptime# runs on all hosts in the 'web' and 'db' roles$> fab uptime --roles lb# runs only on hosts in the 'lb' role

63. Dealing with FailuresBy default Fabric dies if a task fails

64. Use a context manager when failures are anticipated# fabfile.pyfrom __future__ import with_statement # py2.5def symlink_me(): with settings(warn_only=True): run('rm /path/to/symlink') run('ln -s /home/andy /path/to/symlink')

65. Easy sys-adminMake an "invoke" command

66. Great for sys-admin and one-off tasks# fabfile.py@default_roles('all')def invoke(command): "Invoke an arbritrary command" sudo(command)# install new packages on all hosts in one command$> fab invoke:"apt-get install git-core"

67. Real World Tasks$> fab --listAvailable commands:bounce_wsgi_procs Bounce the WSGI procs by touching the filesdeploy Full deploymentdeploy_media Push media to S3invoke Invoke an arbritrary commandmigrate Run any migrations via Southreload_nginx Update Nginx's running configsplash_off Configure Nginx to serve the sitesplash_on Configure Nginx to serve a downed-site pageupdate_repositories Push code to serversupdate_dependencies Update dependencies to third party libs

68. Whiskey's Deploymentdef deploy(splash='no'): "Full deployment" deploy_media() update_cached_repositories() update_dependencies() generate_releases() if splash == 'yes': splash_on() _symlink_code() migrate() bounce_wsgi_procs() if splash == 'yes': splash_off()$> fab deploy:splash=yes

69. Questions?