Reaching
Deployment Nirvana
Using Buildout
Clayton Parker
Plone Conference 2008
nowhere to go but
open source
s ix fe e tup . c om / dw 0 8

Here’s to Plone
si xf eet up .co m /dw 0 8

To-do List
• Buildout
• Parts
• Recipes
• Command Line
• ZopeSkel
• Create Buildouts
• Custom Recipes
si xf eet up .co m /dw 0 8

Why buildout?
si xf eet up .co m /dw 0 8

Eliminate Confusion
si xf eet up .co m /dw 0 8

Tracking Dependencies
si xf eet up .co m /dw 0 8

Roll Your Own
• Installer
• ZopeSkel
si xf eet up .co m /dw 0 8

ZopeSkel
$ easy_install ZopeSkel
$ paster create --list-templates
plone3_buildout: A buildout for Plone 3 projects
$ paster create -t plone3_buildout mybuildout
si xf eet up .co m /dw 0 8

Lingo
si xf eet up .co m /dw 0 8

Syntax
[buildout]
parts =
zope2
plone
instance
[plone]
recipe = plone.recipe.plone
[zope2]
recipe = plone.recipe.zope2install
[instance]
recipe = plone.recipe.zope2instance
user = admin:admin
eggs = ${plone:eggs}
products = ${plone:products}
si xf eet up .co m /dw 0 8

Advanced Syntax
Variable Substitution
${part:option}
${plone:eggs}
${buildout:directory}
option = ${buildout:directory}/somefolder
Option Addition and Removal
options = foo bar
options += baz
options -= foo
si xf eet up .co m /dw 0 8

Reserved Characters
:$ % { }
si xf eet up .co m /dw 0 8

Parts
si xf eet up .co m /dw 0 8

[buildout]
parts =
instance
[instance]
recipe = plone.recipe.zope2instance
user = admin:admin
eggs = ${plone:eggs} archetypes.schemaextender
zcml = archetypes.schemaextender
products = ${plone:products}
environment-vars =
TZ America/Chicago
http-address = 51060
zeo-address = 53060
debug-mode = on
verbose-security = on
event-log-level = debug
si xf eet up .co m /dw 0 8

Recipes
si xf eet up .co m /dw 0 8

[buildout]
parts =
zope2
plone
instance
[plone]
recipe = plone.recipe.plone
[zope2]
recipe = plone.recipe.zope2install
[instance]
recipe = plone.recipe.zope2instance
user = admin:admin
eggs = ${plone:eggs}
products = ${plone:products}
si xf eet up .co m /dw 0 8

Buildout Directory
• bin/
• bootstrap.py
• buildout.cfg
• develop-eggs/
• downloads/
• eggs/
• parts/
• products/
• src/ si xf eet up .co m /dw 0 8

Buildout options
• bin-directory
• develop-eggs-directory
• eggs-directory
• parts
• parts-directory
• log-level
si xf eet up .co m /dw 0 8

Defaults
[buildout]
eggs-directory = /home/clayton/.buildout/eggs
download-cache = /home/clayton/.buildout/downloads
zope-directory = /home/clayton/.buildout/zope
index = http://download.zope.org/ppix
[instance]
event-log-level = debug
si xf eet up .co m /dw 0 8

Recipes
si xf eet up .co m /dw 0 8

Plone Recipes
• plone.recipe.plone
• plone.recipe.zope2install
• plone.recipe.zope2instance
• plone.recipe.squid
si xf eet up .co m /dw 0 8

plone.recipe.zope2install
[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}
fake-zope-eggs = true
additional-fake-eggs = ZODB3
skip-fake-eggs =
zope.testing
zope.component
zope.i18n
si xf eet up .co m /dw 0 8

In The Wild
si xf eet up .co m /dw 0 8

Extending Configuration
• buildout.cfg
• profiles/
• base.cfg
• development.cfg
• debug.cfg
• qa.cfg
• prod.cfg
• versions.cfg
si xf eet up .co m /dw 0 8

buildout.cfg
[buildout]
# the profile we want to use
extends = profiles/development.cfg
find-links =
http://dist.plone.org
http://download.zope.org/distribution/
http://effbot.org/downloads
si xf eet up .co m /dw 0 8

base.cfg
[buildout]
parts =
PILwoTK
plone
zope2
instance
zeoserver
develop = src/my.package
[PILwoTK]
recipe = zc.recipe.egg
find-links = http://download.zope.org/distribution/
[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}
fake-zope-eggs = true
additional-fake-eggs = ZODB3
skip-fake-eggs = zope.testing zope.component zope.i18n
si xf eet up .co m /dw 0 8

[plone]
recipe = plone.recipe.plone
[instance]
recipe = plone.recipe.zope2instance
zeo-client = True
zope2-location = ${zope2:location}
eggs =
PILwoTK
elementtree
${plone:eggs}
my.package
zcml = my.package
products = ${plone:products}
environment-vars =
TZ America/New_York
[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${instance:zope2-location}
zeo-address = ${instance:zeo-address}
si xf eet up .co m /dw 0 8

development.cfg
[buildout]
extends =
base.cfg
debug.cfg
parts += ${debugging:parts} omelette
[instance]
user = admin:admin
http-address = 8080
zeo-address = 8100
debug-mode = on
verbose-security = on
event-log-level = debug
products +=
${buildout:directory}/products
${debugging:debug-products}
eggs += ${debugging:eggs}
zcml += ${debugging:zcml}
[omelette]
recipe = collective.recipe.omelette
eggs = ${instance:eggs}
ignore-develop = True
si xf eet up .co m /dw 0 8
ignores = setuptools

debug.cfg
[debugging]
parts =
debug-products
debug-products-svn
ipzope
zopepy
debug-products =
${debug-products:location}
eggs =
plone.reload
Products.PDBDebugMode
Products.DocFinderTab
Products.Clouseau
Products.PrintingMailHost
zcml =
plone.reload
si xf eet up .co m /dw 0 8

[debug-products]
recipe = plone.recipe.distros
urls =
...dcworkflowgraph-0_3.tgz
...PTProfiler-1.2.tgz
[ipzope]
recipe = zc.recipe.egg
eggs = ipython ${instance:eggs}
initialization =
import sys, os
os.environ[\"SOFTWARE_HOME\"] = \"${instance:zope2-location}/lib/
python\"
os.environ[\"INSTANCE_HOME\"] = \"${instance:location}\"
sys.argv[1:1] = \"-p zope\".split()
extra-paths = ${instance:zope2-location}/lib/python
scripts = ipython=ipzope
[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${instance:zope2-location}/lib/python si xf eet up .co m /dw 0 8
scripts = zopepy

versions.cfg
[versions]
# Use the following from the command line to get the latest versions:
# bin/buildout -vvvvv |sed -ne 's/^Picked: //p' | sort | uniq
PILwoTk = 1.1.6.4
elementtree = 1.2.7-20070827-preview
infrae.subversion = 1.1
plone.recipe.distros = 1.3
plone.recipe.plone = 3.1.6
plone.recipe.zope2install = 2.3
plone.recipe.zope2instance = 2.5
plone.recipe.zope2zeoserver = 0.13
python-openid = 2.2.1
my.package = 1.0
si xf eet up .co m /dw 0 8

prod.cfg
[buildout]
extends = base.cfg versions.cfg
parts += instance2
versions = versions
[instance]
zope2-location = ${zope2:location}
http-address = 9080
zeo-address = 9100
eggs += Products.CacheSetup
z2-log-level = CRITICAL
zodb-cache-size = 15000
[instance2]
recipe = collective.recipe.zope2cluster
instance-clone = instance
http-address = 10080
si xf eet up .co m /dw 0 8

Command Line
si xf eet up .co m /dw 0 8

Baby Steps
$ cd path/to/mybuildout
$ python2.4 bootstrap.py
Creating directory 'mybuildout/bin'.
Creating directory 'mybuildout/parts'.
Creating directory 'mybuildout/develop-eggs'.
Generated script 'mybuildout/bin/buildout'.
$ bin/buildout
$ bin/instance start
si xf eet up .co m /dw 0 8

Options
• -v and -q
• increase and decrease verbosity
• -n and -N
• Newest and non-newest modes
• -O and -o
• online and offline mode
• -t
• socket timeout
si xf eet up .co m /dw 0 8

Update your buildout
$ bin/buildout -v
$ bin/buildout -Nvvv
$ bin/buildout -No
$ bin/buildout -t 60
si xf eet up .co m /dw 0 8

Assignments
$ bin/buildout instance:debug-mode=on
$ bin/buildout buildout:log-level=70
$ bin/buildout -N instance:debug-mode=on -v
si xf eet up .co m /dw 0 8

Commands
$ bin/buildout install
$ bin/buildout -Nv install zope2 instance
$ bin/buildout -nv install instance
si xf eet up .co m /dw 0 8

Create Recipes
$ paster create -t recipe my.recipe.example
si xf eet up .co m /dw 0 8

Recipe
• Recipe class
• constructor
• install
• update
• uninstall (optional)
si xf eet up .co m /dw 0 8

class Recipe:
\"\"\"\"A recipe
\"\"\"\"
def __init__(self, buildout, name, options):
self.buildout = buildout
self.name = name
self.options = options
# gather options from other parts here
options['notmine'] = buildout['someotherpart']['foobar']
def install(self):
\"\"\"\"Install method
\"\"\"
options = self.options
location = options['location']
# must return a string, or an iterable of strings
return location
def update(self):
\"\"\"Update method
\"\"\"
pass
si xf eet up .co m /dw 0 8

class Recipe:
Real World
\"\"\"infrae.subversion recipe.
\"\"\"
def __init__(self, buildout, name, options):
self.buildout = buildout
self.name = name
self.options = options
options['location'] = self.location = os.path.join(
buildout['buildout']['parts-directory'], self.name)
self.urls = [l.split()
for l in options['urls'].splitlines()
if l.strip()]
self.export = options.get('export')
self.newest = (
buildout['buildout'].get('offline', 'false') == 'false'
and
buildout['buildout'].get('newest', 'true') == 'true'
)
self.verbose = buildout['buildout'].get('verbosity', 0)
si xf eet up .co m /dw 0 8

Install Method
def install(self):
\"\"\"Checkout the checkouts.
\"\"\"
for (url, name) in self.urls:
wc = py.path.svnwc(self.location).join(name)
if self.export:
raise Exception('Unimplemented feature')
if self.verbose:
print \"Fetch %s\" % url
wc.checkout(url)
return self.location
si xf eet up .co m /dw 0 8

def update(self):
\"\"\"Update the checkouts\"\"\"
if not self.newest:
return self.location
if self.export:
return self.location
if self.options.get('ignore_updates', False):
return self.location
num_release = re.compile('.*@[0-9]+$')
part = py.path.local(self.location)
for link, sub_path in self.urls:
if num_release.match(link):
if self.verbose:
print \"Given num release for %s, skipping.\" % link
continue
wc = py.path.svnwc(self.location).join(sub_path)
if self.verbose:
print \"Updating %s\" % link
wc.update()
return self.location
si xf eet up .co m /dw 0 8

Wrapping Up
• Buildout
• Parts
• Recipes
• Command Line
• ZopeSkel
• Create Buildouts
• Custom Recipes
si xf eet up .co m /dw 0 8

Plone Deployment Workshop
sixfeetup.com/dw08
Register by October 17
and save $100!

Links
• http://buildout.zope.org
• http://pypi.python.org/pypi/zc.buildout
• https://svn.sixfeetup.com/svn/public/buildout/debug.cfg
• http://www.sixfeetup.com/swag/buildout-quick-reference-card
• http://plone.org/documentation/tutorial/buildout
• http://pypi.python.org
si xf eet up .co m /dw 0 8

Photo Credits
• http://flickr.com/photos/monsieurlam/2645956083/
• http://flickr.com/photos/_boris/2796908072/
• http://flickr.com/photos/b-tal/163450213/
• http://flickr.com/photos/bullish1974/2648544508/
• http://flickr.com/photos/haydnseek/87432002/
• http://flickr.com/photos/disowned/1158260369/
• http://flickr.com/photos/7603557@N08/2662531345/
• http://flickr.com/photos/julishannon/2151986631/
• http://flickr.com/photos/julishannon/2152778524/
• http://flickr.com/photos/lollyknit/1155225799/
• http://flickr.com/photos/binary_koala/86227485/
si xf eet up .co m /dw 0 8