Buildout and Hostout or “How to host an app for $20 in 20min” Dylan Jay [email_address] Technical Solutions Manager PretaWeb (Thanks to Darryl Cousins for slides 1-10 from NZPUG presentation)

A Python Module A block of code imported by some other code

A python module: hello.py We can import the method from the module: def helloworld (): print u"Hello World from hello import helloworld

A Python Package: hello A package is a module that contains other modules: hello/ __init__.py hello.py Now we must import the method from the module within the module. from hello.hello import helloworld

Disutils Distutils was written so we have a unified way to install python modules. python setup.py install

Disutils – creating a distribution To distribute the hello module we need have it in a directory with a setup.py file. workingdir/ setup.py hello/ __init__.py hello.py The setup.py file needs at the least the following. from distutils.core import setup setup(name="hello", )

Disutils creating a distribution continued Now we can create a distribution tarball with disutils. python setup.py sdist Our directory now looks like this workingdir/ setup.py hello/ __init__.py hello.py dist/ hello-1.0.tar.gz If we unpack the source distribution it looks like this: Hello-1.0/ PKG-INFO setup.py hello/ __init__.py hello.py

setuptools • Setuptools is built on top of distutils • uses the setup.py • uses eggs for distribution • allows us to save our modules as eggs to pypi Installing setuptools wget http://peak.telecommunity.com/dist/ez_setup.py python ez_setup.py

Eggs To create an egg change the import line in setup.py from setuptools import setup setup(name="hello", version="1.0", ) We can call that with: python setup.py bdist_egg Which creates a binary egg in our dist directory dist/ hello-1.0-py2.4.egg

pypi If we want that egg available on pypi and we have an account we can do that with a single command. python setup.py sdist upload Which all the world can use easy_install hello

virtualenv If we want to install without affecting whole system $ easy_install virtualenv $ virtualenv myenv $ source myenv/bin/activate (myenv)$ easy_install hello . . . (myenv)$ deactivate

zc.buildout – what is it? “ Buildout is a system of configuring repeatable steps for assembling complicated systems (applications) from multiple parts.” Configure different parts using one config file Glue the parts togeather Specify versions Isolated in one directory Repeat that configuration over and over

Configure different parts using one config file

Glue the parts togeather

Specify versions

Isolated in one directory

Repeat that configuration over and over

Installing buildout $ easy_install zc.buildout $ cd myproj $ buildout init This creates the following directory structure myproj/ bin/ buildout parts/ develop-eggs/ buildout.cfg

buildout.cfg Buildout does very little by itself [buildout] parts = A buildout is made of parts. Parts are instances of recipes. Recipes do the work. [buildout] parts = py [py] recipe = zc.recipe.egg interpreter = py eggs = hello

Recipes – 185+ on pypi

Running buildout $ bin/buildout Installing py. Generated interpreter 'myproj/bin/py'. $ bin/py >>> from hello import helloworld >>> helloworld() Hello World

Buildout - Scripts You can generate scripts [buildout] parts = buildout1 buildout2 [buildout1] recipe = zc.recipe.egg eggs = zc.buildout==1.2.1 scripts = buildout=buildout1 [buildout2] recipe = zc.recipe.egg eggs = zc.buildout<=1.2.0 scripts = buildout=buildout2

Buildout - Versions You can pin versions [buildout] parts = buildout1 buildout2 versions = versions [buildout1] recipe = zc.recipe.egg eggs = zc.buildout scripts = buildout=buildout1 [versions] zc.buildout = 1.2.1

Buildout - Scripts are wrappers $ cat bin/buildout1 #!/bin/python import sys sys.path[0:0] = [ '/download-cache/eggs/zc.buildout-1.2.1-py2.4.egg', '/download-cache/eggs/setuptools-0.6c9-py2.4.egg', ] import zc.buildout.buildout if __name__ == '__main__': zc.buildout.buildout.main()

Buildout – manages installation $ bin/buildout -v Uninstalling py. Installing buildout1. Installing 'zc.buildout==1.2.1'. We have the distribution that satisfies 'zc.buildout==1.2.1'. Adding required 'setuptools' required by zc.buildout 1.2.1. Picked: setuptools = 0.6c9 Generated script 'myproj/bin/buildout1'. Installing buildout2. Installing 'zc.buildout==1.2.0'. We have the distribution that satisfies 'zc.buildout==1.2.0'. Adding required 'setuptools' required by zc.buildout 1.2.0. Picked: setuptools = 0.6c9 Generated script 'myproj/bin/buildout2'.

Buildout – variables and dependencies and distutils [ buildout ] parts = py [ py ] recipe = zc .recipe.egg Interpreter = py extra-paths= ${reportlab:output} [ reportlab ] recipe = collective.recipe. distutils url = http://www.reportlab.org/ftp/ReportLab_2_3.tar.gz output = ${buildout:directory}/parts/site-packages/

Helloworld in Plone src/hello/hello/hello.py from Products.Five.browser import BrowserView class HelloWorld (BrowserView): def __call__ ( self ): return &quot;Hello World&quot; src/hello/hello/configure.zcml <configure xmlns:browser=&quot; http://namespaces.zope.org/browser &quot;> <browser:page name=&quot;hello&quot; class=&quot;.hello.HelloWorld&quot; permission=&quot;zope2.Public&quot; /> </configure>

Helloworld in Plone buildout base.cfg [ buildout ] parts = instance develop = src /hello [zope2] recipe = plone .recipe.zope2install url = http://www.zope.org/Products/Zope/2.10.8/Zope-2.10.8-final.tgz [instance] recipe = plone .recipe.zope2instance zope2-location = ${zope2:location} user = dylan:jay eggs = hello Plone zcml = hello

Run Helloworld $ bin/buildout -c base.cfg Develop: 'myproj/src/hello' Updating zope2. Updating fake eggs Installing instance. $ bin/instance fg 2009-06-08 20:24:54 INFO ZServer HTTP server started at Mon Jun 8 20:24:54 2009 Hostname: localhost Port: 8080 ... 2009-06-08 20:25:44 INFO Zope Ready to handle requests

Production buildout extends base prod.cfg [ buildout ] extends = base. cfg parts += supervisor [instance] zeo -client = True zeo -address = ${zeo:zeo-address} [ zeo ] recipe = plone .recipe.zope2zeoserver zope2-location = ${zope2:location} Zeo -address = 127.0.0.1:9000 [varnish] recipe = plone .recipe.varnish:instance bind = 127.0.0.1:80 mode = foreground backends = ${instance:http-address} daemon = ${varnish-build:location}/ sbin / varnishd [varnish-build] recipe = zc .recipe. cmmi url = http://waix.dl.sourceforge.net/sourceforge/varnish/varnish-2.0.3.tar.gz [supervisor] recipe = collective.recipe.supervisor programs = 10 zeo ${zeo:location}/bin/ runzeo 20 instance1 ${instance:location}/bin/ runzope 70 cache ${buildout:bin-directory}/varnish ${varnish:location} true

Hosting a buildout First get a server with root access ~$20USD p/m for 256mb Slicehost, Amazon ec2, etc

collective.hostout Aims to to make a hosted python app :- as easy to install as a cPanel app as easy to deploy as google app engine keep control of code

Aims to to make a hosted python app :-

as easy to install as a cPanel app

as easy to deploy as google app engine

keep control of code

Put collective.hostout in buildout.cfg [ buildout ] extends = base. cfg parts += hostout [ hostout ] recipe = collective. hostout host= myproj . slicehost . com buildout = prod . cfg start_cmd = ${buildout:bin-directory}/ supervisord stop_cmd = ${buildout:bin-directory}/ supervisorctl shutdown

Run hostout $ bin/buildout Develop: 'myproj/src/hello' Updating zope2. Updating fake eggs Updating instance. Installing hostout. $ bin/hostout running setup script 'src/hello/setup.py'. … creating deployment file: versions.cfg prod.cfg base.cfg hello1.0dev_12...6-py2.4.egg logging into the following hosts as root: myproj.slicehost.com Password: Installing python Bootstrapping buildout in /var/local/buildout Unpacking deployment file Running buildout Running start command

Hostout – with source control Doesn't need access to svn or git on the server

Hostout TODO Multi server Transfer databases Integrate with source control Rollback http://plone.org/products/collective.hostout

Multi server

Transfer databases

Integrate with source control

Rollback

http://plone.org/products/collective.hostout