delivering applications with zc.buildout and a distributed model - Plone Conference 2008

delivering applications with zc.buildout and a distributed model Tarek Ziadé <tarek.ziade@ingeniweb.com> 1

Who am I ? New to Plone (~1 year) -> I worked on peripheral matters Used to Zope (CPS core developer) CTO at Ingeniweb - a Plone company (>20 developers) Python fan - wrote some books about it (even in english, because you don’t hear my accent in a book) involved in plone.org migration, PSC, zc.buildout 2

It will look good on your desk besides Martin Aspeli’s book Special promotion during the PloneConf : buy one book == get a big hug 3

Thanks ! * Andreas Jung * Veda Williams * David Glick * Jeff Kowalczyk * Youenn Boussard * Christian Klinger * Jesse Snyder * Alec Mitchell * John Habermann * Maurits van Rees * Jean-François Roche * Martin Aspeli * Alain Meurant * Aleksandr Vladimirskiy * Jon Stahl * Alexander Limi * Stephen McMahon 4

Part 1 - working with packages Part 2 - working with zc.buildout Part 3 - application lifecycle 5

Part 1 - working with packages 6

distutils builds and distributes a package registers and uploads it to PyPI 7 7

get the screencast at http://ziade.org/ploneconf 8 8

+1 distutils is the standard in the whole Python community. 9 9

-1 distutils is broken (but there’s hope) 10 10

setuptools simple dependencies management, namespaced package egg distribution provides easy_install 11 11

“python setup.py bdist_egg” one single .egg file 12 12

\"Eggs are to Pythons as Jars are to Java...\" egg = deployment format in a zip archive 13 13

“sdist” is enough, though (and should be mandatory) 14 14

get the screencast at http://ziade.org/ploneconf 15 15

easy_install my_package = will get and install “my_package” from “PyPI” 16 16

+1 setuptools is the de facto standard in the Plone community. 17 17

-1 setuptools is broken too (pje says) (but there’s hope) 18 18

packaging future in Python ? 19 19

PJE Debian packagers BSD packagers Us ! Ian Zope 3 guys Repoze Discussions guys Python PEPs core TG2 guys developers Pylons guys Guido Heaven distutils 2 20 20

Packaging future in Python hard to know. but community-driven come sprint with us saturday ! 21 21

It’s OK, fellows we will eventually make it through. 22 22

Problems with packaging ? #1 PyPI == SPOF #2 packages need privacy sometimes #3 plone.org/products is *dying* 23 23

Solutions ? #1 PyPI == SPOF Make a PyPI mirror #2 packages need privacy sometimes Run you own private PyPI #3 plone.org/products is *dying* Make plone.org/products PyPI compatible 24 24

#1 Make a PyPI mirror : Smart mirroring easy_install collective.eggproxy 25 25

26 26

In zc.buildout : the index option [buildout] index = http://my.mirror:8888 all calls will go through the proxy the mirror is filled on-demand 27 27

#2 Run your own private PyPI PloneSoftwareCenter ! 28 28

get the screencast on http://ziade.org/ploneconf 29 29

collective.dist python 2.6 new “register” and “upload” commands “mregister” and “mupload” in 2.4 and 2.5 $ python setup.py mregister sdist mupload $ python setup.py mregister sdist mupload -r plone.org $ python setup.py mregister sdist mupload -r my-company ... 30 30

Developer side : multiple servers with .pypirc [distutils] index-servers = pypi plone.org my-company [pypi] username:tarek password:hahaha [plone.org] username:tarek password:hahaha... repository:http://plone.org/products [my-company] username:tarek password:hahaha..hahaha.haha...ha..pfiiiuuuuuu..and..hahahaahah.i..am..dying.. repository:http://my-company.com/products 31 31

Let’s try our new PyPI with collective.dist 32 32

#3 Make plone.org/products PyPI compatible plone.org was suppose to switch to that for months. Lazy guys 34 34

Let’s try new.plone.org 35 35

The big picture 37 37

Yes !!! 38 developer #1 38

Yepeee !!! Releasing everywhere let’s spam PyPI and plone.org tchak ! tchak ! tchak ! developer #2 39 39

Hahaha ! We have our package all over the place. developer #3 40 40

end of part 1 questions ? #1 Make mirror #2 Run your own PyPI #3 push to several servers #3 use “mregister” and “mupload” 41 41

Part 2 - working with zc.buildout (reminders) 42 42

5 hours in 2006 43 43

5 hours in 2006 install python extra packages get zope install zope create an instance get extra products read extra products doc install extra products dependency install extra products doesn’t work ahhh right, install python-ldap checkout products in development doesn’t work ahhh right, wrong python-ldap version start to work 44 44

Me in 2006 45 45

5 minutes in 2008 46 46

5 minutes in 2008 get the “buildout” $ python bootstrap.py $ bin/buildout start to work 47 47

But that was not the main reason 48 48

2005 Now zope.interface zope.schemas zope.stuff Zope one big-O zope.whatsthat package. “eggification” zope.makecoffee zope.dodishes ... over 100 packages 49 49

2005 Now plone.ista plone.portlets plone.stuff Plone products plone.whatsthat “eggification” plone.makecoffee plone.enolp ... 50 50

Your app.site Killer Plone app.skin App ... 51 51

zope.stuff ... Your plone.stuff Distribution ... app.site app.skin 52 52

Python site-packages +100 packages to run Plone something is Increase conflict going wrong here probabilities 53

zc.buildout = isolated environment 54 54

zc.buildout best practices #1 use the same layout for all your projects #2 make sure all developers have the same environment #3 use one cfg per target 55 55

#1 same layout for all projects project1 docs buildout packages releases project2 docs buildout packages releases .... 56 56

easy_install collective.releaser 57 57

#2 make sure all developers have the same environment Warning Plone buildouts are source based Windows developers Get my Windows installer : python2.4.4-win32.zip Google “An installer for a buildout-ready Windows” 59 59

#3 use one cfg per target Typical buildout layout uses the extends feature buildout.cfg dev.cfg (extends buildout.cfg) prod.cfg (extends buildout.cfg) + bootstrap.py 60 60

buildout.cfg : [buildout] parts = one two [buildout] parts = dev.cfg : one two [buildout] three extends = buildout.cfg develop = parts = ... three develop = ... 61 61

demo creating a fresh Plone 3 buildout (Paste) adding the dev.cfg hooking a new development package adding a prod.cfg 62 62

end of part 2 questions ? #1 use the same layout for all projects #2 make sure all developers have the same environment #3 use one cfg per target 64 64

Part 3 - application lifecycle 65

66

Releasing packages Releasing the buildout 67

Releasing packages for package in packages: raise the version edit CHANGES.txt create a branch (svn) go to that branch remove the dev tag (setup.cfg) release it with “mregister sdist mupload -r somehwere” release it with “mregister sdist mupload -r somehwereelse” 68 68

Releasing packages with collective.releaser: for package in packages: python setup.py release 69 69

Defining targets in .pypirc [distutils] index-servers = pypi plone.org private glob-style = true [pypi] ... release-command = mregister sdist mupload release-packages = iw.* [plone.org] ... release-command = mregister sdist mupload release-packages = plone.* collective.* [private] 70 ... 70

collective.releaser makes a lot of assumptions it’s svn-oriented (the first use case) 72 72

Release the buildout What packages should be frozen ? - recipes - your released packages - exceptions (security fixes, major bug fixes) 73 73

Release the buildout authentication: use lovely.buildouthttp [buildout] ... extensions=lovely.buildouthttp ... repository:http://my-company.com/products HOME/.buildout/.httpauth pypi,http://my-company.com/products,tarek,hahaha pypi,http://plone.org/products,tarek,hahaha 74 74

Release the buildout -> project layout project ... buildout packages release/0.1 <- tag for the buildout $ cd buildout $ svn cp . http://somewhere/releases/0.1 75 75

Release the buildout with collective.releaser : with project_release $ cd buildout $ project_release What version you are releasing ? 0.1 Added version file. 76 76

Build your release distribution - bin/buildout on target system - remove some stuff - offline mode to ‘true’ - tar -czvf release-0.1.tgz release/0.1 77 77

Build your distribution with collective.releaser with project_deploy $ svn co http://somesvn/my_projet/releases/0.1 project $ cd project $ project_deploy prod.cfg 78 78

Upgrading the application 79 79

Upgrading the application : 3 strategies #1 full release #2 diff release, using collective.releaser #3 egg pack release, using collective.releaser 80 80

collective.releaser : the releaser_project template project_release project_deploy project_eggs project_diff 81 81

That’s it ! Thanks ! questions ? 82 82

http://tarekziade.wordpress.com zc.buildout @ PyPI collective.dist @ PyPI collective.releaser @ PyPI http://alterway.fr 83 83

delivering applications with zc.buildout and a distributed model - Plone Conference 2008

0 comments

Notes on slide 1

3 Favorites