1. Practical project
automation
donderdag 19 mei 2011

2. Practical project
automation
• Reinout van Rees
• Nelen & Schuurmans
• Utrecht
• Water consultancy
• Lots of python and
django
donderdag 19 mei 2011

3. Project: more than code
• Projects are important
• Deployment
• Colleagues
• Rebuilding
• Solution: automation
donderdag 19 mei 2011

4. Generic idea: automation
• Cobbler’s children
• Automate your
projects
• Less manual work
• Less menial work
• Fewer errors
• More of the fun work
donderdag 19 mei 2011

5. Build automation
• ./configure;make
• Ant, maven
• Buildout (python)
• Single command to
get started
donderdag 19 mei 2011

6. Build tool: good spot for
more automation
• Most are extensible
• Examples:
• Apache config file
generation
• Setting up cronjobs
donderdag 19 mei 2011

7. Apache config file:
[apacheconf]
recipe = collective.recipe.template
input = ${buildout:directory}/etc/mysite.apache.conf.in
output = ${buildout:directory}/etc/mysite.apache.conf
<VirtualHost *:80>
ServerName mysite.example.org
CustomLog ${buildout:directory}/var/log/access.log combined
ErrorLog ${buildout:directory}/var/log/error.log
DocumentRoot ${buildout:directory}/var/www
...
RewriteRule ^(.*) http://localhost:${django:gunicorn_port}$1 [P]
<VirtualHost *:80>
ServerName mysite.example.org
CustomLog /srv/mysite/var/log/access.log combined
ErrorLog /srv/mysite/var/log/error.log
DocumentRoot /srv/mysite/var/www
...
RewriteRule ^(.*) http://localhost:10003$1 [P]
donderdag 19 mei 2011

8. Prevent missing apache
log dir (automatically!)
[mkdir]
recipe = z3c.recipe.mkdir
paths =
${buildout:directory}/generated
${buildout:directory}/var/geotiffs
${buildout:directory}/var/log
${buildout:directory}/var/media
${buildout:directory}/var/sqlite
${buildout:directory}/var/static
donderdag 19 mei 2011

9. Automatic crontab
[supervisor-cronjob]
recipe = z3c.recipe.usercrontab
times = @reboot
command = ${buildout:bin-directory}/supervisord
[whoosh-cronjob]
recipe = z3c.recipe.usercrontab
times = 13 */8 * * *
command = ${buildout:bin-directory}/django update_index
donderdag 19 mei 2011

10. Test automation
• Jenkins/buildbot
• Deployment ok...
• ... so tests possible
• Prevent mistakes
donderdag 19 mei 2011

11. donderdag 19 mei 2011

12. Extra jenkins automation:
quality checks
• Code coverage
(‘coverage’ on pypi)
• Pyflakes and pep8
(also on pypi)
• Jslint (or jshint)
donderdag 19 mei 2011

13. donderdag 19 mei 2011

14. Quality: make it easy to
do the right thing
• “I really should do x,
but it takes too much
time so I’ll skip it”
• Releasing packages
• Skeleton
donderdag 19 mei 2011

15. donderdag 19 mei 2011

16. Releasing python
packages
• zest.releaser (on pypi)
• prerelease
• release
• postrelease
donderdag 19 mei 2011

17. donderdag 19 mei 2011

18. Practical project
automation
• Make it easier to do
the right thing: let
laziness work for you
• Automate your builds
• Automate tests
• Extend where
practical
donderdag 19 mei 2011

19. Questions?
reinout.vanrees.org
@reinoutvanrees
reinout@vanrees.org
www.nelen-schuurmans.nl
photos.reinout.vanrees.org
donderdag 19 mei 2011