Deployment and Continous Integration of a Zope/Plone application

2,397 views
2,197 views

Published on

Conference given at PyconFR'13

Published in: Technology
2 Comments
4 Likes
Statistics
Notes
No Downloads
Views
Total views
2,397
On SlideShare
0
From Embeds
0
Number of Embeds
92
Actions
Shares
0
Downloads
29
Comments
2
Likes
4
Embeds 0
No embeds

No notes for slide

Deployment and Continous Integration of a Zope/Plone application

  1. 1. Deployment & Continuous integration of a Plone/Zope application Julien Pivotto PyconFR’13 October 26, 2013 ;
  2. 2. whoami sysadmin @ inuits • open-source defender for 7+ years • devops believer • @roidelapluie on twitter/github • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  3. 3. • • • Plone is an Open-Source CMS Written in python Vibrant community Julien Pivotto Deployment and CI of a Plone/Zope application ;
  4. 4. • • • Open-Source Web application server Written in Python Used by Plone Julien Pivotto Deployment and CI of a Plone/Zope application ;
  5. 5. Plone/Zope application • • • Threads = instances Built using Buildout Modules (eggs) Julien Pivotto Deployment and CI of a Plone/Zope application ;
  6. 6. The mission • • • • Several Zope/Plone applications Scalability Automation Reliability Julien Pivotto Deployment and CI of a Plone/Zope application ;
  7. 7. The old days • • • • Manual work Failover HA Supervisord No overview Julien Pivotto Deployment and CI of a Plone/Zope application ;
  8. 8. BUT Run buildout on each server/environment • Compilation error in the middle • Pypi is down • Which commit is faulty? • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  9. 9. The goal • • • • • Less failure during builds More testing Puppetize all the things Reproduce all the builds More env: dev, test, demo, prod Julien Pivotto Deployment and CI of a Plone/Zope application ;
  10. 10. Infrastructure changes • DRBD for PostgreSQL ⇒ PostgreSQL 9 native replication • DRBD for files ⇒ GlusterFS for data, RPM for code • Supervisord to launch instances ⇒ Puppet to start services • Pound as reverse proxy ⇒ Haproxy • Internal mirror of pypi Julien Pivotto Deployment and CI of a Plone/Zope application ;
  11. 11. Puppetization • • • • From nothing to everything Plone, but also apache, haproxy . . . and anything else Mcollective to orchestrate Julien Pivotto Deployment and CI of a Plone/Zope application ;
  12. 12. Development decisions • • • Move away from SVN Write tests Automate them in Jenkins Julien Pivotto Deployment and CI of a Plone/Zope application ;
  13. 13. It takes time • • • • Cultural shift Legacy stuff A lot of surprises Compromises between Dev and Ops Julien Pivotto Deployment and CI of a Plone/Zope application ;
  14. 14. Jenkins • • • • An open-source CI server More than 600 plugins Flexible Jobs & pipelines Julien Pivotto Deployment and CI of a Plone/Zope application ;
  15. 15. Julien Pivotto Deployment and CI of a Plone/Zope application ;
  16. 16. Jobs • • • • Reproducible builds Single place to find build logs History of builds Creating (RPM) artifacts Julien Pivotto Deployment and CI of a Plone/Zope application ;
  17. 17. Pipeline • • • Build is only part of the process Need to package, deploy, . . . Running tests Julien Pivotto Deployment and CI of a Plone/Zope application ;
  18. 18. Tests • • • • • • Running tests on each module Feedback: graphes, mails Code coverage Violations Unit tests zc.recipe.testrunner Julien Pivotto Deployment and CI of a Plone/Zope application ;
  19. 19. Translation in Jobs virtualenv --no-site-packages . bin/python bootstrap.py bin/buildout -c jenkins.cfg buildout:eggs-directory= /var/lib/jenkins/buildout-shared-eggs bin/test-all Julien Pivotto Deployment and CI of a Plone/Zope application ;
  20. 20. Julien Pivotto Deployment and CI of a Plone/Zope application ;
  21. 21. Buildout • • • • Running buildout in a Spec file Creating a RPM Using virtualenv Starting from scratch each time Julien Pivotto Deployment and CI of a Plone/Zope application ;
  22. 22. Ops like RPM • • • • • • Easy to deploy anywhere Consistency and dependencies Find where a file comes from Orchestrate with puppet/mcollective Version number Conf does not belong to package Julien Pivotto Deployment and CI of a Plone/Zope application ;
  23. 23. Time Julien Pivotto Deployment and CI of a Plone/Zope application ;
  24. 24. Deploying Zope • • • • • Building RPM Adding them to a yum repo (with pulp) Cleaning yum cache (with mcollective) Running puppet (with mcollective) dev -> staging -> demo and prod Julien Pivotto Deployment and CI of a Plone/Zope application ;
  25. 25. Reports • • • • Jenkins build log Puppet logs Zope instances logs IRC, mails, . . . Julien Pivotto Deployment and CI of a Plone/Zope application ;
  26. 26. Role of Puppet • • • • Puppet creates the configuration Update the package Restarts the instances Puppet (hiera) creates password files Julien Pivotto Deployment and CI of a Plone/Zope application ;
  27. 27. Database upgrades • • • WIP, no solution chosen yet Postscript in RPM? Commands trigerred by puppet? Julien Pivotto Deployment and CI of a Plone/Zope application ;
  28. 28. Logs Zope/Plone: logs files/instances • 10 instances * 2 servers ⇒ 20 log files • Plus apache logs • Plus system logs • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  29. 29. The deprecated way • • • tail -f multitail clusterssh Julien Pivotto Deployment and CI of a Plone/Zope application ;
  30. 30. Logstash • • • • • An open source tool Manage your logs Input from anywhere Filter Output to anywhere Julien Pivotto Deployment and CI of a Plone/Zope application ;
  31. 31. Logstash Julien Pivotto Deployment and CI of a Plone/Zope application ;
  32. 32. Plone and Logstash • • • • Plone output to syslog Mangle to add good fields Logstash output to ElasticSearch Kibana3 creates dashboards Julien Pivotto Deployment and CI of a Plone/Zope application ;
  33. 33. zope.conf <syslog> address 172.16.28.55:5544 facility local3 format %(asctime)s ZopApp-Server zope[%(process)s]: instance1 [%(levelname)s] %(name)s | %(message)s dateformat %b %d %H:%M:%S level info </syslog> Julien Pivotto Deployment and CI of a Plone/Zope application ;
  34. 34. filter { grok { type => "syslog" pattern => ["(?m)<%= "<%" %>{POSINT:syslog_pri}> %{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program} (?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}"] add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{@source_host}" ] add_tag => "syslog-%{syslog_program}" } } Julien Pivotto Deployment and CI of a Plone/Zope application ;
  35. 35. frametitle{zope.conf} filter { grok { tags => ["syslog-zope"] pattern => ["(?m)(?<instance>[a-z0-9]+) [(?<loglevel>[A-Z]+)] (?<loggername>[A-Za-z0-9.]+) | %{GREEDYDATA:logmessage}"] add_tag => "grokked-zope" } mutate { tags => ["grokked-zope"] rename => [ "@source_host", "application" ] replace => [ "@message", "%{logmessage}" ] add_tag => "loggername-%{loggername}" } } Julien Pivotto Deployment and CI of a Plone/Zope application ;
  36. 36. Kibana3 • • • • • Plays with ElasticSearch and Logstash Easily build dashboards Share the metric and logs with anyone Filter by time, instance, http code. . . View all the logs at the same place Julien Pivotto Deployment and CI of a Plone/Zope application ;
  37. 37. Julien Pivotto Deployment and CI of a Plone/Zope application ;
  38. 38. Conclusions This is a sysadmin point of view • Still a lot of work to be done • From a "black box" . . . To a fully automated environment • Not a unique solution • You need to get it working for you . . . in your way • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  39. 39. This work is being realized with Affinitic SPRL, Plone developers in Belgium http://affinitic.be Julien Pivotto Deployment and CI of a Plone/Zope application ;
  40. 40. Thank you Any question? Julien Pivotto Deployment and CI of a Plone/Zope application ;
  41. 41. Contact Julien Pivotto julien@inuits.eu @roidelapluie INUITS bvba INUITS bvba Belgium Belgium +32 473 441 636 +32 473 441 636 https://inuits.eu https://inuits.eu Julien Pivotto Deployment and CI of a Plone/Zope application ;

×