Deployment and Continous Integration of a Zope/Plone application

  • 1,246 views
Uploaded on

Conference given at PyconFR'13

Conference given at PyconFR'13

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
1,246
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
23
Comments
2
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Deployment & Continuous integration of a Plone/Zope application Julien Pivotto PyconFR’13 October 26, 2013 ;
  • 2. whoami sysadmin @ inuits • open-source defender for 7+ years • devops believer • @roidelapluie on twitter/github • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 3. • • • Plone is an Open-Source CMS Written in python Vibrant community Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 4. • • • Open-Source Web application server Written in Python Used by Plone Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 5. Plone/Zope application • • • Threads = instances Built using Buildout Modules (eggs) Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 6. The mission • • • • Several Zope/Plone applications Scalability Automation Reliability Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 7. The old days • • • • Manual work Failover HA Supervisord No overview Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 8. BUT Run buildout on each server/environment • Compilation error in the middle • Pypi is down • Which commit is faulty? • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 9. The goal • • • • • Less failure during builds More testing Puppetize all the things Reproduce all the builds More env: dev, test, demo, prod Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 10. Infrastructure changes • DRBD for PostgreSQL ⇒ PostgreSQL 9 native replication • DRBD for files ⇒ GlusterFS for data, RPM for code • Supervisord to launch instances ⇒ Puppet to start services • Pound as reverse proxy ⇒ Haproxy • Internal mirror of pypi Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 11. Puppetization • • • • From nothing to everything Plone, but also apache, haproxy . . . and anything else Mcollective to orchestrate Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 12. Development decisions • • • Move away from SVN Write tests Automate them in Jenkins Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 13. It takes time • • • • Cultural shift Legacy stuff A lot of surprises Compromises between Dev and Ops Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 14. Jenkins • • • • An open-source CI server More than 600 plugins Flexible Jobs & pipelines Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 15. Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 16. Jobs • • • • Reproducible builds Single place to find build logs History of builds Creating (RPM) artifacts Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 17. Pipeline • • • Build is only part of the process Need to package, deploy, . . . Running tests Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 18. Tests • • • • • • Running tests on each module Feedback: graphes, mails Code coverage Violations Unit tests zc.recipe.testrunner Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 19. Translation in Jobs virtualenv --no-site-packages . bin/python bootstrap.py bin/buildout -c jenkins.cfg buildout:eggs-directory= /var/lib/jenkins/buildout-shared-eggs bin/test-all Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 20. Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 21. Buildout • • • • Running buildout in a Spec file Creating a RPM Using virtualenv Starting from scratch each time Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 22. Ops like RPM • • • • • • Easy to deploy anywhere Consistency and dependencies Find where a file comes from Orchestrate with puppet/mcollective Version number Conf does not belong to package Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 23. Time Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 24. Deploying Zope • • • • • Building RPM Adding them to a yum repo (with pulp) Cleaning yum cache (with mcollective) Running puppet (with mcollective) dev -> staging -> demo and prod Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 25. Reports • • • • Jenkins build log Puppet logs Zope instances logs IRC, mails, . . . Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 26. Role of Puppet • • • • Puppet creates the configuration Update the package Restarts the instances Puppet (hiera) creates password files Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 27. Database upgrades • • • WIP, no solution chosen yet Postscript in RPM? Commands trigerred by puppet? Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 28. Logs Zope/Plone: logs files/instances • 10 instances * 2 servers ⇒ 20 log files • Plus apache logs • Plus system logs • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 29. The deprecated way • • • tail -f multitail clusterssh Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 30. Logstash • • • • • An open source tool Manage your logs Input from anywhere Filter Output to anywhere Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 31. Logstash Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 32. Plone and Logstash • • • • Plone output to syslog Mangle to add good fields Logstash output to ElasticSearch Kibana3 creates dashboards Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 33. zope.conf <syslog> address 172.16.28.55:5544 facility local3 format %(asctime)s ZopApp-Server zope[%(process)s]: instance1 [%(levelname)s] %(name)s | %(message)s dateformat %b %d %H:%M:%S level info </syslog> Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 34. filter { grok { type => "syslog" pattern => ["(?m)<%= "<%" %>{POSINT:syslog_pri}> %{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program} (?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}"] add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{@source_host}" ] add_tag => "syslog-%{syslog_program}" } } Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 35. frametitle{zope.conf} filter { grok { tags => ["syslog-zope"] pattern => ["(?m)(?<instance>[a-z0-9]+) [(?<loglevel>[A-Z]+)] (?<loggername>[A-Za-z0-9.]+) | %{GREEDYDATA:logmessage}"] add_tag => "grokked-zope" } mutate { tags => ["grokked-zope"] rename => [ "@source_host", "application" ] replace => [ "@message", "%{logmessage}" ] add_tag => "loggername-%{loggername}" } } Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 36. Kibana3 • • • • • Plays with ElasticSearch and Logstash Easily build dashboards Share the metric and logs with anyone Filter by time, instance, http code. . . View all the logs at the same place Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 37. Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 38. Conclusions This is a sysadmin point of view • Still a lot of work to be done • From a "black box" . . . To a fully automated environment • Not a unique solution • You need to get it working for you . . . in your way • Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 39. This work is being realized with Affinitic SPRL, Plone developers in Belgium http://affinitic.be Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 40. Thank you Any question? Julien Pivotto Deployment and CI of a Plone/Zope application ;
  • 41. Contact Julien Pivotto julien@inuits.eu @roidelapluie INUITS bvba INUITS bvba Belgium Belgium +32 473 441 636 +32 473 441 636 https://inuits.eu https://inuits.eu Julien Pivotto Deployment and CI of a Plone/Zope application ;