Plone Deployment:
Secrets & Tricks
Steve McMahon

Plone Symposium East, 2012
On  ly!
                        Tod  ay          ricks!
                                   ts orT
                          t Se cre
                  Wit hou

Plone Deployment:
Secrets & Tricks
Steve McMahon

Plone Symposium East, 2012
You: SysAdmin or Friend of SysAdmin
ix y
               SX , Un
           D, O
       , BS
Li nux

    You: SysAdmin or Friend of SysAdmin
ix y
               SX , Un
           D, O
       , BS
Li nux

    You: SysAdmin or Friend of SysAdmin


              Me: SysAdmin and person
               who talks to SysAdmins
Looking for something more
advanced?
Looking for something more
advanced?


      Clayton is next door!
Looking for something more
advanced?


      Clayton is next door!
       How I secretly rule the world with
          buildout and a pen knife.
How we look to PHP folks
How they look to us.
Real life, for both.
What we want.
Stand-Alone
Zope / Plone                         Internet




✤   This works, but …               Zope / Plone
                                   Port 80 or 8080

✤   Lacks:

    ✤   Versatility

    ✤   SSL
                                   ZODB / Blobs

    ✤   Plays poorly with others
+ Web Server                         Internet




✤   Modern Web Server as Reverse
                                    Web Server
    Proxy                            80 / 443


✤   Handles SSL

✤   Efficiently queues requests
                                   Zope / Plone
                                    Port 8080
✤   Handles URL rewrites

✤   Battle-hardened logging and
    request sanitizing
                                   ZODB / Blobs

✤   Plays well with others
+ Web Apps                                            Internet




                                                     Web Server
                                                      80 / 443



✤   URL Rewriting makes it easy to
    map in other best-of-breed
                                                                    Other
    apps.                             Zope / Plone
                                       Port 8080
                                                                  Web Apps
                                                                   Ports: ?


✤   Can share special authorization
    requirements.
                                      ZODB / Blobs
Optimizations
“Premature optimization is
 the root of all evil.”


                        Donald Knuth
“Measure, measure measure.
 Then act.”

                 E. Leddy
                 Budapest: Keynote Lecture in
                 Appreciation of Donald Knuth
Load Balancing
Load Balancing means always having a canon ready to fire.

L. DaVinci, Authenticity Disputed
+ZEO
                                      ZEO Client 1   ZEO Client 2   ZEO Client …
                                       Port 8081      Port 8082       Ports …



✤   Efficient with modern multi-
    core processors even on single
    machines.
                                                      ZEO Server
                                                       Port 8000
✤   Can scale to multiple machines.

✤   A good option even without
    load balancing: debugging and
    run scripts.                                     ZODB / Blobs



✤   Pack database by command
    line utility.
+Load                              Internet
                                                           Web Server
                                                            80 / 443


 Balancing
                                                             Load
                                                           Balancer
                                                           Port 9000

✤   Distributes work among ZEO
    clients.
                                              ZEO Client   ZEO Client   ZEO Client
                                                  1            2           …
✤   Can help avoid service                    Port 8081    Port 8082     Ports …
    interruptions.

✤   Balancing scheme can matter.                             ZEO
                                                                           ZODB /
                                                            Server
                                                           Port 8000        Blobs
Proxy Caching
“Good caching makes light work.”

John Heywood, 1546, Often Misquoted.
+Server Cache
                                               Web Server
                                    Internet    80 / 443




                                                Proxy
                                                Cache
                                               Port 6000
✤   Reverse-Proxy Cache

✤   Typically between web server
    and load balancer.                           Load
                                               Balancer
                                               Port 9000
✤   Can be on a separate machine.

✤   Cache in memory or on disk.
                                                ZEO
                                               Cluster
“Caching is hard.”


                     D. Glick,
                     Cioppino 2012
plone.app.caching
“plone.app.caching is your frenemy.”

Winston Churchill, 1944, Authenticity Disputed.
Choosing Tools
Take care of your tools and they will take care of you.

Traditional
Web Server
            Internet    80 / 443




                        Proxy
                        Cache

Best of Breed          Port 6000




Approach
                         Load
                       Balancer
                       Port 9000




                        ZEO
                       Cluster
Internet
                       Web Server
                        80 / 443
                                    Nginx




                        Proxy
                        Cache

Best of Breed          Port 6000




Approach
                         Load
                       Balancer
                       Port 9000




                        ZEO
                       Cluster
Internet
                       Web Server
                        80 / 443
                                    Nginx




                        Proxy
                        Cache       Varnish
Best of Breed          Port 6000




Approach
                         Load
                       Balancer
                       Port 9000




                        ZEO
                       Cluster
Internet
                       Web Server
                        80 / 443
                                    Nginx




                        Proxy
                        Cache       Varnish
Best of Breed          Port 6000




Approach
                         Load
                       Balancer
                       Port 9000
                                    HAProxy




                        ZEO
                       Cluster
Web Server                                Internet
                                                     Web Server


Does it All
                                                      80 / 443




Entia non sunt multiplicanda praeter                  Proxy
                                                      Cache
necessitatem.                                        Port 6000

(Entities must not be multiplied beyond
necessity.)
        — William of Occam
            (14th Century SysAdmin)                    Load
                                                     Balancer
                                                     Port 9000
Keep is Simple, Stupid.
      — Occam’s Razor, translated to
         modern sysadmin speak
                                                      ZEO
                                                     Cluster
How To?
http://collective-docs.readthedocs.org/en/latest/hosting/

Is This The Right Place For This Resource? Good Question!
“Backup is serious shit.”


                            E. Leddy,
                            Cioppino 2012
backup, rsync, restore


                         Is there another way?
Logs
“Rotate your logs or your server will die.”

E. Steele, The Missing 18 Minutes
“sudo   apt-get install logrotate”



                        Traditional Proverb,
                        England, 12th Century
logrotate rules

# rotate logs for client #2
/var/db/plone4.1/zeocluster/var/client2/Z2.log
/var/db/plone4.1/zeocluster/var/client2/event.log {
    rotate 5
    weekly
    sharedscripts
    postrotate
       kill -USR2 `cat /var/db/…/client2.pid`
    endscript
}
Database Packing
Early ZODB Packing Tool

Leonardo Da Vinci, ~1500
Better ZODB Packing



 bin/zeopack -h localhost -p 8000 -d 0
Platform Setup & Plone Install
“These days, I use system packages
 as much as possible.
 You should, too.”


                             E. Leddy,
                             Cioppino 2012
Unified Installer:
Least-Used Feature


 --with-python=/usr/bin/python2.7
Unified Installer:
Least-Known Feature


     --clients=client-count
“Never run buildout as root.”


                        S. McMahon,
                        PSE, Today
Questions?
Image Credits
✤   cobol: CC by: http://www.flickr.com/photos/philmanker/3654636770/

✤   complex: CC by: http://www.flickr.com/photos/medea_material/1692344189/

✤   plant crusher: CC by NC ND: http://www.flickr.com/photos/duanemoore/4908036590/

✤   diagram: CC by NC SA: http://www.flickr.com/photos/nataliedowne/6446884983/

✤   blue: CC BY NC ND: http://www.flickr.com/photos/barthanlon/3670183050/

✤   light timer: CC by NC SA: http://www.flickr.com/photos/dorkstyle/3783421698/

✤   chain reaction: CC by NC ND: http://www.flickr.com/photos/acplinfo/2315988501/

✤   Giant Crossbow courtesy of http://www.leonardoda-vinci.org

✤   Centrifuge: CC by SA: http://www.flickr.com/photos/lrosa/515028911/

✤   Mystery: CC by NC ND: http://www.flickr.com/photos/zeptonn/2631113046/

✤   Decoder: CC by NC SA: http://www.flickr.com/photos/curiousexpeditions/962394310/

✤   Screw Gear: CC by NC: http://www.flickr.com/photos/travelinlibrarian/4308521250/

✤   Worm Gear: CC by NC: http://www.flickr.com/photos/travelinlibrarian/4307997887/

✤   Pulleys: CC by NC: http://www.flickr.com/photos/travelinlibrarian/4305293959/

✤   Ball Bearings: CC by: http://www.flickr.com/photos/cristic/312874252/

✤   Multi-Barrel: CC by NC SA: http://www.flickr.com/photos/malcubed/2930190065/

✤   Cannons: CC by: http://www.flickr.com/photos/jeanlouis_zimmermann/5554481148/

✤   Fan: CC by: http://www.flickr.com/photos/jeanlouis_zimmermann/5554419770

✤   Perpetual Motion: CC by NC SA: http://www.flickr.com/photos/grocel/274933518/

Plone Deployment Secrets & Tricks

  • 1.
    Plone Deployment: Secrets &Tricks Steve McMahon Plone Symposium East, 2012
  • 2.
    On ly! Tod ay ricks! ts orT t Se cre Wit hou Plone Deployment: Secrets & Tricks Steve McMahon Plone Symposium East, 2012
  • 3.
    You: SysAdmin orFriend of SysAdmin
  • 4.
    ix y SX , Un D, O , BS Li nux You: SysAdmin or Friend of SysAdmin
  • 5.
    ix y SX , Un D, O , BS Li nux You: SysAdmin or Friend of SysAdmin Me: SysAdmin and person who talks to SysAdmins
  • 6.
    Looking for somethingmore advanced?
  • 7.
    Looking for somethingmore advanced? Clayton is next door!
  • 8.
    Looking for somethingmore advanced? Clayton is next door! How I secretly rule the world with buildout and a pen knife.
  • 9.
    How we lookto PHP folks
  • 10.
  • 11.
  • 12.
  • 13.
    Stand-Alone Zope / Plone Internet ✤ This works, but … Zope / Plone Port 80 or 8080 ✤ Lacks: ✤ Versatility ✤ SSL ZODB / Blobs ✤ Plays poorly with others
  • 14.
    + Web Server Internet ✤ Modern Web Server as Reverse Web Server Proxy 80 / 443 ✤ Handles SSL ✤ Efficiently queues requests Zope / Plone Port 8080 ✤ Handles URL rewrites ✤ Battle-hardened logging and request sanitizing ZODB / Blobs ✤ Plays well with others
  • 15.
    + Web Apps Internet Web Server 80 / 443 ✤ URL Rewriting makes it easy to map in other best-of-breed Other apps. Zope / Plone Port 8080 Web Apps Ports: ? ✤ Can share special authorization requirements. ZODB / Blobs
  • 16.
  • 17.
    “Premature optimization is the root of all evil.” Donald Knuth
  • 18.
    “Measure, measure measure. Then act.” E. Leddy Budapest: Keynote Lecture in Appreciation of Donald Knuth
  • 19.
    Load Balancing Load Balancingmeans always having a canon ready to fire. L. DaVinci, Authenticity Disputed
  • 20.
    +ZEO ZEO Client 1 ZEO Client 2 ZEO Client … Port 8081 Port 8082 Ports … ✤ Efficient with modern multi- core processors even on single machines. ZEO Server Port 8000 ✤ Can scale to multiple machines. ✤ A good option even without load balancing: debugging and run scripts. ZODB / Blobs ✤ Pack database by command line utility.
  • 21.
    +Load Internet Web Server 80 / 443 Balancing Load Balancer Port 9000 ✤ Distributes work among ZEO clients. ZEO Client ZEO Client ZEO Client 1 2 … ✤ Can help avoid service Port 8081 Port 8082 Ports … interruptions. ✤ Balancing scheme can matter. ZEO ZODB / Server Port 8000 Blobs
  • 22.
    Proxy Caching “Good cachingmakes light work.” John Heywood, 1546, Often Misquoted.
  • 23.
    +Server Cache Web Server Internet 80 / 443 Proxy Cache Port 6000 ✤ Reverse-Proxy Cache ✤ Typically between web server and load balancer. Load Balancer Port 9000 ✤ Can be on a separate machine. ✤ Cache in memory or on disk. ZEO Cluster
  • 24.
    “Caching is hard.” D. Glick, Cioppino 2012
  • 25.
    plone.app.caching “plone.app.caching is yourfrenemy.” Winston Churchill, 1944, Authenticity Disputed.
  • 26.
    Choosing Tools Take careof your tools and they will take care of you. Traditional
  • 27.
    Web Server Internet 80 / 443 Proxy Cache Best of Breed Port 6000 Approach Load Balancer Port 9000 ZEO Cluster
  • 28.
    Internet Web Server 80 / 443 Nginx Proxy Cache Best of Breed Port 6000 Approach Load Balancer Port 9000 ZEO Cluster
  • 29.
    Internet Web Server 80 / 443 Nginx Proxy Cache Varnish Best of Breed Port 6000 Approach Load Balancer Port 9000 ZEO Cluster
  • 30.
    Internet Web Server 80 / 443 Nginx Proxy Cache Varnish Best of Breed Port 6000 Approach Load Balancer Port 9000 HAProxy ZEO Cluster
  • 31.
    Web Server Internet Web Server Does it All 80 / 443 Entia non sunt multiplicanda praeter Proxy Cache necessitatem. Port 6000 (Entities must not be multiplied beyond necessity.) — William of Occam (14th Century SysAdmin) Load Balancer Port 9000 Keep is Simple, Stupid. — Occam’s Razor, translated to modern sysadmin speak ZEO Cluster
  • 32.
    How To? http://collective-docs.readthedocs.org/en/latest/hosting/ Is ThisThe Right Place For This Resource? Good Question!
  • 33.
    “Backup is seriousshit.” E. Leddy, Cioppino 2012
  • 34.
    backup, rsync, restore Is there another way?
  • 35.
    Logs “Rotate your logsor your server will die.” E. Steele, The Missing 18 Minutes
  • 36.
    “sudo apt-get install logrotate” Traditional Proverb, England, 12th Century
  • 37.
    logrotate rules # rotatelogs for client #2 /var/db/plone4.1/zeocluster/var/client2/Z2.log /var/db/plone4.1/zeocluster/var/client2/event.log { rotate 5 weekly sharedscripts postrotate kill -USR2 `cat /var/db/…/client2.pid` endscript }
  • 38.
    Database Packing Early ZODBPacking Tool Leonardo Da Vinci, ~1500
  • 39.
    Better ZODB Packing bin/zeopack -h localhost -p 8000 -d 0
  • 40.
    Platform Setup &Plone Install
  • 41.
    “These days, Iuse system packages as much as possible. You should, too.” E. Leddy, Cioppino 2012
  • 42.
    Unified Installer: Least-Used Feature --with-python=/usr/bin/python2.7
  • 43.
  • 44.
    “Never run buildoutas root.” S. McMahon, PSE, Today
  • 45.
  • 46.
    Image Credits ✤ cobol: CC by: http://www.flickr.com/photos/philmanker/3654636770/ ✤ complex: CC by: http://www.flickr.com/photos/medea_material/1692344189/ ✤ plant crusher: CC by NC ND: http://www.flickr.com/photos/duanemoore/4908036590/ ✤ diagram: CC by NC SA: http://www.flickr.com/photos/nataliedowne/6446884983/ ✤ blue: CC BY NC ND: http://www.flickr.com/photos/barthanlon/3670183050/ ✤ light timer: CC by NC SA: http://www.flickr.com/photos/dorkstyle/3783421698/ ✤ chain reaction: CC by NC ND: http://www.flickr.com/photos/acplinfo/2315988501/ ✤ Giant Crossbow courtesy of http://www.leonardoda-vinci.org ✤ Centrifuge: CC by SA: http://www.flickr.com/photos/lrosa/515028911/ ✤ Mystery: CC by NC ND: http://www.flickr.com/photos/zeptonn/2631113046/ ✤ Decoder: CC by NC SA: http://www.flickr.com/photos/curiousexpeditions/962394310/ ✤ Screw Gear: CC by NC: http://www.flickr.com/photos/travelinlibrarian/4308521250/ ✤ Worm Gear: CC by NC: http://www.flickr.com/photos/travelinlibrarian/4307997887/ ✤ Pulleys: CC by NC: http://www.flickr.com/photos/travelinlibrarian/4305293959/ ✤ Ball Bearings: CC by: http://www.flickr.com/photos/cristic/312874252/ ✤ Multi-Barrel: CC by NC SA: http://www.flickr.com/photos/malcubed/2930190065/ ✤ Cannons: CC by: http://www.flickr.com/photos/jeanlouis_zimmermann/5554481148/ ✤ Fan: CC by: http://www.flickr.com/photos/jeanlouis_zimmermann/5554419770 ✤ Perpetual Motion: CC by NC SA: http://www.flickr.com/photos/grocel/274933518/