• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Modern, scalable deployment for plone
 

Modern, scalable deployment for plone

on

  • 1,073 views

Deploying Plone applications is about more than getting a Zope instance running and Apache installed. In this talk I show what we do at current big projects and how we think small projects can ...

Deploying Plone applications is about more than getting a Zope instance running and Apache installed. In this talk I show what we do at current big projects and how we think small projects can benefit.

Statistics

Views

Total Views
1,073
Views on SlideShare
1,058
Embed Views
15

Actions

Likes
0
Downloads
10
Comments
0

1 Embed 15

https://twitter.com 15

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Modern, scalable deployment for plone Modern, scalable deployment for plone Presentation Transcript

    • Scalable, modern deployment for Plone Christian Theune ct@gocept.comTuesday, October 16, 12
    • Tuesday, October 16, 12
    • Tuesday, October 16, 12
    • Tuesday, October 16, 12
    • Tuesday, October 16, 12
    • InfrastructureTuesday, October 16, 12
    • EntropyTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Data centerTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • HardwareTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Virtualization OS Components • Virtualization is very affordable • Platform creates interface between app and infrastructureTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Automation • convergence • idempotency • versioning • everything as code • continuous repairsTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Management • Take care of updates and “production- ready” configuration • Backup and other day-to-day operations • OS and regularly used components • Gradual process moving components from user-space installations to platform-wide featuresTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Monitoring • Nagios :( • (Un-)fortunately: #monitoringsucksTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Support • Request tracker • custom glue code • Daily review • Interrupt shieldTuesday, October 16, 12
    • Platform Application Components Management Automation Monitoring Support OS Virtualization Hardware Data centerTuesday, October 16, 12
    • Application deployment and operationsTuesday, October 16, 12
    • Deployment • Quick • Repeatable • Platform-independent • single-clickTuesday, October 16, 12
    • Batou • Model-based • Use everywhere • No effect on runtime • Orchestration • Convergency • Idempotency • ReusabilityTuesday, October 16, 12
    • class Solr(Component): def configure(self): self.address = Address(self.host.fqdn, 9000) self.provide(solr:server, self.address) self += Buildout(solr, python=2.7) self += Program(solr, command=java, command_absolute=False, args=-jar start.jar, directory=self.expand( {{component.workdir}}/parts/instance), restart=True)Tuesday, October 16, 12
    • [hosts] oshaweb00 = nginx, www, varnish, supervisor, haproxy, docs oshaweb01 = zope, hwthemes, supervisor ... oshaweb20 = zope, hwthemes, supervisor oshaweb21 = zeo, openldap, supervisor oshaweb22 = nginx, media2 oshaweb23 = nginx, birt, lms, ugm, supervisor oshaweb24 = solr, memcached, supervisor oshaweb26 = mailout [component:haproxy] strategy = production_strategy bots = 3 whitelist = 8 default = 3 ...Tuesday, October 16, 12
    • Components good bad nginx apache varnish mysql, BDB haproxy OpenLDAP PostgreSQL memcached supervisor postfixTuesday, October 16, 12
    • Minimal "production- ready" checklist • Sane front-end configuration • System startup/shutdown • Log rotation • Database maintenance • Monitoring: processes and ports • Service userTuesday, October 16, 12
    • Secrets • One AES-encrypted config file per environment • Transparent decryption • Safety-belt to avoid accidental checking in unencrypted files • Future: PGP-encryptionTuesday, October 16, 12
    • Environments • Production vs. Staging • Structurally identical but smaller • Only one of many (dev, integration, ...) • "Fly what you test, test what you fly."Tuesday, October 16, 12
    • Web applications • Shared nothing in your app! • Many small processes • Do not listen early • Never store runtime config in the databaseTuesday, October 16, 12
    • Web applications II • Log long running requests • Avoid synchronous external requests • Avoid accidental write requests. • Use feature switches.Tuesday, October 16, 12
    • Process control • only one component to integrate • turn anything into a daemon • use superlance (httpok, memmon) for active restarts • use 3.0 release candidateTuesday, October 16, 12
    • Python: virtualenv and buildout • use both, combined • buildout might be a hammer, but not everything is a nailTuesday, October 16, 12
    • more buildout • use extends to keep your configs clean and structured • allow-picked-versions=False • -t 3 • buildout >= 1.6 • mr.developer • allowed-hosts = ...Tuesday, October 16, 12
    • and even more buildout • multiple small better than one big • different Python versions • different subsets of egg versions • use mirrors - see pypi-mirrors.org • latency to the PyPI server is the biggest factor in buildout performanceTuesday, October 16, 12
    • Databases • As much RAM as you can afford - seriously • Low latency network (<0.1ms) • Multi-core if necessary • Disk usage < 40%Tuesday, October 16, 12
    • story: more hardware = slower • Application is slow. • Customer buys more hardware. Specifically: RAM. • System is slower now. • System runs many components. Hard to see through.Tuesday, October 16, 12
    • story: more hardware = slower • One number stands out: process exit rate is 20/sec. PosgreSQL. What? • Customer creates new connection for each query. • PostgreSQL forks master for new connection. • Fork went from few 10ms to few 100msTuesday, October 16, 12
    • story: more hardware = slower • Diagnose. Find root cause.Tuesday, October 16, 12
    • HTTP/S termination • HTTP and SSL • Virtual hosting • Redirects • host normalization • Proxy to load balancer • Static filesTuesday, October 16, 12
    • Load balancing • LB + HA • Classify requests • redispatch + retries • balance: leastconn • minconn 1, maxconn 2, threads +2 • Live statistic UITuesday, October 16, 12
    • Monitoring • again :( • processes, ports, logs • model dependencies • check_webpage.rb • Business processes (BPI)Tuesday, October 16, 12
    • Reporting • report + phone call • incidents • changes • availability • performance • trendsTuesday, October 16, 12
    • SLAs • Availability • more keeps your business safe • less reduces costs • Friends dont let friends think in "nines". • Alignment of technical and organisational measuresTuesday, October 16, 12
    • References • "The practice of system and network administration" (Limoncelli, et. al.) • Mark Burgess on config management • "The twelve-factor app" (Heroku) • gocept.net/docTuesday, October 16, 12
    • Conclusion I • Use infrastructure to build platforms • Platforms should be transparent. • Deploy applications on platforms.Tuesday, October 16, 12
    • Conclusion II • Do not cobble together random stuff from the internet. • Careful work pays. Buzzwords dont. • If you hear "enterprise". Run. • Sapere aude.Tuesday, October 16, 12
    • Conclusion III • I dont think IaaS and PaaS in the current extremes are a good answer to many projects. • Many people experiment with ideas. Everything is becoming more flexible. • Lets play!Tuesday, October 16, 12
    • Questions?Tuesday, October 16, 12