• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Plone on Amazon EC2
 

Plone on Amazon EC2

on

  • 4,576 views

A talk from the Plone European Symposium in Sorrento and the Plone Conference in Budapest that explains the advantages of running Plone on Amazon EC2, and some of the things to look out for.

A talk from the Plone European Symposium in Sorrento and the Plone Conference in Budapest that explains the advantages of running Plone on Amazon EC2, and some of the things to look out for.

Statistics

Views

Total Views
4,576
Views on SlideShare
3,541
Embed Views
1,035

Actions

Likes
4
Downloads
0
Comments
0

9 Embeds 1,035

http://blog.jazkarta.com 1004
http://www.slideshare.net 14
http://feeds.jazkarta.com 8
http://translate.googleusercontent.com 3
http://webcache.googleusercontent.com 2
http://www.lmodules.com 1
url_unknown 1
http://ec2.resourcezen.com 1
http://www.cloudcomputingresourcecenter.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Plone on Amazon EC2 Plone on Amazon EC2 Presentation Transcript

    • Plone in the Cloud A CMS hosted on Amazon EC2 Nate Aune (Jazkarta.com) European Plone Symposium 2009 Sorrento, Italy May 13, 2009 Thursday, November 5, 2009 1
    • In the next 45 min... • Amazon Web Services (AWS) • AWS with Python • Plone with AWS • Tools for launching & remote deployment • Challenges of using AWS • Monitoring and maintaining EC2 instances • Admin dashboard for creating new Plone sites Thursday, November 5, 2009 2
    • What is Amazon Web Services? Thursday, November 5, 2009 3
    • What is Amazon Web Services? • Suite of elastic IT infrastructure services • Requisition computing power and storage • Run any application you want on any OS Thursday, November 5, 2009 4
    • Benefits of AWS • Metered pricing (per hour / per GB) • no large up-front capital investments • only pay for what you use • Scaleable hosting architecture • Redundancy for high availability Thursday, November 5, 2009 5
    • AWS Architecture • EC2 = Elastic Computing Cloud • S3 = Simple Storage Solution • EBS = Elastic Block Storage • Elastic IP = Static IP addresses Thursday, November 5, 2009 6
    • Elastic Computing Cloud (EC2) amazon.com/ec2 • Use an existing AMI (Amazon Machine Image) to launch a new instance • Or create your own AMIs (even paid ones) • Attach EBS volumes to your EC2 instance Thursday, November 5, 2009 7
    • Elastic Block Storage (EBS) Thursday, November 5, 2009 8
    • So how does Python play into all this? Thursday, November 5, 2009 9
    • Enter Boto Thursday, November 5, 2009 10
    • What is Boto? • Python package that provides interfaces to Amazon Web Services • Thanks to author Mitch Garnaat! • Boto supports: • S3 via the REST API • EC2 via the Query API • and more... Thursday, November 5, 2009 11
    • Install Boto & Bototools $ sudo easy_install boto $ svn co http://bototools.googlecode.com/svn/trunk/ bototools $ cd bototools $ sudo python setup.py install $ sudo python ./src/bototools/postinstall.py -p /usr/local/bin • This will put these commands in /usr/local/bin: • ec2start, ec2stop, ec2ip, ec2ls, ec2img • s3ls, s3rm, s3cp Thursday, November 5, 2009 12
    • Using Bototools • Set these environment variables: • AWS_ACCESS_KEY • AWS_SECRET_ACCESS_KEY • EC2_SSH_KEY • Can also put in ~/.boto or /etc/boto.cfg Thursday, November 5, 2009 13
    • Find an AMI • We will use the Ubuntu 8.10 Intrepid (32-bit) from Alestic.com - ami-7cfd1a15 • More can be found at: http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=171 Thursday, November 5, 2009 14
    • Launch an instance $ ec2start -a ami-7cfd1a15 -k gsg-keypair Reservation:r-0f26a666 [Instance:i-8e3154e7] $ ec2ls Reservation r-0f26a666 i-8e3154e7 pending None 2009-03-25T01:49:24.000Z ...wait... $ ec2ls Reservation r-0f26a666 i-8e3154e7 running None 2009-03-25T01:49:24.000Z ec2-174-129-238-8.compute-1.amazonaws.com Thursday, November 5, 2009 15
    • Associate a static IP $ ec2ip -g Address:174.129.238.8 $ ec2ip -l 174.129.238.8 -> $ ec2ip -a 174.129.238.8 i-8e3154e7 True $ ec2ip -l 174.129.238.8 -> i-8e3154e7 • Note: once you assign a static IP, you won't be able to access the machine using the original IP. Thursday, November 5, 2009 16
    • Log in $ ec2ls Reservation r-0f26a666 i-8e3154e7 running None 2009-03-25T01:49:24.000Z ec2-174-129-238-8.compute-1.amazonaws.com $ ssh -i ~/id_rsa-gsg-keypair root@ec2-174-129-238-8.compute-1.amazonaws.com Linux domU-12-31-39-00-E5-14 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 root@domU-12-31-39-00-E5-14:~# • Time to play! Thursday, November 5, 2009 17
    • Stop the instance $ ec2stop i-8e3154e7 Stopping ['i-8e3154e7'] $ ec2ls Reservation r-0f26a666 i-8e3154e7 terminated None 2009-03-25T01:49:24.000Z • Be sure to stop the instance or you will be charged! Thursday, November 5, 2009 18
    • Use the AWS Console Thursday, November 5, 2009 19
    • Or use ElasticFox • Firefox extension to start/stop EC2 instances, assign IPs, associate EBS volumes and create keypairs. http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609&categoryID=88 Thursday, November 5, 2009 20
    • Case study Plone on EC2 Thursday, November 5, 2009 21
    • Why Plone on EC2? + • Too few Zope hosting providers • Avoid renting rackspace in co-lo • Flexible and scalable architecture • Complete control of servers Thursday, November 5, 2009 22
    • What is PondCMS? PondCMS = Plone On Demand • Hosted and supported version of Plone • Eliminate the complexities of hosting • Turnkey - automated and repeatable • Bundled with certified add-ons www.jazkarta.com/pondcms Thursday, November 5, 2009 23
    • PondCMS Technology stack HTTP proxy Caching proxy Load Balancer 0%1&2 PondCMS *#"+ ,-./+ + add-ons *#"+ Web App Server ,-./+ Operating System *#"+ ,-./+ Virtual Servers !"#$%&!'() Thursday, November 5, 2009 24
    • One server deployment Apache HTTP (HTTP) Varnish (Caching) Pound (Load balancing) 1 2 n Zope Client #1 Zope Client #1 Zope Client #n Zeo Server Zeo Backup Thursday, November 5, 2009 25
    • Add Zeo clients as demand increases Thursday, November 5, 2009 26
    • Multi server deployment Website visitor HTTP Elastic Block Storage public IP: 72.44.50.237 Apache Frontend server Apache logs Varnish Pound Application Servers 1 2 n zope-1.web.private:8222 zope-2.web.private:8222 zope-n.web.private:8222 Zope client #1 Zope client #2 Zope client #n Elastic Block Storage Database server zeo-1.web.private:8221 Zeo server #1 Data.fs Data Data.fs Backup files Simple Storage Solution (S3) Thursday, November 5, 2009 27
    • Goal: automate everything Scripts for repeatable deployment 1. Bootstrap the server 2. Install dependencies 3. Install PondCMS (includes Plone + add-ons) 4. Configure backups 5. Create the Plone site and Apache vhosts Thursday, November 5, 2009 28
    • 1. Bootstrap server 1. Create the EC2 instance 2. Create an EBS volume 3. Associate volume with EC2 instance 4. Acquire IP address 5. Associate IP with EC2 instance Thursday, November 5, 2009 29
    • 2. Install dependencies 1. Install packages with apt-get • zope, python, varnish, apache2, pound, postfix 2. Initialize EBS volume and mount as ext3 fs 3. Store Apache log files on EBS volume 4. Register server with Landscape 5. Set up Pound to start automatically 6. Set up Pound to point to Zeo clients Thursday, November 5, 2009 30
    • 3. Install PondCMS 1. Make a 'zope' user 2. Create a virtualenv 3. Make Zope data dirs on EBS volume 4. Download PondCMS tarball release (incl. Plone) 5. Install PondCMS into virtualenv Thursday, November 5, 2009 31
    • 4. Configure backups • Configure logfile rotation • Setup backup scripts (using repozo) • Setup sync of backups to S3 (using Duplicity) • Create cron jobs for backups (daily, weekly) Thursday, November 5, 2009 32
    • 5. Create Plone site & vhosts 1. Use Fabric (remote deployment tool) to: • Make Plone site incl. CacheFu setup • Make vhost for accessing site on port 80 2. Create rewrite rules for: • domainname.com/supervisor (start/stop Zope) • domainname.com/admin (ZMI access) Thursday, November 5, 2009 33
    • What is Fabric? Thursday, November 5, 2009 34
    • What is Fabric? http://www.nongnu.org/fab/ • Simple remote deployment tool • Similar to Capistrano (Ruby on Rails) • Deploy applications to multiple similar hosts Thursday, November 5, 2009 35
    • Setup site Fabric script @require('host', 'site_id') @fabricset(host='fab_hosts', user='fab_user', key='fab_key_filename') def createplonesite(host=None, user=None, key=None, site_id=None, public_host=None): """Create a new plone site """ if not site_id: site_id = host installzcml(host=host, user=user, key=key, package='pondcms.basicapp') restart(host=host, user=user, key=key) put('createplonesite.py', '/tmp/createplonesite.py') s = SERVICE+'/bin/utility run /tmp/createplonesite.py -a pondcms.basicapp -s '+site_id if public_host: s += ' --site-url='+public_host run(s) Thursday, November 5, 2009 36
    • Setup vhost Fabric script @require('host', 'vhost_name') @fabricset(host='fab_hosts', user='fab_user', key='fab_key_filename') def setupvhost(host=None, user=None, key=None, vhost_name=None): """Setup an apache2 vhost""" site = VHOST_TEMPL % {'vhost_name': vhost_name} tmpfile = '/tmp/'+get('fab_timestamp')+'_vhost' f = open(tmpfile, 'w') f.write(site) f.close() targetf = vhost_name+'.conf' target = '/etc/apache2/sites-available/' + targetf put(tmpfile, target) run('a2ensite vhost_name') run('/etc/init.d/apache2 reload') Thursday, November 5, 2009 37
    • Running Fabric scripts $ cd fabric $ source bin/active (fabric)$ cd fabs/pondcms (fabric)$ fab createplonesite:host=test.pondcms.net,user=root,key=~/id_rsa-gsg- keypair,site_id=test.pondcms.net,public_host=http:// test.pondcms.net ... (fabric)$ fab setupvhost:host=test.pondcms.net,user=root,key=~/ id_rsa-gsg-keypair,vhost_name=test.pondcms.net Thursday, November 5, 2009 38
    • Challenges • Persistent data must be stored on EBS • Static IPs • Mail sent from EC2 instances flagged as spam • Monitoring and maintenance of VMs • Landscape • ZenOss • Nagios • Hyperic Thursday, November 5, 2009 39
    • Thursday, November 5, 2009 40
    • AWS Status history Thursday, November 5, 2009 41
    • Landscape monitoring / updates • http://landscape.canonical.com Thursday, November 5, 2009 42
    • Landscape update pkgs Thursday, November 5, 2009 43
    • PondCMS v0.1- Dashboard • JazOverseer Thursday, November 5, 2009 44
    • PondCMS 1.0 (mockups) nate (logout) | Security Settings Customers EC2 Plone Instance ID: i-e3de598a RUNNING Elastic IP: 75.101.161.152 EBS: vol-07ce2dfe Supervisor - CMRLS S3 bucket: cmrls.pondcms.net ZMI + staging.cmrls.org Landscape Refresh New EC2 instance SSH + production.cmrls.org EC2 Instances - Oxfam America Instance ID State Launch time Public DNS name + staging.oxfamamerica.org i-e3de598a running 2009-01-08 ec2-75-101-161-152.compute-1.amazonaws.com - production.oxfamamerica.org i-7d573e14 running 2009-01-08 ec2-75-174.129.208.compute-1.amazonaws.com www.oxfamamerica.org blog.oxfamamerica.org - Oxfam Canada - Harvard School of Engineering Refresh New Plone site Customer: CMRLS Plone sites Customer: CMRLS Plone site Plone PondCMS Custom eggs Contact: Rick Levine ashbylibrary.net 3.1.7 0.4.1-20090505 (upgrade) Add egg Phone: (508) 555-1212 fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg Email: rick@cmrls.org taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg Package: PondCMS Small Start date: 01/01/2009 End date: 12/31/2009 Thursday, November 5, 2009 45
    • Reports if Zope is down nate (logout) | Security Settings Customers EC2 Plone Instance ID: i-e3de598a RUNNING NOT RUNNING Elastic IP: 75.101.161.152 EBS: vol-07ce2dfe Supervisor - CMRLS S3 bucket: cmrls.pondcms.net ZMI + staging.cmrls.org Landscape Refresh New EC2 instance SSH + production.cmrls.org EC2 Instances - Oxfam America Instance ID State Launch time Public DNS name + staging.oxfamamerica.org i-e3de598a running 2009-01-08 ec2-75-101-161-152.compute-1.amazonaws.com - production.oxfamamerica.org i-7d573e14 running 2009-01-08 ec2-75-174.129.208.compute-1.amazonaws.com www.oxfamamerica.org blog.oxfamamerica.org - Oxfam Canada - Harvard School of Engineering Refresh New Plone site Customer: CMRLS Plone sites Customer: CMRLS Plone site Plone PondCMS Custom eggs Contact: Rick Levine ashbylibrary.net 3.1.7 0.4.1-20090505 (upgrade) Add egg Phone: (508) 555-1212 fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg Email: rick@cmrls.org taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg Package: PondCMS Small Start date: 01/01/2009 End date: 12/31/2009 Thursday, November 5, 2009 46
    • Launch new EC2 instance nate (logout) | Security Settings Customers EC2 Plone Instance ID: i-e3de598a RUNNING Elastic IP: 75.101.161.152 EBS: vol-07ce2dfe Supervisor - CMRLS S3 bucket: cmrls.pondcms.net ZMI + staging.cmrls.org 2 Landscape Refresh New EC2 instance SSH + production.cmrls.org EC2 Instances New EC2 Instance - Oxfam America Instance IDKeypair: State gsg-keypair Launch time Public DNS name + staging.oxfamamerica.org i-e3de598a running 2009-01-08 ec2-75-101-161-152.compute-1.amazonaws.com Elastic IP: Assign a static IP? - production.oxfamamerica.org i-7d573e14 running 2009-01-08 ec2-75-174.129.208.compute-1.amazonaws.com Instance type: - Small - www.oxfamamerica.org blog.oxfamamerica.org Availability zone: - us-east-1 - - Oxfam Canada Launch instance - Harvard School of Engineering Refresh New Plone site Customer: CMRLS Plone sites Customer: CMRLS Plone site Plone PondCMS Custom eggs Contact: Rick Levine ashbylibrary.net 3.1.7 0.4.1-20090505 (upgrade) Add egg Phone: (508) 555-1212 fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg Email: rick@cmrls.org taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg Package: PondCMS Small Start date: 01/01/2009 End date: 12/31/2009 Thursday, November 5, 2009 47
    • Create new Plone site nate (logout) | Security Settings Customers EC2 Plone Instance ID: i-e3de598a RUNNING Elastic IP: 75.101.161.152 New Plone site EBS: vol-07ce2dfe Supervisor - CMRLS S3 bucket: cmrls.pondcms.net ZMI Site title/host: + staging.cmrls.org Landscape Refresh This essentially maps to the "zope id" for the site which New EC2 instance SSH should also match the public host name that will be used + production.cmrls.org to access this Plone site. EC2 Instances - Oxfam America Manager password: Instance ID State Launch time Public DNS name + staging.oxfamamerica.org Each Plone site gets initiated with a new manager i-e3de598a account with user2009-01-08 This fields sets the running "ploneadmin". ec2-75-101-161-152.compute-1.amazonaws.com password for that account. - production.oxfamamerica.org i-7d573e14 running 2009-01-08 ec2-75-174.129.208.compute-1.amazonaws.com www.oxfamamerica.org Plone theme: - Select One - (theme thumbnail) blog.oxfamamerica.org - Oxfam Canada Create Plone site - Harvard School of Engineering 3 Refresh New Plone site Customer: CMRLS Plone sites Customer: CMRLS Plone site Plone PondCMS Custom eggs Contact: Rick Levine ashbylibrary.net 3.1.7 0.4.1-20090505 (upgrade) Add egg Phone: (508) 555-1212 fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg Email: rick@cmrls.org taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg Package: PondCMS Small Start date: 01/01/2009 End date: 12/31/2009 Thursday, November 5, 2009 48
    • Add eggs to Plone site nate (logout) | Security Settings Customers EC2 Plone Instance ID: i-e3de598a RUNNING Elastic IP: 75.101.161.152 EBS: vol-07ce2dfe Supervisor - CMRLS S3 bucket: cmrls.pondcms.net ZMI + staging.cmrls.org Landscape Refresh New EC2 instance SSH + production.cmrls.org EC2 Instances - Oxfam America Instance ID State Launch time Public DNS name + staging.oxfamamerica.org i-e3de598a running 2009-01-08 ec2-75-101-161-152.compute-1.amazonaws.com - production.oxfamamerica.org i-7d573e14 running 2009-01-08 ec2-75-174.129.208.compute-1.amazonaws.com www.oxfamamerica.org blog.oxfamamerica.org - Oxfam Canada Add eggs - Harvard School of Engineering Available eggs To install PloneFlashUpload Refresh Ploneboard New Plone site > ImageEditor Customer: CMRLS Plone sites slideshowfolder < Customer: CMRLS PloneFormGenPlone site Plone PondCMS Custom eggs 4 Contact: Rick Levine ashbylibrary.net 3.1.7 0.4.1-20090505 (upgrade) Add egg Phone: (508) 555-1212 Egg URL: fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg Email: rick@cmrls.org URL to an egg hosted on the cheeseshop. taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg Package: PondCMS Small Start date: 01/01/2009 Quickinstall: Install the egg in the Plone site. End date: 12/31/2009 Add eggs Thursday, November 5, 2009 49
    • Eucalyptus • Build your own private cloud • Compatible with Amazon EC2 API • Included with Ubuntu 9.0.4 Server http://www.ubuntu.com/products/whatisubuntu/serveredition/cloud/uec http://open.eucalyptus.com/ https://help.ubuntu.com/community/Eucalyptus Thursday, November 5, 2009 50
    • Thank you! Questions? Follow me on Twitter: natea Or email me at: natea (at) jazkarta (dot) com Thursday, November 5, 2009 51
    • More info • Amazon EC2: http://amazon.com/ec2 • Boto: http://code.google.com/p/boto • Bototools: http://code.google.com/p/bototools/ • Fabric: http://www.nongnu.org/fabric • Fabulatr: http://code.google.com/p/fabulatr/ • Plone: http://plone.org • PondCMS: http://jazkarta.com/pondcms Thursday, November 5, 2009 52
    • Other resources • Ubuntu on EC2: http://groups.google.com/group/ec2ubuntu • Ubuntu AMIs: http://www.alestic.com • Beta program: http://www.ubuntu.com/ec2 • Landscape: http://landscape.canonical.com • vm-builder-ec2: http://packages.ubuntu.com/intrepid-updates/utils/python-vm-builder-ec2 • Enomaly's Elastic Computing Platform: http://src.enomaly.com/ Thursday, November 5, 2009 53
    • Photo credits • http://www.flickr.com/photos/docman/66137352/ • http://www.flickr.com/photos/thomashawk/126534906/ • http://www.flickr.com/photos/moeview/498544386/ • http://www.flickr.com/photos/shellysblogger/ 2464969383/ • http://www.flickr.com/photos/mhw/56386662/ • http://www.flickr.com/photos/mrtopf/1541527579/ Thursday, November 5, 2009 54