Plone on Amazon EC2

Plone in the Cloud
A CMS hosted on Amazon EC2

Nate Aune (Jazkarta.com)
European Plone Symposium 2009
Sorrento, Italy
May 13, 2009
Thursday, November 5, 2009 1

In the next 45 min...
• Amazon Web Services (AWS)
• AWS with Python
• Plone with AWS
• Tools for launching & remote deployment
• Challenges of using AWS
• Monitoring and maintaining EC2 instances
• Admin dashboard for creating new Plone sites

What is
Amazon Web Services?


What is Amazon Web Services?

• Suite of elastic IT
infrastructure services
• Requisition computing
power and storage
• Run any application
you want on any OS


Beneﬁts of AWS

• Metered pricing (per hour / per GB)
• no large up-front capital investments

• only pay for what you use

• Scaleable hosting architecture
• Redundancy for high availability


AWS Architecture

• EC2 = Elastic Computing Cloud
• S3 = Simple Storage Solution
• EBS = Elastic Block Storage
• Elastic IP = Static IP addresses


Elastic Computing Cloud (EC2)
amazon.com/ec2

• Use an existing AMI (Amazon Machine Image)
to launch a new instance
• Or create your own AMIs (even paid ones)
• Attach EBS volumes to your EC2 instance


Elastic Block Storage
(EBS)


So how does Python
play into all this?


Enter Boto

What is Boto?
• Python package that provides interfaces to
Amazon Web Services
• Thanks to author Mitch Garnaat!
• Boto supports:
• S3 via the REST API
• EC2 via the Query API
• and more...

Install Boto & Bototools
$ sudo easy_install boto

$ svn co http://bototools.googlecode.com/svn/trunk/ bototools

$ cd bototools

$ sudo python setup.py install

$ sudo python ./src/bototools/postinstall.py -p /usr/local/bin

• This will put these commands in /usr/local/bin:
• ec2start, ec2stop, ec2ip, ec2ls, ec2img
• s3ls, s3rm, s3cp

Using Bototools

• Set these environment variables:
• AWS_ACCESS_KEY
• AWS_SECRET_ACCESS_KEY
• EC2_SSH_KEY
• Can also put in ~/.boto or /etc/boto.cfg


Find an AMI

• We will use the Ubuntu 8.10 Intrepid (32-bit)
from Alestic.com - ami-7cfd1a15
• More can be found at:
http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=171


Launch an instance
$ ec2start -a ami-7cfd1a15 -k gsg-keypair

Reservation:r-0f26a666

[Instance:i-8e3154e7]

$ ec2ls

Reservation r-0f26a666

i-8e3154e7 pending None 2009-03-25T01:49:24.000Z

...wait...

$ ec2ls


i-8e3154e7 running None 2009-03-25T01:49:24.000Z
ec2-174-129-238-8.compute-1.amazonaws.com


Associate a static IP
$ ec2ip -g

Address:174.129.238.8

$ ec2ip -l

174.129.238.8 ->

$ ec2ip -a 174.129.238.8 i-8e3154e7

True

$ ec2ip -l

174.129.238.8 -> i-8e3154e7

• Note: once you assign a static IP, you won't be
able to access the machine using the original IP.

Log in
$ ec2ls


i-8e3154e7 running None 2009-03-25T01:49:24.000Z
ec2-174-129-238-8.compute-1.amazonaws.com

$ ssh -i ~/id_rsa-gsg-keypair root@ec2-174-129-238-8.compute-1.amazonaws.com

Linux domU-12-31-39-00-E5-14 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686

root@domU-12-31-39-00-E5-14:~#

• Time to play!

Stop the instance
$ ec2stop i-8e3154e7

Stopping ['i-8e3154e7']

$ ec2ls


i-8e3154e7 terminated None 2009-03-25T01:49:24.000Z

• Be sure to stop the instance or
you will be charged!


Use the AWS Console


Or use ElasticFox
• Firefox extension to start/stop EC2 instances, assign
IPs, associate EBS volumes and create keypairs.

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609&categoryID=88


Case study
Plone on EC2


Why Plone on EC2?

+

• Too few Zope hosting providers
• Avoid renting rackspace in co-lo
• Flexible and scalable architecture
• Complete control of servers

What is PondCMS?
PondCMS = Plone On Demand

• Hosted and supported version of Plone
• Eliminate the complexities of hosting
• Turnkey - automated and repeatable
• Bundled with certiﬁed add-ons
www.jazkarta.com/pondcms

PondCMS Technology stack
HTTP proxy

Caching proxy

Load Balancer 0%1&2

PondCMS *#"+
,-./+ + add-ons

*#"+
Web App Server ,-./+

Operating System *#"+
,-./+

Virtual Servers !"#$%&!'()


One server deployment
Apache
HTTP
(HTTP)

Varnish
(Caching)

Pound
(Load balancing)

1 2 n
Zope Client #1 Zope Client #1 Zope Client #n

Zeo Server

Zeo Backup


Add Zeo clients as
demand increases


Multi server deployment
Website visitor

HTTP

public IP: 72.44.50.237
Apache
Frontend server Apache logs
Varnish
Pound

Application Servers

1 2 n
zope-1.web.private:8222 zope-2.web.private:8222 zope-n.web.private:8222
Zope client #1 Zope client #2 Zope client #n

Database server

zeo-1.web.private:8221
Zeo server #1 Data.fs

Data
Data.fs Backup ﬁles

Simple Storage Solution (S3)


Goal: automate everything
Scripts for repeatable deployment
1. Bootstrap the server
2. Install dependencies
3. Install PondCMS (includes Plone + add-ons)
4. Conﬁgure backups
5. Create the Plone site and Apache vhosts


1. Bootstrap server
1. Create the EC2 instance
2. Create an EBS volume
3. Associate volume with EC2 instance
4. Acquire IP address
5. Associate IP with EC2 instance


2. Install dependencies
1. Install packages with apt-get
• zope, python, varnish, apache2, pound, postﬁx
2. Initialize EBS volume and mount as ext3 fs
3. Store Apache log ﬁles on EBS volume
4. Register server with Landscape
5. Set up Pound to start automatically
6. Set up Pound to point to Zeo clients


3. Install PondCMS

1. Make a 'zope' user
2. Create a virtualenv
3. Make Zope data dirs on EBS volume
4. Download PondCMS tarball release (incl. Plone)
5. Install PondCMS into virtualenv


4. Configure backups

• Configure logfile rotation
• Setup backup scripts (using repozo)
• Setup sync of backups to S3 (using Duplicity)
• Create cron jobs for backups (daily, weekly)


5. Create Plone site & vhosts
1. Use Fabric (remote deployment tool) to:
• Make Plone site incl. CacheFu setup
• Make vhost for accessing site on port 80

2. Create rewrite rules for:
• domainname.com/supervisor (start/stop Zope)
• domainname.com/admin (ZMI access)


What is Fabric?


What is Fabric?
http://www.nongnu.org/fab/

• Simple remote deployment tool
• Similar to Capistrano (Ruby on Rails)
• Deploy applications to multiple similar hosts


Setup site Fabric script
@require('host', 'site_id')

@fabricset(host='fab_hosts', user='fab_user', key='fab_key_filename')

def createplonesite(host=None, user=None, key=None, site_id=None,

public_host=None):

"""Create a new plone site """

if not site_id:

site_id = host

installzcml(host=host, user=user, key=key, package='pondcms.basicapp')

restart(host=host, user=user, key=key)

put('createplonesite.py', '/tmp/createplonesite.py')

s = SERVICE+'/bin/utility run /tmp/createplonesite.py -a
pondcms.basicapp -s '+site_id

if public_host:

s += ' --site-url='+public_host

run(s)

Setup vhost Fabric script
@require('host', 'vhost_name')
@fabricset(host='fab_hosts', user='fab_user', key='fab_key_filename')
def setupvhost(host=None, user=None, key=None, vhost_name=None):
"""Setup an apache2 vhost"""
site = VHOST_TEMPL % {'vhost_name': vhost_name}
tmpfile = '/tmp/'+get('fab_timestamp')+'_vhost'
f = open(tmpfile, 'w')
f.write(site)
f.close()
targetf = vhost_name+'.conf'
target = '/etc/apache2/sites-available/' + targetf
put(tmpfile, target)
run('a2ensite vhost_name')
run('/etc/init.d/apache2 reload')


Running Fabric scripts
$ cd fabric

$ source bin/active

(fabric)$ cd fabs/pondcms

(fabric)$ fab
createplonesite:host=test.pondcms.net,user=root,key=~/id_rsa-gsg-
keypair,site_id=test.pondcms.net,public_host=http://
test.pondcms.net

...

(fabric)$ fab setupvhost:host=test.pondcms.net,user=root,key=~/
id_rsa-gsg-keypair,vhost_name=test.pondcms.net


Challenges
• Persistent data must be stored on EBS
• Static IPs
• Mail sent from EC2 instances ﬂagged as spam
• Monitoring and maintenance of VMs
• Landscape
• ZenOss
• Nagios
• Hyperic

AWS Status history


Landscape
monitoring / updates

• http://landscape.canonical.com

Landscape update pkgs


PondCMS v0.1- Dashboard

• JazOverseer


PondCMS 1.0 (mockups)
nate (logout) | Security Settings

Customers EC2 Plone Instance ID: i-e3de598a RUNNING
Elastic IP: 75.101.161.152
EBS: vol-07ce2dfe Supervisor
- CMRLS S3 bucket: cmrls.pondcms.net ZMI
+ staging.cmrls.org Landscape
Refresh New EC2 instance
SSH
+ production.cmrls.org
EC2 Instances
- Oxfam America
Instance ID State Launch time Public DNS name
+ staging.oxfamamerica.org
i-e3de598a running 2009-01-08 ec2-75-101-161-152.compute-1.amazonaws.com
- production.oxfamamerica.org i-7d573e14 running 2009-01-08 ec2-75-174.129.208.compute-1.amazonaws.com

www.oxfamamerica.org
blog.oxfamamerica.org
- Oxfam Canada
- Harvard School of Engineering

Refresh New Plone site

Customer: CMRLS Plone sites

Customer: CMRLS Plone site Plone PondCMS Custom eggs
Contact: Rick Levine ashbylibrary.net 3.1.7 0.4.1-20090505 (upgrade) Add egg

Phone: (508) 555-1212 fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg
Email: rick@cmrls.org taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg

Package: PondCMS Small
Start date: 01/01/2009
End date: 12/31/2009


Reports if Zope is down

Customers EC2 Plone Instance ID: i-e3de598a RUNNING NOT RUNNING
Elastic IP: 75.101.161.152
SSH
EC2 Instances
- Oxfam America

- Oxfam Canada





End date: 12/31/2009


Launch new EC2 instance

Elastic IP: 75.101.161.152
+ staging.cmrls.org 2 Landscape
SSH
EC2 Instances
New EC2 Instance
- Oxfam America
Instance IDKeypair:
State gsg-keypair
Launch time Public DNS name
Elastic IP: Assign a static IP?
Instance type: - Small -
blog.oxfamamerica.org Availability zone: - us-east-1 -

- Oxfam Canada Launch instance





End date: 12/31/2009


Create new Plone site

Elastic IP: 75.101.161.152
New Plone site EBS: vol-07ce2dfe Supervisor
Site title/host:
Refresh This essentially maps to the "zope id" for the site which
New EC2 instance
SSH
should also match the public host name that will be used
+ production.cmrls.org to access this Plone site.
EC2 Instances
- Oxfam America Manager password:
+ staging.oxfamamerica.org Each Plone site gets initiated with a new manager
i-e3de598a account with user2009-01-08 This fields sets the
running "ploneadmin". ec2-75-101-161-152.compute-1.amazonaws.com
password for that account.

www.oxfamamerica.org Plone theme: - Select One - (theme
thumbnail)
- Oxfam Canada Create Plone site


3




End date: 12/31/2009


Add eggs to Plone site

Elastic IP: 75.101.161.152
SSH
EC2 Instances
- Oxfam America

- Oxfam Canada Add eggs

Available eggs To install
PloneFlashUpload
Refresh Ploneboard
New Plone site
>
ImageEditor
slideshowfolder <
Customer: CMRLS PloneFormGenPlone site Plone PondCMS Custom eggs
4

Phone: (508) 555-1212 Egg URL: fobeslibrary.org 3.2.2 0.5.0-20090506 Add egg
Email: rick@cmrls.org URL to an egg hosted on the cheeseshop.
taftpubliclibrary.org 3.2.2 0.5.0-20090506 Add egg

Start date: 01/01/2009 Quickinstall: Install the egg in the Plone site.
End date: 12/31/2009
Add eggs


Eucalyptus

• Build your own private cloud
• Compatible with Amazon EC2 API
• Included with Ubuntu 9.0.4 Server
http://www.ubuntu.com/products/whatisubuntu/serveredition/cloud/uec
http://open.eucalyptus.com/
https://help.ubuntu.com/community/Eucalyptus


Thank you!
Questions?

Follow me on Twitter:
natea

Or email me at:
natea (at) jazkarta (dot) com


More info
• Amazon EC2: http://amazon.com/ec2
• Boto: http://code.google.com/p/boto
• Bototools: http://code.google.com/p/bototools/
• Fabric: http://www.nongnu.org/fabric
• Fabulatr: http://code.google.com/p/fabulatr/
• Plone: http://plone.org
• PondCMS: http://jazkarta.com/pondcms

Other resources
• Ubuntu on EC2:
http://groups.google.com/group/ec2ubuntu
• Ubuntu AMIs: http://www.alestic.com
• Beta program: http://www.ubuntu.com/ec2
• Landscape: http://landscape.canonical.com
• vm-builder-ec2:
http://packages.ubuntu.com/intrepid-updates/utils/python-vm-builder-ec2

• Enomaly's Elastic Computing Platform:
http://src.enomaly.com/

Photo credits
• http://www.flickr.com/photos/docman/66137352/
• http://www.flickr.com/photos/thomashawk/126534906/
• http://www.flickr.com/photos/moeview/498544386/
• http://www.flickr.com/photos/shellysblogger/
2464969383/
• http://www.flickr.com/photos/mhw/56386662/
• http://www.flickr.com/photos/mrtopf/1541527579/

Plone on Amazon EC2

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Plone on Amazon EC2

Similar to Plone on Amazon EC2 (20)

More from Jazkarta, Inc.

More from Jazkarta, Inc. (20)

Recently uploaded

Recently uploaded (20)

Plone on Amazon EC2