Fowa Miami 09 Cloud Computing Workshop

How to build your app quickly (and cheaply?) using the Cloud

Intro (Admin, facilities, who I am)

Agenda

Overview

Walk-through deployment on PaaS and IaaS

Break

Considerations

Costs / Economics

Architecture

Discussion

Goals for the workshop

Cloud computing. WTF?

WAN (Frame Relay, ISDN, etc.) Router Router LAN A LAN B

For this workshop, it means: elastic compute resources, charged for like a utility

Why bother?

What’s a “web app”?

http://www.flickr.com/photos/psd/6732568/in/set-666533/

http://www.flickr.com/photos/gopal1035/322084210/

http://www.flickr.com/photos/psd/58054545/

It’s more than static content.

So that means, it’s about any and all commercial web sites, right?

No. Don’t think so.

Griefer alert: is this just flamebait?

No. In the context of cloud computing, it makes a difference.

A big difference.

Apps do something.

Content doesn’t.

Cloud computing is about doing something.

http://www.nytimes.com/2006/06/14/technology/14search.html

“ Microsoft is adding 10k servers a month to their infrastructure” http://blogs.zdnet.com/microsoft/?p=1391

“ That’s one Facebook per month” http://www.datacenterknowledge.com/archives/2008/04/23/facebook-now-running-10000-web-servers/ http://www.istartedsomething.com/20080812/microsoft-enviroment-video-reveals-server-numbers-power/

Amazon talks a lot about the demand…

Animoto went from 50 to 3400 Amazon virtual servers in two days

http://aws.typepad.com/aws/2008/04/animoto---scali.html

The New York Times converted 4TB of TIFFs to PDFs in a day. For $240

http://open.blogs.nytimes.com/2007/11/01/self-service-prorated-super-computing-fun/ http://open.blogs.nytimes.com/2008/05/21/the-new-york-times-archives-amazon-web-services-timesmachine/ http://www.roughtype.com/archives/2008/11/the_new_economi.php

And Amazon shows everyone who will sit still for more than 5 seconds the following picture:

http://aws.typepad.com/aws/2008/05/lots-of-bits.html

Assumption: Mom+Pop ISP ™ cannot compete with this.

Are Mom+Pop doomed?

Ultimately, that depends on you, the developers of web apps.

It’s going to depend on decisions you make about architectural and engineering concerns.

Architecture is about making design choices. Engineering is about knowing your materials.

Making use of the Cloud is an architectural and engineering challenge.

So what are the architectural choices? What are the materials?

http://rationalsecurity.typepad.com/blog/2009/01/cloud-computing-taxonomy-ontology-please-review.html http://cloudcomputing.sys-con.com/node/811519 http://www.collab-ogce.org/gce08/images/7/76/LamiaYouseff.pdf SADIST-PIMP SPI (SaaS, Paas, IaaS)

http://www.mindmeister.com/maps/show_public/15936058

But wait! Once that’s sorted, you have to consider contextual dimensions…

The Radeztsky Cube http://cloudforum.googlegroups.com/web/Metaverse+Decomposition.pdf

http://cloudforum.googlegroups.com/web/Metaverse+Decomposition.pdf

IOW, your choices are influenced by whether you are integrating established apps, or writing a green-field app….

And whether your architecture will be entirely in the public Cloud, or a mix of public and private resources.

For the sake of this workshop, we’re going to refer to the SPI model…

SPI Model SaaS PaaS IaaS

We’re going to assume a green-field app, deployed entirely on the public Cloud…

We’re going to focus on the scenario of “your code, running elsewhere” – not so much on mashups and re-use of SaaS

And we’re going to examine the question of PaaS vs. IaaS for that app.

OK. So what is PaaS?

Platform As A Service

Some examples: Google App Engine, Bungee, Force.com, Heroku

Characterized by…

Constraints on language and design

A high level programming model

A specific model of multi-tenancy

Takes care of low level concerns

Google App Engine

That’s it.

Constraints on language and design (Python + BigTable + Goog Svcs)

A high level programming model (The WebApp (or other Python) framework, Datastore APIs, Memcache, etc.)

A specific model of multi-tenancy (Google’s BigTable + GFS platform)

Takes care of the low level concerns (Scales for you (up and down), distribution across cluster nodes, load balancing, replication of data, etc. )

Pretty cool.

Infrastructure As A Service

Characterized by…

No constraints on language or design

A high level architectural model

A specific model of multi-tenancy

Takes care of very few low level concerns

This is a LOT more work. ;)

Amazon Web Services

Elastic Compute Cloud (EC2)

SimpleDB

Simple Storage Service (S3)

Simple Queue Service (SQS)

EC2 Elastic Block Store (EBS)

Other stuff…

Cloudfront

DevPay

Flexible Payments Service (FPS)

Mechanical Turk

Alexa

By and large, “Amazon” means “EC2”

EC2 is the only AWS service that one deploys to

You just use the other services – whether from an EC2 instance or anywhere else on the Web is irrelevant

The core “unit” of EC2 is the Amazon Machine Image – AMI

An AMI is a virtual machine image – a VM

A VM is just a (very large) file. Like a live ISO disk image. Typically, it is some distro of Linux.

Amazon uses Xen, an open source VM system http://www.xen.org/

The key to IaaS is that you can use any app architecture you like

The drawback with IaaS is that you therefore have to design your own app architecture

Generally speaking, this is the same task (with the same effort) that you would need for physical hardware hosted at an ISP

App servers, load balancers, databases, clusters, replication, networking…

You sort it out yourself

With Amazon, this begins with obtaining your credentials

Then you download the command line tools and set them up…

You use the tools to proceed through the AMI workflow

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ AMI ID

Create keypair

Save the private key locally

ec2-add-keypair <descriptive keypair name>

Launch selected instance

Using AMI ID and keypair name

ec2-run-instances <AMI ID> -k <keypair name>

Returns the Instance ID

Examine the running instance

ec2-describe-instances <instance ID>

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ …

Let’s pick that apart…

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Reservation ID

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … AWS Access Key ID

AWS Access Key ID

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Security Group ID

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Instance ID

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … AMI ID

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … External DNS host name

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Internal DNS host name

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Current state of the instance

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Keypair name

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … AMI Launch Index

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Instance type

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Launch time

http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/ … Availability Zone ID

Availability zones are a bit complicated…

Other IaaS providers may (or may not) have similar capabilities

The point is that, unlike GAE, here is yet another detail that you need to think about

Open network access to the instance

ec2-authorize <security group name> -p <port number>

Can include options like restricting access to a specific (public) IP address

-s <IP address/CIDR subnet range>

Eg. “-s your_public_IP_address/32”* for just your host

At a minimum, you need to configure port 22 for SSH access

Connect to the instance with SSH

ssh –i <private keyfile> root@<public DNS host name>

Note: with “ec2-get-console-output”, you can get (among other things) the SSH host key fingerprint, before logging on with SSH. At login, you can compare to be sure there’s no “man in the middle”

Begin modifying the instance

Once you’ve modified the image, you “bundle” it (which is a kind of snapshot), upload the bundle to S3, and register it with EC2

There are hundreds of publicly available AMIs for use as templates

And, finally, with some set of your own AMIs, you can begin running instances of your app

And that brings us to the runtime considerations…

AKA – how does your app scale horizontally?

A brief scalability refresher: vertical scalability is “get a bigger box”

Horizontal scalability is “add more copies of the same box”

To put it bluntly, if your app can't efficiently scale in a horizontal fashion, you’re wasting your time on the Cloud

On GAE, under the covers tech like BigTable, GFS and the legendary Map/Reduce are taking care of this for you

In an IaaS context like AWS, you have to solve the problem yourself

EC2 has a SOAP API that you can invoke – whether you do so from outside EC2, or from within a running AMI is irrelevant

Invoking the EC2 API to launch and terminate instances from a running instance gives you the ability to create dynamic horizontal scalability

Your load balancer AMI could start extra instances of your web server AMI to meet demand…

And turn them back off, when demand subsides

And there are numerous ways to exploit this capability

From RightScale to Scalr to Gigaspaces, Hadoop, Terracotta and countless others

Combined with the other elements of AWS, like S3, SQS, EBS and SimpleDB, you can design a system that competes favorably with GAE

This is what “elastic” means

To repeat: this is a LOT more work. ;)

No constraints on language or design (AMIs can be any Linux or Windows server platform, and your app can be anything that runs on those platforms)

A high level architectural model (The core services of AWS provide a foundation, and do constrain your design – for example, you need horizontal scalability)

A specific model of multi-tenancy (AMIs are securely isolated from one another, but the underlying hardware is all shared)

Takes care of very few low level concerns (You roll your own)

Before we wrap up “deploying to AWS”, however, let’s look at some alternatives to the command line tools from Amazon

In terms of managing running instances, and the overall configuration of things, AWS provides its own Web UI

https://console.aws.amazon.com/ec2/home#c=EC2&s=Home

https://console.aws.amazon.com/ec2/home#c=EC2&s=Instances

https://console.aws.amazon.com/ec2/home#c=EC2&s=Images

https://console.aws.amazon.com/ec2/home#c=EC2&s=BundleTasks

http://status.aws.amazon.com/

An ecosystem of 3 rd party providers has emerged around AWS

Some of them are specialized in managing VMs

http://elasticserver.com/

http://elasticserver.com/portals

http://elasticserver.com/site/lamp

http://elasticserver.com/build-status

https://elasticserver.com/ec2/

https://elasticserver.com/byo/create

https://elasticserver.com/my/packages

https://elasticserver.com/new-server

There are other vendors offering similar services, eg. rPath

And there is a spectrum between the simplicity of GAE and the complexity of AWS…

Let’s take a look at Elastra, for example

https://cloud.elastra.com/elastra/

Finally, to wrap up our quick overview of IaaS, let’s look at Rightscale…

https://my.rightscale.com/dashboard

But first, a short break… (Jeopardy theme song plays)

What are some of the questions you need to answer, to decide between PaaS and Iaas?

How quickly do you need to go to market?

How fast do you need to iterate versions of the app?

What are your pre-existing platform needs, if any?

What are your security, compliance, regulatory requirements?

What is your capacity for system architecture and design work?

SPI Model Freedom FROM Considerations Freedom TO Differentiate http://twitter.com/gblnetwkr SaaS PaaS IaaS

SPI Model POWER of speed and agility POWER to control http://twitter.com/gblnetwkr SaaS PaaS IaaS

What do things really cost?

Simple LAMP app. 1 box as a load balancer / proxy, 4 web server boxes, and one larger box for a database server

$84 / month (6 x Giant) http://laughingsquid.net/hosting/

$900 / month (1 x Biz1 + 4 x Biz1 + 1 x Enterprise II) http://order.1and1.com:80/xml/order/ServerRoot

So, let’s see what the equivalent setup costs on AWS…

Now, AWS bills in units of things like server hours, IOs per month, GB of storage and bandwidth actually used, etc.

24 hours × 30 days = ---------------------- 720 hours in a month ---------------------- 720 hours × 5 small AWS Instances = ---------------------- 3600 hours ---------------------- + 720 hours of a large AWS instance + 5000 GB network bandwidth + 3200 GB disk space (added to the default space on the instances) + 50 mil. IORs + 30 daily backups ---------------------- roughly equivalent to 1+1 …

$2233 / month http://calculator.s3.amazonaws.com/calc5.html

http://calculator.s3.amazonaws.com/calc5.html

Despite best efforts, this is still quite imprecise, and apples vs. oranges, but…

Even more importantly…

This is an incredibly stupid way to set up and use AWS

$528/ month http://calculator.s3.amazonaws.com/calc5.html

Still not “cheap”, though…

That’s because the key to “elastic computing” isn’t being able to turn on servers at will…

It’s about being able to turn them off .

Let’s rewind to the an earlier part of this discussion…

What’s a web app?

Whatever it is, we said “it’s not static content”…

And I said that makes a big difference

Why is that? Well, the unwritten social contract of the ‘Net says: “you can’t turn static content off”

Apps do something

But they generally don’t do it 24x7

Slicing and dicing your design to strictly demarcate static content from active has always been a Good Idea ™

In a cloud computing context, it’s not just a good idea – it’s imperative

What are your usage patterns? When are your peak loads, on average?

What can you turn off?

And you have to be clever about it…

Your production usage may not even be the place you can save the most money…

Consider: you want to develop the next version of your app

You need resources for your developers to do that, and you need a place for them to test

So, maybe you wind up renting 3 more servers from 1+1 – $300 / month

But, when you examine usage patterns, you find that your developers are only using the boxes for 3 hours each day

$17/ month http://calculator.s3.amazonaws.com/calc5.html

Perhaps even more importantly, you’ve made no commitment here – once your dev / test phase is over, so are the costs

Calculating costs is complex, and entirely context dependent

There are significant potential savings, but only if you’re clever

Since we worked out, sometime in the early ‘90s, what the architecture of a “client / server” system design looked like…

There's been a general consensus about a sort of a canonical architecture for so-called “N-tier systems”

Presentation Service Facáde Application Logic Data Persistence

Web Server App Server Database Server

So, we’ve talked about the need for horizontal scalability in the Cloud

What does that imply?

Well, as already suggested, among other things, parallelism

Parallelism has significant consequences

It leads one to try to avoid stateful interactions

To prefer asynchronous communications (messages)…

One finds oneself on the front lines of the REST War ™ – the battle of the RESTafarians vs. the established IT Universe http://www.dehora.net/journal/2008/07/25/patterns-of-web-architecture/ http://www.dehora.net/journal/2008/08/15/rest-as-an-engineering-discipline/ http://www.infoq.com/articles/webber-rest-workflow/ http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven/ http://www.redmonk.com/jgovernor/2009/02/12/the-rest-of-the-cloud/ http://delicious.com/mastermark/rest/

And it forces one to think strange things about optimal patterns of storing and accessing data

Like sharding one’s data to meet resource demands http://highscalability.com/unorthodox-approach-database-design-coming-shard/

Questions like “is two-phase commit a feature? Or a bug?” begin to seem important

New terms, like CAP, Paxos and BASE creep into conversations about “eventual consistency” http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1495 http://en.wikipedia.org/wiki/Paxos_algorithm http://queue.acm.org/detail.cfm?id=1394128 http://www.allthingsdistributed.com/2008/12/eventually_consistent.html

This was happening anyway, driven by the clash of Web architecture with the established IT universe

Cloud computing’s possibilities are accelerating the process

There is an emerging consensus about what the consequences of all this are for app architectures

“ The canonical cloud architecture that has evolved revolves around dynamically scalable CPUs consuming asynchronous, persistently queued events.” http://highscalability.com/canonical-cloud-architecture

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1632&categoryID=102

Use scalable ingredients

Eg. Hadoop on EC2

Keep ingredients loosely coupled

All communication via persistent messaging

Assume constant failure

Design things to persist state, restart from last known good, and continue their own tasks even if all around them fail

Consider things like re-tries with exponential back-off

Build IN redundancy

Learn about things like the POSA Blackboard pattern, tuplespaces, and Map / Reduce

Read this book! http://www.amazon.com/How-Write-Parallel-Programs-Course/dp/026203171X/

And if your needs / budget require or can accommodate it, consider RAIC

Redundant Array of Independent Cloud providers http://www.jroller.com/MasterMark/entry/raic_pronounce_it_rake_please

http://www.jroller.com/MasterMark/entry/raic_pronounce_it_rake_please

Why bother?

Web Server App Server Database Server

Mainframe

http://www.flickr.com/photos/dvdmerwe/231593469/

http://www.flickr.com/photos/sigkyrre/1091616886/

http://twitter.com/ gblnetwkr http://en.wikipedia.org/wiki/ Consumerization

http://twitter.com/ gblnetwkr

Join the conversation: http://groups.google.com/group/cloud-computing/ http://groups.google.com/group/cloudforum http://tech.groups.yahoo.com/group/cloudcomputing-tech/ … and please come talk to us, as well … http://twitter.com/mastermark http://www.jroller.com/MasterMark/ Thanks!