Trying to stay away from pedantry and introduction.
talk about experiences and open questions.
You can make spreadsheets all day; the big unknown always ends up sysadmin labor.
To the Clouds and Back
To the Cloud(s)
Kyle Cordes Jan 21, 2010
St. Louis Cloud Computing User Group
Image Credit: http://www.ﬂickr.com/photos/vishvarupa/
About Kyle Cordes
• Blah, Blah, Blah.
• Coming soon: consulting
focus on cloudy
• You know where to ﬁnd
• Yet another cloud intro
• Comparison of what N people mean by
• “walkthrough of Amazon Web Services
from a developer perspective”
• Code or Design for parallelism
• Experiences as a cloud customer
• Experiences as a cloud / SaaS provider
• Raise many question, suggest a few answers
• SaaS ﬁrm I co-founded in 2004, sold 2009
• SaaS / Cloud provider
• Mostly ran on owned colo hardware
• Used some S3 and EC2
• Cloud customer
• There could be a good business talk about
this someday; not today.
• Storage is the simplest thing to outsource
to a “cloud”, mature, least to go wrong
• My Story:
• Started with homegrown storage
• Moved to S3
• Moved back.
Why Cloud Storage
• Simpler than running your own robust
internet-accessible ﬁle servers
• Very low startup cost
• Removes a bunch of bandwidth from the
rest of your system
• Very little downtime
Why Not Cloud Storage?
• No so cheap, as the bytes add up
• Consider incremental vs baseline
• Downtime is more obvious
• What about backups?
• Can you ever trust them?
Is S3 cheap or
• Cheap, compared to: • Expensive, compared to:
• Enterprisey sysadmin • Scrappy startup
• Sysadminning other
• Enterprisey storage servers anyway
• Hardware scaled for
• Borrowing money to mostly-archival
buy hardware storage
• Obvious answer: Provider’s Problem
• What if all your (customers’) ﬁles
disappeared one day?
• How many clicks to do this? 5-6?
• Your own backups? Now you have
sysadmin and hardware after all.
Backups are the Provider’s
• What if the data is lost?
• Will your customer / boss / etc. ask why
you didn’t just keep a copy on drives or
• Do you have a survivable answer?
Store low-to-medium importance content
on S3 or similar service, and take the risk of
how well they keep the data. It’s a good trade
... but not by much. Might go the other way,
depending on the details.
Even if you don’t use a public cloud storage,
design your system to isolate bulk data
storage. Do something in-house which
resembles using an outside cloud.
Another gratuitous cloud
Image Credit: http://www.ﬂickr.com/photos/pagedooley/
• Some experience with Amazon EC2
• and a few VPS providers
• To me a “cloud” server is not just any
rented server/VPS; it is something that can
be brought up and down quickly,
automatically, hourly billing.
Why Cloud Servers?
• No up front cost
• No need to ever babysit hardware
• Rapid scaling up or down
• Easy to hand over to someone else
• Keeps you honest in your automation,
management, and architecture
Running Cloud Servers
• Have you set up automated deployment?
• Have you set up automated management?
• Do you hate manual software installation?
• You will!
• Puppet, Chef, a zillion others
• Before EC2:
• Knew we needed to automate
deployment and management
• Never got around to doing it end-to-end
• After EC2:
• Really did it, with apt-get, Puppet, etc.
• Huge beneﬁt for all servers
Why Not Rented / Cloud
• Nailed-up 24x7x365 x N, can get pricey
• Likely cheaper in year 1
• Probably more expensive by year 3
• There is no magic, the provider must charge
you more than the underlying cost.
• But Spot Instances are a little bit magic!
Next time around, I’d most likely use EC2 or
similar for the ﬁrst few years of any new
venture or project.
Buy hardware once you have a proven long
term need - not based on hope, hunch, or
But Wait! (II)
Even if you don’t use a public cloud provider
of CPUs, design your system as if you did.
Use Puppet or whatever. Repave machines
with nary a thought.
Use an in-house cloud.
Design for parallelism.
Farther Up the Stack
Solder Wires ➞ buy servers etc.
Physical servers / nets ➞ IaaS
IaaS ➞ PaaS
PaaS ➞ SaaS
SaaS ➞ BPO
Over time, expect to move up the stack, and expect
to specialize in fewer layers.
Image Credit: http://www.ﬂickr.com/photos/emdot/
• No up front cost • Running the system is
The Vendor’s problem
• Trivially scale
• Hire less sysadmins
• Pay only for actual use
• vendor will instead
• user count
• Specialization is a key
• bytes driver of economic
Focus on your core value
(not on running IT systems)
Outsource the IT systems
(to someone whose whole business it is)