Recommended
More Related Content
Viewers also liked
Viewers also liked (13)
Similar to Arrogant Bastards Guide to Cloud Architecture
Similar to Arrogant Bastards Guide to Cloud Architecture (20)
Recently uploaded
Recently uploaded (20)
Arrogant Bastards Guide to Cloud Architecture
- 2. Primary
- 4. Raffle at 4:15 Your raffle ticket is on your badge. The raffle will be held in theaters #5 and #8
- 7. THANK YOU! RANDALL [AT] ENGINEYARD.COM @DAKSIS HTTP://EVILMARTINI.TUMBLR.COM/
- 8. THE ARROGANT BASTARD’S GUIDE TO CLOUDS
- 12. APPLICATION DESIGN PERSISTENCE
- 13. APPLICATION DESIGN PERSISTENCE ECONOMICS
- 14. APPLICATION DESIGN PERSISTENCE ECONOMICS SECURITY
- 15. AND THAT’S JUST THE BEGINNING...
- 20. PERSISTENCE & MEMORY http://upload.wikimedia.org/wikipedia/commons/d/dc/Salvador_Dali_-_Persistence_Of_Memory_-_Surrealism.jpg/
- 21. RECOVERY ORIENTED COMPUTING HTTP://ROC.CS.BERKELEY.EDU/
- 23. TIME IS _______
- 24. TIME VALUE OF ____
- 27. IN THE CLOUD AUTOMATION IS EVERYTHING
- 28. AND NOW FOR SOMETHING COMPLETELY DIFFERENT...
- 29. A FARMER IN THE DELL http://www.dell.com/us/en/business/virtualization_servers/fs.aspx? refid=virtualization_servers&s=bsd&cs=04
- 30. A FARMER IN THE DELL http://www.dell.com/us/en/business/virtualization_servers/fs.aspx? refid=virtualization_servers&s=bsd&cs=04
- 31. * (*Shameless promotion for Engine Yard in on the next slide) (**More shameless than this slide even)
- 32. A SILVER LINING IN THE CLOUDS http://www.engineyard.com/products/cloud/pricing/full-pricing
- 33. 30 DAYS X 24 HOURS @ $2.55 PER HOUR $1836 PER MONTH
- 34. BY THE WAY THAT'S 26ECUS, 1.6TB STORAGE, 70GB OF RAM
- 36. @DAKSIS FIRST AXIOM OF SCALE: IF YOU CAN’T MEASURE IT YOU CAN’T SCALE IT
- 38. WHAT TO SCALE?
- 39. WHAT NOT TO SCALE?
- 40. PREMATURE SCALING IS THE ROOT OF A LOT OF EVIL...
- 42. IN THE CLOUD AUTOMATION IS EVERYTHING (THIS INCLUDES SECURITY)
- 43. IN THE CLOUD ENCRYPT EVERYTHING
- 44. THE OLD GNU THING http://upload.wikimedia.org/wikipedia/en/4/4e/Paintedpuffy1000X907px.png
- 45. RABID OPINIONS (YOUR MILAGE MAY VARY)
- 46. DEPLOY EARLY DEPLOY OFTEN (UNDEPLOYED CODE DOESN’T EXIST)
- 47. AUTOMATE & ISOLATE: REDUCE THE COST OF CHANGE (AUTOMATION SUPPORTS ISOLATION)
- 48. HELP ME OPEN-WAN YOU’RE MY ONLY HOPE (YOU WON’T RUN OUT OF SERVERS)
- 49. RESIST THE URGE http://commons.wikimedia.org/wiki/File:Joint.jpg
- 50. NO SQL IS NOT A SUBSITITUTE FOR NO KNOWLEDGE (THIS INCLUDES YOU)
- 51. Thank You! randall [at] engineyard.com @daksis http://evilmartini.tumblr.com/
Editor's Notes
- Radically Simple Rails Support
- Otherwise known as what I wish I had known when I started this whole “cloud” thing...
- - This is everyone who’s currently building cloud infrastructure
- * Clouds are different - We all know this, but we continually build applications like it's not the case? We should be smarter than this, but we're not.
- ** How are they different? *** Application design changes - lateral scalability is a key point
- *** Persistence - Persistence doesn't mean the same thing that you think it does. This is a *big* deal for developers. Think about how much time you spend on persistence? ActiveRecord, Mongo, MySQL, File Systems, even abstractions like Mongo - Resources are constrained differently. Usually memory and CPU are broadly available, but constrained. Disk IO can become an issues.
- *** Economics - Cheap means something completely different when you can get a server at $0.11 an hour (we have a whole section on this)
- *** Security - Often over looked, but consider what your attack surface looks like on 10 servers... how about 100 - 1000? - Are you *sure* that virtual disk volume disappeared? Where does the bit bucket on a cloud go?
- This list could go on and on. - Ask the audience, solicit feedback
- Architecture gets a bum rap for two reasons 1) it’s associated with art not science, compare it to a civil engineer 2) Everyone has met an “enterprise architect” Image: http://commons.wikimedia.org/wiki/File:Arc.boutant.cathedrale.Chartres.png
- ** Cloud's are inherently parallel - Most programmers are familiar with iterative/sequential style development - This means that many of the ways that you’re used to doing things either don’t work in a cloud, or they work poorly. - Map Reduce as a common example A small detour into algorithmic complexity
- ** Cloud's _should_ be asynchronous where possible - People overlook this all the time. They assume that a - Latency can be an actual issue in a cloud Message Queues, AMQP, Erlang, Actor Models, Pub/Sub
- ** Cloud's _should_ be asynchronous where possible - People overlook this all the time. They assume that resources in the cloud are always there. - Clouds are powered by fail in a way that normal programmers can’t dream of - Recovery oriented Computing Project at Berkeley
- ** Persistence in the Cloud is Hard... - Forget the old way of doing storage. Fire and forget doesn't work at cloud scale - IO and latency are real issues - Data positioning, where does this data live? Is now a bigger question
- * Economics on clouds are different
- *** Economics - Cheap means something completely different when you can get a server at $0.11 an hour (we have a whole section on this)
- *** Economics - Cheap means something completely different when you can get a server at $0.11 an hour (we have a whole section on this)
- *** Economics - Cheap means something completely different when you can get a server at $0.11 an hour (we have a whole section on this)
- - Effort is the most limited technical resource. I can’t raise effort. VC’s can’t (and won’t) raise you a round of effort *** Economics - Cheap means something completely different when you can get a server at $0.11 an hour (we have a whole section on this) ** You scale the wrong things - Most teams spend time scaling the wrong things *** Time Is ____ / Time Value of _____ / Mo ____ mo' Problems - First and foremost: save you developers *effort*. Consider that a lot of what we do, is about saving developers *effort* *** Developers are LAZY - - Show me a sysadmin who likes doing things manually, and I'll show you an exchange server within arms reach
- ** Automation is Everything - Script it the first time, not the second. If it's not worth scripting, it's not really worth doing - There are no cloud economies of scale without automation! - You are not going to manually configure 20 machines one at a time *** Don't overlook the tools that help - Chef, Puppet, ZSH, bash, EC2 AMI repository, Git
- ** Quick look at the hard Dollar costs of on demand hardware... - Economics for non Geeks - This is how you justify on demand infrastructure to people with MBAs and those who have never undergone the agony of LILO, GRUB, or NTFS vs. FAT32
- * Economics on clouds are different Link: http://www.dell.com/us/en/business/virtualization_servers/fs.aspx?refid=virtualization_servers&s=bsd&cs=04
- * Economics on clouds are different ** Dell Example - Home Page for Dell vitalization applications list prices between $7k and $13k; 12GB of RAM - - Reference: http://www.dell.com/us/en/business/virtualization_servers/fs.aspx?refid=virtualization_servers&s=bsd&cs=04
- * Economics on clouds are different ** Dell Example - Home Page for Dell vitalization applications list prices between $7k and $13k; 12GB of RAM - - Reference: http://www.dell.com/us/en/business/virtualization_servers/fs.aspx?refid=virtualization_servers&s=bsd&cs=04
- ** Automation is Everything - Script it the first time, not the second. If it's not worth scripting, it's not really worth doing - There are no cloud economies of scale without automation! *** Don't overlook the tools that help - Chef, Puppet, ZSH, bash, EC2 AMI repository, Git
- * Cloud's require changing how you think of scale - Scaling is inherently about trading one type of problem for another. - E.g.: Shard the DB or not, use a KV store or not
- * If you can't measure it, you can't scale it - Emacs, Vi(m), TextMate - the age old debates continue - Mongo is faster than, Redis; MySQL ** First Measure factors _outside_ your codebase - People forget that there are costs and scaling issues that start outside the application? How many developers do you have? What's your feature backlog. ** Next: Measure what's in your app - New Relic, Nagios, Splunk, Log Files, CollectD ** Get Help - There are tools to help you measure: New Relic being one of the
- ** Without Metrics You scale the wrong things *** What should you scale? - Good question - Scale available time - Scale developer happiness - Scale client agility - Flexibility *** Scaling is bi-directional - Scaling means going UP AND DOWN! Not just up. - Consider seasonal traffic patterns. - Scaling
- *** What should you scale? - Good question - Scale available time - Scale developer happiness - Scale client agility - Flexibility
- *** Don’t scale the following - Computing time is cheap, don’t save it - Performance tweaks that don’t scale across the network - Infrastructure is cheap, don’t save it
- ** Corollary: Premature scaling is the root of a lot of evil - Not just talking infrastructure, consider adding too many people to a team before the team is ready - Adding too many features before the software is tested - Forrest for the trees
- *** Security - Often over looked, but consider what your attack surface looks like on 10 servers... how about 100 - 1000? - Are you *sure* that virtual disk volume disappeared? Where does the bit bucket on a cloud go? - Security *MUST* be automated and Automatic - Assume that everyone can see everything and everything will be fine...
- - Security *MUST* be automated and Automatic - You are not going to check the logs from 3 servers let alone 300 Tools like - Splunk and Nagios are invaluable. Snort (http://www.snort.org/) - How do you find out what happened on one server out of 30 when that server might go away at any time and destroy the forensics for you?
- - Sometime, somewhere, someone will do something “data stupid” - This includes “forgetting” to sanitize the test data, or posting something they shouldn’t
- *** Security - Encrypt your data at rest
- * Your opinions are nice, but.... here are my ** Rails is opinionated, but infrastructure is unforgiving
- - Stage one mistake: You don’t treat deployment as an extension of your development environment - Second Axiom of Clouds: Undeployed Code is not Code
- - Stage one mistake: You don’t treat deployment as an extension of your development environment - Second Axiom of Clouds: Undeployed Code is not Code
- - Open Technologies help you avoid lock-in
- ** Don't Roll your own - Honestly, just because you can, doesn't mean you should
- - A Broad based rant about not know WTF you’re doing...