Why Startups Need Automated Infrastructures

Why Startups Need Automated Infrastructures Adam Jacob Senior Partner HJK Solutions [email_address] .com

Overview <ul><li>What is an “Automated Infrastructure”? </li></ul><ul><li>Why is it so important? </li></ul><ul><li>MonkeyNews - A Case Study </li></ul><ul><li>Review </li></ul><ul><li>Q & A </li></ul>

Disclaimer <ul><li>These are the tools we use </li></ul><ul><li>They are not the only tools </li></ul><ul><li>They may or may not be the right tools for how you work or think </li></ul>Symbol From IIT Bombay

What is an “Automated Infrastructure”?

Automated <ul><li>”…a process which may once have been performed manually but has been altered in some way which allows a machine or computer to either wholly or partially manipulate the process to save time” </li></ul>

Infrastructure <ul><li>“Infrastructure is generally a set of interconnected structural elements that provide the framework supporting an entire structure” </li></ul>

Automated Infrastructure <ul><li>“Having the basic services necessary for your business to operate largely without the aid of a keeper.” </li></ul>

Why is it so important?

To your Operations Staff <ul><li>Mitigates some failures </li></ul><ul><li>They want to be lazy </li></ul><ul><li>Repetition leads to burnout </li></ul><ul><li>They want to say “Yes” </li></ul>

For Startups <ul><li>Time </li></ul><ul><li>Efficiency </li></ul><ul><li>Scalability </li></ul><ul><li>Economics </li></ul><ul><li>Flexibility </li></ul>

The Jesse Robbins (mis)Quote <ul><li>“ Operations is the act of extracting value from your resources.” </li></ul>

MonkeyNews - A Case Study All the hottest Monkey news, as selected by your monkey-loving peers! Courtesy National Geographic

What is MonkeyNews <ul><li>Small Startup </li></ul><ul><li>“ Like Digg, only about Monkeys” </li></ul><ul><li>Going to sell Advertising as a revenue model </li></ul>

MonkeyNews on Launch Day

Steps to Launch

Steps to Launch - OS Install <ul><li>Manual </li></ul><ul><li>Install each system by hand </li></ul><ul><li>1 hour per system </li></ul>

Steps to Launch - OS Install <ul><li>Automated </li></ul><ul><li>Ops1prod built manually </li></ul><ul><ul><li>1 hour </li></ul></ul><ul><li>Automated install system (PXE) </li></ul><ul><li>EC2, Cloud </li></ul>

Steps to Launch - OS Install

Steps to Launch - DNS <ul><li>Manual </li></ul><ul><li>Update each machine in /etc/hosts </li></ul><ul><li>Automated </li></ul><ul><li>Install a DNS Server </li></ul><ul><li>Use a service provider </li></ul>

Steps to Launch - Server Inventory <ul><li>Manual </li></ul><ul><li>Use a wiki </li></ul><ul><li>Automated </li></ul><ul><li>Use iClassify </li></ul><ul><li>Roll your own </li></ul>

Steps to Launch - Identity Management <ul><li>Manual </li></ul><ul><li>Add each user everywhere </li></ul><ul><li>Automated </li></ul><ul><li>Use LDAP or AD </li></ul>

Steps to Launch - Version Control <ul><li>An Easy One </li></ul><ul><li>You must use version control </li></ul>Symbol From IIT Bombay

Steps to Launch - Configuration Management <ul><li>Manual </li></ul><ul><li>Version Control </li></ul><ul><li>Wiki </li></ul><ul><li>Check in copies of config files </li></ul><ul><li>Try and keep a record of what you have done </li></ul>

Steps to Launch - Configuration Management <ul><li>Automated </li></ul><ul><li>Puppet </li></ul><ul><li>Cfengine </li></ul><ul><li>Bcfg2 </li></ul><ul><li>Infrastructure as Code </li></ul>

Puppet Example - sudo

Steps to Launch - Configuration Management <ul><li>Automation is Key </li></ul><ul><li>50+ Different Applications </li></ul><ul><li>ad, apache2, apt-proxy, beaver, build-essential, capistrano, yum, apt, djbdns, emacs, erubis, rubygems, iclassify, imagemagick, iptables, java, logrotate, man, maradns, memcached, mongrel-runit, munin, mysql, nagios, nscd, ntp, openldap, openssh, perl, perlbal, php, postfix, postgresql, puppet, rails, resolver, rsync, ruby, runit, sqlite, subversion, sudo, trac, zsh </li></ul><ul><li>Why repeat yourself? </li></ul>

Steps to Launch - Monitoring <ul><li>Manual </li></ul><ul><li>Add each server by hand </li></ul><ul><li>Automated </li></ul><ul><li>Use your System Inventory and Configuration Management Tools </li></ul>

Steps to Launch - Trending <ul><li>Manual </li></ul><ul><li>Add each server and graph by hand </li></ul><ul><li>Automated </li></ul><ul><li>Use your System Inventory and Configuration Management Tools </li></ul>

Steps to Launch - Email <ul><li>Another Easy One </li></ul><ul><li>Every server must be able to send email locally. </li></ul>Symbol From IIT Bombay

Steps to Launch - Application Deployment <ul><li>Manual </li></ul><ul><li>Update the application by hand </li></ul><ul><li>Automated </li></ul><ul><li>Use Capistrano </li></ul><ul><li>Roll your own </li></ul><ul><li>Integrate with System Inventory </li></ul>

Launch Day

Tech Crunched! <ul><li>Symptoms </li></ul><ul><li>Monitors alert </li></ul><ul><li>Trending shows the traffic spike </li></ul><ul><li>Your RSS reader has the culprit </li></ul>

Tech Crunched! - EC2 <ul><li>Launch more capacity. </li></ul><ul><li>Two minutes later, they are up </li></ul>

Tech Crunched! - iClassify <ul><li>Classify the Servers </li></ul>

Tech Crunched! - Puppet <ul><li>Puppet applies the class that matches the tag: </li></ul>

Tech Crunched! - Deploy <ul><li>Deploy your application! </li></ul><ul><li>“ cap -S env=prod deploy” </li></ul><ul><li>One command. </li></ul>

Tech Crunched! - Post Mortem <ul><li>Total Time: ~5 - 10 minutes </li></ul><ul><li>4 steps total! </li></ul><ul><li>We doubled capacity </li></ul><ul><li>No configuration during the issue at all </li></ul>

New Hire <ul><li>Issue </li></ul><ul><li>MonkeyNews hires a new engineer </li></ul><ul><li>Resolution </li></ul><ul><li>Add the user to LDAP </li></ul><ul><li>Add the right group privileges </li></ul>

Add SSL Support <ul><li>Issue </li></ul><ul><li>MonkeyNews has started taking micro-payments (for bananas) </li></ul><ul><li>Resolution </li></ul><ul><li>Update Apache Virtual Host in your Configuration Management </li></ul><ul><li>Add the SSL Certificates to Configuration Management </li></ul><ul><li>< 10 minutes from start to finish </li></ul>

Review

Review <ul><li>Automated Infrastructure means: </li></ul><ul><ul><li>“ Having the basic services necessary for your business to operate largely without the aid of a keeper.” </li></ul></ul><ul><li>Automated Infrastructure </li></ul><ul><ul><li>Saves time </li></ul></ul><ul><ul><li>Increases efficiency </li></ul></ul><ul><ul><li>Allows for scalability </li></ul></ul><ul><ul><li>Reduces economic impact </li></ul></ul><ul><ul><li>Improves flexibility </li></ul></ul>

HJK does this for a living. But you can ask me how to do it for free. :) Adam Jacob +1 (206) 508-4759 [email_address]

June 23-24, 2008 20% off discount: “vel08js”