5 Bare Minimum Things a Web startup CTO MUST worry about Indus Khaitan http://khaitan.org [email_address] Twitter: 1ndus *...
What are these? <ul><li>Security </li></ul><ul><li>Availability & Monitoring </li></ul><ul><li>Application Errors </li></u...
Security Threats <ul><li>Your website taken over </li></ul><ul><li>Your database taken over </li></ul><ul><li>Your server ...
Prevention of Security Threats <ul><li>Keep your stack up-to-date. Patch. </li></ul><ul><li>Establish security-aware codin...
Simple TODO List for You <ul><li>Use logwatch and monitor your logs </li></ul><ul><li>Make your Database access local (spe...
A log snapshot of SSH attack <ul><li>Didn't receive an ident from these IPs: </li></ul><ul><li>114.200.199.144: 1 Time(s) ...
Availability & Monitoring <ul><li>Website, Database, SMTP, DNS were down (now up!) </li></ul><ul><li>Poor site performance...
Monitoring – External sample
Monitoring: Internal System Level Monitoring with Nagios
Simple TODO List for You <ul><li>Do some basic external monitoring  </li></ul><ul><ul><li>Zoho does url/5minutes at $4/mon...
Application Errors <ul><li>Bad Code </li></ul><ul><ul><li>function validate($key) { </li></ul></ul><ul><ul><li>global $web...
Application Errors <ul><li>Simple WARNINGS/FATALs lead to bigger problems </li></ul><ul><ul><li>eg. INSERT failed because ...
Simple TODO for You <ul><li>Use a logger like log4j/log4PHP </li></ul><ul><ul><li>Modify the handler to send a real-time e...
Backup <ul><li>Backup before disaster strikes </li></ul><ul><li>Database backups </li></ul><ul><ul><li>Do a dry run of rec...
Simple TODO For You <ul><li>(mysql) Use a slave for a consistent backup. No slave? Then Lock the master before dumping </l...
Source Control: Simple TODO For You <ul><li>Use SVN </li></ul><ul><ul><li>Use hosted… DevGuard..$7/month..cheap! </li></ul...
Summary <ul><li>Know Your Logs! </li></ul><ul><li>Be Security aware </li></ul><ul><ul><li>Lock your SSH. Close Open Ports ...
Sample Advanced Topics & Thanks! <ul><li>Incremental backups, snapshots </li></ul><ul><li>Monitoring Apache Processes, Apa...
Upcoming SlideShare
Loading in …5
×

5 Bare Minimum Things A Web Startup CTO Must Worry About

3,261
-1

Published on

So you have started-it-up and now you are getting good traffic — Thousands of users, etc. etc.
Do you know script kiddies are scanning your website using simple dictionary attacks on SSH ports? Do you know that once in a while there is a Fatal application Error in your PHP log (which may point to bigger problem)? Do you know that the backup you are taking is actually not gonna restore your DB? Do you know that every night at 12 one of the servers has a CPU spike?
It’s a good idea to catch some of the serious problems early on and deploy tools to proactively assess them. In this session we will discuss some very basic things, as a CTO you MUST worry about and proactively solve problems around them.
These are (in the order of decreasing priority):
1. Security
2. Monitoring/Availability/Load (External/System level)
3. Application errors
4. Backup
5. Source control

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,261
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
38
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • 1. How many of you use Windows for your web startup? Hmm..This preso may sound Kanglish to you.
  • 5 Bare Minimum Things A Web Startup CTO Must Worry About

    1. 1. 5 Bare Minimum Things a Web startup CTO MUST worry about Indus Khaitan http://khaitan.org [email_address] Twitter: 1ndus *Not affiliated to any software vendors mentioned in this preso … and implement few basic things to have a good night’s sleep! 18 slides
    2. 2. What are these? <ul><li>Security </li></ul><ul><li>Availability & Monitoring </li></ul><ul><li>Application Errors </li></ul><ul><li>Backup </li></ul><ul><li>Source Control </li></ul>(in order of decreasing priority)
    3. 3. Security Threats <ul><li>Your website taken over </li></ul><ul><li>Your database taken over </li></ul><ul><li>Your server taken over </li></ul><ul><li>(Distributed) Denial of Service </li></ul>
    4. 4. Prevention of Security Threats <ul><li>Keep your stack up-to-date. Patch. </li></ul><ul><li>Establish security-aware coding practice </li></ul><ul><li>Know your Logs! </li></ul><ul><li>Install open source packages for preventive/reactive treatments </li></ul><ul><li>Get a hardware firewall (if you are popular and have money) </li></ul><ul><li>… Subscribe to Securityfocus alerts </li></ul>
    5. 5. Simple TODO List for You <ul><li>Use logwatch and monitor your logs </li></ul><ul><li>Make your Database access local (specific IPs only) </li></ul><ul><li>Secure your sshd </li></ul><ul><ul><li>Password-less login, non-default port, no root login </li></ul></ul><ul><li>Use denyhosts to block dictionary SSH attacks (iptables/netfilter is a good bet, I haven’t tried it) </li></ul><ul><li>Close all ports except SSH, HTTP/HTTPS </li></ul><ul><ul><li>Use nmap to see what “hackers” see! </li></ul></ul>
    6. 6. A log snapshot of SSH attack <ul><li>Didn't receive an ident from these IPs: </li></ul><ul><li>114.200.199.144: 1 Time(s) </li></ul><ul><li>Illegal users from: </li></ul><ul><li>114.200.199.144: 6 times </li></ul><ul><li>alias/password: 1 time </li></ul><ul><li>office/password: 1 time </li></ul><ul><li>recruit/password: 1 time </li></ul><ul><li>sales/password: 1 time </li></ul><ul><li>samba/password: 1 time </li></ul><ul><li>staff/password: 1 time </li></ul><ul><li>Failed logins from: </li></ul><ul><li>211.60.15.30: 1 time </li></ul><ul><li>root/password: 1 time </li></ul><ul><li>219.137.24.12: 1 time </li></ul><ul><li>root/password: 1 time </li></ul>
    7. 7. Availability & Monitoring <ul><li>Website, Database, SMTP, DNS were down (now up!) </li></ul><ul><li>Poor site performance </li></ul><ul><ul><li>Application, Network, or hosting provider? </li></ul></ul><ul><li>CPU, Disk, IO, Memory, Network Interface </li></ul><ul><li>Server down != website down. Put a load balancer </li></ul>
    8. 8. Monitoring – External sample
    9. 9. Monitoring: Internal System Level Monitoring with Nagios
    10. 10. Simple TODO List for You <ul><li>Do some basic external monitoring </li></ul><ul><ul><li>Zoho does url/5minutes at $4/month...cheap! </li></ul></ul><ul><li>Get Nagios for system monitoring </li></ul><ul><li>Use Load Balancer to prevent single server failure </li></ul><ul><ul><li>HTTP, Load Balanced database reads </li></ul></ul>
    11. 11. Application Errors <ul><li>Bad Code </li></ul><ul><ul><li>function validate($key) { </li></ul></ul><ul><ul><li>global $weblog ; </li></ul></ul><ul><ul><li>if (empty($key)) { </li></ul></ul><ul><ul><li>$errorlog->error( &quot;Error : In function validate site key&quot;); </li></ul></ul><ul><ul><li>return FALSE; </li></ul></ul><ul><ul><li>}else{ </li></ul></ul><ul><ul><li>return TRUE; </li></ul></ul><ul><ul><li>} </li></ul></ul><ul><ul><li>} </li></ul></ul><ul><li>Leads to this in phperror log </li></ul><ul><li>[13-Feb-2009 09:41:32] PHP Fatal error: Call to a member function error() on a non-object in /home/padmin/public_html/util/functions.php on line 4 </li></ul>
    12. 12. Application Errors <ul><li>Simple WARNINGS/FATALs lead to bigger problems </li></ul><ul><ul><li>eg. INSERT failed because of duplicate key (was always inserting 0 for the parameter!) </li></ul></ul><ul><li>Apache error_log may show wrong configuration </li></ul><ul><li>Database logs may show a crash (and auto-recovery!) </li></ul>
    13. 13. Simple TODO for You <ul><li>Use a logger like log4j/log4PHP </li></ul><ul><ul><li>Modify the handler to send a real-time email of a desired error level </li></ul></ul><ul><li>Look for Database Error logs, Apache error logs – They will tell you a story! </li></ul><ul><li>Borrow from Security: Use logwatch package </li></ul><ul><li>Review your own application codebase </li></ul>
    14. 14. Backup <ul><li>Backup before disaster strikes </li></ul><ul><li>Database backups </li></ul><ul><ul><li>Do a dry run of recovery at least once </li></ul></ul><ul><ul><li>Ensure consistent, online backups </li></ul></ul><ul><li>Backup your production directories </li></ul>
    15. 15. Simple TODO For You <ul><li>(mysql) Use a slave for a consistent backup. No slave? Then Lock the master before dumping </li></ul><ul><li>Take a backup tar of production </li></ul><ul><ul><li>Preferably backed up every week, and just before a deployment and just after a deployment </li></ul></ul><ul><li>Use S3 to store the files remotely </li></ul>
    16. 16. Source Control: Simple TODO For You <ul><li>Use SVN </li></ul><ul><ul><li>Use hosted… DevGuard..$7/month..cheap! </li></ul></ul><ul><li>Few Developers? Can’t do Linux? No money? Use a local SVN server on Windows. Woorrks! But back-it-up!!! </li></ul><ul><li>Have a prod. deployment strategy </li></ul><ul><ul><li>From SVN, DON’T deploy directly on Prod., use a separate instance and then scp/rsync over </li></ul></ul>
    17. 17. Summary <ul><li>Know Your Logs! </li></ul><ul><li>Be Security aware </li></ul><ul><ul><li>Lock your SSH. Close Open Ports </li></ul></ul><ul><li>Do some basic external monitoring </li></ul><ul><li>Backup your Database & prod directory onto a remote location </li></ul><ul><li>Use SVN </li></ul>
    18. 18. Sample Advanced Topics & Thanks! <ul><li>Incremental backups, snapshots </li></ul><ul><li>Monitoring Apache Processes, Apache IO, Database connections, Load, Query/sec </li></ul><ul><li>Using SSH Tunneling </li></ul><ul><li>Virtual Private & Public LANs </li></ul><ul><li>VPN </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×