5 Bare Minimum Things A Web Startup CTO Must Worry About

5 Bare Minimum Things a Web startup CTO MUST worry about Indus Khaitan http://khaitan.org [email_address] Twitter: 1ndus *Not affiliated to any software vendors mentioned in this preso … and implement few basic things to have a good night’s sleep! 18 slides

What are these?
Security
Availability & Monitoring
Application Errors
Backup
Source Control
(in order of decreasing priority)

Security Threats
Your website taken over
Your database taken over
Your server taken over
(Distributed) Denial of Service

Prevention of Security Threats
Keep your stack up-to-date. Patch.
Establish security-aware coding practice
Know your Logs!
Install open source packages for preventive/reactive treatments
Get a hardware firewall (if you are popular and have money)
… Subscribe to Securityfocus alerts

Simple TODO List for You
Use logwatch and monitor your logs
Make your Database access local (specific IPs only)
Secure your sshd
Password-less login, non-default port, no root login
Use denyhosts to block dictionary SSH attacks (iptables/netfilter is a good bet, I haven’t tried it)
Close all ports except SSH, HTTP/HTTPS
Use nmap to see what “hackers” see!

A log snapshot of SSH attack
Didn't receive an ident from these IPs:
114.200.199.144: 1 Time(s)
Illegal users from:
114.200.199.144: 6 times
alias/password: 1 time
office/password: 1 time
recruit/password: 1 time
sales/password: 1 time
samba/password: 1 time
staff/password: 1 time
Failed logins from:
211.60.15.30: 1 time
root/password: 1 time
219.137.24.12: 1 time
root/password: 1 time

Availability & Monitoring
Website, Database, SMTP, DNS were down (now up!)
Poor site performance
Application, Network, or hosting provider?
CPU, Disk, IO, Memory, Network Interface
Server down != website down. Put a load balancer

Monitoring – External sample

Monitoring: Internal System Level Monitoring with Nagios

Simple TODO List for You
Do some basic external monitoring
Zoho does url/5minutes at $4/month...cheap!
Get Nagios for system monitoring
Use Load Balancer to prevent single server failure
HTTP, Load Balanced database reads

Application Errors
Bad Code
function validate($key) {
global $weblog ;
if (empty($key)) {
$errorlog->error( "Error : In function validate site key");
return FALSE;
}else{
return TRUE;
}
}
Leads to this in phperror log
[13-Feb-2009 09:41:32] PHP Fatal error: Call to a member function error() on a non-object in /home/padmin/public_html/util/functions.php on line 4

Application Errors
Simple WARNINGS/FATALs lead to bigger problems
eg. INSERT failed because of duplicate key (was always inserting 0 for the parameter!)
Apache error_log may show wrong configuration
Database logs may show a crash (and auto-recovery!)

Simple TODO for You
Use a logger like log4j/log4PHP
Modify the handler to send a real-time email of a desired error level
Look for Database Error logs, Apache error logs – They will tell you a story!
Borrow from Security: Use logwatch package
Review your own application codebase

Backup
Backup before disaster strikes
Database backups
Do a dry run of recovery at least once
Ensure consistent, online backups
Backup your production directories

Simple TODO For You
(mysql) Use a slave for a consistent backup. No slave? Then Lock the master before dumping
Take a backup tar of production
Preferably backed up every week, and just before a deployment and just after a deployment
Use S3 to store the files remotely

Source Control: Simple TODO For You
Use SVN
Use hosted… DevGuard..$7/month..cheap!
Few Developers? Can’t do Linux? No money? Use a local SVN server on Windows. Woorrks! But back-it-up!!!
Have a prod. deployment strategy
From SVN, DON’T deploy directly on Prod., use a separate instance and then scp/rsync over

Summary
Know Your Logs!
Be Security aware
Lock your SSH. Close Open Ports
Do some basic external monitoring
Backup your Database & prod directory onto a remote location
Use SVN

Sample Advanced Topics & Thanks!
Incremental backups, snapshots
Monitoring Apache Processes, Apache IO, Database connections, Load, Query/sec
Using SSH Tunneling
Virtual Private & Public LANs
VPN

http://www.sahilparikh.com	52
http://www.slideshare.net	7
http://www.brijj.com	1

5 Bare Minimum Things A Web Startup CTO Must Worry About

by Indus Khaitan on Mar 05, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 60

Statistics

5 Bare Minimum Things A Web Startup CTO Must Worry About — Presentation Transcript