Startup Scalability Strategies Frank Mashraqi

Hello!

Agenda Keeping score: What to measure? Discerning the difference: What to focus on? Finding your path: Which way to go? Choosing your architecture: How to partition? Walking the line: How to balance? Building your team: Who to hire? Thinking ahead: What about the future? Offloading scalability: Is it for me?

Keeping score: What to measure?

Discerning the difference: What to focus on?

Finding your path: Which way to go?

Choosing your architecture: How to partition?

Walking the line: How to balance?

Building your team: Who to hire?

Thinking ahead: What about the future?

Offloading scalability: Is it for me?

Keeping Score: What to Measure? client response time memory utilization throughput cache utilization reads per second thread thrashing disk utilization disk response time failure rates resource utilization IO wait swap utilization cache prunes threshold exceptions exceeding high or low water marks cache hit ratio writes per second threads created transactions per second connections usage distribution of data disk saturation locking statistics growth rate queries per shard threads running CPU Saturation memory/ IO contention connections per shard

cache prunes

cache hit ratio

Performance != High Availability != Scalability

Performance !=

High Availability !=

Scalability

Performance Ability to process or execute a task compared to time and resources used

Performance

High Availability Ability of a system to ensure a certain degree of operational continuity flickr.com/photos/mag3737

High Availability

Scalability Ability to handle growing amounts of traffic in a graceful manner or ability to be readily enlarged : freefoto.ca/key/viewpoint?g2_itemId=7348

Scalability

Pick any two!! C onsistency A vailability P artition-Tolerance Choose if you don’t care about 24/7 availability or accommodating high traffic. Choose for a site that must be available 24/7 Choose for a high traffic website

C onsistency

A vailability

P artition-Tolerance

Choose if you don’t care about 24/7 availability or accommodating high traffic.

Choose for a site that must be available 24/7

Choose for a high traffic website

Vertical or Horizontal? Vertical Aka Scaling up Adding resources to a node Getting a bigger server Using faster CPUs Twice as fast servers can be more than twice expensive Horizontal Aka Scaling out Adding more nodes Cost efficient Commodity hardware increased management complexity “ more complex” programming model Right foundation Throughput and latency between nodes

Vertical

Aka Scaling up

Adding resources to a node

Getting a bigger server

Using faster CPUs

Twice as fast servers can be more than twice expensive

Horizontal

Aka Scaling out

Adding more nodes

Cost efficient

Commodity hardware

increased management complexity

“ more complex” programming model

Right foundation

Throughput and latency between nodes

How to Partition? Functional Partitioning Key based partitioning (users ending in 01 go to server 1) Range based partitioning (records ranging from 2M to 4M go to server 8) Directory server based partitioning (no pre-defined partitioning scheme, instead a lookup is required)

Functional Partitioning

Key based partitioning

(users ending in 01 go to server 1)

Range based partitioning

(records ranging from 2M to 4M go to server 8)

Directory server based partitioning

(no pre-defined partitioning scheme, instead a lookup is required)

How to balance? Balance is easier if the foundation is right Use agile methodologies Technical debt is expensive Technical mortgage is a KILLER!

Balance is easier if the foundation is right

Use agile methodologies

Technical debt is expensive

Technical mortgage is a KILLER!

Before and After What to do before you get big? Lay the right foundation Ability to Shard / Partition Decouple components Effectively cache Have a plan in place What can wait after things start to grow? Now focus on micro optimizations Acquiring and upgrading hardware Performance optimizations and OS tuning Implementing High Availability and Disaster recovery Use CDN

What to do before you get big?

Lay the right foundation

Ability to Shard / Partition

Decouple components

Effectively cache

Have a plan in place

What can wait after things start to grow?

Now focus on micro optimizations

Acquiring and upgrading hardware

Performance optimizations and OS tuning

Implementing High Availability and Disaster recovery

Use CDN

What skills / hires are most crucial to dealing with scalability?

Best Practices Go Asynchronous Go Stateless “ Best IO is No IO” Cache effectively using shared cache and monitor utilization Decouple as much as possible Build using APIs Easy to scale development and deployment and open up your service Virtualize/Abstract everything

Go Asynchronous

Go Stateless

“ Best IO is No IO”

Cache effectively using shared cache and monitor utilization

Decouple as much as possible

Build using APIs

Easy to scale development and deployment and open up your service

Virtualize/Abstract everything

How to blow up?

Can scalability be outsourced? aka Can Cloud fix Twitter? Amazon Google AppEngine Rackspace AppNexus 10Gen Other providers?

Amazon

Google AppEngine

Rackspace

AppNexus

10Gen

Other providers?

Things to take away Focus on scalability, the rest will follow Horizontal is better, Vertical is costly Go Asynchronous Architect so you don’t have to re-architect Choose two out of Consistency, Availability and Partition-Tolerance Measure utilization first, then performance Choose the right infrastructure & invest in right skills

Focus on scalability, the rest will follow

Horizontal is better, Vertical is costly

Go Asynchronous

Architect so you don’t have to re-architect

Choose two out of Consistency, Availability and Partition-Tolerance

Measure utilization first, then performance

Choose the right infrastructure & invest in right skills

Appendix Notes/Tips: http://mashraqi.com/2008/09/startonomics-startup-scalability.html Personal blog: http://mashraqi.com Twitter: http://twitter.com/mashraqi MySQL Blog: http://mysqldatabaseadministration.blogspot.com Email: fmashraqi@yahoo.com

Notes/Tips: http://mashraqi.com/2008/09/startonomics-startup-scalability.html

Personal blog: http://mashraqi.com

Twitter: http://twitter.com/mashraqi

MySQL Blog: http://mysqldatabaseadministration.blogspot.com

Email: fmashraqi@yahoo.com