AJAX for Scalability

AJAX for ScalabilityErik SchultinkPHP Barcelona – 30 Oct, 2009

AJAX
What is AJAX?
“Asynchronous JavaScript and XML”
Paradigm for client-server interaction
Change state on client, without loading a complete HTML page

Traditional HTML Browsing
User clicks link
Browser sends request
Server receives, parses request, generates response
Browser receives response and begins rendering
Dependent objects (images, js, css) load and render
Page appears

AJAX Browsing
User clicks link
Browser sends request
Server receives, parses request, generates response
Browser receives response and begins rendering
Dependent objects (images, js, css) load and render
Page appears

Why Asynchronous?
Not always necessary to wait for server response
Most of response is predictable

Scalability?
Through-put, not speed
Ideally, capacity is linear function of machines
In practice, linear below some boundary

What is Tuenti.com?

Tuenti.com
Started 2007
More than 20 billion pageviews/month
More than 20k/second at peak
Based in Madrid
~80 employees, 40 engineers

How does Tuenti use AJAX?
Only pageloads are login and home page
Loader pulls in all JS/CSS
Afterwards stay within one HTML page, rotating canvas area content

PHOTO UPLOAD

CHAT

Traditional Systems Architecture
www.tuenti.com
Load Balancer
Web server farm
Web server farm
Web server farm

Traditional Systems Architecture
www.tuenti.com
12.45.34.179
12.45.34.178
Load Balancer
Load Balancer
Web server farm
Web server farm
Web server farm
Web server farm

Client-side Routing
www.tuenti.com
wwwb3.tuenti.com
wwwb2.tuenti.com
wwwb1.tuenti.com
wwwb4.tuenti.com
Load Balancer
Load Balancer
Load Balancer
Load Balancer
Web server farm
Web server farm
Web server farm
Web server farm
Linearly scalable …

Client-side Routing
www.tuenti.com
wwwb3.tuenti.com
wwwb2.tuenti.com
wwwb1.tuenti.com
wwwb4.tuenti.com
Load Balancer
Load Balancer
Load Balancer
Load Balancer
Web server farm
Web server farm
Web server farm
Web server farm
Linearly scalable … except for top level

Client-side Routing
www.tuenti.com
wwwb3.tuenti.com
wwwb2.tuenti.com
wwwb1.tuenti.com
wwwb4.tuenti.com
Load Balancer
Load Balancer
Load Balancer
Load Balancer
Web server farm
Web server farm
Web server farm
Web server farm
lots of content creation
= lots of dynamic data

Client-side Routing
www.tuenti.com
wwwb3.tuenti.com
wwwb2.tuenti.com
wwwb1.tuenti.com
wwwb4.tuenti.com
Load Balancer
Load Balancer
Load Balancer
Load Balancer
Web server farm
Web server farm
Web server farm
Web server farm
Cache Farm
lots of dynamic data
= lots of cache
= internal network traffic

Client-side Routing
www.tuenti.com
wwwb3.tuenti.com
wwwb2.tuenti.com
wwwb1.tuenti.com
wwwb4.tuenti.com
Load Balancer
Load Balancer
Load Balancer
Load Balancer
Web server farm
Web server farm
Web server farm
Web server farm
Cache Farm
Cache Farm
Cache Farm
Cache Farm
Partition cache
Route requests to a farm near cache needed to respond

Internal network savings

Balancing Load
Top-level requests to www.tuenti.com
Each request tells client which farm it should be using, based on a mapping
Mapping can be changed to balance load, perform maintenance, etc

Robust Browsing
Don’t tightly couple layers
Client and server are separate layers

Robust Browsing
Client-side code should be robust against server-side failures
What can go wrong?
Client loses connectivity
Server-side error
Web server overloaded
Entire server farm overloaded
Server farm loses connectivity

Robust Browsing
Options:
Retry (carefully)
Switch to alternate farm (carefully)
Detect error, give user feedback

Caching
Think like server-side scalability
Cache data that is likely to be requested in the future
Doesn’t have consistency issues
Doesn’t change

Cache Data in Client Browser
More responsive for user
Fewer requests to your servers

Some details
Lots of dependencies to load initially
Loading can be brittle and slow
Browser performance

Some details
Beware of external libraries
Google JS lib are not static, not indefinitely cacheable

Some details
Friends do have consistency issues
Add/remove friends
Change avatar (rare)

Image Serving
Tuenti serves ~2.5 billion images/day
At peak, this is >6 Gbps and >70k hits/sec
We use CDNs

What is a CDN?
Content Delivery Network

What is a CDN?
Examples: Akamai, Limelight
also dozens more, including Amazon
Big distributed, object cache
Pay per use
either per request, per TB transfer, or per peak Mbps

What is a CDN?
Advantages:
Outsource dev and infrastructure
Geographically distributed
Economies of scale
Disadvantages:
High cost
Less control and transparency
Commitments

What affects image load time?
Client internet connection
Response time of CDN
CDN cache hit rate
Response time of Origin

Monitor Performance from Client
Closer to performance experienced by end-user
Only way to get view of network issues faced by users (ie last mile)

Good Monitoring
Percentiles > averages
Less noise, more stable
Comparable to SLA terms
Establish baselines
performance relative to past more important than absolute
Avoid false positives

How to fix slow ISP?
Choose better transit provider
Set-up peering (or get CDN too)
Traffic management

CDN cache hit rate

Quality of End-User Experience
vs.
Cost

We use multiple CDNs, and shift content based on price/performance.

Know your content

Know your content
30
75
200

Know your content
600

Know your content
120

Know your content

Pre-fetch Content
Exploit predictable user behavior
Ex: clicking to next photo in an album
Simple solution – load next image hidden
Client browser will cache it (next response < 100 ms)
Increase tolerance for slow response time

Pre-fetch Content
More complex solution
Pre-fetch next canvas (full html), render in background – rotate in on Next
Even more complex
Instantiate HTML template w/ data on client
Pre-fetch data X photos in advance, render Y templates in advance with this data

Pre-fetch Content
Problems:
Rendering still takes time
Increases browser load
Need to set cache headers correctly

Image delivery
Small images: High request, low volume
Most cost-effective to cache in memory
Large images: High volume, low requests, greater tolerance for latency

CDN cache hit rate

Monitor Performance from Client
cold servers online

CDN cache hit rate

Deploy cache layers (squid, varnish)
Optimize web servers for static content delivery
More machines
Avoid RAID, file system
Improve Origin Response

Summary
Cache and pre-fetch content
Client-side routing
Client-side monitoring

General caveats
Be careful with browser performance
Look carefully at what’s in the worst 5%
Don’t assume that high values are incorrect

More
jobs.tuenti.com
Presentation
Saturday 14:20
Challenge (win a PS3)
Talk: Continuous Integration
DavideMendolia, Saturday 12:00

More? Ask us!

http://elblogdetuenti.blogspot.com	516
http://blog.tuenti.com	514
http://hoctudau.com	58
http://feeds.feedburner.com	21
http://kartones.net	12
http://www.slideshare.net	10
http://www.elblogdetuenti.blogspot.com	3
http://translate.googleusercontent.com	2
http://webcache.googleusercontent.com	1
http://www.hoctudau.com	1

AJAX for Scalability

by Tuenti on Nov 09, 2009

Accessibility

Upload Details

Usage Rights

10 Embeds 1,138

Statistics

AJAX for Scalability Presentation Transcript