Akamai in Hyperconnected World
Filip Walkowicz – Site Leader & Senior Customer Care Manager
Paweł Kuśmierski – Senior Engineer, Lead, System Operations
Akamai Krakow Center of Excellence
What’s Akamai?
 Founded at MIT in 1998 by prof. Tom Leighton and Danny Lewin
 Akamai has the world’s most distributed Internet platform (over
150.000 servers, deployed in 81 countries)
 The Akamai Intelligent Platform is leading cloud platform delivering up
to 30% of the worldwide web traffic.
 Accelerating Daily Traffic of:

10+ Tbps

20+ million hits per second

2+ trillion deliveries per day

30+ petabytes/day

10+ million concurrent streams
©2013 AKAMAI | FASTER FORWARDTM
Who do we serve?








The top 30 media & entertainment companies
All 20 top global eCommerce sites
7 of the top 10 world banks
9 of the top 10 largest newspapers
9 out of 10 top social media sites
6 of the top 7 computer manufacturers
All of the top anti-virus companies

©2013 AKAMAI | FASTER FORWARDTM
Representative Customers
Adobe
All Nippon Airways (ANA)
Apple (1999)
AUDI
BMW
Department of Defense
Facebook
FedEx
L'Oréal
Marks & Spencer
MTV Networks
MySpace
Microsoft
Netflix

General Motors
Harley-Davidson, Inc
Hitachi
IBM
Intercontinental Hotels Group
Qantas Airways
Red Hat Inc.
Research In Motion (RIM)
Sony Computer Entertainment
Sony Ericsson Mobile Communications
Toyota Motor Sales
TrendMicro
Victoria's Secret
Yahoo!

©2013 AKAMAI | FASTER FORWARDTM
Explosive Growth Online

2012

2020

More People 2.2 Billion1

5 Billion2

More
Devices

8 Billion3

50 Billion4

More Money

€600 Billion1

€2800 Billion2

Sources: (Global) 1 Internet World Stats 2 Akamai Estimates 3 IMS Research 4 Ericsson

©2013 AKAMAI | FASTER FORWARDTM
Trends of mobile devices usage voice/data

View on 2007- 2012 towards data usage
©2013 AKAMAI | FASTER FORWARDTM
Leveraging Key Megatrends

Media

Mobile

Cloud

More than 90% of
Internet traffic will
be video by 20151

10:1 devices per
internet user by
20202

One-third of all
data will live in or
pass through the
cloud by 20201

Sources: 1-Cisco, 2-Akamai 3-IAB, 4-McAfee

©2013 AKAMAI | FASTER FORWARDTM

Big Data
2.5 exabytes of
data are created
every day3

Security
Cyber attacks
cost companies
up to $100,000/min4
10’000 feet view of Akamai

©2013 AKAMAI | FASTER FORWARDTM
Challenges with Cloud Adoption
Akamai Cloud Optimization
The User Always Connects to a Nearby Akamai Server
Akamai Edge Servers
Cloud Datacenter

End User

Cloud servers reside in big
data centers, farther away
from the end user…
...resulting in decreased
performance and security
©2013 AKAMAI | FASTER FORWARDTM
Cloud Optimization: Route Selection
Cloud Datacenter

X

End User

X
Problem 1

Route to datacenter
may perform poorly

©2013 AKAMAI | FASTER FORWARDTM
Cloud Optimization: Route Selection
Akamai Edge Servers
Cloud Datacenter

End User

X
Solution

Problem 1

Route to datacenter
may perform poorly

Akamai SureRoute
to optimize route
©2013 AKAMAI | FASTER FORWARDTM
Akamai SureRoute Makes a Big Difference
Packet loss into India after MidEast cable cut

Packet Loss

50%
40%

30%
20%

10%
0%

Generic Internet
©2013 AKAMAI | FASTER FORWARDTM

Akamai
Cloud Optimization: Communication Protocol
Akamai Edge Servers
Cloud Datacenter

End User

Problem 2

Solution

Akamai Communication
Protocol

Many round trips for
initial large download
©2013 AKAMAI | FASTER FORWARDTM
Attacks on Akamai Customers
600

• Typical Attack Size: 3-10 Gbps

500

• Attacks are originating from all
geographies and are moving between
geographies during the attack

Number of Attacks

• Large Attack Size: 100-200 Gbps

400

300

200

100

0
2009

©2013 AKAMAI | FASTER FORWARDTM

2010

2011
The Threat is Varied & Easier to Launch

Attack Methods

Malware
1%

74% of companies
experienced one or more
DDoS attacks in the past year.

31% of these attacks
resulted in service disruption.
New attack tools such as
Low Orbit Ion Cannon
Users download the tool, insert
the target URL or IP and press
GO!

Insufficient Authentication
1%
DNS Hijacking
1%
Abuse of Functionality
1%
Content Spoofing
1%

OS Commanding
1%

Unknown
10%
Denial of
Service
(DoS)
32%

Predictable Resource Location
1%
Banking Trojan
1%
Stolen Credentials
1%
Misconfiguration
3%
Known Vulnerability
4%
Process Automation
4%
Cross-Site Request Forgery
(CSRF)
4%

Brute Force
4%

Cross-Site
Scripting
(XSS)
9%

SQL Injection
(SQLi)
21%

Source: TrustWave - 2010 - Web Hacking Incident Database

©2013 AKAMAI | FASTER FORWARDTM
Web Application With a Perimeter Defense

(Cloud)
Datacenters

End User
Origin
Traffic

Akamai
Traffic

10000

10000

1000

1000

100

100

10

10

1

1

COVERED

©2013 AKAMAI | FASTER FORWARDTM
July 4th – 7th 2009 DDoS Attack
400,000 Korean Bots Attack Key U.S. Government Web Sites

Peak Traffic

Times Above
Normal Traffic

U.S. Government Customer 1

124 Gbps

598x

U.S. Government Customer 2

32 Gbps

369x

U.S. Government Customer 3

9 Gbps

39x

U.S. Government Customer 4

9 Gbps

19x

U.S. Government Customer 5

2 Gbps

9x

U.S. Government Customer 6

1.9 Gbps

6x

Customer – PROTECTED

©2013 AKAMAI | FASTER FORWARDTM
July 4th – 7th 2009 DDoS Attack
400,000 Korean Bots Attack Key U.S. Government Web Sites

Attack Size — Gbps

125

23:00 Mitigation
measures engaged

100

July 5, 2009

Unique IPs
Spike 1

21:00 Akamai identifies
sources

75

20:00 Attack grows
rapidly

50

16:00 Customer
notified

25

0:00

23:50 Peak
pageviews

Spike 2

08:00

16:00

0:00

©2013 AKAMAI | FASTER FORWARDTM

08:00

16:00

Spike 3
Under the hood

©2013 AKAMAI | FASTER FORWARDTM
Akamai Platform
•
•
•
•
•
•
•

Various web infrastructure services
Over 150,000 machines
Over 1 million distributed components
Over 1000 autonomous systems
24/7/365 operation
Failures, usage changes
Massive, real-time monitoring

©2013 AKAMAI | FASTER FORWARDTM
Query
• Distributed data collection
• Aggregation at several hundred points
• SQL-style interface

©2013 AKAMAI | FASTER FORWARDTM
A Sample Query
SELECT
c.continent_name,
SUM(l.hits) hits
FROM
load_info l,
region_data r,
continent_data c
WHERE
l.georegion=r.id AND
r.continent=c.continent
GROUP BY
c.continent_name
ORDER BY
hits DESC;

c.continent_name
hits
---------------- --------North America 4,620,551
Europe 3,392,102
South America
655,175
Asia
552,258
Africa
106,781
Oceania
39,905
Antarctica
135

©2013 AKAMAI | FASTER FORWARDTM
Query at the Edge
• Each machine collects its own data
• Many processes may publish
• Snapshots every two minutes

©2013 AKAMAI | FASTER FORWARDTM
Cluster proxies
•
•

Collect data for the whole cluster
Include themselves

©2013 AKAMAI | FASTER FORWARDTM
Top-Level Aggregators
• Collect data for the whole network
• Snapshots every two minutes
• Static tables for data that doesn’t change
much

©2013 AKAMAI | FASTER FORWARDTM
SQL parsers
• Get tables from 1 TLA
• Only get the ones we need
• Answer queries based on
them

©2013 AKAMAI | FASTER FORWARDTM
Aggregator Sets
• Span different parts of the network
• Designated for different purposes
• Several replicated TLAs & SQLs
• Combined TLA/SQLs
• Shared hostnames
• Help meet reliability guarantees
• Help tolerate faults & keep localized

©2013 AKAMAI | FASTER FORWARDTM
Scale
•
•
•
•

Several hundred TLAs, SQLs, TLA/SQLs
Thousands of queries per minute
Tens of GB in the system
Up to 16 GB per TLA (and growing fast)
•
•
•
•
•

Internet usage
Network growth
Customer growth
Data/customer
More queries

• Age of data typically a few minutes

©2013 AKAMAI | FASTER FORWARDTM
Download the Akamai
Internet Visualization
app in the Apple store

Result:

Result:

Result:

2-100X

2-100X

2-100X

compression

compression

compression

©2013 AKAMAI | FASTER FORWARDTM
Dziękujemy za uwagę

Filip Walkowicz, fwalkowi@akamai.com
Paweł Kuśmierski, pkusmier@akamai.com

©2013 AKAMAI | FASTER FORWARDTM

Akamai in a hyperconnected world

  • 1.
    Akamai in HyperconnectedWorld Filip Walkowicz – Site Leader & Senior Customer Care Manager Paweł Kuśmierski – Senior Engineer, Lead, System Operations Akamai Krakow Center of Excellence
  • 2.
    What’s Akamai?  Foundedat MIT in 1998 by prof. Tom Leighton and Danny Lewin  Akamai has the world’s most distributed Internet platform (over 150.000 servers, deployed in 81 countries)  The Akamai Intelligent Platform is leading cloud platform delivering up to 30% of the worldwide web traffic.  Accelerating Daily Traffic of:  10+ Tbps  20+ million hits per second  2+ trillion deliveries per day  30+ petabytes/day  10+ million concurrent streams ©2013 AKAMAI | FASTER FORWARDTM
  • 3.
    Who do weserve?        The top 30 media & entertainment companies All 20 top global eCommerce sites 7 of the top 10 world banks 9 of the top 10 largest newspapers 9 out of 10 top social media sites 6 of the top 7 computer manufacturers All of the top anti-virus companies ©2013 AKAMAI | FASTER FORWARDTM
  • 4.
    Representative Customers Adobe All NipponAirways (ANA) Apple (1999) AUDI BMW Department of Defense Facebook FedEx L'Oréal Marks & Spencer MTV Networks MySpace Microsoft Netflix General Motors Harley-Davidson, Inc Hitachi IBM Intercontinental Hotels Group Qantas Airways Red Hat Inc. Research In Motion (RIM) Sony Computer Entertainment Sony Ericsson Mobile Communications Toyota Motor Sales TrendMicro Victoria's Secret Yahoo! ©2013 AKAMAI | FASTER FORWARDTM
  • 5.
    Explosive Growth Online 2012 2020 MorePeople 2.2 Billion1 5 Billion2 More Devices 8 Billion3 50 Billion4 More Money €600 Billion1 €2800 Billion2 Sources: (Global) 1 Internet World Stats 2 Akamai Estimates 3 IMS Research 4 Ericsson ©2013 AKAMAI | FASTER FORWARDTM
  • 6.
    Trends of mobiledevices usage voice/data View on 2007- 2012 towards data usage ©2013 AKAMAI | FASTER FORWARDTM
  • 7.
    Leveraging Key Megatrends Media Mobile Cloud Morethan 90% of Internet traffic will be video by 20151 10:1 devices per internet user by 20202 One-third of all data will live in or pass through the cloud by 20201 Sources: 1-Cisco, 2-Akamai 3-IAB, 4-McAfee ©2013 AKAMAI | FASTER FORWARDTM Big Data 2.5 exabytes of data are created every day3 Security Cyber attacks cost companies up to $100,000/min4
  • 8.
    10’000 feet viewof Akamai ©2013 AKAMAI | FASTER FORWARDTM
  • 9.
    Challenges with CloudAdoption Akamai Cloud Optimization The User Always Connects to a Nearby Akamai Server Akamai Edge Servers Cloud Datacenter End User Cloud servers reside in big data centers, farther away from the end user… ...resulting in decreased performance and security ©2013 AKAMAI | FASTER FORWARDTM
  • 10.
    Cloud Optimization: RouteSelection Cloud Datacenter X End User X Problem 1 Route to datacenter may perform poorly ©2013 AKAMAI | FASTER FORWARDTM
  • 11.
    Cloud Optimization: RouteSelection Akamai Edge Servers Cloud Datacenter End User X Solution Problem 1 Route to datacenter may perform poorly Akamai SureRoute to optimize route ©2013 AKAMAI | FASTER FORWARDTM
  • 12.
    Akamai SureRoute Makesa Big Difference Packet loss into India after MidEast cable cut Packet Loss 50% 40% 30% 20% 10% 0% Generic Internet ©2013 AKAMAI | FASTER FORWARDTM Akamai
  • 13.
    Cloud Optimization: CommunicationProtocol Akamai Edge Servers Cloud Datacenter End User Problem 2 Solution Akamai Communication Protocol Many round trips for initial large download ©2013 AKAMAI | FASTER FORWARDTM
  • 14.
    Attacks on AkamaiCustomers 600 • Typical Attack Size: 3-10 Gbps 500 • Attacks are originating from all geographies and are moving between geographies during the attack Number of Attacks • Large Attack Size: 100-200 Gbps 400 300 200 100 0 2009 ©2013 AKAMAI | FASTER FORWARDTM 2010 2011
  • 15.
    The Threat isVaried & Easier to Launch Attack Methods Malware 1% 74% of companies experienced one or more DDoS attacks in the past year. 31% of these attacks resulted in service disruption. New attack tools such as Low Orbit Ion Cannon Users download the tool, insert the target URL or IP and press GO! Insufficient Authentication 1% DNS Hijacking 1% Abuse of Functionality 1% Content Spoofing 1% OS Commanding 1% Unknown 10% Denial of Service (DoS) 32% Predictable Resource Location 1% Banking Trojan 1% Stolen Credentials 1% Misconfiguration 3% Known Vulnerability 4% Process Automation 4% Cross-Site Request Forgery (CSRF) 4% Brute Force 4% Cross-Site Scripting (XSS) 9% SQL Injection (SQLi) 21% Source: TrustWave - 2010 - Web Hacking Incident Database ©2013 AKAMAI | FASTER FORWARDTM
  • 16.
    Web Application Witha Perimeter Defense (Cloud) Datacenters End User Origin Traffic Akamai Traffic 10000 10000 1000 1000 100 100 10 10 1 1 COVERED ©2013 AKAMAI | FASTER FORWARDTM
  • 17.
    July 4th –7th 2009 DDoS Attack 400,000 Korean Bots Attack Key U.S. Government Web Sites Peak Traffic Times Above Normal Traffic U.S. Government Customer 1 124 Gbps 598x U.S. Government Customer 2 32 Gbps 369x U.S. Government Customer 3 9 Gbps 39x U.S. Government Customer 4 9 Gbps 19x U.S. Government Customer 5 2 Gbps 9x U.S. Government Customer 6 1.9 Gbps 6x Customer – PROTECTED ©2013 AKAMAI | FASTER FORWARDTM
  • 18.
    July 4th –7th 2009 DDoS Attack 400,000 Korean Bots Attack Key U.S. Government Web Sites Attack Size — Gbps 125 23:00 Mitigation measures engaged 100 July 5, 2009 Unique IPs Spike 1 21:00 Akamai identifies sources 75 20:00 Attack grows rapidly 50 16:00 Customer notified 25 0:00 23:50 Peak pageviews Spike 2 08:00 16:00 0:00 ©2013 AKAMAI | FASTER FORWARDTM 08:00 16:00 Spike 3
  • 19.
    Under the hood ©2013AKAMAI | FASTER FORWARDTM
  • 20.
    Akamai Platform • • • • • • • Various webinfrastructure services Over 150,000 machines Over 1 million distributed components Over 1000 autonomous systems 24/7/365 operation Failures, usage changes Massive, real-time monitoring ©2013 AKAMAI | FASTER FORWARDTM
  • 21.
    Query • Distributed datacollection • Aggregation at several hundred points • SQL-style interface ©2013 AKAMAI | FASTER FORWARDTM
  • 22.
    A Sample Query SELECT c.continent_name, SUM(l.hits)hits FROM load_info l, region_data r, continent_data c WHERE l.georegion=r.id AND r.continent=c.continent GROUP BY c.continent_name ORDER BY hits DESC; c.continent_name hits ---------------- --------North America 4,620,551 Europe 3,392,102 South America 655,175 Asia 552,258 Africa 106,781 Oceania 39,905 Antarctica 135 ©2013 AKAMAI | FASTER FORWARDTM
  • 23.
    Query at theEdge • Each machine collects its own data • Many processes may publish • Snapshots every two minutes ©2013 AKAMAI | FASTER FORWARDTM
  • 24.
    Cluster proxies • • Collect datafor the whole cluster Include themselves ©2013 AKAMAI | FASTER FORWARDTM
  • 25.
    Top-Level Aggregators • Collectdata for the whole network • Snapshots every two minutes • Static tables for data that doesn’t change much ©2013 AKAMAI | FASTER FORWARDTM
  • 26.
    SQL parsers • Gettables from 1 TLA • Only get the ones we need • Answer queries based on them ©2013 AKAMAI | FASTER FORWARDTM
  • 27.
    Aggregator Sets • Spandifferent parts of the network • Designated for different purposes • Several replicated TLAs & SQLs • Combined TLA/SQLs • Shared hostnames • Help meet reliability guarantees • Help tolerate faults & keep localized ©2013 AKAMAI | FASTER FORWARDTM
  • 28.
    Scale • • • • Several hundred TLAs,SQLs, TLA/SQLs Thousands of queries per minute Tens of GB in the system Up to 16 GB per TLA (and growing fast) • • • • • Internet usage Network growth Customer growth Data/customer More queries • Age of data typically a few minutes ©2013 AKAMAI | FASTER FORWARDTM
  • 29.
    Download the Akamai InternetVisualization app in the Apple store Result: Result: Result: 2-100X 2-100X 2-100X compression compression compression ©2013 AKAMAI | FASTER FORWARDTM
  • 30.
    Dziękujemy za uwagę FilipWalkowicz, fwalkowi@akamai.com Paweł Kuśmierski, pkusmier@akamai.com ©2013 AKAMAI | FASTER FORWARDTM