Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909
Upcoming SlideShare
Loading in...5
×
 

Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909

on

  • 185 views

 

Statistics

Views

Total Views
185
Views on SlideShare
185
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909 Disaster Recovery in distant DC; people, hardware, latency - PLNOG 200909 Presentation Transcript

  • Disaster Recovery in distant DC; people, hardware, latency.
  • • What is really the Allegro.pl? • Distaster Recovery - true story • Latency - why 20 ms matters • Data Centers topology agenda
  • Allegro ... QXL ? Serwisy Aukcyjne Serwisy Ogłoszeniowe Płatności .pl .cz .ro .ua.bg .hu .sk .lt .pl .pl Sklepy Internetowe View slide
  • Allegro... beginings View slide
  • o Allegro... today
  • o Allegro... today about 1000 servers
  • o Allegro... today about 1000 servers about 300 IT staff
  • o Allegro... today about 1000 servers about 300 IT staff about 40 in infrastructure
  • we are growing Source: http://www.aukcjostat.pl/
  • why bother?
  • Qvo vadis?
  • Qvo vadis?
  • Qvo vadis?
  • Qvo vadis?
  • Qvo vadis?
  • Qvo vadis?
  • 2 hours by plane... ~800 km Go west...
  • 2 hours by plane... ~800 km Go west...
  • WWCKD ?
  • WWCKD ? Hope is the mother of lost battles No guts, no glory. No pain, no game. No risk, no fun.
  • WWCKD ? Hope is the mother of lost battles No guts, no glory. No pain, no game. tiaaaa... No risk, no fun.
  • Building DC3
  • Building DC3
  • DC net topology
  • DC net topology ISP 1 ISP 2 DC2 , Poznań
  • DC net topology ISP 1 ISP 2 ISP 1 ISP 2 DC2 , Poznań DC3 , Frankfurt
  • DC net topology ISP 1 ISP 2 ISP 1 ISP 2 DC2 , Poznań DC3 , Frankfurt 2G, EoMPLS
  • DC net topology ISP 1 ISP 2 ISP 1 ISP 2 DC2 , Poznań DC3 , Frankfurt 2G, EoMPLS 10G, DWDM
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS DNS $ host -t ns allegro.pl allegro.pl name server gtmdc3.allegro.pl. allegro.pl name server gtmdc2.allegro.pl.
  • Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS request
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS response
  • Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS req: www.allegro.pl HTTP traffic
  • Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DNS req: www.allegro.pl DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl
  • Global Traffic Manager DC2 , Poznań DC3 , Frankfurt www.allegro.pl F5 LTM F5 LTM F5 LTM F5 LTM F5 GTM F5 GTM F5 GTM F5 GTM www.allegro.pl DNS req: www.allegro.pl HTTP traffic
  • Latency vs tennis
  • SYN SYN/ACK ACK TCP/IP explained FIN ACK ACK DATA DATA ACK Control packets Data packets bandwidth RTT / latency window size = 64 kBytes Bandwidth-delay Product BDP (bytes) = bandwidth (KBytes/sec) x RTT (ms)
  • 0,0005sec SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,0005sec 0,0005sec ACK DATA DATA ACK 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec Control packets Data packets
  • 0,0005sec SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,0005sec 0,0005sec ACK DATA DATA ACK 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec Control packets Data packets Scenario #1 • bandwidth = 10Gbit/sec • RTT / latency = 1 ms (1/1000 sec) • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B?
  • 0,0005sec SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,0005sec 0,0005sec ACK DATA DATA ACK 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec 0,0005sec Control packets Data packets Scenario #1 • bandwidth = 10Gbit/sec • RTT / latency = 1 ms (1/1000 sec) • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B? 1 sec !!! (640 000 kBytes) / (64 kBytes/ 1ms) = 1000 ms
  • 0,5sek SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,5sek 0,5sek ACK DATA DATA ACK 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek Control packets Data packets
  • 0,5sek SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,5sek 0,5sek ACK DATA DATA ACK 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek Control packets Data packets Scenario #2 • bandwidth = 10Gbit/sec • RTT / latency = 1 sek • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B?
  • 0,5sek SYN SYN/ACK ACK TCP/IP explained FIN ACK 0,5sek 0,5sek ACK DATA DATA ACK 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek 0,5sek Control packets Data packets Scenario #2 • bandwidth = 10Gbit/sec • RTT / latency = 1 sek • window size = 64 kBytes • data to transfer = 640 000 kBytes (1CD) What is the time to deliver CD from point A to B? 1000 sec !!! (640 000 kBytes) / (64 kBytes/1000ms) = 1000 sec
  • 800km distance...
  • 800km distance... storage data base application / transactions AIX Oracle MySQLbackend daemons file transfers server deployment Linux
  • 800km distance... just 20ms...
  • co-operation
  • co-operation developers
  • co-operation developers infrastructure
  • co-operation developers infrastructure
  • co-operation developers infrastructure
  • Recipe for success Disaster recovery ingredients
  • Recipe for success
  • Recipe for success
  • Recipe for success
  • Recipe for success
  • Thank You, Q&A