Clone Your Network with OpenNebula
Thomas Higdon – Akamai Technologies
©2013 AKAMAI | FASTER FORWARDTM
A little about me
• Software engineer at Akamai in the platform
infrastructure team
• We d...
©2013 AKAMAI | FASTER FORWARDTM
Some statistics
Accelerating Daily Traffic of:
• 10+ Tbps
• 25+ million hits per second
• ...
©2013 AKAMAI | FASTER FORWARDTM
A little about me
• I’m addressing the problem of how to get new
platform software out the...
©2013 AKAMAI | FASTER FORWARDTM
Akamai in a nutshell
• Akamai is a globally-distributed cloud of
physical servers
• Machin...
©2013 AKAMAI | FASTER FORWARDTM
Akamai in a nutshell
• Each network runs a different set of software that
depends on the r...
©2013 AKAMAI | FASTER FORWARDTM
Akamai in a nutshell
Infra networkFile transferMonitoringMessaging
Operating system/kernel...
©2013 AKAMAI | FASTER FORWARDTM
Akamai in a nutshell
Infra networkFile transferMonitoringMessaging
Operating system/kernel...
©2013 AKAMAI | FASTER FORWARDTM
Akamai in a nutshell
Infra networkFile transferMonitoringMessaging
Operating system/kernel...
©2013 AKAMAI | FASTER FORWARDTM
Akamai in a nutshell
• State of a network
• Installed software and configuration
• Dynamic...
©2013 AKAMAI | FASTER FORWARDTM
An Akamai “instance”
• A given unique set of Akamai networks that
function together
• Self...
©2013 AKAMAI | FASTER FORWARDTM
Akamai “instances”
Production – 105
SQA – 102
SQA – 102
SQA – 102
Dev – 102
Dev – 102
Dev ...
©2013 AKAMAI | FASTER FORWARDTM
Commoditizing an Akamai instance
• Instances are labor-intensive to manage
• We’d like the...
©2013 AKAMAI | FASTER FORWARDTM
Commoditizing an Akamai instance
• Solution: virtualization with
OpenNebula
• Represent th...
©2013 AKAMAI | FASTER FORWARDTM
Cloning Akamai instances
• Static IP configuration
• Isolation
• traditional NAT
• “revers...
©2013 AKAMAI | FASTER FORWARDTM
Accessing an instance
NAT
SOCKS
authgate192.168.0.0/16
192.168.0.1
NAT
SOCKS
authgate192.1...
©2013 AKAMAI | FASTER FORWARDTM
Clone a machine
1. Power down VM
2. Clone each disk
3. Add new virtual network with given ...
©2013 AKAMAI | FASTER FORWARDTM
Managing the Akamai side
• OpenNebula is great for managing VMs and
their resources
• We n...
©2013 AKAMAI | FASTER FORWARDTM
An Akamai instance service
• Service-oriented
• Language/test harness agnostic
• Share com...
©2013 AKAMAI | FASTER FORWARDTM
An Akamai instance service
Instance
service
OpenNebula
2. “Clone these machines”
3. Create...
©2013 AKAMAI | FASTER FORWARDTM
A well-managed instance
• Leverage expertise from around the company to
create an internal...
©2013 AKAMAI | FASTER FORWARDTM
Addressing existing instances
• Can also address existing instances of physical
machines
•...
©2013 AKAMAI | FASTER FORWARDTM
Testing at Akamai
Unit testing
Dev QA
SQA
Checklisting
Alerting
©2013 AKAMAI | FASTER FORWARDTM
Unify these stages
Dev QA
SQA
Checklisting
• Use the same code to write tests by using a
c...
©2013 AKAMAI | FASTER FORWARDTM
Thank you!
Upcoming SlideShare
Loading in …5
×

OpenNebulaConf 2013 - Keynote: Clone your Network with OpenNebula by Thomas Higdon

660 views

Published on

Akamai, with a network of well over 100,000 globally distributed servers running custom software for optimizing Internet traffic, tackles many problems relating to scale, management, and software quality. Many different groups within the corporate ecosystem maintain largely disparate stacks of software deployed to overlapping subsets of these servers, with complex and loosely-defined interdependencies between many of these software systems. Describing the complete state of a given system, much less a group of these systems, is often difficult or impossible. This makes it difficult for a given team to “stub out” such systems for outside teams to use in their development and testing efforts. Because we’ve found that the use of well-managed, realistic environments is integral to the eventual success of our software deployments, we’ve maintained multiple test networks that are complete copies of the deployed Akamai network for general use internally.

However, as the company grows in size and scope, maintaining these has become an increasingly difficult burden. As a possible solution, we’ve begun to leverage OpenNebula to build multiple well-maintained instances of the groupings of servers that comprise our network. We’ve created the ability to manage and maintain each of these instances as a single independent unit, and to quickly and easily create perfect clones of these instances in our private OpenNebula cloud. Use of these well-maintained cloned instances can then be distributed to various groups, and even individuals, that require them. Groups can integrate their own machines into these cloned instance units to realize the benefits of virtualization as well as the ability to provide their software as infrastructure for other internal development groups.

The talk will discuss some of the technical details of our solution, how it or similar approaches might fit the needs of other companies like Akamai, and how OpenNebula has been instrumental in its development.

Bio:
Thomas Higdon is a Senior System Software Engineer at Akamai Technologies, Inc. in Cambridge, Massachusetts, USA. His past experience has varied from implementing software radio algorithms for cellular infrastructure to kernel-level software load-balancing for HTTPS. He’s currently focused on leveraging virtualization to optimize the ability of engineers to develop, test and deploy software on Akamai’s globally distributed network. He received his B.S. in Electrical Engineering from Purdue University and his M.S.E. in Computer Engineering from the University of Texas.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
660
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

OpenNebulaConf 2013 - Keynote: Clone your Network with OpenNebula by Thomas Higdon

  1. 1. Clone Your Network with OpenNebula Thomas Higdon – Akamai Technologies
  2. 2. ©2013 AKAMAI | FASTER FORWARDTM A little about me • Software engineer at Akamai in the platform infrastructure team • We develop the software that goes on every Akamai server out there (well over 100,000).
  3. 3. ©2013 AKAMAI | FASTER FORWARDTM Some statistics Accelerating Daily Traffic of: • 10+ Tbps • 25+ million hits per second • 1.5+ trillion deliveries/day • 50+ petabytes/day • 10+ million concurrent streams 15-30%+ of Web Traffic Delivering 250,000+ Domains • 5,000+ Customers • 9 of the top 10 Banks • 9 of the top 10 Social Media Sites • All top 30 M&E Companies • All top 60 eCommerce Sites A Global Platform: • 132,000+ Servers • 86 Countries • 650+ Cities • 1,150+ Networks • 2,200+ Locations
  4. 4. ©2013 AKAMAI | FASTER FORWARDTM A little about me • I’m addressing the problem of how to get new platform software out there faster, and with a smaller likelihood of disruption • We’re using OpenNebula!
  5. 5. ©2013 AKAMAI | FASTER FORWARDTM Akamai in a nutshell • Akamai is a globally-distributed cloud of physical servers • Machines at Akamai are grouped into functional units called networks. • Edge networks - CDN services, object/file storage, analytics, etc. • Infrastructure – distributed file transfer, messaging, monitoring, reporting, etc.
  6. 6. ©2013 AKAMAI | FASTER FORWARDTM Akamai in a nutshell • Each network runs a different set of software that depends on the role of the network in the ecosystem. • Each also runs a common set of platform software • Generally OS software and Akamai-specific File transferMessaging Monitorin g Network-specific software OS
  7. 7. ©2013 AKAMAI | FASTER FORWARDTM Akamai in a nutshell Infra networkFile transferMonitoringMessaging Operating system/kernel Edge network Network-specific software
  8. 8. ©2013 AKAMAI | FASTER FORWARDTM Akamai in a nutshell Infra networkFile transferMonitoringMessaging Operating system/kernel Edge network Network-specific software
  9. 9. ©2013 AKAMAI | FASTER FORWARDTM Akamai in a nutshell Infra networkFile transferMonitoringMessaging Operating system/kernel Edge network Network-specific software Edge network Network-specific software Edge network Network-specific software
  10. 10. ©2013 AKAMAI | FASTER FORWARDTM Akamai in a nutshell • State of a network • Installed software and configuration • Dynamic configuration/metadata • Persistent (across installation) filesystem state • Applied manual changes • Each network tends to have built-up state necessary for proper functionality • not well-documented or defined
  11. 11. ©2013 AKAMAI | FASTER FORWARDTM An Akamai “instance” • A given unique set of Akamai networks that function together • Self-contained with limited external dependencies
  12. 12. ©2013 AKAMAI | FASTER FORWARDTM Akamai “instances” Production – 105 SQA – 102 SQA – 102 SQA – 102 Dev – 102 Dev – 102 Dev – 102
  13. 13. ©2013 AKAMAI | FASTER FORWARDTM Commoditizing an Akamai instance • Instances are labor-intensive to manage • We’d like the ability to create these as a commodity • Automatically bootstrapping from scratch is possible, but difficult.
  14. 14. ©2013 AKAMAI | FASTER FORWARDTM Commoditizing an Akamai instance • Solution: virtualization with OpenNebula • Represent the state of an instance using virtual machines. • Save the virtual machines and clone them. SQA – 102 SQA copy– 102 SQA – 102
  15. 15. ©2013 AKAMAI | FASTER FORWARDTM Cloning Akamai instances • Static IP configuration • Isolation • traditional NAT • “reverse” NAT • SOCKS proxy • Akamai “authgate”
  16. 16. ©2013 AKAMAI | FASTER FORWARDTM Accessing an instance NAT SOCKS authgate192.168.0.0/16 192.168.0.1 NAT SOCKS authgate192.168.0.0/16 192.168.0.1 Client 172.26.238.10 172.26.238.20
  17. 17. ©2013 AKAMAI | FASTER FORWARDTM Clone a machine 1. Power down VM 2. Clone each disk 3. Add new virtual network with given VLAN. 4. Construct new VM template • Using cloned disks • Using new virtual network 5. Reboot old VM – still functioning 6. Power up new VM
  18. 18. ©2013 AKAMAI | FASTER FORWARDTM Managing the Akamai side • OpenNebula is great for managing VMs and their resources • We needed another level of management • “Akamai” networks • Akamai instances • Expose specific Akamai machine types and services
  19. 19. ©2013 AKAMAI | FASTER FORWARDTM An Akamai instance service • Service-oriented • Language/test harness agnostic • Share common resources
  20. 20. ©2013 AKAMAI | FASTER FORWARDTM An Akamai instance service Instance service OpenNebula 2. “Clone these machines” 3. Creates resources 4. “Here’s your instance!” 5. “Ok, let’s use it.” Gateway User1. “Give me an instance”
  21. 21. ©2013 AKAMAI | FASTER FORWARDTM A well-managed instance • Leverage expertise from around the company to create an internal “master” instance • Each additional network gets it closer to a “real” instance.
  22. 22. ©2013 AKAMAI | FASTER FORWARDTM Addressing existing instances • Can also address existing instances of physical machines • Production • Lab instances
  23. 23. ©2013 AKAMAI | FASTER FORWARDTM Testing at Akamai Unit testing Dev QA SQA Checklisting Alerting
  24. 24. ©2013 AKAMAI | FASTER FORWARDTM Unify these stages Dev QA SQA Checklisting • Use the same code to write tests by using a common interface to refer to an Akamai instance.
  25. 25. ©2013 AKAMAI | FASTER FORWARDTM Thank you!

×