Build vs. Buy: Internet Datacenter


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Build vs. Buy: Internet Datacenter

  1. 1. Build vs. Buy: Internet Datacenter William B. Norton – Equinix [email_address] Michael Lucking – Epoch Internet
  2. 2. Goals <ul><li>Create a community white paper on issues associated with the building & scaling of Internet Datacenters </li></ul><ul><li>Identify the issues that folks should consider when making the build vs. buy decision </li></ul><ul><li>Tonight: </li></ul><ul><li>Demonstrate the document & models to date and get feedback </li></ul>
  3. 3. Non-Goals (Out-of-Order) <ul><li>Not to advocate building you own rack space </li></ul><ul><li>Not to advocate buying rack space </li></ul><ul><li>(Rather – document the scaling of each and the criteria involved in the decision) </li></ul><ul><li>(Warning: Tradeoffs are Religious Topics for many.) </li></ul>Background…
  4. 4. Toured a lot of data centers
  5. 5. Some with very Secure Fiber Access… Source: Technology Center of the Americas Tour
  6. 6. Some with less so Source: Don’t Ask.
  7. 7. Some Very Secure…
  8. 8. Some Cat 5 Hurricane proof…
  9. 9. Some not quite cat proof. Loading Dock
  10. 10. Great religious debate over ties Velcro vs. Plastic cable ties vs. String
  11. 11. Time to white paper what I’ve seen & Facilitate BOF Discussion <ul><li>Datacenter BOF I NANOG Oct ’98 Atlanta </li></ul><ul><li>Build vs. Buy White Paper </li></ul><ul><li>Internet Datacenter Model </li></ul><ul><ul><li>Definition of Internet Datacenter </li></ul></ul><ul><li>Focused discussion of Each Component of the Model </li></ul>
  12. 12. Internet Datacenter Model Data Center Operations Core Infrastructure Real Estate Network Transport Business Models Internet Service Models Content & Support Dependence Relationship Increased Value
  13. 13. Internet Datacenter Model Example Data Center Operations Core Infrastructure Real Estate Transport Mkt: UA WCom Level3 AtlMedia... Transit Mkt: C&W Nacamar TeleGlobe AT&T... Value Dependence CP/e*/ASP/Support: BMA IBM Schwab Yahoo! Akamai Site Smith
  14. 14. Ultimate Goal <ul><li>Build Financial models that allow an ISP to say: </li></ul><ul><li>“ Given these parameters and our required footprint, it makes sense to build our own.” </li></ul><ul><li>Or </li></ul><ul><li>“ Given these parameters, it probably makes sense to outsource this to someone who can realize these economies of scale.” </li></ul>+ + = Buy Build
  15. 15. Real Estate for Internet Datacenter Issues <ul><li>Place to put stuff </li></ul><ul><li>Proximity to Fiber </li></ul><ul><li>Space Size </li></ul><ul><li>Utility Power </li></ul><ul><li>Facilities Yard </li></ul>Data Center Operations Core Infrastructure Real Estate Increased Value Relationship Scaling of Datacenter Real Estate?
  16. 16. Scaling of the Real Estate Model # Sq. Ft. Premium Real Estate Market (Tokyo, NYC,…) Real Estate Market (everywhere else) $/sq. ft.
  17. 17. Real Estate Size is Important <ul><li>Choose Too Small: Run out of space, lose economies of scale </li></ul><ul><li>Choose Too Big: Cost per Rack very high </li></ul><ul><li>Vs. </li></ul><ul><li>Right of First Refusal (some Internet Datacenters allow this) </li></ul><ul><li>Expandability of Space </li></ul>
  18. 18. Commitment to DataCenter <ul><li>1 yr contract </li></ul><ul><li>Vs. </li></ul><ul><li>10 year lease </li></ul>Time to Market Issue – 6 – 9 months to build
  19. 19. Hidden Charges <ul><li>Meet Me Rooms </li></ul><ul><li>Required products: </li></ul><ul><ul><li>Must buy port on switch or certain volume of BW </li></ul></ul><ul><li>Cross Connect Fees </li></ul>
  20. 20. Size & Finding Real Estate <ul><li>100,000 sq ft in downtown DC? </li></ul><ul><li>10,000 sq ft in downtown DC </li></ul><ul><li>Vs. </li></ul><ul><li>Fiber Miles to Internet Datacenter </li></ul>
  21. 21. Infrastructure Scaling <ul><li>Heating Venting Air Conditioning (HVAC) </li></ul><ul><li>Security Infrastructure </li></ul><ul><li>Power Infrastructure </li></ul><ul><li>Fire Suppression </li></ul>Data Center Operations Core Infrastructure Real Estate
  22. 22. Heating, Venting, Air Conditioning Infrastructure <ul><li>Controlled Humidity ~45% </li></ul><ul><li>Too much Humidity: System Failure </li></ul><ul><li>Too Little Humidity: Static Electricity </li></ul><ul><li>Air Temperature:~72 (IBM: 65-85) </li></ul><ul><li>Solutions: </li></ul><ul><li>DX Systems </li></ul><ul><li>Air Cooled </li></ul><ul><li>Water-Based </li></ul><ul><li>How do these Scale? </li></ul>
  23. 23. HVAC Scaling Direct Expansion Air Cooled Chilled Water System HVAC Cost # Racks $1,000/ton $478/ton $250/ton HVAC System Cost: Economies of Scale DX Systems: 25-300 tons of Cooling ~ 10K sq ft or ~150-200 racks
  24. 24. Crossover point where Chilled Water lowest per-rack solution
  25. 25. Security Infrastructure <ul><li>24x7 Manned </li></ul><ul><li>Access: Biometrics, card keys </li></ul><ul><li>Video Surveillance </li></ul>Sliding Glass
  26. 26. Power Infrastructure <ul><li>Monitoring of UPS, DC Power, PDUs, </li></ul><ul><li>N+1 Redundancy </li></ul><ul><li>Power Draw? </li></ul><ul><ul><li>20K=500Kw </li></ul></ul><ul><ul><li>(Jason Colangelo, APC) </li></ul></ul>
  27. 27. Fuel Cells for Generators
  28. 28. Fire Suppression Infrastructure <ul><li>VESPA air sampling </li></ul><ul><li>Dry pipe, pre action water system vs. FM-200 </li></ul>
  29. 29. Infrastructure Operations <ul><li>Operations of HVAC, Security, Power systems </li></ul><ul><li>N+1 Redundancy </li></ul>Data Center Operations Core Infrastructure Real Estate
  30. 30. Transport <ul><li>Dark Fiber Provider(s), Carrier(s), etc. </li></ul>
  31. 31. Internet Service <ul><li>ISP(s), Aggregator(s) </li></ul>
  32. 32. Content Providers <ul><li>Content Distributor(s), Hoster(s), Manager(s) </li></ul>
  33. 33. Sample Crank of the Model: Assumptions <ul><li>Both: $400/sq foot build out costs for conditioned space </li></ul><ul><li>Buy: Equinix retail price $1,500/rack used for comparison, </li></ul><ul><li>Buy: No discount for buying more or larger cages </li></ul><ul><li>Build: Building Real Estate comes in 10,000 sq foot increments </li></ul><ul><li>Both: Conditioned space includes infrastructure (power, HVAC, Security systems, etc.) </li></ul><ul><li>Both: Does not include network bandwidth, hosting equipment, managed services, etc. </li></ul><ul><li>Build: Financing of build costs is available, </li></ul><ul><li>Build: Adequate building is available, and build time (time to market) is not an issue </li></ul><ul><li>Build: Sufficient network capacity (transport and transit) is available and not an issue </li></ul><ul><li>Both: Facility population is not an issue – ignoring collocation benefits and don’t expect to buy or sell services at the facility. </li></ul>
  34. 34. Sample Crank of the Model: Internet Datacenter Build vs. Buy Note: Fictitious
  35. 35. References – Don’t Squint… <ul><li>Ripin & Sayles, Insider Strategies for Outsourcing Information Systems , 1999 Oxford University Press </li></ul><ul><li>Greaver, Strategic Outsourcing: A structured Approach to Outsourcing Decisions and Initiatives , 1999 AMACOM </li></ul><ul><li>Frenza, J.P. , Buying Web Services , 1999 Wiley Computer Publishing, particularly Chapter 1: An Outsourcing Overview by Frenza and Chapter 5:Infrastructure and Site Hosting by Chris Bryant. </li></ul><ul><li>Norton, Donelan, Adelson, Newton, Data Center Needs Birds of a Feather meeting at NANOG 14 – Atlanta, Georgia – November 9, 1998 7:30-9PM, http://www. nanog .org/ mtg -9811/data. bof .html </li></ul><ul><li>Schroepfer, Andrew M., Communication Service Providers, Equity Research, U.S. Bancorp Piper Jaffray Equity Research, December 1999, particularly Ch 12: Facilities Hosting Providers. </li></ul><ul><li>“ What every CEO needs to know about Electronic Business”, Business Week,, Marsh 22, 1999 </li></ul><ul><li>IBM White Paper: Building a shared services data center, ibm .com/ asus / cwp - liushrdsvcs .html , focuses on shared services enterprise Data Center </li></ul><ul><li>Murray, Steve, “Web hosting Strategies: Market Review and Forecast, 1998-2003”, IDC Report #20953, December 1999. </li></ul><ul><li>Search for “Data Center Construction” on http://www. google .com . </li></ul><ul><li> ibm .com/ asus / nordstrom .html </li></ul><ul><li>http://www. gruber .com/ compfac / datal . htm </li></ul><ul><li>http://www. edgemail .com/ whitepapers / </li></ul><ul><li>Dialtone Internet: http:// dialtoneinternet .net/tour/page0. htm </li></ul>
  36. 36. References – cont’d <ul><li>Yahoo! Directory of Colocation Providers: Colocation / </li></ul><ul><li>Yahoo! Directory of web hosting providers: WhitePages / wp _0.html </li></ul><ul><li>Search for Internet Data Center on </li></ul><ul><li>AT&T Internet Data Centers: http://www. ipservices . att .com/ datatour / idc _dedicated.html </li></ul><ul><li>Web hosting directory site: http://www. webhostdir .com/ </li></ul><ul><li>Exodus Internet Data Center descriptions and virtual tour: idcs / </li></ul><ul><li>.Equinix Internet Business Exchange (IBX) virtual tour http://www. equinix .com </li></ul><ul><li>PSINet Colocation Product: http://www. psinet .com. hk /hosting/co_location.html </li></ul><ul><li>Flashhost data centers http://www. flashhost .com/data_center.html </li></ul><ul><li>Hosting Internet Data Center in Zaire: za /solution/hosting/IDC2.html </li></ul><ul><li>Level3 colocation facilities: colocation </li></ul><ul><li>Frontier Global Center (now global crossing) datacenters . shtml </li></ul><ul><li>Exodus Communications  Q1 2000 Quarterly Earnings Announcement  4/20/2000 2:00 PM Pacific Time: http:// webevents financecalls /event/index.asp? EarningsID =766& src = StreetFusion </li></ul><ul><li>Houses are designed by Geniuses & built by Gorillas , by Bob Johnson, ISBN: 1-886110-48-4 </li></ul><ul><li>Facilities Operations & Engineering Reference , The Certified Plant Engineer Reference, R.S. Means Company, Inc., ISBN: 0-87629-462-X </li></ul><ul><li>Network Equipment-Building System (NEBS) Requirements: Physical Protection , Bellcore GR-63-CORE, Issue 1, October 1995. A module of LSSGR, FR-64; TSGR, FR-440; and NEBSFR, FR-2063 </li></ul><ul><li>Central-Station Alarm Services, UL 827 Standard for Safety , Underwriters Laboratories, Inc. ISBN 0-7629-0067-9 , Sixth Edition ©1997 </li></ul><ul><li>  </li></ul>
  37. 37. Document Availability <ul><li>All references are in the BvB white paper </li></ul><ul><li>Send e-mail to [email_address] </li></ul>
  38. 38. Thanks to <ul><li>Sean Donelan (Equinix) </li></ul><ul><li>Jason Colangelo (APC) </li></ul><ul><li>Stephen Stuart/Phil Steffora (MFN) </li></ul><ul><li>All Colo/ISPs tour facilitators </li></ul><ul><li>Jay Adelson (Equinix) </li></ul><ul><li>Josh Horn (NOTA) </li></ul><ul><li>Justin Newton (NetZero) </li></ul><ul><li>Friends at Exodus, Frontier Global Center, Level3, etc. </li></ul><ul><li>Daniel Golding (NetRail) </li></ul><ul><li>Everyone here! Please e-mail feedback. </li></ul>