3. Data Centers and Interconnection
Michael Terlizzi
Executive Vice President, Operations
3
3
4. Interconnection & Co-location Facilities
The Most Strategic Facilities at Crossroads of the World’s Best Networks
Chicago
• 2 Datacenters
California • 62,222 Square Feet
• 3 Datacenters
• 48,834 Square Feet
New York Metro
New York, NY • 4 Datacenters
• 252,151
Chicago, IL Clifton, NJ
San Francisco, CA Square Feet
Weehawken, NJ
Santa Clara, CA
Charlotte, NC
Los Angeles , CA Phoenix, AZ Atlanta, GA
Southeast
Dallas, TX • 3 Datacenters
• 177,824 Square Feet
Population per square mile
Total
250 or more
50 – 249.9
Dallas / Phoenix Miami, FL
10 – 49.9
• 3 Datacenters
Less than 10 • 83,264 Square Feet
Note: As of 04/15/10, population in 000s
4
5. Interconnection Center Launch Process
Objective:
To effectively construct and maintain a highly redundant, available, and
energy-efficient Interconnection Center worthy of our customer‟s business.
Process:
1. Site Selection
2. Planning / Design
3. Construction
4. Commissioning
5. Transition to Operations (Production)
5
6. Site Selection
•Gating predecessor – Market analysis including current supply /
demand, competition, space / power requirements, past performance, etc.
Primary Areas of Focus
1. Property Review (Structural, Environmental)
•Floor loads, ceiling heights, column spacing, distribution pathways, site
considerations
2. Utility Power
•Capacity available / potential to increase, redundancy, rate class / tariff
analysis, voltage available, Green quota / renewable energy options
3. Fiber Proximity
•Review of carriers in and around the facility
4. LEED Considerations
6
7. Planning & Design
1. Owner‟s Project Requirements (OPR):
• A document that details the functional requirements of a project and the
expectations of how it will be used and operated. This includes project
and design goals, measurable performance
criteria, budgets, schedules, success criteria, and supporting information.
Inputs from Sales, Marketing, Engineering and Operations department
considered.
2. Basis of Design (BOD) Document
• A document that records the concepts, calculations, decisions, and
product selections used in the design to meet the Owner‟s Project
Requirements and to satisfy applicable regulatory
requirements, standards, and guidelines.
3. Project Team Selection
• Internal resources assigned and stakeholders identified
• Engineering partner selected
• CM partner and CX Agent selected in competitive bid process based on
construction documentation
4. Pre-purchase Specification Released
• Mission critical equipment specifications created and released to suitable
vendors for bid
7
8. Construction
Construction Phase
1. Detailed Project Plan Created
2. Construction Documents Distributed to trades for bid
3. Pre-bid Meeting held to address points of clarification
4. Contract award made
5. Budget updated with bid values
6. Permits secured
7. Trades coordinated / construction begins
8. Equipment Deliveries / Site Inspections
9. Construction Completion
10. Punch list
(Note: Commissioning detailed separately)
8
9. Commissioning
The basic purpose of commissioning is to provide documented confirmation that
mission critical systems function in compliance with criteria set forth in the project
contract documents to satisfy the operational needs.
1. Factory Witness Tests (Level 1)
•Manufacturer demonstrates equipment performance prior to shipping
2. Component Verification (Level 2)
•Was everything shipped correctly and in good condition?
3. Static tests (such as duct leakage tests)
4. Completion of Pre-Functional Checklists (Level 3)
•Is everything connected and operational?
5. Start-up
6. Control point-to-point checks
7. Balancing
8. System Test Procedures (Level 4)
•Dynamic testing of systems (rather than just components) under load
9. Integrated Systems Tests (Level 5)
•“Pull-the-plug”
9
10. Turnover to Operations
1. Staff Appropriately – 24/7/365 SMEs and supporting personnel
2. Confirm all work has been accurately captured on As-Built drawings and reviewed by
local operations team.
• Inclusion of Operations Team is pre-design, construction, Cx is imperative
3. Confirm equipment list has been populated with warranty information
4. Owner training of all equipment, maintenance procedures, operating procedures
5. Review of sequence of operation on all control systems (BMS) has been fine tuned
to meet efficiency criteria.
6. Create and Implement detailed Preventive Maintenance program on all systems
2011 Completed Training:
CNET- Certified Data Center Design Professional Liebert CRAC / CRAH
TPI - DC Power Engineering Cummins 1000 / 1750 / 2250kW Generator
BOMI International- DOM Building Principles Caterpillar 2000kw Generator
BOMI International- Env. Health & Saftey Stulz CRAH / CRAH
OSHA 10/30/100 Eaton 9395 UPS
DCEP Generalist EVAPCO Fluid Cooler
GR-1275-CORE Marley Fluid Cooler
VFD Standard of Operation Andover Contnuum Ops & Provisioning
DCR – Optimization of Cooling Data Centers FieldView Ops & Provisioning
Siemens FLSA Fire Systems
Liebert NXL UPS
10
11. Energy Efficiency
• Corporate focus for improvement
• Performance monitoring of mission critical facilities
• Measure / Trend Energy Consumption –FieldView
• Closely manage the infrastructure capacity to
prevent critical power thresholds from being
breached to operate at efficient densities
• ~$700K over last 24 months in rebates
Resource Utilization - Aperture
• Manage physical assets and resources / space v.
power utilization
• Defer capital expenditures
Process Efficiency
• Optimize human capital in support data center
operation
• Automated data collection and consolidated
reporting
• Having a single comprehensive presentation
ensures that trends are identified and overall meaning
is incorporated into effective polices and strategies.
11
12. Interconnection
Christopher Eldredge
President & GM, Ethernet Exchange
and Product Management
12
12
13. State of Interconnection Market
Business Drivers
Outsourcing the Data Center
Choice of Networks – Local and
Long Loop Providers
Competitive Pricing
Technology Drivers
Public Network Access
Ethernet Exchanges
Private Network Connectivity for
Mission Critical Applications
Secure and Reliable
Connectivity
Industry Trends
Virtualization
Cloud Computing
Thin Clients
Proximity
Mobility
13
14. Interconnection Use Cases
Private Network Access
Telx Dynamic Connect
Network Extensions
Local, National
International
Infrastructure Construction
Telx Direct Connect
Network Node InterConnection
Direct NNI
Cloud Construction
Network Reach
Telx Network Connect
Disaster Recovery
Capacity Management
Enhanced Choice
Public Network Access
Telx IP Connect
Peering for Proximity
Public Access to Remote Elements
14
15. Ethernet Exchange and the Interconnection Center
Connecting Global Networks
15 Strategically Located Data Centers
Cloud
Providers
(Multinational)
The Evolution of Ethernet Exchange 2010 to 2011
2010 Business Drivers and Market Message
1. Focused on supporting the Ethernet Private Line Market Only
2. Solve complex Network to Network Connectivity for WAN connectivity
3. Stabilize disparate QoS delivery techniques
4. Speed time to Market
2011 Business Drivers and Market Message
1. Create a common platform for Ethernet service providers to conduct business
2. Provide access to Services and Technologies stored or created in the Data Center
3. Create an Engine for content acquisition, distribution and creative collaboration
4. Front door to Cloud and an Enabler of services
15
16. Ethernet Exchange Drivers
Ethernet Exchange Provides a Community to
Access Applications
Online Place to Conduct Business
Bring all Services under one unified Product
Management Umbrella Telecommunications Unified Communications
Access to over 950+ Unique Networks Infrastructure
Understand Service Dependencies Development IPTV Data Services
Messaging MSO
Carrier Networks and Data Center Services Co- SaaS Provider/
Convergence Providers Location Cloud
Network Connectivity to Data Centers Teleconferencing
Conferencing 15- Services Providers
Data Centers Host Services
Facilities
Services Drive Bandwidth Storage
Storage Enterprise
VOIP
Bandwidth Consumption Drives
TELX Geo
Interconnection Products & Wireless
MPLS Services Services Community Locations
Communications
Verticals Services
National Industry
Verticals International
Media and Entertainment Vertical Carrier
Carrier
Storage and Archiving
Cloud and SaaS Data
Center Internet Ethernet Private Line
Financial Services
Mobility LAN
Fixed-line Voice Cable & Satellite Services
Services Providers
Hosting; ISP/Data/Voice
16
17. Telx Cloud Connection Center
Communities
Ethernet
Xchange
Data Centers
Services
Financial
Xchange
Cloud
Xchange
InterConnection
Carrier
Xchange
Media
Xchange
Xchange
Channel
Partners
Telx Connect
Marketplace
17
19. Concerns
Lack of IT Security
Data Management
Control
Trust
SLAs
Interoperability and portability
Vendor lock-in
ISV software licensing
Loss of Control
Standards
0% 10% 20% 30% 40% 50% 60%
Source: The 451 Group Cloud Adoption Survey 2010
19
20. Opportunity
External/public Internal/private
Consumers Applications, hosted
Billions of users desktops, household
savings
SMEs Rapid T&D resources
Millions of users Disaster readiness/
resilience/DR
Capacity expansion
“Legacy” migration
VM management
Enterprises Cost/ecosystem mgmt
Thousands of users Production applications
0% 20% 40% 60% 80% 100%
..there’s lots of opportunity for everyone..
20
22. The Solution
Hybrids are
- Multiple Cloud ‘houses’
- Private
- Public
- Hybrid
- Connected Solutions
- Carrier backbones
- Enterprise backbones
- Internet Networks
- Each other
Enterprise Class Hybrid Cloud Solution - Bridges
- Public-Private-Own
- Choices
- Networks
- Locations
- App. Providers
Source: The 451Group : Cloudscape - Integrated
- New-Old-Legacy
- Authentication
On premise + Colocation + Cloud - Sensitive Data
22
23. Benefits
• Deploy cloud platforms where they are needed
• Close to the user base
• In proximity to IT/Managed Services
• Highly Connected
• Put the cloud IN THE NETWORK – not just on it
• Improved Performance
• Avoid Internet Congestion
• Leverage QoS over private connectivity
• Prioritize business data services
• Control Costs
• Reduce WAN and Internet backhaul
• Eliminate local access „tax‟
• Improved Security with built-in DR
• Private interconnection reduces „Internet exposure‟
• HUBs in data centers benefit from data center resilience & SLAs
• Choice, Choice and More Choice
• Choose the right network for the application, location and user base
23
Telx maintains diverse customer base spanning multiple verticals so design needs to be flexible to adapt to end-user.It is imperative that our design teams and builders see the space through our eyes and we take great care facilitate that at the beginning of each project.OPR is inclusive of performance criteria and modeled energy demand to facilitate the right-sizing of gear and design PUEPre-purchase specification includes efficiency options selected of mission critical gear – Unity power factor UPS, ECO-Mode, Intelligent paralleling on electrical; plug fans, VFD, etc. on mechanical
Managing vendors to strict adherence to design to ensure optimal facility performance.
Integrated Systems Test: Integrated System Testing is functional testing that tests interfaces among infrastructure systems and is performed after successful completion of Functional Testing. Integrated Testing is to establish the integration, compatibility, and interaction of all critical infrastructure systems (electrical and mechanical). Integrated Testing requires all systems to be tested not only in the standard operating modes, but also in failure modes (i.e.: incoming utility power failure, bus failure, etc) to ensure that there is redundancy within, and between these systems. There are multiple opportunities in this phase to establish benchmarks to which future performance can be compared. Example 1) Level 4 allows for testing part-load efficiencies conform to manufactures specifications 2) Measure PUE at various loads during IST.
Having a comprehensive real-time look at capacity and redundancy not only allows us the ability to make the most informed decisions but by virtue of sharing this information with our customers, they can better manage their portion of the environment driving greater efficiencies which is atypical in multi-user scenario. Additionally, sharing this information internally allows our Sales and SE team(s) to engage with their customers with opportunities to enhance their presence with additional offerings.
Increase in data center demandAccess to 950+ NetworksLower Overall Interconnection costs
Dynamic Connect Virtual Exchange, EE, Video conference from Diverse networks different standardsPhysical connect traditional cross connect and fiber Riser between suitesNetwork Connect Metro Cross connect facilities together etc.IP Connect TIE and DIA
Network traffic growth resulting from numerous sourcesIncreased broadband penetrationPopularity of online and over-the-top video content and use of peer-to-peer web applications Adoption of connected devices such as smart phonesNetwork based business applications (e.g. SAAS) & Cloud ComputingIP traffic will quadruple by 2014Data Center Market Evolution Enterprise Data Center Consolidation: Centralization of infrastructure computing with VirtualizationHigh Density Power/CoolingBusiness Continuity Cloud HostingScalable Technology – Carrier Ethernet Increased network capacity and efficiency requirements have led to the emergence of Carrier Ethernet as a prominent industry standard among Telco's and enterprisesEmerging requirement for private connectivity to the CloudNetwork traffic growth resulting from numerous sourcesIncreased broadband penetrationPopularity of online and over-the-top video content and use of peer-to-peer web applications Adoption of connected devices such as smart phonesNetwork based business applications (e.g. SAAS) & Cloud ComputingIP traffic will quadruple by 2014Data Center Market Evolution Enterprise Data Center Consolidation: Centralization of infrastructure computing with VirtualizationHigh Density Power/CoolingBusiness Continuity Cloud HostingScalable Technology – Carrier Ethernet Increased network capacity and efficiency requirements have led to the emergence of Carrier Ethernet as a prominent industry standard among Telco's and enterprisesEmerging requirement for private connectivity to the Cloud