This document provides an overview of data center design considerations for a green and agile environment. It discusses the need for increased energy efficiency in data centers and standards like TIA-942, PUE, and green building initiatives. Specific strategies are presented for efficient power usage, cooling, and cabling infrastructure to support virtualization, consolidation and high-speed networking.
To meet growing bandwidth demands, carriers need networks that can adapt to new business models, scale on demand to support new services and customers as well as protect margins. Achieving this can be very challenging — especially for those with a large and sprawling installed base of legacy infrastructure.
This whitepaper addresses the following challenges: • Sprawling network infrastructure
• Inflexible networking equipment
• Business risks posed by unsupported legacy infrastructure
• Unreliable performance and Quality of Service (QoS)
• Inefficient networking with poor resource utilisation
• Costly and inefficient use of facilities
Energy in Factory Automation and the Role of Industrial Networks ControlEng
In today’s world energy cost big dollars for manufactures and the fact is most plants don’t know where there energy is being used. To help with this problem the Industrial Network communities are providing common interfaces to gather and control energy in the industrial space. This presentation will focus on aspects of Energy where it relates to Industrial Automation and some of the challenges we face. We will also cover upcoming initiative for interfacing to the smart grid for demand response request.
- Improving density and overall data center throughput providing a better environment for highly virtualized or cloud computing projects
- Optimizing data center space and rack unit availability providing the space necessary to support rapidly expanding infrastructure to support new mobile and “consumerized” applications
- Optimizing staff resources in the data center
Reducing operating costs/expenses for staff, energy, and data center components
- Minimizing downtime with better cable infrastructure
- Delivery of fully integrated solutions, that are co-engineered to work seamlessly, providing faster deployment
Open Compute has had a revolutionary impact on data center design ,offering many simplicity and cost saving benefits. To become more mainstream, we believe critical questions need to be answered about the implications of OCP on the upstream power infrastructure. Can data centers still achieve Tier 3 redundancy? How do I support traditional and OCP loads in the same data center? This presentation will address these and other questions, and propose a series of simplified architectures to support OCP. We will discuss redundancy, availability, and flexibility implications, as well as present a cost analysis. Schneider Electric embraces the spirit of OCP by openly sharing this analysis and making these simplified reference designs freely available
[Case study] Dakota Electric Association: Solutions to streamline GIS, design...Schneider Electric
Applications:
Integration of GIS-based processes makes existing circuits and proposed circuits
available in the same system so operations staff can work in parallel with the designers
rather than in succession.
Customer benefits
• Model, design and manage critical infrastructure
• Highly configurable
• Easily adapted for multiple uses
• Proactively identify needed repairs and replacements well in advance
To meet growing bandwidth demands, carriers need networks that can adapt to new business models, scale on demand to support new services and customers as well as protect margins. Achieving this can be very challenging — especially for those with a large and sprawling installed base of legacy infrastructure.
This whitepaper addresses the following challenges: • Sprawling network infrastructure
• Inflexible networking equipment
• Business risks posed by unsupported legacy infrastructure
• Unreliable performance and Quality of Service (QoS)
• Inefficient networking with poor resource utilisation
• Costly and inefficient use of facilities
Energy in Factory Automation and the Role of Industrial Networks ControlEng
In today’s world energy cost big dollars for manufactures and the fact is most plants don’t know where there energy is being used. To help with this problem the Industrial Network communities are providing common interfaces to gather and control energy in the industrial space. This presentation will focus on aspects of Energy where it relates to Industrial Automation and some of the challenges we face. We will also cover upcoming initiative for interfacing to the smart grid for demand response request.
- Improving density and overall data center throughput providing a better environment for highly virtualized or cloud computing projects
- Optimizing data center space and rack unit availability providing the space necessary to support rapidly expanding infrastructure to support new mobile and “consumerized” applications
- Optimizing staff resources in the data center
Reducing operating costs/expenses for staff, energy, and data center components
- Minimizing downtime with better cable infrastructure
- Delivery of fully integrated solutions, that are co-engineered to work seamlessly, providing faster deployment
Open Compute has had a revolutionary impact on data center design ,offering many simplicity and cost saving benefits. To become more mainstream, we believe critical questions need to be answered about the implications of OCP on the upstream power infrastructure. Can data centers still achieve Tier 3 redundancy? How do I support traditional and OCP loads in the same data center? This presentation will address these and other questions, and propose a series of simplified architectures to support OCP. We will discuss redundancy, availability, and flexibility implications, as well as present a cost analysis. Schneider Electric embraces the spirit of OCP by openly sharing this analysis and making these simplified reference designs freely available
[Case study] Dakota Electric Association: Solutions to streamline GIS, design...Schneider Electric
Applications:
Integration of GIS-based processes makes existing circuits and proposed circuits
available in the same system so operations staff can work in parallel with the designers
rather than in succession.
Customer benefits
• Model, design and manage critical infrastructure
• Highly configurable
• Easily adapted for multiple uses
• Proactively identify needed repairs and replacements well in advance
More Electric:
Our world is becoming More Electric. Almost everything we interact with today is either already electric or becoming electric. Think about it. From the time you start your day in the morning to the time you finish your day – your home, your car, your work, your devices, your entertainment – almost everything is electric. Imagine the energy needed to power this. Electricity consumption will increase by 80% in next 25 years
More Connected: Our lives are also becoming more connected. The Internet has already transformed the way we live, work and play. Now the Connected Things is going to take this to a brand new level. 50 billion things connected in the next 5 years.
More Distributed: With such a widespread electrification and connectivity, energy models need rethinking as well. Which is why the generation of power needs to be closer to users. Distributed Energy is rapidly evolving globally. This is positive energy – renewable. In 2014 , Renewables overtook fossil fuels in investment value, with $295bn invested in renewables compared to $289bn invested in fossil fuels. And it is getting cheaper to do this.
More Efficient: When our world is more electric, more connected and more distributed, new opportunities emerge and allows us to tap into even more efficiency – in industrial processes, in the energy value chain, in buildings, in transportation, in the global supply chain and even in the comfort and peace-of-mind of our homes.
Presentation from the EPRI-Sandia Symposium on Secure and Resilient Microgrids: IEEE 1547 and Microgrids, presented by Tom Key, EPRI, Baltimore, MD, August 29-31, 2016.
The ever-increasing demand for various multimedia applications and services as well as the need for bandwidth expansions and faster data rates are becoming a challenge for every data center everywhere.
Oliver spoke about the challenges and opportunities associated with transitioning from WPD’s role as a Distribution Network Operator (DNO) to a Distribution System Operator (DSO).
The ever-increasing demand for various multimedia applications and services as well as the need for bandwidth expansions and faster data rates are becoming a challenge for every data center everywhere. Follow this link https://www.cbo-it.de/
Renewable energy and grid integration energy transitionNarinporn Malasri
Energy Regulatory Commission Thailand : Energy-Related Policies and Activities
Renewable energy and grid integration and energy transition in Thailand. IMPACT of Renewable Connection such as FULL GRID CONNECTION CAPACITIES, REVERSE POWER FLOWS, VOLTAGE CONTROL IN DISTRIBUTION GRID
PROTECTION COORDINATION IN DISTRIBUTION GRID.
Please take a moment to have a virtual tour of CoreXchange\'s Dallas InfoMart Datacenter. Call 214-646-1952 or email tpersky@corexchange.com to setup a physical tour
Renewable Energy Sources are being used in Off-Grid mode. By integrating all these sources to a common point energy efficiency can be improved and frequent dynamic faults can be avoided. This approach needs to implement smart grid and technologies.
GRID INTERCONNECTION OF RENEWABLE ENERGY SOURCES AT DISTRIBUTION LEVEL WITH P...Pradeep Avanigadda
Renewable energy resources (RES) are being increasingly connected in distribution systems utilizing power electronic converters. This project presents a novel control strategy for achieving maximum benefits from these grid-interfacing inverters when installed in 3-phase 4-wire distribution systems. The inverter is controlled to perform as a multi-function device by incorporating active power filter functionality. The inverter can thus be utilized as: 1) power converter to inject power generated from RES to the grid, and 2) shunt APF to compensate current unbalance, load current harmonics, load reactive power demand and load neutral current. All of these functions may be accomplished either individually or simultaneously. With such a control, the combination of grid-interfacing inverter and the 3-phase 4-wire linear/non-linear unbalanced load at point of common coupling appears as balanced linear load to the grid. This new control concept is demonstrated with extensive MATLAB/ Simulink simulation studies and validated through digital signal processor-based laboratory experimental results.
More Electric:
Our world is becoming More Electric. Almost everything we interact with today is either already electric or becoming electric. Think about it. From the time you start your day in the morning to the time you finish your day – your home, your car, your work, your devices, your entertainment – almost everything is electric. Imagine the energy needed to power this. Electricity consumption will increase by 80% in next 25 years
More Connected: Our lives are also becoming more connected. The Internet has already transformed the way we live, work and play. Now the Connected Things is going to take this to a brand new level. 50 billion things connected in the next 5 years.
More Distributed: With such a widespread electrification and connectivity, energy models need rethinking as well. Which is why the generation of power needs to be closer to users. Distributed Energy is rapidly evolving globally. This is positive energy – renewable. In 2014 , Renewables overtook fossil fuels in investment value, with $295bn invested in renewables compared to $289bn invested in fossil fuels. And it is getting cheaper to do this.
More Efficient: When our world is more electric, more connected and more distributed, new opportunities emerge and allows us to tap into even more efficiency – in industrial processes, in the energy value chain, in buildings, in transportation, in the global supply chain and even in the comfort and peace-of-mind of our homes.
Presentation from the EPRI-Sandia Symposium on Secure and Resilient Microgrids: IEEE 1547 and Microgrids, presented by Tom Key, EPRI, Baltimore, MD, August 29-31, 2016.
The ever-increasing demand for various multimedia applications and services as well as the need for bandwidth expansions and faster data rates are becoming a challenge for every data center everywhere.
Oliver spoke about the challenges and opportunities associated with transitioning from WPD’s role as a Distribution Network Operator (DNO) to a Distribution System Operator (DSO).
The ever-increasing demand for various multimedia applications and services as well as the need for bandwidth expansions and faster data rates are becoming a challenge for every data center everywhere. Follow this link https://www.cbo-it.de/
Renewable energy and grid integration energy transitionNarinporn Malasri
Energy Regulatory Commission Thailand : Energy-Related Policies and Activities
Renewable energy and grid integration and energy transition in Thailand. IMPACT of Renewable Connection such as FULL GRID CONNECTION CAPACITIES, REVERSE POWER FLOWS, VOLTAGE CONTROL IN DISTRIBUTION GRID
PROTECTION COORDINATION IN DISTRIBUTION GRID.
Please take a moment to have a virtual tour of CoreXchange\'s Dallas InfoMart Datacenter. Call 214-646-1952 or email tpersky@corexchange.com to setup a physical tour
Renewable Energy Sources are being used in Off-Grid mode. By integrating all these sources to a common point energy efficiency can be improved and frequent dynamic faults can be avoided. This approach needs to implement smart grid and technologies.
GRID INTERCONNECTION OF RENEWABLE ENERGY SOURCES AT DISTRIBUTION LEVEL WITH P...Pradeep Avanigadda
Renewable energy resources (RES) are being increasingly connected in distribution systems utilizing power electronic converters. This project presents a novel control strategy for achieving maximum benefits from these grid-interfacing inverters when installed in 3-phase 4-wire distribution systems. The inverter is controlled to perform as a multi-function device by incorporating active power filter functionality. The inverter can thus be utilized as: 1) power converter to inject power generated from RES to the grid, and 2) shunt APF to compensate current unbalance, load current harmonics, load reactive power demand and load neutral current. All of these functions may be accomplished either individually or simultaneously. With such a control, the combination of grid-interfacing inverter and the 3-phase 4-wire linear/non-linear unbalanced load at point of common coupling appears as balanced linear load to the grid. This new control concept is demonstrated with extensive MATLAB/ Simulink simulation studies and validated through digital signal processor-based laboratory experimental results.
Sklubi AlumniWeekend 23.10.2010:
Reijo Maihaniemi
Electricity Consumption: General
Electricity Savings Through DC Power Feed
DC Data Center Projects in the World
ICT Energy saving actions
Virtualization for efficiency: by Kathrin Winkler, The green gridDCC Mission Critical
The green grid is a not-for-profit global consortium focused on driving energy efficiency in the computing ecosystem
Developing meaningful and user-centric metrics to help IT and Facilities better manage their computing resources
Developing and promoting standards, measurement methods, best practices and technologies that support these metrics.
The data center market has expanded dramatically in the past few years, and it doesn’t show signs of slowing down. Many clients and building owners are requesting modular data centers, which can be placed anywhere data capacity is needed. Modular data centers can help cash-strapped building owners add a new data center (or more capacity) to their site, and can assist facilities with unplanned outages, such as disruptions due to storms. Owners look to modular data centers to accelerate the “floor ready” date as compared to a traditional brick and mortar.
OIF's Network Operator Working Group Chair Junjie Li spoke on delivering 400ZR, FlexE, and 112Gbps electrical to the datacom/telecom industry on September 4, 2019 during CIOE(China International Optoelectronic Exposition) event?
#400ZR #FlexE #112G #datacom #telecom #interoperability
DesignCon 2019 112-Gbps Electrical Interfaces: An OIF Update on CEI-112GLeah Wilkinson
DesignCon 2019
112-Gbps Electrical Interfaces: An OIF Update on CEI-112G
Brian Holden, Kandou Bus
Cathy Liu, Broadcom
Steve Sekel, Keysight
Nathan Tracy, TE Connectivity
Delivering Carrier Grade OCP for Virtualized Data CentersRadisys Corporation
This webinar explores the requirements for carrier grade Open Compute Project (OCP) infrastructure for virtualized telecom data centers delivering SDN and NFV for digital services.
3. Technology Leadership
• TIA
• ISO/IEC
• Ethernet Alliance
• IEEE
• ANZ standards committees
• BICSI
• US Green Building Council
• Various World Green Initiatives
• Uptime Institute
• ASHRAE
• AFCOM
• 7x24Exchange
• GreenDCA.org, Data Center Pulse
• Etc.
4. Siemon Culture 3300 Acres of
Conserved
Forest
ISO 14001
Environmentally
Responsible 220 Kilowatt
Solar
330% Carbon Power Plant
Negative
“Zero Landfill”
5. Power
• According to the Uptime Institute the three year cost of
powering and cooling servers is currently one and a half times
the cost of purchasing the server hardware.
• Server energy demand has doubled from 2000-2005
• 1.2% of the US electrical usage
• Equal to 5 1,000 MW power plants (study by LBL and Stanford)
• Cooling and electrical costs represent up to 44 -50% of a data
centers TCO
• EU directive to drive a 20% reduction in energy by 2020
• U.S. Federal Executive Order 13423
– Improve energy efficiency and reduce greenhouse gas
emissions through reduction of energy intensity by:
– 3 percent annually through the end of fiscal year 2015 for
a total reduction of 30%
6. IT Power
• 2% (now) of overall power and emissions
– 50% of power is cooling
– 29% is servers
– 5% is networking
7. EPA Announces Data Centers Can Now Earn Energy Star
Label
EPA Announces Data Centers Can Now Earn Energy Star Label
Release date: 06/07/2010
Contact Information: Enesta Jones, jones.enesta@epa.gov, 202-564-7873, 202-564-4355
WASHINGTON – The U.S. Environmental Protection Agency (EPA) announced today that stand-alone data centers and
buildings that house large data centers can now earn the Energy Star label. To earn the label, data centers must be in the top
25 percent of their peers in energy efficiency according to EPA’s energy performance scale. By improving efficiency, centers
can save energy and money and help fight climate change.
EPA uses a commonly accepted measure for energy efficiency, the Power Usage Effectiveness metric, to determine
whether a data center qualifies for the Energy Star label. Before being awarded the Energy Star, a licensed professional must
independently verify the energy performance of these buildings and sign and seal the application document that is sent to
EPA for review and approval.
Data centers are found in nearly every sector of the economy and deliver vital information technology services, including data
storage, communications and internet accessibility. Data centers use a significant amount of energy, accounting for 1.5
percent of total U.S. electricity consumption at a cost of $4.5 billion annually, an amount that is expected to almost double
over the next five years.
Significant energy and cost savings are possible through modest gains in efficiency. The energy consumed by data centers is
growing every year. Based on the latest available data, improving the energy efficiency of America’s data centers by just 10
percent would save more than 6 billion kilowatt-hours each year, enough to power more than 350,000 homes and save more
than $450 million annually.
Through Energy Star, EPA provides a proven energy management strategy and free tools for public and private organizations
to save energy and money through increased energy efficiency.
Data centers can improve energy efficiency in many ways, such as purchasing Energy Star qualified servers and ensuring
that all HVAC equipment functions properly.
8. PUE
• Also adopted in EU and Japan
• Power Usage Effectiveness
• Defacto standard
• Total power / IT load
• Most data centers are in the 1.9-2.6 range
• The closer to 1 the better
• Requires Intelligent PDU’s or some means to measure USED
power
• DCIE (Data Center Infrastructure Effectiveness) is 1/PUE to give
percentage
• PUE Version 2 kWH instead of kW
9. Green Grid
• The Green Grid is proposing the use of a new metric
• Data center compute efficiency (DCcE)
• and its underlying sub-metric, server compute efficiency (ScE).
• These metrics will enable data center operators to determine the
efficiency of their compute resources, which allows them to identify
areas of inefficiency.
• Using DCcE and ScE, data center operators can continually refine
and increase the efficiency of their compute resources in the same
way that they use power usage effectiveness (PUE) to improve data
center infrastructure.
• CUE (Carbon Usage Effectiveness)
• All are relatively new
10. Ethernet, Energy Efficient Ethernet, Data Center
Ethernet
• All things Ethernet!
• EEE is a game changer, especially for 10GBASE-T
– Provides a true idle state
– Significantly lowers power
• Data Center Ethernet collapses the backbone structure – layer 2
instead of layer 3
• Idea is to increase speed (bridge not route) and provide priority
• Ethernet has a LOT of overhead
• Not intelligent
12. The need for speed
• Virtualization
• Consolidation
• Increasing storage requirements
• More analytics
• Blade technologies
• Instant demand for data
14. TIA-942
• Horizontal and Vertical channels shall be run accommodating growth so these
areas do not have to be revisited
• No more bus or direct connections unless specifically REQUIRED by the
manufacturer
• No shared sheath media
• Category 6A recommended (echoed by Cisco® 6A or 7) Note: this is an update
from the original TIA-942 document now TIA 942-2 soon to be combined into TIA
942-A
• OM3 Minimum recommended fiber/ may change to OM4
ISO/IEC 24764
• Horizontal and Vertical channels shall be run accommodating growth so
these areas do not have to be revisited
• OM3 minimum recommended figer
• All systems shall be connected via a structured cabling system, point to
point connections are allowed for short run cables in server cabinets only
• Category 6A minimum (UTP or F/UTP) Class EA
• Category 7 and 7A/Class F and FA
15. Remember
• 6A and 7A support short reach mode
(low power mode) for a savings
estimated at about 1W per port
• In a data center with a PUE of 1.9, 1W
saved at the server = 2.84W through all
systems
16. TIA 942, 942-2, 942-A
• This Standard (942-A) replaces ANSI/TIA-942
dated April 12, 2005 and its addenda.
• This Standard incorporates and refines the
technical content of:
– ANSI/TIA-942, Addendum 1, Data center coaxial
cabling specifications and application 311 distances
312
– ANSI/TIA-942, Addendum 2, Additional guidelines for
data centers
19. Summary of Changes
• TIA-942-A is harmonized with the TIA-568-C standard series including the topology, terms and environmental
classifications described in 568-C.0, as well as component specifications in TIA-568-C.2 and C.3.
• The addenda, TIA-942-1 and TIA-942-2, have been incorporated into 942-A; those two addenda will be
superseded by TIA-942-A.
• Grounding and bonding content from TIA-942 has been removed and incorporated into TIA-607-B.
• Administration content has been removed and incorporated into the proposed TIA-606-B.
• Most content regarding cabinets/racks and power/telecommunications separation has been removed and
incorporated into the proposed TIA-569-C.
• Outside-plant pathways content has been removed and incorporated into TIA-758-B.
• The 100-meter length limitation for optical-fiber horizontal cabling has been removed. Horizontal cabling
distances for optical fiber are based on individual application requirements.
• Category 3 and Category 5e are no longer recognized for horizontal cabling. The draft recognizes Category 6
and Category 6A balanced twisted-pair cable types for horizontal cabling. Both types of cabling are still
permitted for backbone cabling.
• The recognized multimode optical fiber cable for horizontal and backbone cabling has been changed to OM3
and OM4 850-nm laser-optimized 50/125-micron multimode fiber cable. OM1 and OM2 are no longer
recognized in TIA-942-A.
• The recognized optical-fiber connectors are LC for one or two fibers and MPO for more than two fibers.
• The intermediate distribution area (IDA) has been added to the data center topology.
• An allowance for midspan powering equipment in the zone distribution area (ZDA) has been removed.
• A section on energy efficiency has been added.
• The terms "equipment outlet" (EO) and "external network interface" (ENI) from the ISO/IEC 24764 standard
have been added.
20. TR 942 Design Considerations
Top of Rack (called point to point) is Allowed as an exception here only!
21. Data Center Design – The Standard
ISO-IEC 24764
Top of Rack (called point to point) is Allowed as an exception here
only and must be in rack or close proximity!
24. Intel® Ethernet 10GBASE-T Transition
From 25 watts to less than 5 watts per port
25W /port
Intel® 10 Gigabit AT Server Adapter
25W • 10GBASE-T Single Port
• 82598 + Gen1 PHY
• 25W / port
Intel® 10 Gigabit AT2 Server Adapter
15W • 10GBASE-T Single Port
• 82598 + Gen2 PHY 10W /port
• 15W / port
15W /port
Intel® Ethernet Server Adapter X520
10W How we reduce power… • 10GBASE-T Dual Port
• 82599 + Gen2 PHY <5w / port
• More efficient integrated PHYs • 10W / port
• Smaller die size (40nm)
• Adaptive Power Modes Intel® Ethernet Controller
5W • Other Technologies •
•
10GBASE-T Single/Dual Port
Fully Integrated Mac + PHY
• Svr LAN on Motherboard (LOM)
• < 5W / port
2007 2008 2009 2010 2011
All unreleased products, computer systems, dates, and figures specified are preliminary based on current
expectations, and are subject to change without notice
24
26. Why Use a Structured System?
• Greatest Versatility over time
• Significant costs savings in switches day one
• Significant savings in power and maintenance costs day two
• Point to point connections create spaghetti
• Point to point can lock you into a single vendor solution
• Cable abatement is difficult in a point to point system
• Pathways are properly sized and filled
27.
28. Passive copper HSI assemblies
• Small Form Factor (SFF) connectors terminated to twinaxial cable
that facilitate direct connections between switch, server, and storage
equipment
SFP+ 8470 (CX4) QSFP+
(2-pairs) (8-pairs) (8-pairs)
The three most commonly deployed passive copper HSI assemblies are shown here. These assemblies are used to support short length direct connections, which are typically 7 meters or less. These assemblies feature small form factor connector interfaces that are terminated to twinaxial cable. The type of passive copper HSI cable assembly required for a given project is dependent upon the application supported by the network equipment and the connection distance. Note that the gage of the twinaxial cable that these assemblies are constructed from typically varies from 30 to 24 AWG and the gage size specified is related to the application support distance. For instance, 30 gage assemblies support a reach or 3 meters, which is ideal for Top of Rack deployments and 24 gage assemblies support a reach of 7 meters, which is ideal for Middle of Row deployment. Unlike structured balanced twisted-pair copper cabling systems, which support bi-directional transmission, high speed interconnects only support transmission in one direction. Therefore, in a manner that is similar to optical fiber media, a minimum of two pairs are required to transmit and receive data between two connected network devices.Siemon Interconnect Solutions supports 2-pair SFP+, 8-pair 8470 or CX4, and 8-pair QSFP+ passive copper high speed interconnect interfaces. 2-pair SFP+ assemblies are a cost-effective and lower-power alternative to optical cables for short reach links and support data transfer rates up to 10 Gb/s over both the transmit and receive pairs. 8-pair 8470 or CX4 assemblies are designed to support an array of infrastructures such as high-performance computing, enterprise networking, data centers, network storage, and Ethernet CX4 up to 10Gb/s over the four transmit pairs and the four receive pairs. In some implementations of InfiniBand, 8470 assemblies can support data rates up to 40 Gigabits per second. QSFP+ or Quad SFP+ represents the state-of-the art in high speed interconnect technology and can support transmit and receive speeds of up to 40 Gb/s over the four transmit pairs and the four receive pairs. It is important to note that not all SFP+ cable assemblies are interoperable and some equipment (such as HP's) may have been programmed or encrypted to work only with the brand or specific cable assemblies that were packaged with the equipment.
40Gig Ethernet on multimode is similar to 4 lane (4x) Infiniband. 4 strands transmit and 4 strands receive. In 40GbE the data is divided into packets and each is numbered. As they transmit down the fibers, the receiving end reads the packet segment numbers and puts the packet back together. MTP trunks fully support 40GbE without the need for zero bit skew trunks because of this.
When we move to 100GbE there are actually 3 options for MMF. A user can have one 24 strand MTP trunk (we ignore the strands at the outside) or two 12 strand MTP trunks mounted over and under or side by side in the electronics. Due to the existing number of 12 strand trunks already installed, you will likely see all of these options