What is Adaptive Routing ?
•Switch networks typically provide many paths between any two points
•In an adaptively routed network
routers make packet by packet decisions on the route to use based on
–Queue occupancy
–Channel usage
–Error rates and state
–Class of traffic
QsNetIII, An HPC Interconnect For Peta Scale SystemsFederica Pisani
QsNetIII Network
–Multi-stage switch network
–Evolution of the QsNetIIdesign
–Increased use of commodity hardware
–Increasing support for standard software
•QsNetIII Components
–ASICs Elan5 and Elite5
–Adapters, switches, cables
–Firmware, drivers, libraries
–Diagnostics, documentation
Slawomir Janukowicz, Juniper Networks
Juniper Day, Praha, 13.5.2015
Jestliže SlideShare nezobrazí prezentaci korektně, můžete si ji stáhnout ve formátu .ppsx nebo .pdf (kliknutím na tlačitko v dolní liště snímků).
QsNetIII, An HPC Interconnect For Peta Scale SystemsFederica Pisani
QsNetIII Network
–Multi-stage switch network
–Evolution of the QsNetIIdesign
–Increased use of commodity hardware
–Increasing support for standard software
•QsNetIII Components
–ASICs Elan5 and Elite5
–Adapters, switches, cables
–Firmware, drivers, libraries
–Diagnostics, documentation
Slawomir Janukowicz, Juniper Networks
Juniper Day, Praha, 13.5.2015
Jestliže SlideShare nezobrazí prezentaci korektně, můžete si ji stáhnout ve formátu .ppsx nebo .pdf (kliknutím na tlačitko v dolní liště snímků).
Building DataCenter networks with VXLAN BGP-EVPNCisco Canada
The session specifically covers the requirements and approaches for deploying the Underlay, Overlay as well as the inter-Fabric connectivity of Data Center Networks or Fabrics. Within the VXLAN BGP-EVPN based Overlay, we focus on the insights like forwarding and control plane functions which are critical to the simplicity operation of the architecture in achieving scale, small failure domains and consistent configuration. To complete the overlay view on VXLAN BGP-EVPN, we are going to the insides of BGP and its EVPN address-familiy and extend to about how multiple DC Fabric can be interconnected within, either as stretched Fabrics or with true DCI. The session concludes with a brief overview of manageability functions, network orchestration capabilities and multi-tenancy details. This Advanced session is intended for network, design and operation engineers from Enterprises to Service Providers.
Performance Improved Network on Chip Router for Low Power ApplicationsIJTET Journal
Abstract— On chip routers typically have buffers dedicated to their input or output ports for temporarily storing packets in case contention occurs. Buffers consume significant portions of router area. While running a traffic trace, however not all input ports of routers have incoming packets needed to be transferred simultaneously. So large numbers of buffer queues in the network are empty and other queues are mostly busy. This observation motivates us to design Router architecture with Shared Queues (RoShaQ), router architecture that maximizes buffer utilization by allowing the sharing multiple buffer queues among input ports. In the network design of the NoC the most essential things are a network topology and a routing algorithm. Routers route the packets based on the algorithm that they use. Every system has its own requirements for the routing algorithm. A new adaptive weighted XY routing algorithm for eight port router Architecture is proposed in order to decrease the latency of the network on chip router.
A novel way of creating overlay networks for OpenNebula is presented here. Using BGP Ethernet VPN (EVPN) with VXLAN data-plane encapsulation. This provides scalable Layer 2 over IP networks.
Demystifying EVPN in the data center: Part 1 in 2 episode seriesCumulus Networks
Network operators are slowly but surely embracing L3-based leaf-spine designs. However, either due to legacy applications or certain multi-tenancy requirements, the need for L2 across racks is still present. How do you solve the problem of providing L2 across multiple racks? EVPN is quickly emerging as the best answer to this question.
In this episode of our 2-part series on EVPN, we start with a discussion of the use cases, a review of the technologies EVPN competes with, and dive into an evaluation of the pros and cons of each.
For a recording of the live event, go to http://go.cumulusnetworks.com/l/32472/2017-09-22/95t27t
CCNA DC ,CCNP DC ,CCIE DC ,CCIE DC RACK RENTALS ,CCIE DC LEARNING PPT ,CCIE DC ONLINE TRAINING.
UCS RACK RENTALS ,MDS RACK RENTALS ,NEXUS 7000 RACK RENALS
Hardware in clouds in commonly connected by Ethernet LAN. There are alternatives but all are much more expensive by comparison. The problem in Ethernet used to be its CSMA/CD protocol which helps detect and resolve collisions. Even though modern switches are 100\% collision-free, high-rate multiparty chatter over the shared medium creates congestion and reduced overall utilization efficiency. One way to drastically improve efficiency is to revert to the old technology of circuit switching. This paper looks into possibilities of implementing a distributed scheduler which would emulate a circuit switching environment for multiparty communications. The immediate practical application in mind is access to local shared storage in clouds, or specifically, data centers.
Infrastructure as a Service (IaaS) for cloud environments provides compute processing, storage, networks, and other fundamental computing resources. To support multi-tenant cloud environments, IaaS utilizes the various advantages of the virtualization, but con-ventional virtual (overlay) network architectures for IaaS have been a direct cause of scalability limitations in multi-tenant cloud environments. In other words, IaaS’s virtual networks have the limitations due to the problems of high availability and load bal-ancing, etc. To solve these problems, we present EYWA, a virtual network architecture that scales to support huge data centers with high availability, load balancing and large layer-2 semantics. The design of EYWA overcomes the limitations by accommodating (1)a large number of tenants (about 224 = 16,777,216) by using virtual LANs such as logically isolated network with its own IP range in the cloud service providers’ view, and providing (2)public network service per tenant without throughput bottleneck and single point of failure (SPOF) on Source and Destination Network Address Translation (SNAT/DNAT) and (3)a single large IP subnet per tenant by using large layer-2 semantics in the consumers’ view. EYWA combines existing techniques into a decentralized scale-out control and data plane. The only component of EYWA is an agent in every hypervisor host that can control packets and the agents act as distributed controller. As a result, EYWA can be deployed into all the multi-tenant cloud environments today.
Building DataCenter networks with VXLAN BGP-EVPNCisco Canada
The session specifically covers the requirements and approaches for deploying the Underlay, Overlay as well as the inter-Fabric connectivity of Data Center Networks or Fabrics. Within the VXLAN BGP-EVPN based Overlay, we focus on the insights like forwarding and control plane functions which are critical to the simplicity operation of the architecture in achieving scale, small failure domains and consistent configuration. To complete the overlay view on VXLAN BGP-EVPN, we are going to the insides of BGP and its EVPN address-familiy and extend to about how multiple DC Fabric can be interconnected within, either as stretched Fabrics or with true DCI. The session concludes with a brief overview of manageability functions, network orchestration capabilities and multi-tenancy details. This Advanced session is intended for network, design and operation engineers from Enterprises to Service Providers.
Performance Improved Network on Chip Router for Low Power ApplicationsIJTET Journal
Abstract— On chip routers typically have buffers dedicated to their input or output ports for temporarily storing packets in case contention occurs. Buffers consume significant portions of router area. While running a traffic trace, however not all input ports of routers have incoming packets needed to be transferred simultaneously. So large numbers of buffer queues in the network are empty and other queues are mostly busy. This observation motivates us to design Router architecture with Shared Queues (RoShaQ), router architecture that maximizes buffer utilization by allowing the sharing multiple buffer queues among input ports. In the network design of the NoC the most essential things are a network topology and a routing algorithm. Routers route the packets based on the algorithm that they use. Every system has its own requirements for the routing algorithm. A new adaptive weighted XY routing algorithm for eight port router Architecture is proposed in order to decrease the latency of the network on chip router.
A novel way of creating overlay networks for OpenNebula is presented here. Using BGP Ethernet VPN (EVPN) with VXLAN data-plane encapsulation. This provides scalable Layer 2 over IP networks.
Demystifying EVPN in the data center: Part 1 in 2 episode seriesCumulus Networks
Network operators are slowly but surely embracing L3-based leaf-spine designs. However, either due to legacy applications or certain multi-tenancy requirements, the need for L2 across racks is still present. How do you solve the problem of providing L2 across multiple racks? EVPN is quickly emerging as the best answer to this question.
In this episode of our 2-part series on EVPN, we start with a discussion of the use cases, a review of the technologies EVPN competes with, and dive into an evaluation of the pros and cons of each.
For a recording of the live event, go to http://go.cumulusnetworks.com/l/32472/2017-09-22/95t27t
CCNA DC ,CCNP DC ,CCIE DC ,CCIE DC RACK RENTALS ,CCIE DC LEARNING PPT ,CCIE DC ONLINE TRAINING.
UCS RACK RENTALS ,MDS RACK RENTALS ,NEXUS 7000 RACK RENALS
Hardware in clouds in commonly connected by Ethernet LAN. There are alternatives but all are much more expensive by comparison. The problem in Ethernet used to be its CSMA/CD protocol which helps detect and resolve collisions. Even though modern switches are 100\% collision-free, high-rate multiparty chatter over the shared medium creates congestion and reduced overall utilization efficiency. One way to drastically improve efficiency is to revert to the old technology of circuit switching. This paper looks into possibilities of implementing a distributed scheduler which would emulate a circuit switching environment for multiparty communications. The immediate practical application in mind is access to local shared storage in clouds, or specifically, data centers.
Infrastructure as a Service (IaaS) for cloud environments provides compute processing, storage, networks, and other fundamental computing resources. To support multi-tenant cloud environments, IaaS utilizes the various advantages of the virtualization, but con-ventional virtual (overlay) network architectures for IaaS have been a direct cause of scalability limitations in multi-tenant cloud environments. In other words, IaaS’s virtual networks have the limitations due to the problems of high availability and load bal-ancing, etc. To solve these problems, we present EYWA, a virtual network architecture that scales to support huge data centers with high availability, load balancing and large layer-2 semantics. The design of EYWA overcomes the limitations by accommodating (1)a large number of tenants (about 224 = 16,777,216) by using virtual LANs such as logically isolated network with its own IP range in the cloud service providers’ view, and providing (2)public network service per tenant without throughput bottleneck and single point of failure (SPOF) on Source and Destination Network Address Translation (SNAT/DNAT) and (3)a single large IP subnet per tenant by using large layer-2 semantics in the consumers’ view. EYWA combines existing techniques into a decentralized scale-out control and data plane. The only component of EYWA is an agent in every hypervisor host that can control packets and the agents act as distributed controller. As a result, EYWA can be deployed into all the multi-tenant cloud environments today.
A quick simple presentation about how a company needs to use the OSI Model to look at building their network. Power, Cabling, Routers, and Switches are the most important items to start with; they are the foundation of your companies infrastructure!
Introducing the Future of Data Center Interconnect NetworksADVA
Our ADVA FSP 3000 CloudConnect™ is the future of Data Center Interconnect (DCI) networks. It’s a highly scalable, energy efficient and truly open platform. With our DCI technology, there are no more limits, no more restrictions. A new era of possibilities has arrived.
Harvard HPC Seminar Series
Theresa Kaltz, PhD, High Performance Technical Computing, FAS, Harvard
Due to the wide availability and low cost of high speed networking, commodity clusters have become the de facto standard for building high performance parallel computing systems. This talk will introduce the leading technology for high speed interconnects called Infiniband and compare its deployment and performance to Ethernet. In addition, some emerging interconnect technologies and trends in cluster networking will be discussed.
A Survey on Wireless Mesh Networks (WMN)Eyob Sisay
The network architectures of WMNs, Critical factors influencing protocol design or its design factors and Open Areas for Research on WMNs are discussed in this slide.
A Whole Lot of Ports: Juniper Networks QFabric System AssessmentJuniper Networks
Juniper Networks commissioned Network Test to assess the performance, interoperability, and usability of its QFabric System, a converged switch fabric for cloud and large data center applications tested with 1,536 10-Gbit/s Ethernet ports.
Even at this unprecedented scale – by far the largest ever in a public switch test – this project loaded the QFabric System to only one-quarter of its maximum capacity of 6,144 10-Gbit/s Ethernet ports.
Using industry-standard RFC benchmarks representing the most rigorous possible test cases, engineers stress-tested QFabric System performance in terms of unicast and multicast throughput and latency with separate events for Layer 2 and Layer 3 traffic. Engineers also assessed interoperability, a key consideration when adding QFabric technology incrementally into existing data center networks, and evaluated device management.
IBM Power9 Servers are here! Launched this week, the AC922 POWER9 servers will form the basis of the world’s fastest “Coral” supercomputers coming to ORNL and LLNL. Built specifically for compute-intensive AI workloads, the new POWER9 systems are capable of improving the training times of deep learning frameworks by nearly 4x allowing enterprises to build more accurate AI applications, faster.
Listen to the Radio Free HPC podcast on Power9: https://insidehpc.com/2017/12/radio-free-hpc-looks-new-power9-titan-v-snapdragon-845/
Learn more: https://www.ibm.com/us-en/marketplace/power-systems-ac922
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
High-performance 32G Fibre Channel Module on MDS 9700 Directors:Tony Antony
To better serve the new application requirements, Cisco is introducing a New high-performance Analytics ready 32G Fibre Channel Module on MDS 9700 Directors and a new 32G Host Bus Adapter for UCS C-series. The end to end 32G FC support across Cisco DC platforms set new standards for Storage Networking providing customers with choice. Along with this announcement, Cisco is also announcing NVMe over Fabric support on MDS 9000 Series enabling customers to take advantage of the performance and low latency benefits offered by the new technology to scale efficiently in the post-flash environments.
Ariel Waizel discusses the Data Plane Development Kit (DPDK), an API for developing fast packet processing code in user space.
* Who needs this library? Why bypass the kernel?
* How does it work?
* How good is it? What are the benchmarks?
* Pros and cons
Ariel worked on kernel development at the IDF, Ben Gurion University, and several companies. He is interested in networking, security, machine learning, and basically everything except UI development. Currently a Solution Architect at ConteXtream (an HPE company), which specializes in SDN solutions for the telecom industry.
Cloud Networking is not Virtual Networking - London VMUG 20130425Greg Ferro
Talking how and why virtual networking that we use today is not suitable for use in Cloud deployments. First I talk about the gap between "server" & "networks", then discuss the problems of virtual networking that we use today. Then into using software appliances instead of physical devices by highlighting the good & bad.
Then a brief overview of Software Defined Networking and how it will impact Cloud Networking in the next two years,
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
QsNetIII Adaptively Routed Network For HPC
1. QsNetIII an Adaptively Routed Network for
High Performance Computing
Duncan Roweth, Quadrics Ltd
Hot Interconnects August 2008
28/8/2008 Quadrics Ltd 1
2. Quadrics Background
• Develops interconnect products for the HPC market
– HPC Linux systems
– AlphaServer SC systems
• Quadrics is owned by the Finmeccanica group
• Quadrics was 12 years old in July
28/8/2008 Quadrics Ltd 2
5. Quadrics Networks
• Elan1 / Elite1, 1994, Meiko Computing Surface 2
– Source chooses between pre-defined routes
• Elan3 / Elite3, 2000, first Quadrics product, QsNet
– First use of packet-by-packet adaptive routing
– Crosspoint router, x8
• Elan4 / Elite4, 2004, QsNetII
– Reduced latency, increased bandwidth
– Increased support for offloading collectives
• Elan5 / Elite5, 2008, QsNetIII
– General purpose crosspoint router, increased radix, x32
– Highly programmable adapter
28/8/2008 Quadrics Ltd 5
6. What is Adaptive Routing ?
• Switch networks typically provide many
paths between any two points
• In an adaptively routed network
routers make packet by packet decisions
on the route to use based on
– Queue occupancy
– Channel usage
– Error rates and state
– Class of traffic
28/8/2008 Quadrics Ltd 6
7. Why is Adaptive Routing Important ?
• Most HPC networks are statically routed
– They use pre-determined paths between nodes
• Static routing can work well
– If traffic pattern is known in advance
– If traffic pattern is persistent
– If traffic pattern is uniform (i.e. application is load balanced)
– If there are no errors
• These conditions are not met by real codes on production
HPC systems {see LLNL and Sandia results}
• Adaptive routing solves these problems
– Delivering significantly better aggregate bandwidths and worst
case latencies on real systems running real codes
28/8/2008 Quadrics Ltd 7
8. Benefits of Adaptive Routing
• Bandwidth achieved
when 1024 nodes all
communicate at the
same time
• Plots show the
distribution of
measured bandwidths
System Interconnect Min Max Average
Atlas Infiniband 95 762 263
QsNetII
Thunder 248 403 369
Data from Lawrence Livermore National Lab, published at the Sonoma OpenFabrics workshop April 2007
28/8/2008 Quadrics Ltd 8
10. Ordering Considerations
• Adaptively routed packets can arrive out of order
– Problems for stream devices, e.g. multipath Ethernet
• Message ordering is required in HPC
– But within a message we are free to deliver the bulk data in
arbitrary order
Get it there as fast as possible then tell me that it is done
• QsNet ordering
– Packets contain the destination virtual address at which to write
the data
– Bulk data transfers can arrive out of order and can be replayed
– Atomic transactions are sequenced
28/8/2008 Quadrics Ltd 10
11. Adaptive Routing in QsNetIII
• More flexible than QsNetII
– Operates over arbitrary sets of links
– More opportunities to use the technique
– Higher radix switches
• Select a subset of lightly loaded output ports based on:
– Destination
– Link state, errors etc
– Number of pending acks (programmable threshold)
• Programmable algorithm for selecting from this subset:
– First free, next free, random
28/8/2008 Quadrics Ltd 11
12. Adaptive Routing: standard case
– All top switches are equivalent, select one
– Adaptive routing selects a lightly loaded path
28/8/2008 Quadrics Ltd 12
13. Implementation of Fat Tree Networks
• Connect M×N-way node switches by N×M-way top switches
• In this case M = 16, N = 4
28/8/2008 Quadrics Ltd 13
14. Adaptive Routing in the Top Switch
• If top switch radix ≤ router radix / 2
– i.e. 16 for Elite5, 2048-way networks
• Router provides multiple top switches
– Select which to use based on load
• Example:
– Traffic from A to B via routers 210 and
300 is blocked by traffic between 300
and 200.
– The router providing 300, 301, 302 and
303 can select a different path
28/8/2008 Quadrics Ltd 14
15. Adaptive Routing on the Final Hop
• Multiple connections to a node
• Switch can select a free path
• Reduces end-point contention
• Simple case is not optimal
• Spreading the connections
– Improves fault tolerance
– Reduces network contention
• Routing decision is made higher
in the network
28/8/2008 Quadrics Ltd 15
16. Adaptive routing in the presence of errors
• In a production system with 1000s
of links it is not uncommon for a
small number to be broken – until
the next maintenance slot
• Adaptive routing minimises the
impact
• Example:
– Link between routers 10 and 20 is
broken
– Router 10 dynamically selects paths
via 21,22,23 spreading the load.
– Reverse case, avoid sending to 10
via 20. Reset 20’s links or update
switches 11,12,13.
28/8/2008 Quadrics Ltd 16
17. Small Packet Support
• Aim to get as close to line rate as possible with small packets
• For example:
– Small put
– 32 byte packet
• Adapter has multiple packet engines
• Adapters support up to 64 outstanding packets per link
– Doubles if we use both links
• Switches provide 32 virtual channels per output link
• Prioritisation – buffering on input to the router
28/8/2008 Quadrics Ltd 17
18. Barrier & Broadcast Support
• Switches broadcast over
a range of output links
• Combine Acks / Nacks
• Contiguous in QsNetII
• Sparse in QsNetIII
• Barrier implementation
– Network conditional
– Broadcast release
28/8/2008 Quadrics Ltd 18
19. Elan5 – Device Overview
CX4/ CX4/
• 2× QSNetIII QSNetIII
QsNetIII links
– 20Gbit/s/direction after protocol
Elan5 Adapter
Link Link
• PCIe, PCIe2 host interface
• Multiple packet engines
Packet Engine Packet Engine Packet Engine Packet Engine Packet Engine Packet Engine Packet Engine
16K inst cache 16K inst cache 16K inst cache 16K inst cache 16K inst cache 16K inst cache 16K inst cache
9K data buffers 9K data buffers 9K data buffers 9K data buffers 9K data buffers 9K data buffers 9K data buffers
• 512KB of high bandwidth on
Fabric
chip local memory x8
• SDRAM interface to optional Bridge
Host I/F Local Memory Local Functions
Object Cache Tags
TLB
local memory Buffer Manager External cache
Cmd Launch
SDRAM i/f Ext i/f
Free List
PCIe
• Buffer manager, object 16K x 8 x 8 banks = 1MB ECC RAM PLL
SERDES
cache External EEPROM Clocks
PCIe
• Details in ISC Dresden
DDRII
16 Lanes
Paper
28/8/2008 Quadrics Ltd 19
20. Elite5 – Device Overview
• 64 × 32 crosspoint router
– Direct & buffered input from each link
– 8K of input buffering per link
• 32 virtual channels per link
• Physical layer DDR XAUI (6.25GHz)
• Adaptive routing
• Hardware barrier and broadcast
• Memory mapped stats & error
counters accessed out-of-band
28/8/2008 Quadrics Ltd 20
22. QsNetIII Implementation
• Node switch chassis
– 128 links down to the nodes
– 128 links up to the top switches
– Backplane connects 2 sets of cards
• Top switches QsNetIII switch
– 256 links down to the node switches logical design
– Range of system sizes:
Ports Radix Per Chassis
512 4 64
QsNetIII switch
1024 8 32
implementation
2048 16 16
4096 32 8
28/8/2008 Quadrics Ltd 22
23. QsNetIII Network 1024–way
• Fat tree, constructed from 8 × 128-way node switches connected by
128 × 8-way top switches
28/8/2008 Quadrics Ltd 23
24. QsNetIII Implementation – Cables
• QSFP connectors throughout
• Copper cables (e.g. Gore) 1-10m
• Active copper cables (e.g. Gore), 8-20m
• Optical cables (e.g. Luxtera), 5-300m
– PVDF Plenum rated
– LSZH available as an option
• No longer Quadrics proprietary
• Likely usage:
– Short copper cables from nodes
– Optical cables between switches
28/8/2008 Quadrics Ltd 24
25. QsNetIII Fault Tolerance
• All of the QsNetII Features
– CRCs on every packet
– Automatic retransmission
– Redundant routes
– Adaptive routing avoids failed links
– Redundant, hot plugable, PSUs and fans
+ Line rate testing of each link as it comes up
– Switches generate CRPAT, CJPAT or PRBS packets
– Links are only added to the route tables when they are (a) up, (b)
connect to the right place, and (c) can transfer data at full line rate
without error.
28/8/2008 Quadrics Ltd 25
26. QsNetIII Implementation – HP BladeSystem
Elan5 mezzanine adapter
Elite5 switch module
2 QsNet links, PCI-E x8 Gen2
Full bandwidth
128 MB of memory
16 links to the blades (via backplane)
16 links to back of the module
28/8/2008 Quadrics Ltd 26
27. Current Status
• Elite5 silicon in Bristol
• Elan5 at TSMC, first parts expected
in 3-4 weeks
• Switch PCBs, chassis, backplane,
controllers are working
• First adapter PCBs are ready
– PCI-Express x16, HP Blade,
ExpressModule (Sun Blade)
• We are porting the QsNetII software
• Components at SC08 in Austin
• First customer shipment in Q1 of 2009
28/8/2008 Quadrics Ltd 27
28. Future Work
• QsNetIII hardware
– Low cost 32-way switch
– 1024-way single chassis switch
• QsNetIII Software
– General framework for optimised collectives
– Support for “multiport” networks - “fat” nodes have multiple
connections to the same rail
– Ethernet firmware for the network adapter
28/8/2008 Quadrics Ltd 28
29. Conclusions
• Adaptive routing underwrites the scalability of HPC systems
designed to run a single large application
• Adaptive routing has been a feature of QsNet systems since 2000
• QsNetIII offers significant enhancements over both QsNetII and
competing products
28/8/2008 Quadrics Ltd 29
32. Packet Format
• Packet size of up to 4K made up of 256 byte packet segment and
continuations, 8 byte ACK
28/8/2008 Quadrics Ltd 32
33. Impact of static routing on latency
Data from Thunderbird cluster, Sandia National Lab
Big increases in worst case latency with number of nodes
28/8/2008 Quadrics Ltd 33
34. Impact of static routing on latency
Data from Thunderbird cluster, Sandia National Lab
Big variation in worst case latency across a large job
28/8/2008 Quadrics Ltd 34
35. Software Model – Firmware & Drivers
• Base firmware in the ROMs
• Firmware modules loadable with the device driver
– Elan, OpenFabrics, 10GE Ethernet, …
• Kernel modules
– elan5, elan, rms
• Device dependent library (libelan5)
• Device independent library (libelan)
• User libraries
28/8/2008 Quadrics Ltd 35
36. Software Model – Elan Libraries
• Point-to-point message • Optimised collectives
passing • Locks and atomics ops
• One-sided put/get • Global memory allocation
• Transparent rail striping
28/8/2008 Quadrics Ltd 36
37. QsNetIII Performance Summary
• Similar latencies to QsNetII
– The 1.3 to 2 microsecs of latency is mostly in the host PCI and
memory system
• Higher issue rates
– Improved link utilisation on small transfers
• Higher bandwidths
– 1.5 to 2.25 GB/sec/link depending on host interface
• Bi-directional host interface
– 2 x improvement over QsNetII
• Broadcast and barrier in hardware
• Continued development of adaptive routing underwrites scaling
to high node counts
28/8/2008 Quadrics Ltd 37