SlideShare a Scribd company logo

MPWide: A light-weight communication library for wide area message passing and code coupling

MPWide is a light-weight communication library for wide area message passing and code coupling. We originally developed MPWide to efficiently exchange data between supercomputers, allowing us to run a cosmological simulation in parallel across multiple machines in different countries. However, we also used MPWide for other purposes, such as coupling the 3D HemeLB bloodflow solver. Overall MPWide allows users to obtain excellent performance, incurring as little as 7% overhead for the cosmological simulations and 1% overhead in the multiscale bloodflow simulations. We manage to obtain this performance despite the presence of background traffic and, in some cases, non-optimized network configurations. I gave this presentation during the MAPPER Seasonal School in Barcelona in 2013. For more information, please feel free to contact me on Twitter or refer to my webpage.

1 of 14
Download to read offline
MPWide: A communication library
for wide area message passing

Derek Groen
Centre for Computational Science
Overview







The networking landscape
Using wide area networks
MPWide
Example applications
Uses for multiscale modelling
Questions
The networking landscape


The networks connecting grid sites and
supercomputers are highly heterogeneous.
−
−
−

Configurations differ at end points.
Shared paths vs. Dedicated paths
Optical interconnects vs. Regular interconnects.

A

C
B
The networking landscape


Fundamental issue: Networks configurations
tend to be node-specific, not path-specific.
−

What do we do when a node has multiple paths?


(most nodes nowadays do)

A

C
B
Using wide area networks (WANs)


Solution 1: Apply a homogeneous configuration
for all paths.
−

Could work for nodes with similar path lengths.


−
−

Not common for WAN communication nodes.

Inefficient for the TCP protocol, where the optimal
config is dependent on the path length.
Requires admin privileges on all end-points.

A

A
A
Using WANs


Solution 2: Adopt a different protocol.
−
−
−
−

May accomodate heterogeneous configs.
New protocol, new list of potential issues.
Interplay between protocols on shared networks.
Time-consuming and politically heavyweight
process.

խ

ե
Ր

Recommended

Computing localized power efficient data
Computing localized power efficient dataComputing localized power efficient data
Computing localized power efficient dataambitlick
 
An Adaptive Routing Algorithm for Communication Networks using Back Pressure...
An Adaptive Routing Algorithm for Communication Networks  using Back Pressure...An Adaptive Routing Algorithm for Communication Networks  using Back Pressure...
An Adaptive Routing Algorithm for Communication Networks using Back Pressure...IJMER
 
Link Capacity Estimation in SDN-based End-hosts
Link Capacity Estimation in SDN-based End-hostsLink Capacity Estimation in SDN-based End-hosts
Link Capacity Estimation in SDN-based End-hostsFarzaneh Pakzad
 
Mobile data gathering with load balanced
Mobile data gathering with load balancedMobile data gathering with load balanced
Mobile data gathering with load balancedjpstudcorner
 
MANET Experiment - I (Using Network Simulator NetSim -www.tetcos.com)
MANET Experiment - I (Using Network Simulator NetSim -www.tetcos.com)MANET Experiment - I (Using Network Simulator NetSim -www.tetcos.com)
MANET Experiment - I (Using Network Simulator NetSim -www.tetcos.com)Amulya Naik
 
JPN1406 Snapshot and Continuous Data Collection in Probabilistic Wireless S...
JPN1406   Snapshot and Continuous Data Collection in Probabilistic Wireless S...JPN1406   Snapshot and Continuous Data Collection in Probabilistic Wireless S...
JPN1406 Snapshot and Continuous Data Collection in Probabilistic Wireless S...chennaijp
 

More Related Content

What's hot

An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...
An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...
An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...Editor IJCATR
 
SCOR: Constraint Programming-based Northbound Interface for SDN
SCOR: Constraint Programming-based Northbound Interface for SDNSCOR: Constraint Programming-based Northbound Interface for SDN
SCOR: Constraint Programming-based Northbound Interface for SDNFarzaneh Pakzad
 
Evaluation of mininet WiFi integration via ns-3
Evaluation of mininet WiFi integration via ns-3Evaluation of mininet WiFi integration via ns-3
Evaluation of mininet WiFi integration via ns-3Farzaneh Pakzad
 
SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...
SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...
SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...IJCNCJournal
 
M phil-computer-science-mobile-computing-projects
M phil-computer-science-mobile-computing-projectsM phil-computer-science-mobile-computing-projects
M phil-computer-science-mobile-computing-projectsVijay Karan
 
M.E Computer Science Mobile Computing Projects
M.E Computer Science Mobile Computing ProjectsM.E Computer Science Mobile Computing Projects
M.E Computer Science Mobile Computing ProjectsVijay Karan
 
Mobile data gathering with load balanced
Mobile data gathering with load balancedMobile data gathering with load balanced
Mobile data gathering with load balancedjpstudcorner
 
Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...
Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...
Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...1crore projects
 
Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...
Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...
Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...Md Nazrul Islam Roxy
 
Clustering and data aggregation scheme in underwater wireless acoustic sensor...
Clustering and data aggregation scheme in underwater wireless acoustic sensor...Clustering and data aggregation scheme in underwater wireless acoustic sensor...
Clustering and data aggregation scheme in underwater wireless acoustic sensor...TELKOMNIKA JOURNAL
 
IEEE 2014 NS2 NETWORKING PROJECTS Fast regular expression matching using sma...
IEEE 2014 NS2 NETWORKING PROJECTS  Fast regular expression matching using sma...IEEE 2014 NS2 NETWORKING PROJECTS  Fast regular expression matching using sma...
IEEE 2014 NS2 NETWORKING PROJECTS Fast regular expression matching using sma...IEEEBEBTECHSTUDENTPROJECTS
 
Data gathering in wireless sensor networks using intermediate nodes
Data gathering in wireless sensor networks using intermediate nodesData gathering in wireless sensor networks using intermediate nodes
Data gathering in wireless sensor networks using intermediate nodesIJCNCJournal
 
Basic communication operations - One to all Broadcast
Basic communication operations - One to all BroadcastBasic communication operations - One to all Broadcast
Basic communication operations - One to all BroadcastRashiJoshi11
 
Cross layer design of congestion control and power control in fast-fading wir...
Cross layer design of congestion control and power control in fast-fading wir...Cross layer design of congestion control and power control in fast-fading wir...
Cross layer design of congestion control and power control in fast-fading wir...Ecway Technologies
 

What's hot (14)

An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...
An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...
An Improved Energy Efficient Wireless Sensor Networks Through Clustering In C...
 
SCOR: Constraint Programming-based Northbound Interface for SDN
SCOR: Constraint Programming-based Northbound Interface for SDNSCOR: Constraint Programming-based Northbound Interface for SDN
SCOR: Constraint Programming-based Northbound Interface for SDN
 
Evaluation of mininet WiFi integration via ns-3
Evaluation of mininet WiFi integration via ns-3Evaluation of mininet WiFi integration via ns-3
Evaluation of mininet WiFi integration via ns-3
 
SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...
SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...
SECTOR TREE-BASED CLUSTERING FOR ENERGY EFFICIENT ROUTING PROTOCOL IN HETEROG...
 
M phil-computer-science-mobile-computing-projects
M phil-computer-science-mobile-computing-projectsM phil-computer-science-mobile-computing-projects
M phil-computer-science-mobile-computing-projects
 
M.E Computer Science Mobile Computing Projects
M.E Computer Science Mobile Computing ProjectsM.E Computer Science Mobile Computing Projects
M.E Computer Science Mobile Computing Projects
 
Mobile data gathering with load balanced
Mobile data gathering with load balancedMobile data gathering with load balanced
Mobile data gathering with load balanced
 
Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...
Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...
Mobile Data Gathering with Load Balanced Clustering and Dual Data Uploading i...
 
Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...
Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...
Location and Mobility Aware Resource Management for 5G Cloud Radio Access Net...
 
Clustering and data aggregation scheme in underwater wireless acoustic sensor...
Clustering and data aggregation scheme in underwater wireless acoustic sensor...Clustering and data aggregation scheme in underwater wireless acoustic sensor...
Clustering and data aggregation scheme in underwater wireless acoustic sensor...
 
IEEE 2014 NS2 NETWORKING PROJECTS Fast regular expression matching using sma...
IEEE 2014 NS2 NETWORKING PROJECTS  Fast regular expression matching using sma...IEEE 2014 NS2 NETWORKING PROJECTS  Fast regular expression matching using sma...
IEEE 2014 NS2 NETWORKING PROJECTS Fast regular expression matching using sma...
 
Data gathering in wireless sensor networks using intermediate nodes
Data gathering in wireless sensor networks using intermediate nodesData gathering in wireless sensor networks using intermediate nodes
Data gathering in wireless sensor networks using intermediate nodes
 
Basic communication operations - One to all Broadcast
Basic communication operations - One to all BroadcastBasic communication operations - One to all Broadcast
Basic communication operations - One to all Broadcast
 
Cross layer design of congestion control and power control in fast-fading wir...
Cross layer design of congestion control and power control in fast-fading wir...Cross layer design of congestion control and power control in fast-fading wir...
Cross layer design of congestion control and power control in fast-fading wir...
 

Viewers also liked

Scholarly publishing a perspective from an early career academic
Scholarly publishing  a perspective from an early career academicScholarly publishing  a perspective from an early career academic
Scholarly publishing a perspective from an early career academicDerek Groen
 
Supercomputing 2013 slides
Supercomputing 2013 slidesSupercomputing 2013 slides
Supercomputing 2013 slidesDerek Groen
 
HemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slidesHemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slidesDerek Groen
 
On science hackathons univercite 2016
On science hackathons univercite 2016On science hackathons univercite 2016
On science hackathons univercite 2016Derek Groen
 
Multiscale computing in 15 minutes
Multiscale computing in 15 minutesMultiscale computing in 15 minutes
Multiscale computing in 15 minutesDerek Groen
 
The Building Blocks of Science
The Building Blocks of ScienceThe Building Blocks of Science
The Building Blocks of ScienceDerek Groen
 

Viewers also liked (6)

Scholarly publishing a perspective from an early career academic
Scholarly publishing  a perspective from an early career academicScholarly publishing  a perspective from an early career academic
Scholarly publishing a perspective from an early career academic
 
Supercomputing 2013 slides
Supercomputing 2013 slidesSupercomputing 2013 slides
Supercomputing 2013 slides
 
HemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slidesHemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slides
 
On science hackathons univercite 2016
On science hackathons univercite 2016On science hackathons univercite 2016
On science hackathons univercite 2016
 
Multiscale computing in 15 minutes
Multiscale computing in 15 minutesMultiscale computing in 15 minutes
Multiscale computing in 15 minutes
 
The Building Blocks of Science
The Building Blocks of ScienceThe Building Blocks of Science
The Building Blocks of Science
 

Similar to MPWide: A light-weight communication library for wide area message passing and code coupling

Hybrid networking and distribution
Hybrid networking and distribution Hybrid networking and distribution
Hybrid networking and distribution vivek pratap singh
 
Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...
Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...
Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...CSCJournals
 
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSRIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSijwmn
 
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSRIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSijwmn
 
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSRIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSijwmn
 
MuMHR: Multi-path, Multi-hop Hierarchical Routing
MuMHR: Multi-path, Multi-hop Hierarchical RoutingMuMHR: Multi-path, Multi-hop Hierarchical Routing
MuMHR: Multi-path, Multi-hop Hierarchical RoutingM H
 
Multi-Channel Multi-Interface Wireless Network Architecture
Multi-Channel Multi-Interface Wireless Network ArchitectureMulti-Channel Multi-Interface Wireless Network Architecture
Multi-Channel Multi-Interface Wireless Network Architectureijsrd.com
 
Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...
Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...
Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...CSCJournals
 
A Survey on Wireless Mesh Networks (WMN)
A Survey on Wireless Mesh Networks (WMN)A Survey on Wireless Mesh Networks (WMN)
A Survey on Wireless Mesh Networks (WMN)Eyob Sisay
 
Ieee transactions 2018 topics on wireless communications for final year stude...
Ieee transactions 2018 topics on wireless communications for final year stude...Ieee transactions 2018 topics on wireless communications for final year stude...
Ieee transactions 2018 topics on wireless communications for final year stude...tsysglobalsolutions
 
Ieee transactions 2018 on wireless communications Title and Abstract
Ieee transactions 2018 on wireless communications Title and AbstractIeee transactions 2018 on wireless communications Title and Abstract
Ieee transactions 2018 on wireless communications Title and Abstracttsysglobalsolutions
 
M.Phil Computer Science Mobile Computing Projects
M.Phil Computer Science Mobile Computing ProjectsM.Phil Computer Science Mobile Computing Projects
M.Phil Computer Science Mobile Computing ProjectsVijay Karan
 
HPC with Clouds and Cloud Technologies
HPC with Clouds and Cloud TechnologiesHPC with Clouds and Cloud Technologies
HPC with Clouds and Cloud TechnologiesInderjeet Singh
 
DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...
DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...
DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...IEEEGLOBALSOFTTECHNOLOGIES
 
JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...
JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...
JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...IEEEGLOBALSOFTTECHNOLOGIES
 
Mobile relay configuration in data intensive wireless sensor networks
Mobile relay configuration in data intensive wireless sensor networksMobile relay configuration in data intensive wireless sensor networks
Mobile relay configuration in data intensive wireless sensor networksIEEEFINALYEARPROJECTS
 
Application Aware Topology Generation for Surface Wave Networks-on-Chip
Application Aware Topology Generation for Surface Wave Networks-on-ChipApplication Aware Topology Generation for Surface Wave Networks-on-Chip
Application Aware Topology Generation for Surface Wave Networks-on-Chipzhao fu
 

Similar to MPWide: A light-weight communication library for wide area message passing and code coupling (20)

Hybrid networking and distribution
Hybrid networking and distribution Hybrid networking and distribution
Hybrid networking and distribution
 
Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...
Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...
Energy Behavior in Ad Hoc Network Minimizing the Number of Hops and Maintaini...
 
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSRIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
 
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSRIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
 
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONSRIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
RIVERBED-BASED NETWORK MODELING FOR MULTI-BEAM CONCURRENT TRANSMISSIONS
 
MuMHR: Multi-path, Multi-hop Hierarchical Routing
MuMHR: Multi-path, Multi-hop Hierarchical RoutingMuMHR: Multi-path, Multi-hop Hierarchical Routing
MuMHR: Multi-path, Multi-hop Hierarchical Routing
 
Multi-Channel Multi-Interface Wireless Network Architecture
Multi-Channel Multi-Interface Wireless Network ArchitectureMulti-Channel Multi-Interface Wireless Network Architecture
Multi-Channel Multi-Interface Wireless Network Architecture
 
Gurpinder_Resume
Gurpinder_ResumeGurpinder_Resume
Gurpinder_Resume
 
Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...
Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...
Energy Consumption in Ad Hoc Network With Agents Minimizing the Number of Hop...
 
A Survey on Wireless Mesh Networks (WMN)
A Survey on Wireless Mesh Networks (WMN)A Survey on Wireless Mesh Networks (WMN)
A Survey on Wireless Mesh Networks (WMN)
 
Ieee transactions 2018 topics on wireless communications for final year stude...
Ieee transactions 2018 topics on wireless communications for final year stude...Ieee transactions 2018 topics on wireless communications for final year stude...
Ieee transactions 2018 topics on wireless communications for final year stude...
 
Ieee transactions 2018 on wireless communications Title and Abstract
Ieee transactions 2018 on wireless communications Title and AbstractIeee transactions 2018 on wireless communications Title and Abstract
Ieee transactions 2018 on wireless communications Title and Abstract
 
M.Phil Computer Science Mobile Computing Projects
M.Phil Computer Science Mobile Computing ProjectsM.Phil Computer Science Mobile Computing Projects
M.Phil Computer Science Mobile Computing Projects
 
HPC with Clouds and Cloud Technologies
HPC with Clouds and Cloud TechnologiesHPC with Clouds and Cloud Technologies
HPC with Clouds and Cloud Technologies
 
Mq3624532158
Mq3624532158Mq3624532158
Mq3624532158
 
DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...
DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...
DOTNET 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data i...
 
JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...
JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...
JAVA 2013 IEEE MOBILECOMPUTING PROJECT Mobile relay configuration in data int...
 
Mobile relay configuration in data intensive wireless sensor networks
Mobile relay configuration in data intensive wireless sensor networksMobile relay configuration in data intensive wireless sensor networks
Mobile relay configuration in data intensive wireless sensor networks
 
[IJCT-V3I2P21] Authors: Swati Govil, Dr.Paramjeet Rawat
[IJCT-V3I2P21] Authors: Swati Govil, Dr.Paramjeet Rawat[IJCT-V3I2P21] Authors: Swati Govil, Dr.Paramjeet Rawat
[IJCT-V3I2P21] Authors: Swati Govil, Dr.Paramjeet Rawat
 
Application Aware Topology Generation for Surface Wave Networks-on-Chip
Application Aware Topology Generation for Surface Wave Networks-on-ChipApplication Aware Topology Generation for Surface Wave Networks-on-Chip
Application Aware Topology Generation for Surface Wave Networks-on-Chip
 

Recently uploaded

Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerSaiLinnThu2
 
Pragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdfPragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdfinfogdgmi
 
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...SearchNorwich
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxNeo4j
 
Huntly presentation deck design for Behance
Huntly presentation deck design for BehanceHuntly presentation deck design for Behance
Huntly presentation deck design for Behancewhalesdesign
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanDatabarracks
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Product School
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
 
Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, MonzoRevolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, MonzoProduct School
 
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)François
 
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...Cprime
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...DianaGray10
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationAppsthentic Technology
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceVijayananda Mohire
 
Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...Product School
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Jay Zhao
 
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Product School
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxNeo4j
 
Q4 2023 Quarterly Investor Presentation - FINAL.pdf
Q4 2023 Quarterly Investor Presentation - FINAL.pdfQ4 2023 Quarterly Investor Presentation - FINAL.pdf
Q4 2023 Quarterly Investor Presentation - FINAL.pdfTejal81
 
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Product School
 

Recently uploaded (20)

Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
 
Pragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdfPragmatic UI testing with Compose Semantics.pdf
Pragmatic UI testing with Compose Semantics.pdf
 
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
 
Huntly presentation deck design for Behance
Huntly presentation deck design for BehanceHuntly presentation deck design for Behance
Huntly presentation deck design for Behance
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response Plan
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficient
 
Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, MonzoRevolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
 
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
 
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
Improving IT Investment Decisions and Business Outcomes with Integrated Enter...
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
 
Mastering Play Store App Listing and Optimization
Mastering Play Store App Listing and OptimizationMastering Play Store App Listing and Optimization
Mastering Play Store App Listing and Optimization
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial Intelligence
 
Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
 
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
 
Q4 2023 Quarterly Investor Presentation - FINAL.pdf
Q4 2023 Quarterly Investor Presentation - FINAL.pdfQ4 2023 Quarterly Investor Presentation - FINAL.pdf
Q4 2023 Quarterly Investor Presentation - FINAL.pdf
 
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
Cultivating Entrepreneurial Mindset in Product Management: Strategies for Suc...
 

MPWide: A light-weight communication library for wide area message passing and code coupling

  • 1. MPWide: A communication library for wide area message passing Derek Groen Centre for Computational Science
  • 2. Overview       The networking landscape Using wide area networks MPWide Example applications Uses for multiscale modelling Questions
  • 3. The networking landscape  The networks connecting grid sites and supercomputers are highly heterogeneous. − − − Configurations differ at end points. Shared paths vs. Dedicated paths Optical interconnects vs. Regular interconnects. A C B
  • 4. The networking landscape  Fundamental issue: Networks configurations tend to be node-specific, not path-specific. − What do we do when a node has multiple paths?  (most nodes nowadays do) A C B
  • 5. Using wide area networks (WANs)  Solution 1: Apply a homogeneous configuration for all paths. − Could work for nodes with similar path lengths.  − − Not common for WAN communication nodes. Inefficient for the TCP protocol, where the optimal config is dependent on the path length. Requires admin privileges on all end-points. A A A
  • 6. Using WANs  Solution 2: Adopt a different protocol. − − − − May accomodate heterogeneous configs. New protocol, new list of potential issues. Interplay between protocols on shared networks. Time-consuming and politically heavyweight process. խ ե Ր
  • 7. Using WANs  Solution 3: User-space tuning through software. − Limited space for tuning.  − − Some adjustments require admin rights. Use TCP protocol and existing configurations. No special privileges required. X A Y Y B Z X C Z
  • 8. MPWide   MPWide is a communication library which allows for user-space tuning of individual paths. For each path it can: − Use 1 or multiple tcp streams.  − − Good performance obtained with up to 128 streams/path. Configure different buffer and packet sizes. Apply software-based packet pacing to reduce load.  Also improves performance on long networks (Yoshino et al. 2008).
  • 9. Example: cosmological N-body      One simulation, parallelized across supercomputers. Uses the SUSHI code, which is a cross-site adaptation of GreeM. Models dark matter structure formation over 13.4 billion years. Algorithm: Tree + Particle-mesh. Adaptive load-balancing between sites. →
  • 10. Example: cosmological N-body  Using 2 to 4 supercomputers simultaneously. −     Up to 2048 cores total. MPI within each site. Custom MPWide connections between sites. MPWide Forwarder procs bypass connectivity restrictions. 2048 cores, 3 sites, 7% comm. overhead. 10 Gbps lightpath
  • 12. Example: multiscale bloodflow   pyNS (1D) coupled to HemeLB (3D). 400.000 time steps, 4000 velocity exchanges − − with 1.2% comm. overhead (512+1 cores, 2298 s), and 5% comm. overhead (2048+1 cores, 907 s.).
  • 13. Uses for multiscale modelling  Can be used for performance critical cyclic coupling over wide area networks. −     High-performance, simple low-level interface. Contains an mpw-cp file transfer client to accelerate file-based couplings. Supports C, C++, Python. Trivial to install and intended for users without administrative privileges. Is being integrated into MUSCLE 2 to improve its coupling performance.
  • 14. Thank you!  MPWide website: −  More on the multiscale bloodflow application: −  Groen et al., Interface Focus 3(2), 2013. More on the cosmological N-body application: −  http://castle.strw.leidenuniv.nl/software/mpwide.html Groen et al., INFOCOMP 2011, ArXiv:1109.5559. Thanks go out to Steven Rieder, Simon Portegies Zwart, Tomoaki Ishiyama, Keigo Nitadori, Joris Borgdorff, Rupert Nash and the MAPPER consortium as a whole.