SlideShare a Scribd company logo

Gravitational Billion Body Project

The document summarizes research on running cosmological N-body simulations across multiple supercomputers located in different countries. Key points: 1) Researchers were able to run simulations on up to 750 cores distributed across 4 supercomputers in different locations, achieving up to 92% performance of a single supercomputer. 2) They developed software called SUSHI (Simulating Universe Structure formation on Heterogeneous Infrastructures) that uses tree and particle-mesh algorithms to simulate structure formation across supercomputers over long and short distances. 3) Timing results showed simulations spent about 90% of time on calculations when run across 2-3 supercomputer sites separated by up to 30,000 km of network cable.

1 of 1
Download to read offline
The Gravitational Billion Body Project
a

a

a

Authors: Derek Groen , Steven Rieder , Simon Portegies Zwart ,
b
b
Tomoaki Ishiyama , Jun Makino

Introduction

The Application

We report on cosmological N-body
simulations which run over up to 4
supercomputers across the globe. We
achieved to run simulations on 60 to 750
cores distributed over a variety of
supercomputers. Regardless of the
network latency of 0.32 s and the
communication over 30.000 km of optical
network cable we are able to achieve up
to 92% of the performance compared to
an equal number of cores on a single
supercomputer.

Our cosmological code simulates structure
formation in the universe by integrating the
gravitational forces between dark matter
particles over time. This code, which is based on
1
2
GreeM , uses Barnes-Hut Tree integration to
resolve force inter-actions over short distances,
3
and Particle-Mesh integration to resolve
interactions over long distances. We have
4
coupled our code with MPWide to enable
simulations across supercomputers. Our code is
named SUSHII.

Results
Timing results of our simulations per step
averaged over 10 steps can be found in
the table. Here, the simulations spend
about 90% of the total time on calcutions
when performed across 2 or 3 sites.
For our experiments we have used one
IBM Power6 supercomputer and three
Cray-XT4 machines. The IBM resides at
SARA in Amsterdam(NL) and the Cray
machines reside at EPCC in Edinburgh
(UK), CSC in Espoo (FI) and CFCA in
Tokyo (JP).

I)

SUSHI stands for Simulating Universe Structure formation on
Heterogeneous Infrastructures.

particles procs sites time/step comm. time
#
#
#
[s]
[s]
5123
120
1
71,04
1,05
5123
120
2
61,62
4,59
5123
120
3
56,31
4,78
5123
120
4
70,1
19,26
10243
240
1
272
3,4
10243
240
2
252
21,98
10243
240
3
294,7
31,28
20483
750
2
483,4
46,5
Runs over 1 site were performed at SARA
only, runs over 2 sites also at EPCC, and
runs over 3 sites also at CSC. The 20483
run was performed at SARA and CFCA.

The picture in the background is a colorized density plot
of the simulation data at redshift z=5.65. We performed
the run over 4 sites and the slices are colored to match
volumes residing at CFCA, CSC, EPCC and SARA.

120 cores total

Higher accuracy tree integration
using an opening angle of 0.3.

Lower accuracy tree integration
using an opening angle of 0.5.

3

Timings per step of a 512 run over 3 sites.

Network Setup
All supercomputers have been connected by
optical networks. The DEISA network is shared
with other users. Communication nodes are
shown by the green boxes.

Conclusion
We have run cosmological simulations efficiently
across multiple supercomputers. The scale of our
experiments is constrained by the political
overhead of scheduling the application across
supercomputers. The use of a meta-scheduler
and reservation system that works across sites
will enable us to perform long-lasting and large
production runs on a grid of supercomputers.

References
1. T. Ishiyama, T. Fukushige, and J. Makino, “GreeM : Massively ParallelTreePM
Code for Large Cosmological N-body Simulations,” accepted by PASJ.
2. J. Barnes and P. Hut, “A Hierarchical O(NlogN) Force-Calculation Algorithm,”
Nature, vol. 324, pp. 446–449, Dec. 1986.
3. R. Hockney and J. Eastwood, “Computer Simulation Using Particles”, New
York: McGraw-Hill, 1981.
4. D. Groen, S. Rieder, P. Grosso, C. de Laat and S. Portegies Zwart, “A lightweight communication library for distributed computing,” (submitted to CSD).

Affiliations
A. Leiden Observatory, Leiden University, Leiden, the Netherlands.
B. Center for Computational Astrophysics, Mitaka, Tokyo, Japan.

Acknowledgements
This research is supported by the Netherlands organization for Scientific
research (NWO) grant #639.073.803, #643.200.503 and #643.000.803, the
Stichting Nationale Computerfaciliteiten (project #SH-095-08), NAOJ, SURFNet
(GigaPort project), the Netherlands Advanced School for Astronomy (NOVA)
and the Leids Kerkhoven-Bosscha fonds (LKBF). We thank the DEISA
Consortium (www.deisa.eu), co-funded through the EU FP6 project RI-031513
and the FP7 project RI-222919, for support within the DEISA Extreme
Computing Initiative.
We thank the network facilities of SURFnet,, IEEAF, WIDE, Northwest Gigapop
and the Global Lambda Integrated FAcility (GLIF) GOLE of Trans-Light Cisco on
National LambdaRail, TransLight, StarLight, NetherLight, T-LEX, Pacific and
Atlantic Wave.

Recommended

More Related Content

What's hot

Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...
Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...
Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...idescitation
 
Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)
Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)
Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)Ahmed Abdel-Basit
 
Accelerating Science with Generative Adversarial Networks
Accelerating Science with Generative Adversarial NetworksAccelerating Science with Generative Adversarial Networks
Accelerating Science with Generative Adversarial NetworksMichela Paganini
 
Portable Energy-Aware Cluster-Based Edge Computers
Portable Energy-Aware Cluster-Based Edge ComputersPortable Energy-Aware Cluster-Based Edge Computers
Portable Energy-Aware Cluster-Based Edge ComputersThomas Rausch
 
Unsupervised Learning: Similarities and distance functions for IoT data
Unsupervised Learning: Similarities and distance functions for IoT dataUnsupervised Learning: Similarities and distance functions for IoT data
Unsupervised Learning: Similarities and distance functions for IoT dataGiampaoloCasolla
 
Cluster formation over huge volatile robotic data
Cluster formation over huge volatile robotic data Cluster formation over huge volatile robotic data
Cluster formation over huge volatile robotic data Eirini Ntoutsi
 
Storm surge simulation
Storm surge simulationStorm surge simulation
Storm surge simulationFei Liu
 
Scaling Deep Learning Models for Large Spatial Time-Series Forecasting
Scaling Deep Learning Models for Large Spatial Time-Series ForecastingScaling Deep Learning Models for Large Spatial Time-Series Forecasting
Scaling Deep Learning Models for Large Spatial Time-Series ForecastingZainab Abbas
 
Data Science in the Rough
Data Science in the RoughData Science in the Rough
Data Science in the RoughEli Bressert
 
Magnetic Field Line Tangling and Topological Entropy
Magnetic Field Line Tangling and Topological EntropyMagnetic Field Line Tangling and Topological Entropy
Magnetic Field Line Tangling and Topological EntropySimon Candelaresi
 
CLUSTERING HYPERSPECTRAL DATA
CLUSTERING HYPERSPECTRAL DATACLUSTERING HYPERSPECTRAL DATA
CLUSTERING HYPERSPECTRAL DATAcsandit
 
Processing of raw astronomical data of large volume by map reduce model
Processing of raw astronomical data of large volume by map reduce modelProcessing of raw astronomical data of large volume by map reduce model
Processing of raw astronomical data of large volume by map reduce modelSergey Gerasimov
 
Weather forecasting by anfis
Weather forecasting by anfisWeather forecasting by anfis
Weather forecasting by anfisDUAA J. IBRAHIM
 
Machine learning astronomical structure
Machine learning astronomical structureMachine learning astronomical structure
Machine learning astronomical structurePanditNitesh
 
Machine learning and climate and weather research
Machine learning and climate and weather researchMachine learning and climate and weather research
Machine learning and climate and weather researchDrMariaMorenodeCastr
 

What's hot (20)

Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...
Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...
Comparison of Proposed Kite Architecture with P- Hexagon for Directional Sens...
 
Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)
Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)
Ahmed-Abdul-Basit-Mahommed-June.-2016 (1)
 
Accelerating Science with Generative Adversarial Networks
Accelerating Science with Generative Adversarial NetworksAccelerating Science with Generative Adversarial Networks
Accelerating Science with Generative Adversarial Networks
 
Portable Energy-Aware Cluster-Based Edge Computers
Portable Energy-Aware Cluster-Based Edge ComputersPortable Energy-Aware Cluster-Based Edge Computers
Portable Energy-Aware Cluster-Based Edge Computers
 
Unsupervised Learning: Similarities and distance functions for IoT data
Unsupervised Learning: Similarities and distance functions for IoT dataUnsupervised Learning: Similarities and distance functions for IoT data
Unsupervised Learning: Similarities and distance functions for IoT data
 
Cluster formation over huge volatile robotic data
Cluster formation over huge volatile robotic data Cluster formation over huge volatile robotic data
Cluster formation over huge volatile robotic data
 
Storm surge simulation
Storm surge simulationStorm surge simulation
Storm surge simulation
 
Scaling Deep Learning Models for Large Spatial Time-Series Forecasting
Scaling Deep Learning Models for Large Spatial Time-Series ForecastingScaling Deep Learning Models for Large Spatial Time-Series Forecasting
Scaling Deep Learning Models for Large Spatial Time-Series Forecasting
 
sp2015ren
sp2015rensp2015ren
sp2015ren
 
Data Science in the Rough
Data Science in the RoughData Science in the Rough
Data Science in the Rough
 
Magnetic Field Line Tangling and Topological Entropy
Magnetic Field Line Tangling and Topological EntropyMagnetic Field Line Tangling and Topological Entropy
Magnetic Field Line Tangling and Topological Entropy
 
Machine Learning in Astrophysics
Machine Learning in AstrophysicsMachine Learning in Astrophysics
Machine Learning in Astrophysics
 
CLUSTERING HYPERSPECTRAL DATA
CLUSTERING HYPERSPECTRAL DATACLUSTERING HYPERSPECTRAL DATA
CLUSTERING HYPERSPECTRAL DATA
 
Processing of raw astronomical data of large volume by map reduce model
Processing of raw astronomical data of large volume by map reduce modelProcessing of raw astronomical data of large volume by map reduce model
Processing of raw astronomical data of large volume by map reduce model
 
Eeuc111
Eeuc111Eeuc111
Eeuc111
 
P1151133713
P1151133713P1151133713
P1151133713
 
Weather forecasting by anfis
Weather forecasting by anfisWeather forecasting by anfis
Weather forecasting by anfis
 
Poster: MMSP 2008
Poster: MMSP 2008Poster: MMSP 2008
Poster: MMSP 2008
 
Machine learning astronomical structure
Machine learning astronomical structureMachine learning astronomical structure
Machine learning astronomical structure
 
Machine learning and climate and weather research
Machine learning and climate and weather researchMachine learning and climate and weather research
Machine learning and climate and weather research
 

Viewers also liked

10 dicas para o noivo no dia do casamento
10 dicas para o noivo no dia do casamento10 dicas para o noivo no dia do casamento
10 dicas para o noivo no dia do casamentoBomEvento
 
Botar o desodorizante
Botar o desodorizanteBotar o desodorizante
Botar o desodorizanteticceeabarcia
 
ост 1 11319 73
ост 1 11319 73ост 1 11319 73
ост 1 11319 73Grant Cole
 
1901097rev1 apostilaeletrodosrevestidos ok
1901097rev1 apostilaeletrodosrevestidos ok1901097rev1 apostilaeletrodosrevestidos ok
1901097rev1 apostilaeletrodosrevestidos okjcmora01
 
Belarus construction regulations 737
Belarus construction regulations 737Belarus construction regulations 737
Belarus construction regulations 737Grant Cole
 
Modul media pembelajaran
Modul media pembelajaranModul media pembelajaran
Modul media pembelajarannurhasniatin072
 
Evaluation questions
Evaluation questionsEvaluation questions
Evaluation questionsgreenie46
 
Business russia 1387
Business russia 1387Business russia 1387
Business russia 1387Grant Cole
 
10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...
10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...
10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...b2bhub
 
Conociendolavidauniversitaria 121020140654-phpapp01
Conociendolavidauniversitaria 121020140654-phpapp01Conociendolavidauniversitaria 121020140654-phpapp01
Conociendolavidauniversitaria 121020140654-phpapp01nilda2019
 

Viewers also liked (19)

200902 france nicolas_duntze_droitsbafouéshaldelà
200902 france nicolas_duntze_droitsbafouéshaldelà200902 france nicolas_duntze_droitsbafouéshaldelà
200902 france nicolas_duntze_droitsbafouéshaldelà
 
Tutorial 6
Tutorial 6Tutorial 6
Tutorial 6
 
Practica palomino
Practica palominoPractica palomino
Practica palomino
 
10 dicas para o noivo no dia do casamento
10 dicas para o noivo no dia do casamento10 dicas para o noivo no dia do casamento
10 dicas para o noivo no dia do casamento
 
Botar o desodorizante
Botar o desodorizanteBotar o desodorizante
Botar o desodorizante
 
ост 1 11319 73
ост 1 11319 73ост 1 11319 73
ост 1 11319 73
 
παρουσίαση
παρουσίασηπαρουσίαση
παρουσίαση
 
950946
950946950946
950946
 
1901097rev1 apostilaeletrodosrevestidos ok
1901097rev1 apostilaeletrodosrevestidos ok1901097rev1 apostilaeletrodosrevestidos ok
1901097rev1 apostilaeletrodosrevestidos ok
 
Belarus construction regulations 737
Belarus construction regulations 737Belarus construction regulations 737
Belarus construction regulations 737
 
Modul media pembelajaran
Modul media pembelajaranModul media pembelajaran
Modul media pembelajaran
 
Evaluation questions
Evaluation questionsEvaluation questions
Evaluation questions
 
Joseph_Resume
Joseph_ResumeJoseph_Resume
Joseph_Resume
 
Sistema operativo
Sistema operativoSistema operativo
Sistema operativo
 
Business russia 1387
Business russia 1387Business russia 1387
Business russia 1387
 
10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...
10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...
10.12 SMMForum 2015: Панель 1/ Fitness Geek Camp: Как использовать личный бр...
 
Ukrsepro 1360
Ukrsepro 1360Ukrsepro 1360
Ukrsepro 1360
 
Diaguitas
Diaguitas Diaguitas
Diaguitas
 
Conociendolavidauniversitaria 121020140654-phpapp01
Conociendolavidauniversitaria 121020140654-phpapp01Conociendolavidauniversitaria 121020140654-phpapp01
Conociendolavidauniversitaria 121020140654-phpapp01
 

Similar to Gravitational Billion Body Project

Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...
Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...
Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...cscpconf
 
NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...
NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...
NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...csandit
 
The Einstein Toolkit: A Community Computational Infrastructure for Relativist...
The Einstein Toolkit: A Community Computational Infrastructure for Relativist...The Einstein Toolkit: A Community Computational Infrastructure for Relativist...
The Einstein Toolkit: A Community Computational Infrastructure for Relativist...University of Illinois at Urbana-Champaign
 
Information 2014_5_28-100
Information 2014_5_28-100Information 2014_5_28-100
Information 2014_5_28-100Subrata Ghosh
 
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYIMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYcsandit
 
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYIMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYcscpconf
 
Improving of artifical neural networks performance by using gpu's a survey
Improving of artifical neural networks performance by using gpu's  a surveyImproving of artifical neural networks performance by using gpu's  a survey
Improving of artifical neural networks performance by using gpu's a surveycsandit
 
Science and Cyberinfrastructure in the Data-Dominated Era
Science and Cyberinfrastructure in the Data-Dominated EraScience and Cyberinfrastructure in the Data-Dominated Era
Science and Cyberinfrastructure in the Data-Dominated EraLarry Smarr
 
PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」
PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」
PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」PC Cluster Consortium
 
3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...
3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...
3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...Sérgio Sacani
 
Complexity and Quantum Information Science
Complexity and Quantum Information ScienceComplexity and Quantum Information Science
Complexity and Quantum Information ScienceMelanie Swan
 
High-throughput computation and machine learning methods applied to materials...
High-throughput computation and machine learning methods applied to materials...High-throughput computation and machine learning methods applied to materials...
High-throughput computation and machine learning methods applied to materials...Anubhav Jain
 
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...Frank Wuerthwein
 
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...Igor Sfiligoi
 
The Transformation of Systems Biology Into A Large Data Science
The Transformation of Systems Biology Into A Large Data ScienceThe Transformation of Systems Biology Into A Large Data Science
The Transformation of Systems Biology Into A Large Data ScienceRobert Grossman
 
International Journal of Computer Science and Security Volume (2) Issue (5)
International Journal of Computer Science and Security Volume (2) Issue (5)International Journal of Computer Science and Security Volume (2) Issue (5)
International Journal of Computer Science and Security Volume (2) Issue (5)CSCJournals
 
Black holes as tools for quantum computing by advanced extraterrestrial civil...
Black holes as tools for quantum computing by advanced extraterrestrial civil...Black holes as tools for quantum computing by advanced extraterrestrial civil...
Black holes as tools for quantum computing by advanced extraterrestrial civil...Sérgio Sacani
 
"Building and running the cloud GPU vacuum cleaner"
"Building and running the cloud GPU vacuum cleaner""Building and running the cloud GPU vacuum cleaner"
"Building and running the cloud GPU vacuum cleaner"Frank Wuerthwein
 

Similar to Gravitational Billion Body Project (20)

Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...
Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...
Neural Networks for High Performance Time-Delay Estimation and Acoustic Sourc...
 
NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...
NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...
NEURAL NETWORKS FOR HIGH PERFORMANCE TIME-DELAY ESTIMATION AND ACOUSTIC SOURC...
 
The Einstein Toolkit: A Community Computational Infrastructure for Relativist...
The Einstein Toolkit: A Community Computational Infrastructure for Relativist...The Einstein Toolkit: A Community Computational Infrastructure for Relativist...
The Einstein Toolkit: A Community Computational Infrastructure for Relativist...
 
Information 2014_5_28-100
Information 2014_5_28-100Information 2014_5_28-100
Information 2014_5_28-100
 
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYIMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
 
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEYIMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
IMPROVING OF ARTIFICIAL NEURAL NETWORKS PERFORMANCE BY USING GPU’S: A SURVEY
 
Improving of artifical neural networks performance by using gpu's a survey
Improving of artifical neural networks performance by using gpu's  a surveyImproving of artifical neural networks performance by using gpu's  a survey
Improving of artifical neural networks performance by using gpu's a survey
 
Science and Cyberinfrastructure in the Data-Dominated Era
Science and Cyberinfrastructure in the Data-Dominated EraScience and Cyberinfrastructure in the Data-Dominated Era
Science and Cyberinfrastructure in the Data-Dominated Era
 
PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」
PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」
PCCC22:筑波大学計算科学研究センター テーマ2「学際計算科学による最新の研究成果」
 
3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...
3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...
3D-Spatiotemporal forecasting the expansion of supernova shells using deep le...
 
THz Plasmonics
THz PlasmonicsTHz Plasmonics
THz Plasmonics
 
Complexity and Quantum Information Science
Complexity and Quantum Information ScienceComplexity and Quantum Information Science
Complexity and Quantum Information Science
 
High-throughput computation and machine learning methods applied to materials...
High-throughput computation and machine learning methods applied to materials...High-throughput computation and machine learning methods applied to materials...
High-throughput computation and machine learning methods applied to materials...
 
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
 
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
Running a GPU burst for Multi-Messenger Astrophysics with IceCube across all ...
 
Cyberinfrastructure for Einstein's Equations and Beyond
Cyberinfrastructure for Einstein's Equations and BeyondCyberinfrastructure for Einstein's Equations and Beyond
Cyberinfrastructure for Einstein's Equations and Beyond
 
The Transformation of Systems Biology Into A Large Data Science
The Transformation of Systems Biology Into A Large Data ScienceThe Transformation of Systems Biology Into A Large Data Science
The Transformation of Systems Biology Into A Large Data Science
 
International Journal of Computer Science and Security Volume (2) Issue (5)
International Journal of Computer Science and Security Volume (2) Issue (5)International Journal of Computer Science and Security Volume (2) Issue (5)
International Journal of Computer Science and Security Volume (2) Issue (5)
 
Black holes as tools for quantum computing by advanced extraterrestrial civil...
Black holes as tools for quantum computing by advanced extraterrestrial civil...Black holes as tools for quantum computing by advanced extraterrestrial civil...
Black holes as tools for quantum computing by advanced extraterrestrial civil...
 
"Building and running the cloud GPU vacuum cleaner"
"Building and running the cloud GPU vacuum cleaner""Building and running the cloud GPU vacuum cleaner"
"Building and running the cloud GPU vacuum cleaner"
 

More from Derek Groen

On science hackathons univercite 2016
On science hackathons univercite 2016On science hackathons univercite 2016
On science hackathons univercite 2016Derek Groen
 
FabSim: Facilitating computational research through automation on large-scale...
FabSim: Facilitating computational research through automation on large-scale...FabSim: Facilitating computational research through automation on large-scale...
FabSim: Facilitating computational research through automation on large-scale...Derek Groen
 
Scholarly publishing a perspective from an early career academic
Scholarly publishing  a perspective from an early career academicScholarly publishing  a perspective from an early career academic
Scholarly publishing a perspective from an early career academicDerek Groen
 
Multiscale computing in 15 minutes
Multiscale computing in 15 minutesMultiscale computing in 15 minutes
Multiscale computing in 15 minutesDerek Groen
 
The Building Blocks of Science
The Building Blocks of ScienceThe Building Blocks of Science
The Building Blocks of ScienceDerek Groen
 
Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...
Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...
Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...Derek Groen
 
HemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slidesHemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slidesDerek Groen
 
Supercomputing 2013 slides
Supercomputing 2013 slidesSupercomputing 2013 slides
Supercomputing 2013 slidesDerek Groen
 
MPWide: A light-weight communication library for wide area message passing an...
MPWide: A light-weight communication library for wide area message passing an...MPWide: A light-weight communication library for wide area message passing an...
MPWide: A light-weight communication library for wide area message passing an...Derek Groen
 
Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...
Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...
Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...Derek Groen
 

More from Derek Groen (10)

On science hackathons univercite 2016
On science hackathons univercite 2016On science hackathons univercite 2016
On science hackathons univercite 2016
 
FabSim: Facilitating computational research through automation on large-scale...
FabSim: Facilitating computational research through automation on large-scale...FabSim: Facilitating computational research through automation on large-scale...
FabSim: Facilitating computational research through automation on large-scale...
 
Scholarly publishing a perspective from an early career academic
Scholarly publishing  a perspective from an early career academicScholarly publishing  a perspective from an early career academic
Scholarly publishing a perspective from an early career academic
 
Multiscale computing in 15 minutes
Multiscale computing in 15 minutesMultiscale computing in 15 minutes
Multiscale computing in 15 minutes
 
The Building Blocks of Science
The Building Blocks of ScienceThe Building Blocks of Science
The Building Blocks of Science
 
Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...
Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...
Evolution of the HemeLB Parallel Simulation Environment for Human Brain Blood...
 
HemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slidesHemeLB weighted decomposition EASC 2014 slides
HemeLB weighted decomposition EASC 2014 slides
 
Supercomputing 2013 slides
Supercomputing 2013 slidesSupercomputing 2013 slides
Supercomputing 2013 slides
 
MPWide: A light-weight communication library for wide area message passing an...
MPWide: A light-weight communication library for wide area message passing an...MPWide: A light-weight communication library for wide area message passing an...
MPWide: A light-weight communication library for wide area message passing an...
 
Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...
Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...
Multiscale modelling of brain bloodflow using HemeLB and Python Navier Stokes...
 

Recently uploaded

The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxNeo4j
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...htrindia
 
IT Nation Evolve event 2024 - Quarter 1
IT Nation Evolve event 2024  - Quarter 1IT Nation Evolve event 2024  - Quarter 1
IT Nation Evolve event 2024 - Quarter 1Inbay UK
 
LF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIELF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIEDanBrown980551
 
Roundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfRoundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfMostafa Higazy
 
KUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionKUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionNeo4j
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Product School
 
Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, MonzoRevolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, MonzoProduct School
 
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...SearchNorwich
 
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Umar Saif
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxNeo4j
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...Neo4j
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Jay Zhao
 
New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024ThousandEyes
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerSaiLinnThu2
 
AI for Educators - Integrating AI in the Classrooms
AI for Educators - Integrating AI in the ClassroomsAI for Educators - Integrating AI in the Classrooms
AI for Educators - Integrating AI in the ClassroomsPremsankar Chakkingal
 
Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...Product School
 
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)François
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxInfosec
 

Recently uploaded (20)

The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
 
IT Nation Evolve event 2024 - Quarter 1
IT Nation Evolve event 2024  - Quarter 1IT Nation Evolve event 2024  - Quarter 1
IT Nation Evolve event 2024 - Quarter 1
 
LF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIELF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIE
 
Roundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfRoundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdf
 
KUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionKUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ion
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
 
Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, MonzoRevolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
Revolutionizing The Banking Industry: The Monzo Way by CPO, Monzo
 
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
 
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
Progress Report: Ministry of IT under Dr. Umar Saif Aug 23-Feb'24
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
 
New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024
 
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-ManagerCentralized TLS Certificates Management Using Vault PKI + Cert-Manager
Centralized TLS Certificates Management Using Vault PKI + Cert-Manager
 
AI for Educators - Integrating AI in the Classrooms
AI for Educators - Integrating AI in the ClassroomsAI for Educators - Integrating AI in the Classrooms
AI for Educators - Integrating AI in the Classrooms
 
Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...Launching New Products In Companies Where It Matters Most by Product Director...
Launching New Products In Companies Where It Matters Most by Product Director...
 
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
Mind your App Footprint 🐾⚡️🌱 (@FlutterHeroes 2024)
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptx
 
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
 

Gravitational Billion Body Project

  • 1. The Gravitational Billion Body Project a a a Authors: Derek Groen , Steven Rieder , Simon Portegies Zwart , b b Tomoaki Ishiyama , Jun Makino Introduction The Application We report on cosmological N-body simulations which run over up to 4 supercomputers across the globe. We achieved to run simulations on 60 to 750 cores distributed over a variety of supercomputers. Regardless of the network latency of 0.32 s and the communication over 30.000 km of optical network cable we are able to achieve up to 92% of the performance compared to an equal number of cores on a single supercomputer. Our cosmological code simulates structure formation in the universe by integrating the gravitational forces between dark matter particles over time. This code, which is based on 1 2 GreeM , uses Barnes-Hut Tree integration to resolve force inter-actions over short distances, 3 and Particle-Mesh integration to resolve interactions over long distances. We have 4 coupled our code with MPWide to enable simulations across supercomputers. Our code is named SUSHII. Results Timing results of our simulations per step averaged over 10 steps can be found in the table. Here, the simulations spend about 90% of the total time on calcutions when performed across 2 or 3 sites. For our experiments we have used one IBM Power6 supercomputer and three Cray-XT4 machines. The IBM resides at SARA in Amsterdam(NL) and the Cray machines reside at EPCC in Edinburgh (UK), CSC in Espoo (FI) and CFCA in Tokyo (JP). I) SUSHI stands for Simulating Universe Structure formation on Heterogeneous Infrastructures. particles procs sites time/step comm. time # # # [s] [s] 5123 120 1 71,04 1,05 5123 120 2 61,62 4,59 5123 120 3 56,31 4,78 5123 120 4 70,1 19,26 10243 240 1 272 3,4 10243 240 2 252 21,98 10243 240 3 294,7 31,28 20483 750 2 483,4 46,5 Runs over 1 site were performed at SARA only, runs over 2 sites also at EPCC, and runs over 3 sites also at CSC. The 20483 run was performed at SARA and CFCA. The picture in the background is a colorized density plot of the simulation data at redshift z=5.65. We performed the run over 4 sites and the slices are colored to match volumes residing at CFCA, CSC, EPCC and SARA. 120 cores total Higher accuracy tree integration using an opening angle of 0.3. Lower accuracy tree integration using an opening angle of 0.5. 3 Timings per step of a 512 run over 3 sites. Network Setup All supercomputers have been connected by optical networks. The DEISA network is shared with other users. Communication nodes are shown by the green boxes. Conclusion We have run cosmological simulations efficiently across multiple supercomputers. The scale of our experiments is constrained by the political overhead of scheduling the application across supercomputers. The use of a meta-scheduler and reservation system that works across sites will enable us to perform long-lasting and large production runs on a grid of supercomputers. References 1. T. Ishiyama, T. Fukushige, and J. Makino, “GreeM : Massively ParallelTreePM Code for Large Cosmological N-body Simulations,” accepted by PASJ. 2. J. Barnes and P. Hut, “A Hierarchical O(NlogN) Force-Calculation Algorithm,” Nature, vol. 324, pp. 446–449, Dec. 1986. 3. R. Hockney and J. Eastwood, “Computer Simulation Using Particles”, New York: McGraw-Hill, 1981. 4. D. Groen, S. Rieder, P. Grosso, C. de Laat and S. Portegies Zwart, “A lightweight communication library for distributed computing,” (submitted to CSD). Affiliations A. Leiden Observatory, Leiden University, Leiden, the Netherlands. B. Center for Computational Astrophysics, Mitaka, Tokyo, Japan. Acknowledgements This research is supported by the Netherlands organization for Scientific research (NWO) grant #639.073.803, #643.200.503 and #643.000.803, the Stichting Nationale Computerfaciliteiten (project #SH-095-08), NAOJ, SURFNet (GigaPort project), the Netherlands Advanced School for Astronomy (NOVA) and the Leids Kerkhoven-Bosscha fonds (LKBF). We thank the DEISA Consortium (www.deisa.eu), co-funded through the EU FP6 project RI-031513 and the FP7 project RI-222919, for support within the DEISA Extreme Computing Initiative. We thank the network facilities of SURFnet,, IEEAF, WIDE, Northwest Gigapop and the Global Lambda Integrated FAcility (GLIF) GOLE of Trans-Light Cisco on National LambdaRail, TransLight, StarLight, NetherLight, T-LEX, Pacific and Atlantic Wave.