In this video from the HPC User Forum at Argonne, Dr. Brett Bode from NCSA presents: Research on Blue Waters.
"Blue Waters is one of the most powerful supercomputers in the world and is one of the fastest supercomputers on a university campus. Scientists and engineers across the country use the computing and data power of Blue Waters to tackle a wide range of challenging problems, from predicting the behavior of complex biological systems to simulating the evolution of the cosmos."
Watch the video: https://wp.me/p3RLHQ-kYx
Learn more: http://www.ncsa.illinois.edu/enabling/bluewaters
and
http://hpcuserforum.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Using the Pacific Research Platform for Earth Sciences Big DataLarry Smarr
Grand Challenge Lecture
Big Data and the Earth Sciences: Grand Challenges Workshop
Calit2’s Qualcomm Institute
University of California, San Diego
May 31, 2017
Opening Keynote Lecture
15th Annual ON*VECTOR International Photonics Workshop
Calit2’s Qualcomm Institute
University of California, San Diego
February 29, 2016
The Pacific Research Platform: A Regional-Scale Big Data Analytics Cyberinfra...Larry Smarr
National Ocean Exploration Forum 2017
Ocean Exploration in a Sea of Data
Calit2’s Qualcomm Institute
University of California, San Diego
October 21, 2017
Can a Greener Internet Help Us Moderate Climate Change?Larry Smarr
09.05.13
High Definition Remote Presentation to the
Monash Undergraduate Research Projects Abroad (MURPA) Program, located at Monash University, Australia
Title: Can a Greener Internet Help Us Moderate Climate Change?
La Jolla, CA
The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way t...Larry Smarr
05.06.14
Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers
Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid
San Diego, CA
Using the Pacific Research Platform for Earth Sciences Big DataLarry Smarr
Grand Challenge Lecture
Big Data and the Earth Sciences: Grand Challenges Workshop
Calit2’s Qualcomm Institute
University of California, San Diego
May 31, 2017
Opening Keynote Lecture
15th Annual ON*VECTOR International Photonics Workshop
Calit2’s Qualcomm Institute
University of California, San Diego
February 29, 2016
The Pacific Research Platform: A Regional-Scale Big Data Analytics Cyberinfra...Larry Smarr
National Ocean Exploration Forum 2017
Ocean Exploration in a Sea of Data
Calit2’s Qualcomm Institute
University of California, San Diego
October 21, 2017
Can a Greener Internet Help Us Moderate Climate Change?Larry Smarr
09.05.13
High Definition Remote Presentation to the
Monash Undergraduate Research Projects Abroad (MURPA) Program, located at Monash University, Australia
Title: Can a Greener Internet Help Us Moderate Climate Change?
La Jolla, CA
The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way t...Larry Smarr
05.06.14
Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers
Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid
San Diego, CA
Positioning University of California Information Technology for the Future: S...Larry Smarr
05.02.15
Invited Talk
The Vice Chancellor of Research and Chief Information Officer Summit
“Information Technology Enabling Research at the University of California”
Title: Positioning University of California Information Technology for the Future: State, National, and International IT Infrastructure Trends and Directions
Oakland, CA
A National Big Data Cyberinfrastructure Supporting Computational Biomedical R...Larry Smarr
Invited Presentation
Symposium on Computational Biology and Bioinformatics:
Remembering John Wooley
National Institutes of Health
Bethesda, MD
July 29, 2016
Toward a Global Interactive Earth Observing CyberinfrastructureLarry Smarr
05.01.12
Invited Talk to the 21st International Conference on Interactive Information Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology Held at the 85th AMS Annual Meeting
Title: Toward a Global Interactive Earth Observing Cyberinfrastructure
San Diego, CA
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.15
Eighth Lecture in the
Australian American Leadership Dialogue Scholar Tour
Australian National University
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Canberra, Australia
Building an Information Infrastructure to Support Microbial Metagenomic SciencesLarry Smarr
06.01.14
Presentation for the Microbe Project Interagency Team
Title: Building an Information Infrastructure to Support Microbial Metagenomic Sciences
La Jolla, CA
Set My Data Free: High-Performance CI for Data-Intensive ResearchLarry Smarr
10.11.03
Keynote Speaker
Cyberinfrastructure Days
University of Michigan
Title: Set My Data Free: High-Performance CI for Data-Intensive Research
Ann Arbor, MI
Analyzing Large Earth Data Sets: New Tools from the OptiPuter and LOOKING Pro...Larry Smarr
05.05.03
Presentation to 3rd Annual GEON Meeting
Bahia Resort
Title: Analyzing Large Earth Data Sets: New Tools from the OptiPuter and LOOKING Projects
San Diego, CA
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.13
Sixth Lecture in the
Australian American Leadership Dialogue Scholar Tour
University of Technology Sydney
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Sydney, Australia
A High-Performance Campus-Scale Cyberinfrastructure: The Technical, Political...Larry Smarr
10.10.11
Presentation by Larry Smarr to the NSF Campus Bridging Workshop
Title: A High-Performance Campus-Scale Cyberinfrastructure: The Technical, Political, and Economic
Anaheim, CA
From Jisc's campus network engineering for data-intensive science workshop on 19 October 2016.
https://www.jisc.ac.uk/events/campus-network-engineering-for-data-intensive-science-workshop-19-oct-2016
In this deck from the Stanford HPC Conference, Shahin Khan from OrionX describes major market Shifts in IT.
"We will discuss the digital infrastructure of the future enterprise and the state of these trends."
"We work with clients on the impact of Digital Transformation (DX) on them, their customers, and their messages. Generally, they want to track, in one place, trends like IoT, 5G, AI, Blockchain, and Quantum Computing. And they want to know what these trends mean, how they affect each other, and when they demand action, and how to formulate and execute an effective plan. If that describes you, we can help."
Watch the video: https://wp.me/p3RLHQ-lPP
Learn more: http://orionx.net
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Preparing to program Aurora at Exascale - Early experiences and future direct...inside-BigData.com
In this deck from IWOCL / SYCLcon 2020, Hal Finkel from Argonne National Laboratory presents: Preparing to program Aurora at Exascale - Early experiences and future directions.
"Argonne National Laboratory’s Leadership Computing Facility will be home to Aurora, our first exascale supercomputer. Aurora promises to take scientific computing to a whole new level, and scientists and engineers from many different fields will take advantage of Aurora’s unprecedented computational capabilities to push the boundaries of human knowledge. In addition, Aurora’s support for advanced machine-learning and big-data computations will enable scientific workflows incorporating these techniques along with traditional HPC algorithms. Programming the state-of-the-art hardware in Aurora will be accomplished using state-of-the-art programming models. Some of these models, such as OpenMP, are long-established in the HPC ecosystem. Other models, such as Intel’s oneAPI, based on SYCL, are relatively-new models constructed with the benefit of significant experience. Many applications will not use these models directly, but rather, will use C++ abstraction libraries such as Kokkos or RAJA. Python will also be a common entry point to high-performance capabilities. As we look toward the future, features in the C++ standard itself will become increasingly relevant for accessing the extreme parallelism of exascale platforms.
This presentation will summarize the experiences of our team as we prepare for Aurora, exploring how to port applications to Aurora’s architecture and programming models, and distilling the challenges and best practices we’ve developed to date. oneAPI/SYCL and OpenMP are both critical models in these efforts, and while the ecosystem for Aurora has yet to mature, we’ve already had a great deal of success. Importantly, we are not passive recipients of programming models developed by others. Our team works not only with vendor-provided compilers and tools, but also develops improved open-source LLVM-based technologies that feed both open-source and vendor-provided capabilities. In addition, we actively participate in the standardization of OpenMP, SYCL, and C++. To conclude, I’ll share our thoughts on how these models can best develop in the future to support exascale-class systems."
Watch the video: https://wp.me/p3RLHQ-lPT
Learn more: https://www.iwocl.org/iwocl-2020/conference-program/
and
https://www.anl.gov/topic/aurora
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck, Greg Wahl from Advantech presents: Transforming Private 5G Networks.
Advantech Networks & Communications Group is driving innovation in next-generation network solutions with their High Performance Servers. We provide business critical hardware to the world's leading telecom and networking equipment manufacturers with both standard and customized products. Our High Performance Servers are highly configurable platforms designed to balance the best in x86 server-class processing performance with maximum I/O and offload density. The systems are cost effective, highly available and optimized to meet next generation networking and media processing needs.
“Advantech’s Networks and Communication Group has been both an innovator and trusted enabling partner in the telecommunications and network security markets for over a decade, designing and manufacturing products for OEMs that accelerate their network platform evolution and time to market.” Said Advantech Vice President of Networks & Communications Group, Ween Niu. “In the new IP Infrastructure era, we will be expanding our expertise in Software Defined Networking (SDN) and Network Function Virtualization (NFV), two of the essential conduits to 5G infrastructure agility making networks easier to install, secure, automate and manage in a cloud-based infrastructure.”
In addition to innovation in air interface technologies and architecture extensions, 5G will also need a new generation of network computing platforms to run the emerging software defined infrastructure, one that provides greater topology flexibility, essential to deliver on the promises of high availability, high coverage, low latency and high bandwidth connections. This will open up new parallel industry opportunities through dedicated 5G network slices reserved for specific industries dedicated to video traffic, augmented reality, IoT, connected cars etc. 5G unlocks many new doors and one of the keys to its enablement lies in the elasticity and flexibility of the underlying infrastructure.
Advantech’s corporate vision is to enable an intelligent planet. The company is a global leader in the fields of IoT intelligent systems and embedded platforms. To embrace the trends of IoT, big data, and artificial intelligence, Advantech promotes IoT hardware and software solutions with the Edge Intelligence WISE-PaaS core to assist business partners and clients in connecting their industrial chains. Advantech is also working with business partners to co-create business ecosystems that accelerate the goal of industrial intelligence."
Watch the video: https://wp.me/p3RLHQ-lPQ
* Company website: https://www.advantech.com/
* Solution page: https://www2.advantech.com/nc/newsletter/NCG/SKY/benefits.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
The Incorporation of Machine Learning into Scientific Simulations at Lawrence...inside-BigData.com
In this deck from the Stanford HPC Conference, Katie Lewis from Lawrence Livermore National Laboratory presents: The Incorporation of Machine Learning into Scientific Simulations at Lawrence Livermore National Laboratory.
"Scientific simulations have driven computing at Lawrence Livermore National Laboratory (LLNL) for decades. During that time, we have seen significant changes in hardware, tools, and algorithms. Today, data science, including machine learning, is one of the fastest growing areas of computing, and LLNL is investing in hardware, applications, and algorithms in this space. While the use of simulations to focus and understand experiments is well accepted in our community, machine learning brings new challenges that need to be addressed. I will explore applications for machine learning in scientific simulations that are showing promising results and further investigation that is needed to better understand its usefulness."
Watch the video: https://youtu.be/NVwmvCWpZ6Y
Learn more: https://computing.llnl.gov/research-area/machine-learning
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
How to Achieve High-Performance, Scalable and Distributed DNN Training on Mod...inside-BigData.com
In this deck from the Stanford HPC Conference, DK Panda from Ohio State University presents: How to Achieve High-Performance, Scalable and Distributed DNN Training on Modern HPC Systems?
"This talk will start with an overview of challenges being faced by the AI community to achieve high-performance, scalable and distributed DNN training on Modern HPC systems with both scale-up and scale-out strategies. After that, the talk will focus on a range of solutions being carried out in my group to address these challenges. The solutions will include: 1) MPI-driven Deep Learning, 2) Co-designing Deep Learning Stacks with High-Performance MPI, 3) Out-of- core DNN training, and 4) Hybrid (Data and Model) parallelism. Case studies to accelerate DNN training with popular frameworks like TensorFlow, PyTorch, MXNet and Caffe on modern HPC systems will be presented."
Watch the video: https://youtu.be/LeUNoKZVuwQ
Learn more: http://web.cse.ohio-state.edu/~panda.2/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...inside-BigData.com
In this deck from the Stanford HPC Conference, Nick Nystrom and Paola Buitrago provide an update from the Pittsburgh Supercomputing Center.
Nick Nystrom is Chief Scientist at the Pittsburgh Supercomputing Center (PSC). Nick is architect and PI for Bridges, PSC's flagship system that successfully pioneered the convergence of HPC, AI, and Big Data. He is also PI for the NIH Human Biomolecular Atlas Program’s HIVE Infrastructure Component and co-PI for projects that bring emerging AI technologies to research (Open Compass), apply machine learning to biomedical data for breast and lung cancer (Big Data for Better Health), and identify causal relationships in biomedical big data (the Center for Causal Discovery, an NIH Big Data to Knowledge Center of Excellence). His current research interests include hardware and software architecture, applications of machine learning to multimodal data (particularly for the life sciences) and to enhance simulation, and graph analytics.
Watch the video: https://youtu.be/LWEU1L1o7yY
Learn more: https://www.psc.edu/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck from the Stanford HPC Conference, Ryan Quick from Providentia Worldwide describes how DNNs can be used to improve EDA simulation runs.
"Systems Intelligence relies on a variety of methods for providing insight into the core mechanisms for driving automated behavioral changes in self-healing command and control platforms. This talk reports on initial efforts with leveraging Semiconductor Electronic Design Automation (EDA) telemetry data from cross-domain sources including power, network, storage, nodes, and applications in neural networks as a driving method for insight into SI automation systems."
Watch the video: https://youtu.be/2WbR8tq-XbM
Learn more: http://www.providentiaworldwide.com/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoringinside-BigData.com
In this deck from the Stanford HPC Conference, Nicole Xu from Stanford University describes how she transformed a common jellyfish into a bionic creature that is part animal and part machine.
"Animal locomotion and bioinspiration have the potential to expand the performance capabilities of robots, but current implementations are limited. Mechanical soft robots leverage engineered materials and are highly controllable, but these biomimetic robots consume more power than corresponding animal counterparts. Biological soft robots from a bottom-up approach offer advantages such as speed and controllability but are limited to survival in cell media. Instead, biohybrid robots that comprise live animals and self- contained microelectronic systems leverage the animals’ own metabolism to reduce power constraints and body as an natural scaffold with damage tolerance. We demonstrate that by integrating onboard microelectronics into live jellyfish, we can enhance propulsion up to threefold, using only 10 mW of external power input to the microelectronics and at only a twofold increase in cost of transport to the animal. This robotic system uses 10 to 1000 times less external power per mass than existing swimming robots in literature and can be used in future applications for ocean monitoring to track environmental changes."
Watch the video: https://youtu.be/HrmJFyvInj8
Learn more: https://sanfrancisco.cbslocal.com/2020/02/05/stanford-research-project-common-jellyfish-bionic-sea-creatures/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck from the Stanford HPC Conference, Peter Dueben from the European Centre for Medium-Range Weather Forecasts (ECMWF) presents: Machine Learning for Weather Forecasts.
"I will present recent studies that use deep learning to learn the equations of motion of the atmosphere, to emulate model components of weather forecast models and to enhance usability of weather forecasts. I will than talk about the main challenges for the application of deep learning in cutting-edge weather forecasts and suggest approaches to improve usability in the future."
Peter is contributing to the development and optimization of weather and climate models for modern supercomputers. He is focusing on a better understanding of model error and model uncertainty, on the use of reduced numerical precision that is optimised for a given level of model error, on global cloud- resolving simulations with ECMWF's forecast model, and the use of machine learning, and in particular deep learning, to improve the workflow and predictions. Peter has graduated in Physics and wrote his PhD thesis at the Max Planck Institute for Meteorology in Germany. He worked as Postdoc with Tim Palmer at the University of Oxford and has taken up a position as University Research Fellow of the Royal Society at the European Centre for Medium-Range Weather Forecasts (ECMWF) in 2017.
Watch the video: https://youtu.be/ks3fkRj8Iqc
Learn more: https://www.ecmwf.int/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck, Gilad Shainer from the HPC AI Advisory Council describes how this organization fosters innovation in the high performance computing community.
"The HPC-AI Advisory Council’s mission is to bridge the gap between high-performance computing (HPC) and Artificial Intelligence (AI) use and its potential, bring the beneficial capabilities of HPC and AI to new users for better research, education, innovation and product manufacturing, bring users the expertise needed to operate HPC and AI systems, provide application designers with the tools needed to enable parallel computing, and to strengthen the qualification and integration of HPC and AI system products."
Watch the video: https://wp.me/p3RLHQ-lNz
Learn more: http://hpcadvisorycouncil.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Today RIKEN in Japan announced that the Fugaku supercomputer will be made available for research projects aimed to combat COVID-19.
"Fugaku is currently being installed and is scheduled to be available to the public in 2021. However, faced with the devastating disaster unfolding before our eyes, RIKEN and MEXT decided to make a portion of the computational resources of Fugaku available for COVID-19-related projects ahead of schedule while continuing the installation process.
Fugaku is being developed not only for the progress in science, but also to help build the society dubbed as the “Society 5.0” by the Japanese government, where all people will live safe and comfortable lives. The current initiative to fight against the novel coronavirus is driven by the philosophy behind the development of Fugaku."
Initial Projects
Exploring new drug candidates for COVID-19 by "Fugaku"
Yasushi Okuno, RIKEN / Kyoto University
Prediction of conformational dynamics of proteins on the surface of SARS-Cov-2 using Fugaku
Yuji Sugita, RIKEN
Simulation analysis of pandemic phenomena
Nobuyasu Ito, RIKEN
Fragment molecular orbital calculations for COVID-19 proteins
Yuji Mochizuki, Rikkyo University
In this deck from the Performance Optimisation and Productivity group, Lubomir Riha from IT4Innovations presents: Energy Efficient Computing using Dynamic Tuning.
"We now live in a world of power-constrained architectures and systems and power consumption represents a significant cost factor in the overall HPC system economy. For these reasons, in recent years researchers, supercomputing centers and major vendors have developed new tools and methodologies to measure and optimize the energy consumption of large-scale high performance system installations. Due to the link between energy consumption, power consumption and execution time of an application executed by the final user, it is important for these tools and the methodology used to consider all these aspects, empowering the final user and the system administrator with the capability of finding the best configuration given different high level objectives.
This webinar focused on tools designed to improve the energy-efficiency of HPC applications using a methodology of dynamic tuning of HPC applications, developed under the H2020 READEX project. The READEX methodology has been designed for exploiting the dynamic behaviour of software. At design time, different runtime situations (RTS) are detected and optimized system configurations are determined. RTSs with the same configuration are grouped into scenarios, forming the tuning model. At runtime, the tuning model is used to switch system configurations dynamically.
The MERIC tool, that implements the READEX methodology, is presented. It supports manual or binary instrumentation of the analysed applications to simplify the analysis. This instrumentation is used to identify and annotate the significant regions in the HPC application. Automatic binary instrumentation annotates regions with significant runtime. Manual instrumentation, which can be combined with automatic, allows code developer to annotate regions of particular interest."
Watch the video: https://wp.me/p3RLHQ-lJP
Learn more: https://pop-coe.eu/blog/14th-pop-webinar-energy-efficient-computing-using-dynamic-tuning
and
https://code.it4i.cz/vys0053/meric
Sign up for our insideHPC Newsletter: http://insidehpc.com/newslett
In this deck from GTC Digital, William Beaudin from DDN presents: HPC at Scale Enabled by DDN A3i and NVIDIA SuperPOD.
Enabling high performance computing through the use of GPUs requires an incredible amount of IO to sustain application performance. We'll cover architectures that enable extremely scalable applications through the use of NVIDIA’s SuperPOD and DDN’s A3I systems.
The NVIDIA DGX SuperPOD is a first-of-its-kind artificial intelligence (AI) supercomputing infrastructure. DDN A³I with the EXA5 parallel file system is a turnkey, AI data storage infrastructure for rapid deployment, featuring faster performance, effortless scale, and simplified operations through deeper integration. The combined solution delivers groundbreaking performance, deploys in weeks as a fully integrated system, and is designed to solve the world's most challenging AI problems.
Watch the video: https://wp.me/p3RLHQ-lIV
Learn more: https://www.ddn.com/download/nvidia-superpod-ddn-a3i-ai400-appliance-with-the-exa5-filesystem/
and
https://www.nvidia.com/en-us/gtc/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck, Paul Isaacs from Linaro presents: State of ARM-based HPC. This talk provides an overview of applications and infrastructure services successfully ported to Aarch64 and benefiting from scale.
"With its debut on the TOP500, the 125,000-core Astra supercomputer at New Mexico’s Sandia Labs uses Cavium ThunderX2 chips to mark Arm’s entry into the petascale world. In Japan, the Fujitsu A64FX Arm-based CPU in the pending Fugaku supercomputer has been optimized to achieve high-level, real-world application performance, anticipating up to one hundred times the application execution performance of the K computer. K was the first computer to top 10 petaflops in 2011."
Watch the video: https://wp.me/p3RLHQ-lIT
Learn more: https://www.linaro.org/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Versal Premium ACAP for Network and Cloud Accelerationinside-BigData.com
Today Xilinx announced Versal Premium, the third series in the Versal ACAP portfolio. The Versal Premium series features highly integrated, networked and power-optimized cores and the industry’s highest bandwidth and compute density on an adaptable platform. Versal Premium is designed for the highest bandwidth networks operating in thermally and spatially constrained environments, as well as for cloud providers who need scalable, adaptable application acceleration.
Versal is the industry’s first adaptive compute acceleration platform (ACAP), a revolutionary new category of heterogeneous compute devices with capabilities that far exceed those of conventional silicon architectures. Developed on TSMC’s 7-nanometer process technology, Versal Premium combines software programmability with dynamically configurable hardware acceleration and pre-engineered connectivity and security features to enable a faster time-to- market. The Versal Premium series delivers up to 3X higher throughput compared to current generation FPGAs, with built-in Ethernet, Interlaken, and cryptographic engines that enable fast and secure networks. The series doubles the compute density of currently deployed mainstream FPGAs and provides the adaptability to keep pace with increasingly diverse and evolving cloud and networking workloads.
Learn more: https://insidehpc.com/2020/03/xilinx-announces-versal-premium-acap-for-network-and-cloud-acceleration/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Zettar: Moving Massive Amounts of Data across Any Distance Efficientlyinside-BigData.com
In this video from the Rice Oil & Gas Conference, Chin Fang from Zettar presents: Moving Massive Amounts of Data across Any Distance Efficiently.
The objective of this talk is to present two on-going projects aiming at improving and ensuring highly efficient bulk transferring or streaming of massive amounts of data over digital connections across any distance. It examines the current state of the art, a few very common misconceptions, the differences among the three major type of data movement solutions, a current initiative attempting to improve the data movement efficiency from the ground up, and another multi-stage project that shows how to conduct long distance large scale data movement at speed and scale internationally. Both projects have real world motivations, e.g. the ambitious data transfer requirements of Linac Coherent Light Source II (LCLS-II) [1], a premier preparation project of the U.S. DOE Exascale Computing Initiative (ECI) [2]. Their immediate goals are described and explained, together with the solution used for each. Findings and early results are reported. Possible future works are outlined.
Watch the video: https://wp.me/p3RLHQ-lBX
Learn more: https://www.zettar.com/
and
https://rice2020oghpc.rice.edu/program-2/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck from the Rice Oil & Gas Conference, Bradley McCredie from AMD presents: Scaling TCO in a Post Moore's Law Era.
"While foundries bravely drive forward to overcome the technical and economic challenges posed by scaling to 5nm and beyond, Moore’s law alone can provide only a fraction of the performance / watt and performance / dollar gains needed to satisfy the demands of today’s high performance computing and artificial intelligence applications. To close the gap, multiple strategies are required. First, new levels of innovation and design efficiency will supplement technology gains to continue to deliver meaningful improvements in SoC performance. Second, heterogenous compute architectures will create x-factor increases of performance efficiency for the most critical applications. Finally, open software frameworks, APIs, and toolsets will enable broad ecosystems of application level innovation."
Watch the video:
Learn more: http://amd.com
and
https://rice2020oghpc.rice.edu/program-2/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
CUDA-Python and RAPIDS for blazing fast scientific computinginside-BigData.com
In this deck from the ECSS Symposium, Abe Stern from NVIDIA presents: CUDA-Python and RAPIDS for blazing fast scientific computing.
"We will introduce Numba and RAPIDS for GPU programming in Python. Numba allows us to write just-in-time compiled CUDA code in Python, giving us easy access to the power of GPUs from a powerful high-level language. RAPIDS is a suite of tools with a Python interface for machine learning and dataframe operations. Together, Numba and RAPIDS represent a potent set of tools for rapid prototyping, development, and analysis for scientific computing. We will cover the basics of each library and go over simple examples to get users started. Finally, we will briefly highlight several other relevant libraries for GPU programming."
Watch the video: https://wp.me/p3RLHQ-lvu
Learn more: https://developer.nvidia.com/rapids
and
https://www.xsede.org/for-users/ecss/ecss-symposium
Sign up for our insideHPC Newsletter: http://insidehp.com/newsletter
In this deck from FOSDEM 2020, Colin Sauze from Aberystwyth University describes the development of a RaspberryPi cluster for teaching an introduction to HPC.
"The motivation for this was to overcome four key problems faced by new HPC users:
* The availability of a real HPC system and the effect running training courses can have on the real system, conversely the availability of spare resources on the real system can cause problems for the training course.
* A fear of using a large and expensive HPC system for the first time and worries that doing something wrong might damage the system.
* That HPC systems are very abstract systems sitting in data centres that users never see, it is difficult for them to understand exactly what it is they are using.
* That new users fail to understand resource limitations, in part because of the vast resources in modern HPC systems a lot of mistakes can be made before running out of resources. A more resource constrained system makes it easier to understand this.
The talk will also discuss some of the technical challenges in deploying an HPC environment to a Raspberry Pi and attempts to keep that environment as close to a "real" HPC as possible. The issue to trying to automate the installation process will also be covered."
Learn more: https://github.com/colinsauze/pi_cluster
and
https://fosdem.org/2020/schedule/events/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck from ATPESC 2019, Ken Raffenetti from Argonne presents an overview of HPC interconnects.
"The Argonne Training Program on Extreme-Scale Computing (ATPESC) provides intensive, two-week training on the key skills, approaches, and tools to design, implement, and execute computational science and engineering applications on current high-end computing systems and the leadership-class computing systems of the future."
Watch the video: https://wp.me/p3RLHQ-luc
Learn more: https://extremecomputingtraining.anl.gov/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
"Impact of front-end architecture on development cost", Viktor Turskyi
Research on Blue Waters
1. Research on Blue Waters
Dr. Brett Bode (brett@Illinois.edu)
National Center for Supercomputing Applications, University of Illinois
http://bluewaters.ncsa.illinois.edu
2. Brief History of Blue Waters
2007 – NSF Proposal and Award for a Broad use
“leadership” computing system
- Funding shared 2007-2019 NSF (~60%), University of
Illinois (~25%) and State of Illinois (~15%)
2008-11 – “Co-Design”, Development, NPCF completion
2012 – Build, deploy, test BW
2013 Q1 – Early Science Operations
2013 - 2019 – Full Service for Open, Broad, Frontier
Science Use
See annual reports for science accomplishments
https://bluewaters.ncsa.illinois.edu/annual-report
2HPC User Forum - Sept. 2019
3. Blue Waters Computing System
bluewaters.ncsa.illinois.edu
HPC User Forum - Sept. 2019
Sonexion: 26
usable PB
>1 TB/sec
100 GB/sec
10/40/100 Gb
Ethernet Switch
Going to 450 Gb/sec
IB Switch
External Servers
1.66PB Globally Addressable Memory
13.1 Peak PF
450 Gbps WAN
Spectra Logic: 300
usable PB
Full Scale use
across all
ranges of
research
Measured,
Sustained 1.3
PF/s over 14
benchmarks
The largest
System Cray
has ever built –
45% larger
than Titan
Largest
Memory of any
system in open
science – 1.66
PB
Most
networked
facility in open
science
Not listed on
the Top500 on
purpose
3
5. 5HPC User Forum - Sept. 2019
Blue Waters is a Multi-Focused Program
Sustained Petascale - Blue Waters Base System – Processors, Memory, Interconnect, On-
line Storage, System Software, Programming Environment
Value added Software – Collaborations
Value added hardware and software (HPSS, archive HW, LAN/WAN Networking, etc.)
NCSA Petascale Application Improvement Program
Major Petascale Applications Point of Contact Advanced
Engagement
(Computing Resource Allocations)
Outstanding User and Production Support
WAN connections, Consulting, System Management, Security,
Operations, …
National Petascale Computing Facility
Petascale
Education,
Industry
and
Outreach
Great Lakes
Consortium
for
Petascale
Computing
6. National Petascale Computing Facility
6HPC User Forum - Sept. 2019
• Only Facility in the world of this
scale on an Academic Campus
• Capable of sustained 24 MW today
• Expandable in space, power and
cooling [50,000 ft2 (4,645+ m2) machine
room gallery and sustained 100 MW]
• Modern Data Center
• 90,000+ ft2 (8,360+ m2) total
• 30,000 ft2 (2,790+ m2) raised floor
20,000 ft2 (1,860+ m2) machine room gallery
• Energy Efficiency
• LEED certified Gold
• Power Utilization Efficiency, PUE = 1.1–1.2
• Staff participating in Energy Efficient HPC
working group
7. The Blue Waters Project
• Comprehensive development, deployment and service phases with co-design and other aspects
• Blue Waters is a Universal Research Instrument for frontier computational and data science that has delivered
over 1.3 Billion node hours/ 41 Billion core hour equivalents
• Make teams highly productive creating insights that cannot be done in other ways
• Diverse Science teams are able to make excellent use of those capabilities due to the system’s flexibility and
emphasis on sustained performance.
• The Blue Waters system is a top ranked system in all aspects of its capabilities. – Intentionally not listed on
Top500 list
7HPC User Forum - Sept. 2019
• 45% larger than any system Cray has ever built. The next closest Cray system was #1 on the Top500 (Titan at
ORNL).
• Ranks in the top systems in the world in peak performance – despite being over seven years old
• Largest memory capacity (1.66 PetaBytes) of any HPC system in the world between 2012 and 2019!
• One of the fastest file systems (>1 TB/s) in the world between 2012 and 2019!
• Fastest external network capability (450 Gb/s) of any open science site
• 147.4 Tb/s computer room network capability
• Largest nearline tape system (>250 PB) in the world at introduction
9. Science & Engineering on Blue Waters
Molecular Science Weather & Climate Forecasting
Earth ScienceAstro* Health
Blue Waters will enable advances in a broad range of science and
engineering disciplines. Examples include:
HPC User Forum - Sept. 2019 9
10. Petascale Simulation of Turbulent Stellar
Hydrodynamics
10HPC User Forum - Sept. 2019
Paul Woodward PI (U of Minn)
Code PPM
Study flash events in early generation
stars, which result in expulsion of heavy
elements formed in the so-called s-
process.
To fully understand these flashes and to
estimate reliably the enrichment of the
heavy elements in the early interstellar
medium that they cause, the effort must
follow the star through the entire two-
year duration of a flash, which is well
beyond any previous calculation.
Possible other uses as well.
11. Petascale Simulation of Turbulent Stellar
Hydrodynamics
11HPC User Forum - Sept. 2019
Paul Woodward PI (U of Minn)
Code PPM
1.5 Pflop/s sustained on Blue Waters
10,5603 grid
A Trillion Cell, Multifluid CFD Simulation
21,962 XE nodes; 702,784 interger cores; 1331 I/Os;
11 MW
All message passing and all I/O overlapped with
computation
12% theoretical peak performance sustained 41 hrs
1.02 PB data written and archived; 16.5 TB per dump.
Ran over 12 days in 6-hour increments
12. Enabling Breakthrough Kinetic Simulations of the
Magnetosphere via Petascale Computing
• PI - Homa Karimabadi -UCSD
• Code PPM
• Background “When Space Weather Attacks” –
Washington Post July 13, 2013
• The Earth’s magnetic field, for the most part,
shields us from the direct effects of the solar
wind. However, this shielding is not perfect and
the term “space weather” has been coined to
describe the conditions in space that affect the
Earth and its technological system.
• At risk Global Positioning System satellites,
geosynchronous communication and weather
satellites, large-scale power distribution grids,
and navigation and communications systems
through the ionosphere.
12HPC User Forum - Sept. 2019
See http://www.washingtonpost.com/blogs/wonkblog/wp/2013/07/13/when-
space-weather-attacks/
13. Example: The Computational Microscope: NAMD
1.Simulated flexibility of ribosome trigger factor complex
at full length and obtained better starting configuration of
trigger factor model (simulated to 80ns)
2.100ns simulation of cylindrical HIV 'capsule’ of CA
proteins revealed it is stabilized by hydrophobic interactions
between CA hexamers; maturation involves detailed
remodeling rather than
disassembly/re-assembly of CA lattice, as had been proposed.
200ns simulation of CA pentamer surrounded by CA hexamers suggested interfaces in
hexamer-hexamer and hexamer-pentamer pairings involve different patterns of
interactions
3.Simulated photosynthetic membrane of a chromatophore
in bacterium Rps. photometricum for 20 ns -- simulation of a
few hundred nanoseconds will be needed
13
PI: Klaus Schulten, University of Illinois at Urbana-Champaign
HPC User Forum - Sept. 2019
14. First Unprecedented Result
Computational Microscope
• Klaus Schulten (PI) and the
NAMD group - Code
NAMD/Charm++
• Completed the highest
resolution study of the
mechanism of HIV cellular
infection.
• Are now able to develop
mitigation and drug strategies
• May 30, 2013 Cover of Nature
• Orders of magnitude increase
in number of atoms –
resolution at about 1
angstrom
14HPC User Forum - Sept. 2019
15. Earthquake System
Science (PressOn)
Science Goals
Improve resolution of dynamic rupture simulations by ~10X to investigate realistic
friction laws, near-fault stress states, & off-fault plasticity
Investigate upper frequency limit of deterministic ground-motion prediction by
simulating strong motions up to 3 Hz using realistic 3D structural models for
Southern California
Improved understanding of engineering response to earthquakes by modeling
response of buildings to broadband ground motions
Three seismic & engineering codes for Blue Waters
1. AWP-Olsen: finite difference dynamic rupture & wave propagation
2. Hercules: finite element wave propagation
3. OpenSees: structure modeling for earthquake engineering
Tom Jordan, Phil Maechling
HPC User Forum - Sept. 2019 15
16. Using Petascale Computing Capabilities to Address Climate Change
Uncertainties Goals
PI – Don Weubbles – UIUC
The purpose to address key uncertainties associated with the numerical modeling of the Earth’s climate system and the ability
to accurately analyze past and projected future changes in climate.
A unique aspect of the project is coupling of CESM with the newly built Cloud-Aerosol-Radiation (CAR) ensemble modeling system.
Model development which is complementary to the main NCAR/CESM effort
The specific scientific objectives are:
evaluate the model with different model dynamical cores and the effects of going to much higher horizontal and vertical resolution.
very high resolution (10-30 km horizontal resolution) in coupled climate models is motivated evidence that increased resolution leads to
better simulations both of transient small-scale activity, e.g. eddies and waves, and of large-scale features of the mean climate.
New model dynamical cores allow for enhanced resolution as well as the use of the extensive number of the petascale capabilities (i.e., the
cores scale to tens-of-thousands of processors).
Examine uncertainties associated with the representations of processes and interactions occurring between clouds, aerosols,
and radiative transfer in these models and how these uncertainties influence the climate sensitivity (see slides 2 and 3).
16HPC User Forum - Sept. 2019
17. High Resolution Atmospheric Component of
CESM: Projected Changes in Tropical Cyclones
High resolution
(0.25o) atmosphere
simulations produce
an excellent global
hurricane climatology
Courtesy of
Michael Wehner, LBNL
Observations
0.25o FV -CAM5.1
HPC User Forum - Sept. 2019 17
18. Understanding Tornadoes and Their Parent
Supercells Through Ultra-High Resolution
Simulation/Analysis
Goal
Simulate development, structure, & demise of
large damaging tornadoes in supercells at
resolution sufficient to capture low-level
tornado inflow, thin precipitation shaft that
forms “hook echo” adjacent to tornado, and
other smaller scale structures
Approach
Simulate storms with model
that solves (using finite differences) the
equations of motion for air and water
substances (droplets, rain, ice, …)
Bob Wilhelmson (UIUC), Leigh Orf, et al
HPC User Forum - Sept. 2019 18
19. Strong Scaling of Reservoir Simulations
PI: Vadim Dyadechko
ExxonMobil Upstream Research:
Vadim Dyadechko, Sumant Prasad and John Kuzan
NCSA/UIUC:
JaeHyuk Kwack, Greg Bauer and Seid Koric
Blue Waters Allocation (Type/Size):
Industry 188K node-hours (>6 million core hours)
19HPC User Forum - Sept. 2019
21. 21HPC User Forum - Sept. 2019
2018 Reports by Field of Science
• 141 Projects
Biology,
Chemistry &
Health, 37
Computer
Science &
Engineering,
10
Geoscience,
20
Physics &
Engineering,
42
Social
Science,
Economics &
Humanities,
1
Space
Science , 31
22. 22HPC User Forum - Sept. 2019
Self-Identified Categories of Methods
Label Label Number Criteria
DI Data Intensive 61 Uses large numbers of files, e.g. large disk space/bandwidth, or automated workflows/off-site transfers.
GA GPU-Accelerated 35 Written to run faster on XK nodes than on XE nodes
TN Thousand Node 61 Scales to at least 1,000 nodes for production science
MI Memory Intensive 18 Uses at least 50 percent of available memory on 1,000-node-runs
BW Blue Waters 33 Research only possible on Blue Waters
MP
Multi-Physics/multi-
scale
59 Job spans multiple length/timescales or physical/chemical processes
ML Machine Learning 17 Employs deep learning or other techniques, includes “big data”
CI
Communication-
Intensive
31 Requires high-bandwidth/low-latency interconnect for frequent, tightly coupled messaging
IA Industrial Application 10 Researcher has private sector collaborators or results directly applicable to industry
FS
First of Its Kind
Frontier Science
Outcome
25
NSF BI NSF Big Ideas 66
23. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
AUTOMATIC KNOWLEDGE BASE CONSTRUCTION AND HYPOTHESIS GENERATION:
ANTIBIOTIC RESISTANCE MECHANISM FOR ESCHERICHIA COLI
Schematic diagram of the knowledge base construction and novel
hypothesis generation framework.
Antibiotic resistance is is one of the leading threats to global health and food security and its direct
consequences include longer hospital stays, higher medical costs, and increased mortality rates.
Various factors have been preventing the development of machine-learnable knowledge bases that
could help better understand and fight antibiotic resistance.
The team built an extensive knowledge base for E. coli antibiotic resistance
mechanisms that are specifically structured for efficient machine learning. The
knowledge base incorporates a total of one million triples from 10 sources
where the five distinctive durations of antibiotic exposure exist, ranging from
30 minutes to 7 days. The proposed framework can accelerate knowledge
discovery by automating the process of generation of novel hypotheses from
knowledge bases.
The team built a generic framework that constructs an inconsistency-free
knowledge graph based on the multilayered machine-learning method and
then trains the hypothesis generator using this graph. The team optimized the
code to best utilize its principles for the task.
The amount of computation and data processing necessary for the project makes Blue Waters critical to the success of this project. Project staff helped the team
utilize Blue Waters’ computational resources efficiently.
Allocation: NSF PRAC/350 Knh
PI: Ilias Tagkopoulos
University of California, Davis
Biology, Chemistry & Health
ML 2018 BLUE WATERS BOOK
24. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
ENABLING REDISTRICTING REFORM: A COMPUTATIONAL STUDY OF ZONING
OPTIMIZATION
Performance comparison: Spatial Path Relinking crossover operator and
the basic overlay + expansion crossover. Both take the same input. The
difference in fitness between the input and output solution is plotted. A
negative difference value indicates fitness improvement.
Political redistricting is intended to provide fair representation in Congress to all communities and
interest groups. Gerrymandering occurs when districts are drawn in a manner that discriminates
against a partisan or racial group. Due to a lack of sufficient tools to analyze and synthesize
redistricting data, the Supreme Court has found it difficult to identify a workable standard by which
we might regulate gerrymandering. This team has developed a computational redistricting tool
utilizing massively parallel high-performance computing for redistricting optimization and analysis.
The approach is designed to identify redistricting maps that satisfy a set of
user-defined criteria by leveraging and enhancing a scalable Parallel Genetic
Algorithm (PGA) library to develop PEAR for the computationally intensive
redistricting problem. PEAR provides a novel and powerful tool that harnesses
massive computing power to handle spatial characteristics and the associated
computational challenges. The project has been the subject of numerous
amicus briefs and has been discussed in oral arguments before the Supreme
Court.
The algorithm, PEAR, or Parallel Evolutionary Algorithm for Redistricting, uses
MPI nonblocking functions for asynchronous migration, and the C SPRNG 2.0
library to provide a unique random number sequence for each MPI process.
The design of the evolutionary operators incorporates spatial characteristics to
effectively search the solution space. The parallelization of the algorithm
maximizes the overlapping of computing and communication at runtime.
The PEAR library is designed for extreme-scale redistricting applications. The computational approach implemented in the solution generates a very large number
of electoral maps for the quantitative study of redistricting. Identifying quality electoral maps is a very large combinatorial optimization problem that is
computationally complex. Generating a large number of feasible and statistically independent maps is possible only on a supercomputer at the scale of Blue
Waters.
Allocation: Illinois/400 Knh
PI: Wendy K. Tam Cho
University of Illinois at Urbana-Champaign
Social Science, Economics, & Humanities
DI TN MI BW ML CI FS NSF BI
2018 BLUE WATERS BOOK
25. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
TINY GALAXIES HOST THE FIRST GIANT BLACK HOLES: BLUE TIDES SIMULATION MAKES
CONTACT WITH THE FIRST 700 MILLION YEARS OF COSMIC HISTORY
Conical gas outflows around the most massive black hole are powered by the
quasar, which energizes and blows gas out from the host galaxy and also
quenches star formation. The insert shows a histogram of the gas velocities
around the black hole. The simulations predict this outflow should be observable
around the first massive black holes
Understanding and detecting the first galaxies and black holes formed in the first billion years, is one of the main observational
and theoretical challenges in galaxy formation. The fundamental challenge in simulations trying to understand this epoch of the
Universe is that extremely large volumes need to be analyzed as the first objects are rare, while at the same time extremely high
resolution is required as the first galaxies and quasars are expected to be small and compact. A complete simulation of the
universe at the epochs being studied, requires a small enough particle mass to model the first galaxies. It also requires an
enormous volume, of the order of 1 cubic gigaparsec (1 Gpc3 is 3 × 1028 cubic light years) to capture the rarest and brightest
objects. The first requirement is therefore equivalent to a high particle density, and the second to a large volume. The BlueTides
(BT) simulations on Blue Waters have successfully answered these challenges.
• BT is the only simulation making contact with the recently discovered supermassive black hole
and can now make predictions regarding its formation, history, and observation signatures by
next-generation telescopes. BT is also the first and only cosmological simulation of structure
formation that has run on the full BW machine.
• The host galaxy of the most distant known quasar (from 690 million years after the Big Bang),
was explored in the large-volume cosmological hydrodynamic simulation.
• BT simulations studied the feedback around the highest redshift quasar in and predicted that
there are significant outflows observable at radio wavelenghs around this quasar.
• The BT simulations have been made possible with the use of the new
cosmological hydrodynamic simulation code (MP)-Gadget which is
massively parallel
• Radical updates to the code efficiency, the Smooth Particle Hydrodynamics
formulation, and star formation modeling, have met the challenge of
simulating the next-generation space telescope fields and the effective use
of the full BW machine.
• The size of the simulations being performed, meant that BW was the only system that could solve it, both in terms of memory and compute cores. Simulations
essentially used the whole BW.
• Assistance by BW team in scheduling, file system tuning and I/O, and run time problem analysis, were also very important and beneficial.
Allocation: NSF PRAC/1856 Knh
PI: Tiziana DiMatteo
Carnegie Melon University
Space Science
DI TN MI BW MP CI FS 2018 BLUE WATERS BOOK
26. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
SEDIMENT TRANSPORT IN ESTUARIES: ESTIMATING BED SHEAR STRESS DISTRIBUTION
FROM NUMERICALLY MODELED TIDES IN AN ENERGETIC ESTUARY
The Piscataqua River-Great Bay estuary at mid-tide—108.8 hours into the
model run. Darker blue colors indicate higher water levels than lighter blue
colors. The differences in the tidal level from the Atlantic Ocean to the
estuary drive large tidal currents that are important for sediment
transport.
The rise in land development and associated increases in impervious surface cover have led to a
decline in water quality and estuarine health by depositing higher loads of sediment, nutrients, and
pollutants. Numerical modeling is a cost-effective way to predict the impact of sediments on estuary-
wide nutrient loads, a potentially significant but so far largely neglected source of nutrients.
The results suggest that nutrient fluxes derived from sediment resuspension
during a typical tidal cycle are potentially significant and should be considered
when estimating nutrient loads in estuaries. This is particularly important for
estuaries with tidal mudflats, as mud and cohesive sediment tend to trap more
nutrients and pollutants. Scientists, land managers, and legislators should
incorporate this finding into nutrient load estimates, especially from non-point
sources. This is highly relevant for setting project budgets, regulatory limits, and
determining best practices for estuarine management.
This project used the Regional Ocean Modeling System (ROMS) within the
Coupled–Ocean–Atmosphere–Wave–Sediment Transport (COAWST) coupled
modeling framework. The model was forced by a combination of tidal and
meteorological boundary conditions to reflect real world conditions. Time
series analysis and statistical methods were used to determine the best-fit
bottom boundary condition for future model runs. The next phase of the
project will incorporate waves and sediment transport.
Blue Waters system provided the necessary computational power to test models using a higher-resolution 10-meter grid, which was previously infeasible for this
study. Further, the project support staff were an invaluable asset in getting this project up and running on Blue Waters.
2017-2018 BW Grad Fellow
Salme Cook
University of New Hampshire
Geoscience
BW MP CI 2018 BLUE WATERS BOOK
27. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
FACTOR SEPARATION ANALYSIS OF URBAN IMPACTS ON SIMULATED SUPERCELL
Earth's population is increasingly concentrated in urban areas, with nearly two-thirds of the world's population expected to live in urban areas by 2050. However,
interactions between urban areas and synoptically-active convection, such as supercells, remain relatively unexamined. In order to truly understand the nature
of these interactions, and thus provide city planning recommendations for the future, it is important to ascertain whether the urban heat island or slower winds
induced by increased urban surface roughness result in greater storm modifications.
• Full- and single-physics urban simulations were compared to CTRLE, with
the aid of hierarchical clustering analysis (HCA) to form statistically similar
groups of simulations; Investigated the effects of the storm having various
city-relative path and the storm lifecycle stage during urban interactions;
• Result suggests that urban surface roughness may play a greater role in
modifying supercell characteristics than the better-known urban heat
island effect.
• Weather Research and Forecasting (WRF) model is used for a total of 334 simulations of a
supercell thunderstorm to quantify the impacts of a large Great Plains urban area on the
evolution and strength of a supercell thunderstorm
• All simulations were run on a 500-m horizontal grid over a 250-km x 250-km grid to properly
resolve complex urban structure.
• 120 vertical grid points, with 20 of those points in the lowest 1.5 km above ground were used to
resolve the atmospheric boundary layer. More than 29.7 million points over 75,600 timesteps
were used for each simulation.
The large computational and storage requirements of the analysis has made Blue Waters vital to this work. While each simulation was relatively small, the large
quantity of of simulations needed to produce significant results required the large computational and data storage capacities of Blue Waters.
2015-2016 BW Grad Fellow
Larissa Reames
University of Oklahoma
Geoscience
DI MP 2018 BLUE WATERS BOOK
28. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
HIGH-ORDER METHODS FOR TURBULENT TRANSPORT IN ENGINEERING AND
GEOSCIENCES
Left: velocity magnitude from a NekNek simulation of a turbulent jet
entering a container. Right: two subdomains (orange and black)
decomposed from the global domain.
The goal of this effort is to develop new algorithms for large-scale high-fidelity turbulence simulations
for different research problems. The team applies the new apparatus to:
• Better understand how biofilm growth affects the flow hydrodynamics and fine-particle transport
at the unprecedent spatial and temporal resolutions.
• Address problems that either have complicated meshes, involve disparate spatial scales, or have
moving boundaries that would require the remeshing of standard conforming meshes
The team conducted simulations of the flow over biofilm with a bulk Reynolds
number of 8,000 with mesh resolutions ranging from 20 million to 200 million
grid points. The results show the coherent turbulent structures in the flow
caused by the biofilm. The team also used a recently developed method called
NekNek to simulate a jet flowing into a tank. The results showed appreciable
agreement among simulations with Nek5000 and NekNek, with around 20%
reduction in computation cost.
The turbulence simulations are based on the open-source spectral element
code Nek5000. The spectral element method (SEM) is a domain-decomposition
approach in which the solution is represented by tensor-product polynomials
that cover the entire domain. The team conducted for the first time spectrally
accurate direct numerical simulations of a channel with complex natural
roughness.
The team conducted simulations using up to 296 million computational points, with the code demonstrating linear speed-up for this problem out to 32,768 MPI
ranks. High computation requirements combined with the need for fast turnaround times for the parameter sweep made Blue Water ideal for the task.
Allocation: BW Professor/300 Knh
PI: Paul Fischer
University of Illinois at Urbana-Champaign
Physics & Engineering
TN MP CI FS NSF BI 2018 BLUE WATERS BOOK
29. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
QUANTIFYING FOREST DROUGHT RESISTANCE
(a) Canopy structure of monitored trees within the Tapajós National
Forest. (b) Mean daily potential evapotranspiration for both wet
(blue) and dry (red) seasons. (c) Dots represent individual trees in
the domain with color indicating rooting depth. (d) Scenarios of
lateral spread, which vary with tree diameter at breast height (DBH).
Over the past two decades, the Amazon Basin region has been hit with multiple drought events that
were triggered by strong shifts in sea surface temperature caused by the El Niño–Southern
Oscillation. The increased frequency and severity of droughts and their regional consequences have
highlighted the potential vulnerability of the Amazon to heat- and drought-induced stress. To
adequately capture the response of tropical rainforests to water limitation, mechanistic models that
incorporate diverse plant morphology and hydraulic function are needed.
Simulations encompass more than 3,000 individuals of varying size and water
demands. Preliminary results highlight the contribution of root traits to both
individual and community integrity. Analysis is ongoing, but this work
represents one of the largest modeling studies of three-dimensional root
water uptake ever attempted. Results from this work can enhance next-
generation earth system models by identifying key traits for water uptake
processes.
Root architectures that represent the structural and spatial distribution of
roots have been modeled using the open source RootBox model. Each tree
system was assigned hydraulic parameterization based on statistically
generated water usage strategies. Root water uptake was coupled with the
massively parallel flow and transport model PFLOTRAN, using hybridization
techniques. This project is exploring how tree roots contribute to forest
drought resilience in areas of the Amazon rainforest.
Blue Waters is critical to the ongoing success of this project. Simulations of this complexity and scale require the computational power of this system to make
meaningful analyses. Not only are the simulation domains complex, multiple simulations are needed to account for system uncertainty. The enormous biodiversity
and harsh environmental conditions of tropical forests hinder data collection needed for model parameterization. Scalable, physically based models provide a
necessary tool with which to explore modes of uncertainty and help target data collection efforts.
2016-2017 BW Grad Fellow
Elizabeth Agee
University of Michigan
Geoscience
DI 2018 BLUE WATERS BOOK
30. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
MAGNETIC RECONNECTION IN LASER-DRIVEN PLASMAS: FROM ASTROPHYSICS TO THE
LABORATORY IN SILICO
(a) Charge density for a simulation of the Weibel instability calculated
using SIC (simplex-in-cell) and zoomed in on a filament. (b) Momentum
distribution at the location indicated by the green circle in (a). (c) Stream
number field over the same region and time shown in (a). (d) Transverse
momentum along a lineout indicated by the red dotted line in (c).
Magnetic reconnection is a fundamental plasma process that converts magnetic field energy into
plasma kinetic energy though the breaking and rearrangement of magnetic field lines. The goal of this
project is to use simulations to study particles from reconnection in varied plasma conditions, and in
particular to investigate whether laser-driven plasma experiments could be used to study the particle
acceleration properties of reconnection in the laboratory.
• The team demonstrated that electrons can be accelerated by reconnection with
sufficient quantity and energy to be detected in the laboratory.
• Coulomb collisions, included in their simulations, have shown the importance of
collisionality on the structure of the reconnection layer
• Simulation results are being used to guide several experimental programs in the
United States.
• The team has been developing simplex-in-cell (SIC), a novel method for plasma
simulation that, under the right conditions, has been shown to reduce the
number of required simulation particles by a factor of 1,000.
One of the most powerful tools for ab initio plasma simulation is the particle-
in-cell (PIC) method, which treats the plasma as a collection of discrete
simulation particles that interact via electromagnetic forces. The simulations
for this project were run using the massively parallel, fully relativistic PIC code
OSIRIS, and match the experimental conditions produced by the most
energetic laser systems in the world, such as the National Ignition Facility.
This project required the use of large-scale 2D and 3D simulations with sufficient size and resolution to bridge the multiscale physics, from fluid dynamics to the
kinetic microscopic processes. These computationally demanding simulations can require billions of simulation particles, and demand the cores, memory, and
communication performance available on Blue Waters.
2015-2016 BW Grad Fellow
Sam Totorica
Stanford University
Space Science
TN MP CI
2018 BLUE WATERS BOOK
31. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
TOWARD PREDICTIVE COMPUTATIONAL DESIGN OF PRECISION MOLECULAR
OPTOELECTRONICS
A correlated random walk of an electron pair in imaginary time,
efficiently solving the fundamental equation of motion of quantum
chemistry numerically.
Chemical technology can fabricate solid-state materials made of organic components with precise
dimensions for optoelectronic device and catalytic applications. An even greater advance occurs
when this synthetic capability is coupled with computational machinery that can predict their
properties and functions quantitatively. Ab initio many-electron theory provides the necessary
foundation, but its legacy algorithms based on matrix algebra are poorly scalable for larger molecules
or larger computers. This project introduces completely new and scalable stochastic algorithms.
The developed MC-GF2-F12 method efficiently executes on both XE and XK
nodes. It enables an exact (CBS-limit) GF2 calculation of electron-detachment
energies for a wide range of large conjugated organic molecules. The largest
calculation for C70 with 1,610 basis functions was run on 128 GPUs for the GF2
portion and on 896 CPUs for the F12 part. The implemented two-level
parallelism enhances the performance of the redundant-walker algorithm on
GPUs beyond the degree that is possible by merely running it on many CPUs.
The usual sum-of-products matrix expressions of second-order MBGF (GF2)
theory and its complete-basis-set (CBS) correction by explicitly correlated (F12)
ansätze are mathematically transformed into high-dimensional integrals, which
are then evaluated by a highly scalable Metropolis Monte Carlo (MC) method.
The resulting stochastic methods—MC-GF2 and MC-GF2-F12—can compute
energy differences directly without a sign problem in a scalable manner with
respect to both compute size and system size.
The stability and ease of use of Blue Waters as well as the balanced deployment of CPUs and GPUs are all essential for rapid coding/profiling of new scalable
algorithms from scratch and their capacity testing.
Allocation: BW Professor/200 Knh
PI: So Hirata
University of Illinois at Urbana-Champaign
Biology, Chemistry & Health
GA TN 2018 BLUE WATERS BOOK
32. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
IMAGE PROCESSING TO BUILD A MULTI-TEMPORAL VEGETATION ELEVATION
ECOSYSTEM MODEL (MTVEEM) OF THE GREAT LAKES BASIN (GLB)
Top left: June 2016 WorldView-3 image; Top center: Stereo derived Digital
Surface Model (DSM) from the image date on the left; Bottom left: August
2016 WorldView-3; Bottom center: Stereo derived DSM from the image date
on the left; Right: Difference raster between the two DSMs in the center.
How are the ecosystems across the GLB changing? Stereo submeter satellite imagery is used
for the generation of multi-temporal, highly accurate surface elevation models which are highly
demanded by natural resource managers. This research will identify both natural changes (e.g.
flooding, forest blowdown, fire, insect and disease damage) and anthropogenic changes (e.g.
harvest and land-cover change). MTVEEM will improve habitat and biological modeling for
mapping and monitoring canopy change in forested and wetland habitats across the GLB.
The final product of this research, a seamless and registered surface elevation
ecosystem model (MTVEEM) across the GLB, will enable a large range of
science activities and provide substantially higher resolution data than
currently available. These canopy maps and change detection products will
provide positional accuracies of less than a couple meters with the added
ground control points. Preliminary results show great promise for providing
valuable data to myriad of coastal and terrestrial ecosystem science
researchers and decision-makers across the entire GLB.
*In 2017, 83,000 stereo pairs were processed; in 2018 50,000 pairs are expected
*Each job converts the input image pairs to GeoTIFF and calls the elevation
extraction software (SETSM)
*Each pair is run on a single node, submitted in batches of 2-100 tasks; each
task takes an average of 12 NH (6 NH charged due to low-priority discount)
*Additional 150,000–200,000 NH to process and classify ortho-images
The amount of stereo imagery the GLB and the computational burden to process this stereo imagery is well beyond those available from academic, private, and
government systems. Since these image sets are flexible for processing, these researchers work with the Polar Geospatial Center to backfill using Blue Waters
existing and new stereo imagery over the entire GLB. Each stereo pair is about 30GB and the total number of pairs processed to date is about 100,000 which moves
this volume alone into the petascale environment.
Allocation: GLCPC/540 Knh
PI: Jennifer Corcoran
University of Minnesota, Twin Cities
Geoscience
DI 2018 BLUE WATERS BOOK
33. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
IDENTIFICATION OF MISSING VARIANTS IN ALZHEIMER’S DISEASE, AND THE NEW
STANDARDS FOR GENOMIC VARIANT IDENTIFICATION IN LARGE COHORTS
To study the effects of sample size, we performed joint genotyping
on variable-size batches subsampled from 10,000 whole exomes. We
subsampled each batch size five times, and ran joint genotyping to
see how many variants could be recovered. See
https://github.com/dpwickland/JointGenotyping_BW.
Alzheimer’s disease (AD) is a neurodegenerative dementia that affects more than five million
Americans and more than 35 million people worldwide. This project will deliver novel genomic
variants that have remained hitherto undetected by the standard workflow in AD. These variants will
be posted into public databases for use by researchers and clinicians worldwide to improve
understanding of the genomic underpinnings of AD, as well as drug development and treatment
outcome prediction.
By combining two read aligners and several variant callers into the workflow, they
were able to recover 50% of the variants in the ADSP data that were missed by the
standard protocol. They further annotated SNPs, or genetic variations in a single
DNA building block, as synonymous or nonsynonymous, and assessed the
proportion of alternate alleles between cases and controls. They discovered SNPs
within genes previously reported to interact with Alzheimer's-related proteins or to
function in the brain.
The team tested a select number of steps and parameters in the variant detection
pipeline in the context of sample sizes. They grouped the Alzheimer Disease
Sequencing Project (ADSP) samples into different samples sizes of 50, 500, 1000,
2000, 5000, and 10,000. For each sample size, they tested two different aligners,
three different variant callers, multi- vs. single-sample variant calling, and five
different parameter settings in the variant calling and quality control process. The
goal was to build a new set of guidelines for variant discovery based on project size.
The team tested multiple combinations of steps and parameters using data from the Alzheimer's Disease Sequencing Project consisting of over 10,000 whole exome
sequencing samples. The total amount of time that would be required to complete this project on a single server would be 109 years. On Blue Waters, they were
able to run a single workflow on the entire set of 10,000 AD samples by parallelizing across thousands of nodes.
Allocation: Illinois/350 Knh
PI: Liudmila Mainzer
University of Illinois at Urbana-Champaign
Biology, Chemistry & Health
DI BW NSF BI 2018 BLUE WATERS BOOK
34. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
DEEP LEARNING FOR MULTIMESSENGER ASTROPHYSICS: REAL TIME DISCOVERY AT
SCALE
CNNs can be used for both detection and parameter estimation of
GW signals in raw LIGO data.
Matched-filtering searches, the most sensitive gravitational wave (GW) detection algorithms used by
LIGO, currently target a 4D parameter. Extending these template-matching searches to target the 9D
parameter space available to GW detectors is computationally prohibitive. To address these
limitations, we pioneered the use of GPU-accelerated deep learning algorithms that employ a system
of two deep convolution neural networks (CNNs) that take time-series inputs for both classification
and regression.
Deep Filtering demonstrated, for the first time, that machine learning can
detect and recover true parameters of real GW signals observed by LIGO and
achieve performance comparable to matched-filtering methods, several
orders of magnitude faster and far more resilient to transient noise artifacts.
We showed that after a single training process, Deep Filtering can
automatically generalize to noise having new Power Spectral Densities (PSDs)
from different LIGO events, without retraining.
We used the Wolfram Language neural network functionality, built using the
open-source MXNet framework, that uses the cuDNN library for accelerating
training on NVIDIA GPUs. The learning algorithm was ADAM. While training, we
used the curriculum learning strategy to improve the performance and reduce
training times of the CNNs while retaining performance at very high signal-to-
noise ratios.
Blue Waters played a critical role in creating the numerical relativity waveforms used to train and test deep learning algorithms. In recent developments, Blue
Waters has provided the required scale and computational power to construct deep neural networks using distributed learning involving over 1,024 GPUs.
Allocation: Illinois/570 Knh
PI: Eliu Huerta
University of Illinois at Urbana-Champaign
Astrophysics
GA BW ML FS NSF BI 2018 BLUE WATERS BOOK
35. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
COUPLED MULTIPHYSICS OF ADVANCED MOLTEN-SALT NUCLEAR REACTORS
The coupled fast neutron flux and temperature in a 3D cuboidal model of a
molten-salt reactor. Moltres coupled results have been validated against
experimental results from the Molten-Salt Reactor Experiment.
Nuclear power provides 19% of the total electricity generated in the United States and is our largest
source of clean energy. Advanced molten-salt reactor (MSR) technology represents numerous distinct
advantages over the light-water reactor (LWR) used today. Current state of the art in advanced
nuclear reactor simulation focuses primarily on LWR. This work extends the state of the art by
enabling modeling and high-fidelity simulation for MSR designs. This requires development of models
and tools for representing unique materials, geometries, and physical phenomena.
Developed in the open, this effort enables both transparency and distributed
collaboration on promising nuclear reactor concepts. It can improve designs, help
characterize performance, inform reactor safety margins, and enable validation of
numerical modeling techniques for unique physics.
MSR technology is exciting and well worth advancing. MSRs are a compelling next-
generation nuclear reactor technology as many MSR designs promise improved
passive safety, fissile material utilization, recycling flexibility, and power generation
responsiveness appropriate for an electric grid involving variable renewable
electricity generation.
Two new simulation tools were developed to enable advanced reactor and fuel
cycle simulations. First, the finite-element-based physics application, Moltres,
couples the thermal–hydraulics and neutronics of molten-salt flow in high-
temperature liquid-fueled reactor designs. Second, a Python package for
modeling fuel-salt reprocessing, Saltproc. It relies on full-core high-fidelity
Monte Carlo simulations to perform depletion computations that require
significant computational time and memory.
Accurate simulations at realistic spatial and temporal resolution are only possible with the aid of high-performance computing resources. To assess nuclear reactor
performance under a variety of conditions and dynamic transients, myriad 2D and 3D finite element simulations must be conducted. Transient and multiscale
simulations, which require greater capability per simulation, are on the horizon for our work. These may occupy up to 100,000 CPU cores at a time.
Allocation: BW Professor/60 Knh
PI: Kathryn Huff
University of Illinois at Urbana-Champaign
Physics & Engineering
TN MI MP IA NSF BI
2018 BLUE WATERS BOOK
36. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
HPC-BASED HIGH-RESOLUTION QUANTITATIVE MRI
(Top) MPRAGE, FLAIR, and MRE-Stiffness images from a patient with mesial temporal lobe
sclerosis. (Bottom) The ratio of stiffness of epileptogenic to nonaffected hippocampus (HC)
is significantly greater compared to controls. The ROC curve indicates that the ratio of HC
stiffness dissociates patients from controls with high accuracy.
The team is a leader in Magnetic Resonance Elastography (MRE) development and
application, enabling the study of the mechanical properties of the brain. They have
been able to map the brain at unprecedented resolution and to probe specific
neuroanatomical structures. The method will benefit investigators of both
neurodegenerative disease and epileptologists looking at long-term changes to tissue.
MRE is a useful tool for identifying patients with mesial temporal lobe epilepsy
due to an asymmetry in hippocampal scarring. The research work has revealed
significant softening of the nonsclerotic hippocampus compared to healthy
controls, which may provide an important biomarker for early disease
detection, which is critical for successful treatment before significant and
irreversible damage occurs. The team now has sufficient pilot data to write a
grant to translate this technology into the clinic for early detection.
PowerGrid, an open source, GPU-accelerated, MRI image reconstruction toolkit,
computes of a series of displacement encoded images per subject to enable MRE.
PowerGrid leverages GPUs to apply parallel imaging, corrections for imperfect
magnetic fields, and subject motion. Nonlinear Inversion (NLI) for iterative MRE
material property estimation from displacement encoded images, divides the brain
into smaller regions for Finite-Element Method (FEM) meshing and optimization of
properties at a local level, which are then reconstituted as a global solution, allowing
parallelization across hundreds to thousands of CPU cores.
The Blue Waters system provides a unique resource for MRE because it allows for rapid transfer of imaging data from the scanners and the low-latency memory
transfers necessary for highly parallel computations. For PowerGrid, Blue Waters provides an environment with a large number of GPUs as well as support for MPI
for leveraging GPUs across nodes.
Allocation: Illinois/100 Knh
PI: Brad Sutton
University of Illinois at Urbana-Champaign
Biology, Chemistry & Health
DI GA 2018 BLUE WATERS BOOK
37. Research Challenge
Results & ImpactMethods & Codes
Why Blue Waters
Surface topography is among the most fundamental Earth Science data sets, essential to a wide
range of research activities, including ice mass-balance, hazard assessment and mitigation,
hydrologic modeling, solid earth dynamics, and many others. The National Geospatial-Intelligence
Agency, DigitalGlobe, and the Polar Geospatial Center built a near-seamless archive of polar sub-
meter stereo imagery that consists of millions of stereo pair images from the Worldview-1, -2, and 3
satellites. Using photogrammetric algorithms, they are able to construct digital elevation models
from the stereo pairs, enabling mapping of surface features at the 2-meter scale for the first time.
ENHANCED DIGITAL ELEVATION MODEL FOR THE ARCTIC
The poles now have better time-dependant topography than almost
anywhere else on earth. In addition, The ice on earth has better
topography than the land surface on Earth. Our team has produced
over 200,000,000 km2 2m posting topography covering the Arctic over 8
times. The data has been adopted by the Greenland, Iceland and
Canada as their national standard elevation data.
The Surface Extraction from TIN-based Search-space Minimization
(SETSM) algorithm, initially designed to extract elevation data over ice
sheets, has been refined and optimized to handle stereoscopic imagery
over any landcover. After an initial preprocessing step that corrects the
source imagery for sensor-specific detector alignment artifacts, SETSM
takes the two source images and derives increasingly detailed elevation
models using its pyramid-based approach.
No other open research/academic computer has the capacity for this project. Over 17 million node hours (more than ½ billion core hours) were used to
process the archive of stereoscopic imagery that was made available. Additionally, the staff at the Blue Waters Project were invaluable in adapting the
leadership system to handle the single-node, high-throughput ArcticDEM workflow.
Allocation: NSF PRAC/9696 Knh
PI: Paul Morin
University of Minnesota
Geoscience
DI TN BW FS NSF BI
2018 BLUE WATERS BOOK
A shaded relief map of the final release of
ArcticDEM.
38. 38HPC User Forum - Sept. 2019
2019 NSF Allocations
• The last round of NSF PRAC allocations on Blue Waters provided
large allocations for a small set of teams to use in nine months.
• “Accretion Dynamics of Black Hole Binaries”, Manuela Campanelli,
16MNH
• “Supernova Explosion Simulations”, Adam Burrows, 15MNH
• Nuclear QCD, Phiala Shanahan, 14MNH
• “Petascale Supercell Simulations”, Leigh Orf, 21MNH
• “Marine Aerosol Surfactant Partitioning”, Kimberly Prather, 6.4MNH
• “Hippocampal network optimization”, Ivan Soltesz, 8.1MNH
• “Petascale World Topography Production”, Paul Morin, 25MNH
• …
39. Communicating Blue Waters Science
• All current and past science teams
• https://bluewaters.ncsa.illinois.edu/science-teams
• Usage by science teams
• https://bluewaters.ncsa.illinois.edu/status-usage
• Impact Stories
• https://bluewaters.ncsa.illinois.edu/impact-overview
• Public Video releases
• Full Dome and HD documentaries
• http://www.ncsa.illinois.edu/enabling/vis/cadens/documentary/solar
_superstorms
• Solar Superstorm, Seeing the Beginning of Time, Super
Tornadoes, Birth of Planet Earth, The Jupiter Engima
• Upcoming - Atlas of a Changing Earth – working title
• Geospatial, mapping and polar
• Science project and training videos
HPC User Forum - Sept. 2019 39
41. • Blue Waters operations will
continue through 2020 funded
by the National Geospatial-
Intelligence Agency (NGA)
• Open research, but focused
on projects of interest to NGA.
• Primary use will be DEM
production.
41HPC User Forum - Sept. 2019
Blue Waters in 2020
42. Blue Waters Reliability
The reliability of Blue Waters remains excellent!
• Overall node failure rate for the past year: 2.07 nodes/day – 0.008%
• MTBF for a node: ~83 years
• GPUs failed at a rate of 0.7%/year
• Hard drive failure rate: ~3/week - <1%/year
• Failure rates are stable.
42HPC User Forum - Sept. 2019
43. 43HPC User Forum - Sept. 2019
Blue Waters Use in the NGA Operations Phase
• 90% of Blue Waters is available December 20, 2019 – December 31, 2020 for
use by projects of interest to NGA
• ~180 MNH/5.7Billion Core Hours
• Staff to support development/porting/optimization of applications and workflows on
the system.
• Allocation requests will be accepted on a continuous basis with a light-weight
submission (~2-5 page)
• Describe the work being performed
• How the work relates to the NGA mission
• The computation and data needs
• Expert assistance needed in areas of application develop, porting, scaling and
overall application workflow
• Requests will be reviewed by a steering committee.
• Details will be announced soon.
44. 44HPC User Forum - Sept. 2019
Scope
• Geospatial projects in general
• Mapping and Modeling of the Earth
• Large majority of time expected to be used for DEM production.
• Education projects
• In support of classes, workshops and training events
45. 46HPC User Forum - Sept. 2019
Acknowledgments
• This research is part of the Blue Waters sustained-petascale computing
project, which is supported by the National Science Foundation (awards OCI-
0725070 and ACI-1238993) and the state of Illinois. Blue Waters is a joint effort
of the University of Illinois at Urbana-Champaign and its National Center for
Supercomputing Applications.
• Personnel: NCSA Blue Waters team, Cray site team