The document summarizes recent advances at the Argonne Leadership Computing Facility (ALCF). It describes the ALCF's Intrepid Blue Gene/P system, resources like storage and visualization systems, and services provided to users. It highlights several large science projects utilizing ALCF resources across many domains like climate modeling, materials science, and biology. The document also discusses the INCITE and ALCC programs that allocate computing time and the ALCF's upcoming BG/Q system called Mira.
Blue Waters and Resource Management - Now and in the Futureinside-BigData.com
In this presentation from Moabcon 2013, Bill Kramer from NCSA presents: Blue Waters and Resource Management - Now and in the Future.
Watch the video of this presentation: http://insidehpc.com/?p=36343
Scalable Deep Learning in ExtremeEarth-phiweek19ExtremeEarth
This document summarizes a presentation about scalable deep learning techniques for analyzing Copernicus Earth observation data using the Hopsworks platform. The presentation discusses Hopsworks' end-to-end machine learning pipelines, feature engineering capabilities, distributed deep learning techniques like data parallel training, and applications of these techniques to challenges in classifying satellite imagery like sea ice mapping. Deep learning architectures, preprocessing steps, and distributed training methods are highlighted as areas of ongoing work and improvement for analyzing large volumes of remote sensing data on Hopsworks.
In this ACM Tech Talk, Doug Kothe from ORNL presents: The Exascale Computing Project and the future of HPC.
"The mission of the US Department of Energy (DOE) Exascale Computing Project (ECP) was initiated in 2016 as a formal DOE project and extends through 2022. The ECP is designing the software infrastructure to enable the next generation of supercomputers—systems capable of more than 1018 operations per second—to effectively and efficiently run applications that address currently intractable problems of strategic importance. The ECP is creating and deploying an expanded and vertically integrated software stack on US Department of Energy (DOE) HPC exascale and pre-exascale systems, thereby defining the enduring US exascale ecosystem."
Watch the video: https://wp.me/p3RLHQ-kep
Learn more: https://www.exascaleproject.org/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck from the DDN User Group at SC19, Gael Delbray from CEA presents: Optimizing Flash at Scale. CEA, a major player in research and innovation, has been recognized as an expert in HPC through the momentum of the "Simulation Programme" supported by its Direction des Applications Militaires (CEA / DAM) and implemented by the Department of Simulation Sciences and Information (DSSI).
"The major challenges that the HPC will face in the coming years are manifold, such as the development of hardware and software architectures able to deliver very high computing power, modelling methods combining different scales and physical models and the management of huge volumes of numerical data.
High performance computing for numerical simulation has become an essential tool in scientific and technological research, as well as for industrial applications. Simulation can replace experiments that are too dangerous (accidental situations), beyond reach in terms of time or scale (climate, astrophysics) or banned (nuclear tests). Simulation is also time-saving and leverages productivity in many situations."
Watch the video: https://wp.me/p3RLHQ-li0
Learn more: https://www.ddn.com/company/events/user-group-sc/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facilityinside-BigData.com
In this deck from the Swiss HPC Conference, Mark Wilkinson presents: 40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility.
"DiRAC is the integrated supercomputing facility for theoretical modeling and HPC-based research in particle physics, and astrophysics, cosmology, and nuclear physics, all areas in which the UK is world-leading. DiRAC provides a variety of compute resources, matching machine architecture to the algorithm design and requirements of the research problems to be solved. As a single federated Facility, DiRAC allows more effective and efficient use of computing resources, supporting the delivery of the science programs across the STFC research communities. It provides a common training and consultation framework and, crucially, provides critical mass and a coordinating structure for both small- and large-scale cross-discipline science projects, the technical support needed to run and develop a distributed HPC service, and a pool of expertise to support knowledge transfer and industrial partnership projects. The on-going development and sharing of best-practice for the delivery of productive, national HPC services with DiRAC enables STFC researchers to produce world-leading science across the entire STFC science theory program."
Watch the video: https://wp.me/p3RLHQ-k94
Learn more: https://dirac.ac.uk/
and
http://hpcadvisorycouncil.com/events/2019/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Getting Access to ALCF Resources and Servicesdavidemartin
This document provides an overview of the resources and services available at the Argonne Leadership Computing Facility (ALCF). It summarizes the ALCF's computing systems, including the Intrepid and Eureka supercomputers. It also describes the various allocation programs that provide computing time, such as INCITE and ALCC. Finally, it outlines the services and support offered by the ALCF, including programming models, libraries, performance engineering, and user support.
Polar Use Case - ExtremeEarth Open WorkshopExtremeEarth
This document provides an overview of an ExtremeEarth project that aims to apply deep learning techniques to classify sea ice in polar regions using satellite imagery. The project has received funding from the European Union. It discusses challenges in classifying sea ice from SAR imagery compared to optical imagery. It outlines user requirements for sea ice products, including high resolution (300m or better) and frequent updates (near real-time). The document describes workflows using the Polar Thematic Exploitation Platform (Polar TEP) for large-scale sea ice mapping using Copernicus satellite data and machine learning algorithms. It also discusses exploitation of results, including the impact of Polar TEP and efforts to facilitate the polar machine learning community.
Paul Messina presented this deck at the HPC User Forum in Austin. "The Exascale Computing Project (ECP) is a collaborative effort of two US Department of Energy (DOE) organizations – the Office of Science (DOE-SC) and the National Nuclear Security Administration (NNSA). As part of President Obama’s National Strategic Computing initiative, ECP was established to develop a new class of high-performance computing systems whose power will be a thousand times more powerful than today’s petaflop machines. ECP’s work encompasses applications, system software, hardware technologies and architectures, and workforce development to meet the scientific and national security mission needs of DOE."
Watch the video presentation: http://wp.me/p3RLHQ-fIC
Learn more: http://insidehpc.com/ecp
Blue Waters and Resource Management - Now and in the Futureinside-BigData.com
In this presentation from Moabcon 2013, Bill Kramer from NCSA presents: Blue Waters and Resource Management - Now and in the Future.
Watch the video of this presentation: http://insidehpc.com/?p=36343
Scalable Deep Learning in ExtremeEarth-phiweek19ExtremeEarth
This document summarizes a presentation about scalable deep learning techniques for analyzing Copernicus Earth observation data using the Hopsworks platform. The presentation discusses Hopsworks' end-to-end machine learning pipelines, feature engineering capabilities, distributed deep learning techniques like data parallel training, and applications of these techniques to challenges in classifying satellite imagery like sea ice mapping. Deep learning architectures, preprocessing steps, and distributed training methods are highlighted as areas of ongoing work and improvement for analyzing large volumes of remote sensing data on Hopsworks.
In this ACM Tech Talk, Doug Kothe from ORNL presents: The Exascale Computing Project and the future of HPC.
"The mission of the US Department of Energy (DOE) Exascale Computing Project (ECP) was initiated in 2016 as a formal DOE project and extends through 2022. The ECP is designing the software infrastructure to enable the next generation of supercomputers—systems capable of more than 1018 operations per second—to effectively and efficiently run applications that address currently intractable problems of strategic importance. The ECP is creating and deploying an expanded and vertically integrated software stack on US Department of Energy (DOE) HPC exascale and pre-exascale systems, thereby defining the enduring US exascale ecosystem."
Watch the video: https://wp.me/p3RLHQ-kep
Learn more: https://www.exascaleproject.org/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
In this deck from the DDN User Group at SC19, Gael Delbray from CEA presents: Optimizing Flash at Scale. CEA, a major player in research and innovation, has been recognized as an expert in HPC through the momentum of the "Simulation Programme" supported by its Direction des Applications Militaires (CEA / DAM) and implemented by the Department of Simulation Sciences and Information (DSSI).
"The major challenges that the HPC will face in the coming years are manifold, such as the development of hardware and software architectures able to deliver very high computing power, modelling methods combining different scales and physical models and the management of huge volumes of numerical data.
High performance computing for numerical simulation has become an essential tool in scientific and technological research, as well as for industrial applications. Simulation can replace experiments that are too dangerous (accidental situations), beyond reach in terms of time or scale (climate, astrophysics) or banned (nuclear tests). Simulation is also time-saving and leverages productivity in many situations."
Watch the video: https://wp.me/p3RLHQ-li0
Learn more: https://www.ddn.com/company/events/user-group-sc/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facilityinside-BigData.com
In this deck from the Swiss HPC Conference, Mark Wilkinson presents: 40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility.
"DiRAC is the integrated supercomputing facility for theoretical modeling and HPC-based research in particle physics, and astrophysics, cosmology, and nuclear physics, all areas in which the UK is world-leading. DiRAC provides a variety of compute resources, matching machine architecture to the algorithm design and requirements of the research problems to be solved. As a single federated Facility, DiRAC allows more effective and efficient use of computing resources, supporting the delivery of the science programs across the STFC research communities. It provides a common training and consultation framework and, crucially, provides critical mass and a coordinating structure for both small- and large-scale cross-discipline science projects, the technical support needed to run and develop a distributed HPC service, and a pool of expertise to support knowledge transfer and industrial partnership projects. The on-going development and sharing of best-practice for the delivery of productive, national HPC services with DiRAC enables STFC researchers to produce world-leading science across the entire STFC science theory program."
Watch the video: https://wp.me/p3RLHQ-k94
Learn more: https://dirac.ac.uk/
and
http://hpcadvisorycouncil.com/events/2019/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Getting Access to ALCF Resources and Servicesdavidemartin
This document provides an overview of the resources and services available at the Argonne Leadership Computing Facility (ALCF). It summarizes the ALCF's computing systems, including the Intrepid and Eureka supercomputers. It also describes the various allocation programs that provide computing time, such as INCITE and ALCC. Finally, it outlines the services and support offered by the ALCF, including programming models, libraries, performance engineering, and user support.
Polar Use Case - ExtremeEarth Open WorkshopExtremeEarth
This document provides an overview of an ExtremeEarth project that aims to apply deep learning techniques to classify sea ice in polar regions using satellite imagery. The project has received funding from the European Union. It discusses challenges in classifying sea ice from SAR imagery compared to optical imagery. It outlines user requirements for sea ice products, including high resolution (300m or better) and frequent updates (near real-time). The document describes workflows using the Polar Thematic Exploitation Platform (Polar TEP) for large-scale sea ice mapping using Copernicus satellite data and machine learning algorithms. It also discusses exploitation of results, including the impact of Polar TEP and efforts to facilitate the polar machine learning community.
Paul Messina presented this deck at the HPC User Forum in Austin. "The Exascale Computing Project (ECP) is a collaborative effort of two US Department of Energy (DOE) organizations – the Office of Science (DOE-SC) and the National Nuclear Security Administration (NNSA). As part of President Obama’s National Strategic Computing initiative, ECP was established to develop a new class of high-performance computing systems whose power will be a thousand times more powerful than today’s petaflop machines. ECP’s work encompasses applications, system software, hardware technologies and architectures, and workforce development to meet the scientific and national security mission needs of DOE."
Watch the video presentation: http://wp.me/p3RLHQ-fIC
Learn more: http://insidehpc.com/ecp
Exascale Computing Project - Driving a HUGE Change in a Changing Worldinside-BigData.com
In this video from the OpenFabrics Workshop in Austin, Al Geist from ORNL presents: Exascale Computing Project - Driving a HUGE Change in a Changing World.
"In this keynote, Mr. Geist will discuss the need for future Department of Energy supercomputers to solve emerging data science and machine learning problems in addition to running traditional modeling and simulation applications. In August 2016, the Exascale Computing Project (ECP) was approved to support a huge lift in the trajectory of U.S. High Performance Computing (HPC). The ECP goals are intended to enable the delivery of capable exascale computers in 2022 and one early exascale system in 2021, which will foster a rich exascale ecosystem and work toward ensuring continued U.S. leadership in HPC. He will also share how the ECP plans to achieve these goals and the potential positive impacts for OFA."
Learn more: https://exascaleproject.org/
and
https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: https://www.openfabrics.org/index.php/abstracts-agenda.html
High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biom...Larry Smarr
11.04.06
Joint Presentation
UCSD School of Medicine Research Council
Larry Smarr, Calit2 & Phil Papadopoulos, SDSC/Calit2
Title: High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biomedical Sciences
High Performance Cyberinfrastructure Enabling Data-Driven Science Supporting ...Larry Smarr
11.05.13
Invited Presentation
Sanford Consortium for Regenerative Medicine
Salk Institute, La Jolla
Larry Smarr, Calit2 & Phil Papadopoulos, SDSC/Calit2
Title: High Performance Cyberinfrastructure Enabling Data-Driven Science Supporting Stem Cell Research
In this deck from the 2017 MVAPICH User Group, Adam Moody from Lawrence Livermore National Laboratory presents: MVAPICH: How a Bunch of Buckeyes Crack Tough Nuts.
"High-performance computing is being applied to solve the world's most daunting problems, including researching climate change, studying fusion physics, and curing cancer. MPI is a key component in this work, and as such, the MVAPICH team plays a critical role in these efforts. In this talk, I will discuss recent science that MVAPICH has enabled and describe future research that is planned. I will detail how the MVAPICH team has responded to address past problems and list the requirements that future work will demand."
Watch the video: https://wp.me/p3RLHQ-hp6
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
Geofizyka Krakow Selects Panasas for Simplicity and PerformancePanasas
Geofizyka Krakow’s customers rely on the critical data they receive from the company to be competitive in a very demanding industry. The company needed an IT storage solution that could manage the its demanding computing work workload while retaining management simplicity. With Panasas Geofizyka Krakow is seeing up to 6X faster completion of
seismic processing jobs, improved image results, and more.
The document describes the Green Computing Observatory (GCO), which monitors energy usage at a large computing center. The GCO collects data from sensors that measure energy usage of individual components. It publishes the data through XML files to analyze energy efficiency. The GCO aims to address barriers to improved energy efficiency by collecting detailed usage data and defining semantics for the data through an ontology. It currently collects data from sensors on servers, power distribution units, and the Ganglia monitoring system. Future work includes developing a consistent XML schema and calculating total energy consumption.
f people think about big data, they always think about Twitter or Facebook. But there are other areas where much more data amounts incurred and the analyzes are more complex. In this talk, we talk about a real example from bioinformatics. We will explain the actual scenario and how the various Microsoft platforms from SQL Server to Azure Analytics and HDInsight could help us – or not.
TPCx-HS is the first vendor-neutral benchmark focused on big data systems – which have become a critical part of the enterprise IT ecosystem.
Watch the video presentation: http://wp.me/p3RLHQ-cLY
Learn more: http://www.tpc.org/tpcx-hs
SCFE 2020 OpenCAPI presentation as part of OpenPWOER TutorialGanesan Narayanasamy
This document introduces hardware acceleration using FPGAs with OpenCAPI. It discusses how classic FPGA acceleration has issues like slow CPU-managed memory access and lack of data coherency. OpenCAPI allows FPGAs to directly access host memory, providing faster memory access and data coherency. It also introduces the OC-Accel framework that allows programming FPGAs using C/C++ instead of HDL languages, addressing issues like long development times. Example applications demonstrated significant performance improvements using this approach over CPU-only or classic FPGA acceleration methods.
A Semantic-Based Approach to Attain Reproducibility of Computational Environm...Idafen Santana Pérez
Slides from our presentation at the 1st International Workshop on Reproducibility in Parallel Computing (REPPAR'14) in conjunction with Euro-Par 2014 (August 25-29)
Genomics at the Speed of Light: Understanding the Living OceanLarry Smarr
06.07.17-19
Invited Talk
The Gordon and Betty Moore Foundation 2nd Annual Marine Microbiology Investigator Symposium The Golden Gate Club, The Presidio of San Francisco
Title: Genomics at the Speed of Light: Understanding the Living Ocean
San Francisco, CA
CINECA for HCP and e-infrastructures infrastructuresCineca
Sanzio Bassini. Head of the HPC Department of Cineca. Cineca is the technological partner of the Ministry of Education, and takes part in the Italian commitment for the development of e-infrastrcuture in Italy and in Europe for HCP and HCP technologies; scientific data repository and management, cloud computing for industries and Public administration, for the development of computing intensive and data intensive methods for science and engineering
Cineca offers a unique offer for: open access of integrated tier0 and tier1 HCP national infrastructure; of education and training activities under the umbrella of PRACE Training
advanced center action; integrated help desk and scale up process for HCP users support
Preparing to program Aurora at Exascale - Early experiences and future direct...inside-BigData.com
In this deck from IWOCL / SYCLcon 2020, Hal Finkel from Argonne National Laboratory presents: Preparing to program Aurora at Exascale - Early experiences and future directions.
"Argonne National Laboratory’s Leadership Computing Facility will be home to Aurora, our first exascale supercomputer. Aurora promises to take scientific computing to a whole new level, and scientists and engineers from many different fields will take advantage of Aurora’s unprecedented computational capabilities to push the boundaries of human knowledge. In addition, Aurora’s support for advanced machine-learning and big-data computations will enable scientific workflows incorporating these techniques along with traditional HPC algorithms. Programming the state-of-the-art hardware in Aurora will be accomplished using state-of-the-art programming models. Some of these models, such as OpenMP, are long-established in the HPC ecosystem. Other models, such as Intel’s oneAPI, based on SYCL, are relatively-new models constructed with the benefit of significant experience. Many applications will not use these models directly, but rather, will use C++ abstraction libraries such as Kokkos or RAJA. Python will also be a common entry point to high-performance capabilities. As we look toward the future, features in the C++ standard itself will become increasingly relevant for accessing the extreme parallelism of exascale platforms.
This presentation will summarize the experiences of our team as we prepare for Aurora, exploring how to port applications to Aurora’s architecture and programming models, and distilling the challenges and best practices we’ve developed to date. oneAPI/SYCL and OpenMP are both critical models in these efforts, and while the ecosystem for Aurora has yet to mature, we’ve already had a great deal of success. Importantly, we are not passive recipients of programming models developed by others. Our team works not only with vendor-provided compilers and tools, but also develops improved open-source LLVM-based technologies that feed both open-source and vendor-provided capabilities. In addition, we actively participate in the standardization of OpenMP, SYCL, and C++. To conclude, I’ll share our thoughts on how these models can best develop in the future to support exascale-class systems."
Watch the video: https://wp.me/p3RLHQ-lPT
Learn more: https://www.iwocl.org/iwocl-2020/conference-program/
and
https://www.anl.gov/topic/aurora
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
This document discusses evaluating FPGA acceleration for real-time unstructured search. It motivates the need for high-performance and energy-efficient data analysis due to the explosion of data. It describes using FPGAs to accelerate unstructured search workloads like document filtering and profile matching. It details the workload algorithm, hardware platform with four FPGAs, synthetic datasets used, and experimental parameters. Performance results show the FPGA acceleration provides speedups of 23X-38X and energy efficiency improvements of 31X-40X compared to optimized multi-threaded CPU implementations.
TAU Performance System and the Extreme-scale Scientific Software Stack (E4S) aim to improve productivity for HPC and AI workloads. TAU provides a portable performance evaluation toolkit, while E4S delivers modular and interoperable software stacks. Together, they lower barriers to using software tools from the Exascale Computing Project and enable performance analysis of complex, multi-component applications.
Hadoop as a Platform for Genomics - Strata 2015, San JoseAllen Day, PhD
Personalized medicine holds much promise to improve the quality of human life.
However, personalizing medicine depends on genome analysis software that does not scale well. Given the potential impact on society, genomics takes first place among fields of science that can benefit from Hadoop.
A single human genome contains about 3 billion base pairs. This is less than 1 gigabyte of data but the intermediate data produced by a DNA sequencer, required to produce a sequenced human genome, is many hundreds of times larger. Beyond the huge storage requirement, deep genomic analysis across large populations of humans requires enormous computational capacity as well.
Interestingly enough, while genome scientists have adopted the concept of MapReduce for parallelizing I/O, they have not embraced the Hadoop ecosystem. For example, the popular Genome Analysis Toolkit (GATK) uses a proprietary MapReduce implementation that can scale vertically but not horizontally.
The document discusses plans to upgrade the CMS Event Builder for the long shutdown in order to accommodate higher data rates expected during the CMS upgrade. It describes testing advanced networking technologies like 10/40GbE and Infiniband to replace the existing Myrinet-based system. Preliminary measurements are being made to evaluate these technologies for reliably transmitting the larger event fragments at 100kHz in the upgraded event builder network. The CMS online software framework called XDAQ provides a flexible architecture to interface the new networking technologies through pluggable peer transport layers and remain largely unchanged despite the underlying network upgrade.
Technology Development and Commercialization at Argonne National Laboratoryasauers
Argonne National Laboratory is a multidisciplinary science and engineering research center operated by the University of Chicago for the U.S. Department of Energy. With an annual budget of $650 million and about 3,000 employees, including 1,000 scientists and engineers, Argonne conducts basic and applied research across various fields such as materials science, biology, computing, and energy technologies. Argonne operates several scientific user facilities and is a leader in high-performance computing. The laboratory works to transfer its science and technologies to industry through various collaborative mechanisms including licensing agreements, joint development, and proprietary work arrangements.
Pacific Northwest National Labs - Electric Vehicle ProjectsForth
Pacific Northwest National Laboratory (PNNL) conducts electric vehicle (EV) research projects. PNNL manages the Electric Infrastructure Operations Center and has studied how EVs can help integrate renewable energy by charging from the electric grid. Studies show the existing grid has capacity to charge a high percentage of EVs each night without overloading transformers. PNNL has developed smart charging technologies that can provide grid services by modulating charging based on grid needs. These technologies help lower emissions and electricity rates by filling valleys in demand. PNNL is also testing electric buses and vehicle-to-grid systems that allow EVs to discharge power back to the grid.
Exascale Computing Project - Driving a HUGE Change in a Changing Worldinside-BigData.com
In this video from the OpenFabrics Workshop in Austin, Al Geist from ORNL presents: Exascale Computing Project - Driving a HUGE Change in a Changing World.
"In this keynote, Mr. Geist will discuss the need for future Department of Energy supercomputers to solve emerging data science and machine learning problems in addition to running traditional modeling and simulation applications. In August 2016, the Exascale Computing Project (ECP) was approved to support a huge lift in the trajectory of U.S. High Performance Computing (HPC). The ECP goals are intended to enable the delivery of capable exascale computers in 2022 and one early exascale system in 2021, which will foster a rich exascale ecosystem and work toward ensuring continued U.S. leadership in HPC. He will also share how the ECP plans to achieve these goals and the potential positive impacts for OFA."
Learn more: https://exascaleproject.org/
and
https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: https://www.openfabrics.org/index.php/abstracts-agenda.html
High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biom...Larry Smarr
11.04.06
Joint Presentation
UCSD School of Medicine Research Council
Larry Smarr, Calit2 & Phil Papadopoulos, SDSC/Calit2
Title: High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biomedical Sciences
High Performance Cyberinfrastructure Enabling Data-Driven Science Supporting ...Larry Smarr
11.05.13
Invited Presentation
Sanford Consortium for Regenerative Medicine
Salk Institute, La Jolla
Larry Smarr, Calit2 & Phil Papadopoulos, SDSC/Calit2
Title: High Performance Cyberinfrastructure Enabling Data-Driven Science Supporting Stem Cell Research
In this deck from the 2017 MVAPICH User Group, Adam Moody from Lawrence Livermore National Laboratory presents: MVAPICH: How a Bunch of Buckeyes Crack Tough Nuts.
"High-performance computing is being applied to solve the world's most daunting problems, including researching climate change, studying fusion physics, and curing cancer. MPI is a key component in this work, and as such, the MVAPICH team plays a critical role in these efforts. In this talk, I will discuss recent science that MVAPICH has enabled and describe future research that is planned. I will detail how the MVAPICH team has responded to address past problems and list the requirements that future work will demand."
Watch the video: https://wp.me/p3RLHQ-hp6
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
Geofizyka Krakow Selects Panasas for Simplicity and PerformancePanasas
Geofizyka Krakow’s customers rely on the critical data they receive from the company to be competitive in a very demanding industry. The company needed an IT storage solution that could manage the its demanding computing work workload while retaining management simplicity. With Panasas Geofizyka Krakow is seeing up to 6X faster completion of
seismic processing jobs, improved image results, and more.
The document describes the Green Computing Observatory (GCO), which monitors energy usage at a large computing center. The GCO collects data from sensors that measure energy usage of individual components. It publishes the data through XML files to analyze energy efficiency. The GCO aims to address barriers to improved energy efficiency by collecting detailed usage data and defining semantics for the data through an ontology. It currently collects data from sensors on servers, power distribution units, and the Ganglia monitoring system. Future work includes developing a consistent XML schema and calculating total energy consumption.
f people think about big data, they always think about Twitter or Facebook. But there are other areas where much more data amounts incurred and the analyzes are more complex. In this talk, we talk about a real example from bioinformatics. We will explain the actual scenario and how the various Microsoft platforms from SQL Server to Azure Analytics and HDInsight could help us – or not.
TPCx-HS is the first vendor-neutral benchmark focused on big data systems – which have become a critical part of the enterprise IT ecosystem.
Watch the video presentation: http://wp.me/p3RLHQ-cLY
Learn more: http://www.tpc.org/tpcx-hs
SCFE 2020 OpenCAPI presentation as part of OpenPWOER TutorialGanesan Narayanasamy
This document introduces hardware acceleration using FPGAs with OpenCAPI. It discusses how classic FPGA acceleration has issues like slow CPU-managed memory access and lack of data coherency. OpenCAPI allows FPGAs to directly access host memory, providing faster memory access and data coherency. It also introduces the OC-Accel framework that allows programming FPGAs using C/C++ instead of HDL languages, addressing issues like long development times. Example applications demonstrated significant performance improvements using this approach over CPU-only or classic FPGA acceleration methods.
A Semantic-Based Approach to Attain Reproducibility of Computational Environm...Idafen Santana Pérez
Slides from our presentation at the 1st International Workshop on Reproducibility in Parallel Computing (REPPAR'14) in conjunction with Euro-Par 2014 (August 25-29)
Genomics at the Speed of Light: Understanding the Living OceanLarry Smarr
06.07.17-19
Invited Talk
The Gordon and Betty Moore Foundation 2nd Annual Marine Microbiology Investigator Symposium The Golden Gate Club, The Presidio of San Francisco
Title: Genomics at the Speed of Light: Understanding the Living Ocean
San Francisco, CA
CINECA for HCP and e-infrastructures infrastructuresCineca
Sanzio Bassini. Head of the HPC Department of Cineca. Cineca is the technological partner of the Ministry of Education, and takes part in the Italian commitment for the development of e-infrastrcuture in Italy and in Europe for HCP and HCP technologies; scientific data repository and management, cloud computing for industries and Public administration, for the development of computing intensive and data intensive methods for science and engineering
Cineca offers a unique offer for: open access of integrated tier0 and tier1 HCP national infrastructure; of education and training activities under the umbrella of PRACE Training
advanced center action; integrated help desk and scale up process for HCP users support
Preparing to program Aurora at Exascale - Early experiences and future direct...inside-BigData.com
In this deck from IWOCL / SYCLcon 2020, Hal Finkel from Argonne National Laboratory presents: Preparing to program Aurora at Exascale - Early experiences and future directions.
"Argonne National Laboratory’s Leadership Computing Facility will be home to Aurora, our first exascale supercomputer. Aurora promises to take scientific computing to a whole new level, and scientists and engineers from many different fields will take advantage of Aurora’s unprecedented computational capabilities to push the boundaries of human knowledge. In addition, Aurora’s support for advanced machine-learning and big-data computations will enable scientific workflows incorporating these techniques along with traditional HPC algorithms. Programming the state-of-the-art hardware in Aurora will be accomplished using state-of-the-art programming models. Some of these models, such as OpenMP, are long-established in the HPC ecosystem. Other models, such as Intel’s oneAPI, based on SYCL, are relatively-new models constructed with the benefit of significant experience. Many applications will not use these models directly, but rather, will use C++ abstraction libraries such as Kokkos or RAJA. Python will also be a common entry point to high-performance capabilities. As we look toward the future, features in the C++ standard itself will become increasingly relevant for accessing the extreme parallelism of exascale platforms.
This presentation will summarize the experiences of our team as we prepare for Aurora, exploring how to port applications to Aurora’s architecture and programming models, and distilling the challenges and best practices we’ve developed to date. oneAPI/SYCL and OpenMP are both critical models in these efforts, and while the ecosystem for Aurora has yet to mature, we’ve already had a great deal of success. Importantly, we are not passive recipients of programming models developed by others. Our team works not only with vendor-provided compilers and tools, but also develops improved open-source LLVM-based technologies that feed both open-source and vendor-provided capabilities. In addition, we actively participate in the standardization of OpenMP, SYCL, and C++. To conclude, I’ll share our thoughts on how these models can best develop in the future to support exascale-class systems."
Watch the video: https://wp.me/p3RLHQ-lPT
Learn more: https://www.iwocl.org/iwocl-2020/conference-program/
and
https://www.anl.gov/topic/aurora
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
This document discusses evaluating FPGA acceleration for real-time unstructured search. It motivates the need for high-performance and energy-efficient data analysis due to the explosion of data. It describes using FPGAs to accelerate unstructured search workloads like document filtering and profile matching. It details the workload algorithm, hardware platform with four FPGAs, synthetic datasets used, and experimental parameters. Performance results show the FPGA acceleration provides speedups of 23X-38X and energy efficiency improvements of 31X-40X compared to optimized multi-threaded CPU implementations.
TAU Performance System and the Extreme-scale Scientific Software Stack (E4S) aim to improve productivity for HPC and AI workloads. TAU provides a portable performance evaluation toolkit, while E4S delivers modular and interoperable software stacks. Together, they lower barriers to using software tools from the Exascale Computing Project and enable performance analysis of complex, multi-component applications.
Hadoop as a Platform for Genomics - Strata 2015, San JoseAllen Day, PhD
Personalized medicine holds much promise to improve the quality of human life.
However, personalizing medicine depends on genome analysis software that does not scale well. Given the potential impact on society, genomics takes first place among fields of science that can benefit from Hadoop.
A single human genome contains about 3 billion base pairs. This is less than 1 gigabyte of data but the intermediate data produced by a DNA sequencer, required to produce a sequenced human genome, is many hundreds of times larger. Beyond the huge storage requirement, deep genomic analysis across large populations of humans requires enormous computational capacity as well.
Interestingly enough, while genome scientists have adopted the concept of MapReduce for parallelizing I/O, they have not embraced the Hadoop ecosystem. For example, the popular Genome Analysis Toolkit (GATK) uses a proprietary MapReduce implementation that can scale vertically but not horizontally.
The document discusses plans to upgrade the CMS Event Builder for the long shutdown in order to accommodate higher data rates expected during the CMS upgrade. It describes testing advanced networking technologies like 10/40GbE and Infiniband to replace the existing Myrinet-based system. Preliminary measurements are being made to evaluate these technologies for reliably transmitting the larger event fragments at 100kHz in the upgraded event builder network. The CMS online software framework called XDAQ provides a flexible architecture to interface the new networking technologies through pluggable peer transport layers and remain largely unchanged despite the underlying network upgrade.
Technology Development and Commercialization at Argonne National Laboratoryasauers
Argonne National Laboratory is a multidisciplinary science and engineering research center operated by the University of Chicago for the U.S. Department of Energy. With an annual budget of $650 million and about 3,000 employees, including 1,000 scientists and engineers, Argonne conducts basic and applied research across various fields such as materials science, biology, computing, and energy technologies. Argonne operates several scientific user facilities and is a leader in high-performance computing. The laboratory works to transfer its science and technologies to industry through various collaborative mechanisms including licensing agreements, joint development, and proprietary work arrangements.
Pacific Northwest National Labs - Electric Vehicle ProjectsForth
Pacific Northwest National Laboratory (PNNL) conducts electric vehicle (EV) research projects. PNNL manages the Electric Infrastructure Operations Center and has studied how EVs can help integrate renewable energy by charging from the electric grid. Studies show the existing grid has capacity to charge a high percentage of EVs each night without overloading transformers. PNNL has developed smart charging technologies that can provide grid services by modulating charging based on grid needs. These technologies help lower emissions and electricity rates by filling valleys in demand. PNNL is also testing electric buses and vehicle-to-grid systems that allow EVs to discharge power back to the grid.
The document discusses pre-mobilization planning and safety requirements for a characterization project at ANL Bldg 310, including developing a contract-specific safety plan, job safety analyses, and ensuring required training. It also summarizes the scope of work, which includes alpha and beta surveys, sampling of various materials and structures within the building, and mitigations for associated hazards.
Idaho National Labs - Overview & Electric Vehicle ActivitiesForth
The Idaho National Laboratory is a government-owned, contractor-operated institution that has been involved in nuclear energy research and development since the 1940s. It is the U.S. Department of Energy's lead laboratory for nuclear energy and conducts important work in electric vehicles and infrastructure testing. The laboratory maintains extensive facilities on its 890 square mile site and has tested over 11,600 electric vehicles and infrastructure components, accumulating over 122 million test miles. It works with partners to study grid integration and provides independent analysis to support further electric vehicle adoption.
The document provides an overview of NASA Ames Research Center, including its history, missions, programs, facilities, and educational activities. It notes that NASA Ames conducts applied research and develops critical technologies to enable NASA missions. Some key areas of focus include space, Earth, and life sciences; astrobiology; small satellites; aviation and aeronautics; exploration systems; and educational outreach. It also summarizes several past and current NASA Ames missions.
This document discusses using R to scrape baseball box score data from the website baseball-reference.com. It describes breaking down the scraping process into steps: iterating through each MLB team from 2005-2010, each date from April to October, and accessing the box score URLs and tables. The document outlines the R code used to systematically scrape over 78,000 URLs and extract the data tables. It concludes that R provides a powerful platform for web scraping and accessing large amounts of publicly available data.
The 7th US/German Workshop on Salt Repository Research, Design, and Operation was held in September 2016. Over 50 participants from government agencies, universities, and companies in the US and Germany discussed progress on salt repository programs. Key topics included the safety case, operational safety at WIPP, geomechanics modeling and testing, plugging and sealing drifts, and percolation of fluids in salt. The workshop proceedings document these technical discussions and identify ongoing collaborations and an agenda for future research. Knowledge sharing on salt repository programs contributes to safe disposal of nuclear waste.
High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Scien...Larry Smarr
11.03.28
Remote Luncheon Presentation from Calit2@UCSD
National Science Board
Expert Panel Discussion on Data Policies
National Science Foundation
Title: High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Science and Engineering
Arlington, Virginia
The document discusses the National Research Platform (NRP), specifically the 4th NRP workshop. It provides an overview of NRP's Nautilus, a multi-institution hypercluster connected by optical networks across 25 partner campuses. In 2022, Nautilus comprised ~200 computing nodes and 4000TB of rotating storage. The document highlights several large research projects from different domains that utilize Nautilus, including particle physics, telescopes, biomedical applications, earth sciences, and visualization. These applications demonstrate how Nautilus enables data-intensive and collaborative multi-campus research at national scale.
How to Terminate the GLIF by Building a Campus Big Data Freeway SystemLarry Smarr
12.10.11
Keynote Lecture
12th Annual Global LambdaGrid Workshop
Title: How to Terminate the GLIF by Building a Campus Big Data Freeway System
Chicago, IL
The ALCF provides leading-edge high-performance computing resources and services to support scientific research. It operates two main systems, Intrepid and Eureka, that provide over 500 teraflops of computing power and petabytes of storage. The ALCF supports projects through the DOE INCITE and ALCC programs, which make large allocations of computing time available through a peer-review process. It also offers director's discretionary allocations. The ALCF's integrated services include performance engineering, visualization support, training, and project management to help researchers maximize the impact of their work.
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...confluent
The Oak Ridge Leadership Facility (OLCF) in the National Center for Computational Sciences (NCCS) division at Oak Ridge National Laboratory (ORNL) houses world-class high-performance computing (HPC) resources and has a history of operating top-ranked supercomputers on the TOP500 list, including the world's current fastest, Summit, an IBM AC922 machine with a peak of 200 petaFLOPS. With the exascale era rapidly approaching, the need for a robust and scalable big data platform for operations data is more important than ever. In the past when a new HPC resource was added to the facility, pipelines from data sources spanned multiple data sinks which oftentimes resulted in data silos, slow operational data onboarding, and non-scalable data pipelines for batch processing. Using Apache Kafka as the message bus of the division's new big data platform has allowed for easier decoupling of scalable data pipelines, faster data onboarding, and stream processing with the goal to continuously improve insight into the HPC resources and their supporting systems. This talk will focus on the NCCS division's transition to Apache Kafka over the past few years to enhance the OLCF's current capabilities and prepare for Frontier, OLCF's future exascale system; including the development and deployment of a full big data platform in a Kubernetes environment from both a technical and cultural shift perspective. This talk will also cover the mission of the OLCF, the operational data insights related to high-performance computing that the organization strives for, and several use-cases that exist in production today.
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...Databricks
In this session, you will learn how CERN easily applied end-to-end deep learning and analytics pipelines on Apache Spark at scale for High Energy Physics using BigDL and Analytics Zoo open source software running on Intel Xeon-based distributed clusters.
Technical details and development learnings will be shared using an example of topology classification to improve real-time event selection at the Large Hadron Collider experiments. The classifier has demonstrated very good performance figures for efficiency, while also reducing the false positive rate compared to the existing methods. It could be used as a filter to improve the online event selection infrastructure of the LHC experiments, where one could benefit from a more flexible and inclusive selection strategy while reducing the amount of downstream resources wasted in processing false positives.
This is part of CERN’s research on applying Deep Learning and Analytics using open source and industry standard technologies as an alternative to the existing customized rule based methods. We show how we could quickly build and implement distributed deep learning solutions and data pipelines at scale on Apache Spark using Analytics Zoo and BigDL, which are open source frameworks unifying Analytics and AI on Spark with easy to use APIs and development interfaces seamlessly integrated with Big Data Platforms.
The document summarizes IBM's Blue Gene/L supercomputer. It discusses that Blue Gene/L was created in 1999 through a partnership between IBM and Lawrence Livermore National Laboratory to build a scalable supercomputer optimized for bandwidth. It had 65,536 dual-processor nodes, 512 MB of memory per node, and used a 3D torus network topology to interconnect the nodes. Its main applications included protein folding, modeling the human brain, and other scientific problems.
The document discusses EDF's use of a data lake and data lab to optimize operations and safety at their nuclear power plants. It describes how EDF is building a Hadoop-based data lake called ESPADON to store sensor and operational data from their 59 nuclear plants. They are developing data science algorithms to analyze the data from the whole fleet to improve maintenance and operations. EDF is also creating a data lab team and architecture to develop analytics and quantify the value of these initiatives.
The document summarizes plans for two new Calit2 buildings at UC San Diego and UC Irvine that will provide laboratories for research in areas like nanotechnology, biomedical engineering, computer chips, and more. The buildings will be linked via high-speed optical networks and will support over 1000 researchers. Key aspects include ultra high-speed networking capabilities up to 10 gigabits per second, advanced visualization resources, and proposals to extend this infrastructure to enable new collaborative research projects.
Overview of challenges being faced by the AI community to achieve high-performance, scalable and distributed DNN training on Modern HPC systems with both scale-up and scale-out strategies. After that, the talk will focus on a range of solutions being carried out in my group to address these challenges. The solutions will include: 1) MPI-driven Deep Learning, 2) Co-designing Deep Learning Stacks with High-Performance MPI, 3) Out-of-core DNN training, and 4) Hybrid (Data and Model) parallelism. Case studies to accelerate DNN training with popular frameworks like TensorFlow, PyTorch, MXNet and Caffe on modern HPC systems will be presented.
A High-Performance Campus-Scale Cyberinfrastructure: The Technical, Political...Larry Smarr
The document discusses UCSD's plans to build a high-performance campus-scale cyberinfrastructure to support data-intensive research. It outlines investments in fiber networks and large-scale storage resources to connect researchers and instruments campus-wide. A key part of the plan is the Triton resource, a shared cluster and storage system hosted at SDSC to enable large-scale data analysis across campus.
06.07.26
Invited Talk
Cyberinfrastructure for Humanities, Arts, and Social Sciences, A Summer Institute, SDSC
Title: The OptIPuter and Its Applications
La Jolla, CA
Positioning University of California Information Technology for the Future: S...Larry Smarr
05.02.15
Invited Talk
The Vice Chancellor of Research and Chief Information Officer Summit
“Information Technology Enabling Research at the University of California”
Title: Positioning University of California Information Technology for the Future: State, National, and International IT Infrastructure Trends and Directions
Oakland, CA
Why Researchers are Using Advanced NetworksLarry Smarr
07.07.03
Remote Talk from Calit2 to:
Building KAREN Communities for Collaboration Forum
KIWI Advanced Research and Education Network
University of Auckland, Auckland City, New Zealand
Title: Why Researchers are Using Advanced Networks
La Jolla, CA
The document summarizes Dr. Larry Smarr's presentation on the Pacific Research Platform (PRP) and its role in working toward a national research platform. It describes how PRP has connected research teams and devices across multiple UC campuses for over 15 years. It also details PRP's innovations like Flash I/O Network Appliances (FIONAs) and use of Kubernetes to manage distributed resources. Finally, it outlines opportunities to further integrate PRP with the Open Science Grid and expand the platform internationally through partnerships.
Archiving data from Durham to RAL using the File Transfer Service (FTS)Jisc
From Jisc's campus network engineering for data-intensive science workshop on 19 October 2016.
https://www.jisc.ac.uk/events/campus-network-engineering-for-data-intensive-science-workshop-19-oct-2016
The document discusses the emerging LambdaCloud optical network infrastructure that enables collaborative data-intensive research across institutions. Key points:
- The LambdaCloud provides 10Gbps lightpaths connecting over 50 OptiPortals worldwide, allowing shared access to high-performance computing resources, data repositories, and research instruments.
- Projects include the OptIPlaner for global collaborations, the GreenLight project to measure energy costs of cloud computing for specific research communities, and plans for the Triton supercomputer at UCSD to connect biomedical researchers and instruments.
- Early testbeds show the ability to sort over a billion records across multiple sites sustaining over 5Gbps of network throughput.
Similar to Advances at the Argonne Leadership Computing Center (20)
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Essentials of Automations: Exploring Attributes & Automation ParametersSafe Software
Building automations in FME Flow can save time, money, and help businesses scale by eliminating data silos and providing data to stakeholders in real-time. One essential component to orchestrating complex automations is the use of attributes & automation parameters (both formerly known as “keys”). In fact, it’s unlikely you’ll ever build an Automation without using these components, but what exactly are they?
Attributes & automation parameters enable the automation author to pass data values from one automation component to the next. During this webinar, our FME Flow Specialists will cover leveraging the three types of these output attributes & parameters in FME Flow: Event, Custom, and Automation. As a bonus, they’ll also be making use of the Split-Merge Block functionality.
You’ll leave this webinar with a better understanding of how to maximize the potential of automations by making use of attributes & automation parameters, with the ultimate goal of setting your enterprise integration workflows up on autopilot.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Discover top-tier mobile app development services, offering innovative solutions for iOS and Android. Enhance your business with custom, user-friendly mobile applications.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/how-axelera-ai-uses-digital-compute-in-memory-to-deliver-fast-and-energy-efficient-computer-vision-a-presentation-from-axelera-ai/
Bram Verhoef, Head of Machine Learning at Axelera AI, presents the “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision” tutorial at the May 2024 Embedded Vision Summit.
As artificial intelligence inference transitions from cloud environments to edge locations, computer vision applications achieve heightened responsiveness, reliability and privacy. This migration, however, introduces the challenge of operating within the stringent confines of resource constraints typical at the edge, including small form factors, low energy budgets and diminished memory and computational capacities. Axelera AI addresses these challenges through an innovative approach of performing digital computations within memory itself. This technique facilitates the realization of high-performance, energy-efficient and cost-effective computer vision capabilities at the thin and thick edge, extending the frontier of what is achievable with current technologies.
In this presentation, Verhoef unveils his company’s pioneering chip technology and demonstrates its capacity to deliver exceptional frames-per-second performance across a range of standard computer vision networks typical of applications in security, surveillance and the industrial sector. This shows that advanced computer vision can be accessible and efficient, even at the very edge of our technological ecosystem.
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor IvaniukFwdays
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
JavaLand 2024: Application Development Green Masterplan
Advances at the Argonne Leadership Computing Center
1. Recent Advances at the Argonne
Leadership Computing Facility
David Martin, dem@alcf.anl.gov
with thanks to Paul Messina
2. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Argonne Leadership Computing Facility
ALCF was established in 2006 at Argonne to provide the
computational science community with a leading-edge computing
capability dedicated to breakthrough science and engineering
One of two DOE national Leadership Computing Facilities (the other
is the National Center for Computational Sciences at Oak Ridge
National Laboratory)
Supports the primary mission of DOE’s Office of Science Advanced
Scientific Computing Research (ASCR) program to discover,
develop, and deploy the computational and networking tools that
enable researchers in the scientific disciplines to analyze, model,
simulate, and predict complex phenomena important to DOE.
Intrepid Allocated: 60% INCITE, 30% ALCC, 10% Discretionary
2
3. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010 3
Argonne Leadership Computing Facility
Intrepid - ALCF Blue Gene/P System:
– 40,960 nodes / 163,840 PPC cores
– 80 Terabytes of memory
– Peak flop rate: 557 Teraflops
– Linpack flop rate: 450.3
– #13 on the Top500 list
Eureka - ALCF Visualization System:
– 100 nodes / 800 2.0 GHz Xeon cores
– 3.2 Terabytes of memory
– 200 NVIDIA FX5600 GPUs
– Peak flop rate: 100 Teraflops
Storage:
– 6+ Petabytes of disk storage with an I/O rate of 80 GB/s
– 5+ Petabytes of archival storage (10,000 volume tape
archive)
5. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010 5
• Startup assistance
• User administration assistance
• Job management services
• Technical support (Standard and
Emergency)
ALCF
Services
• ALCF science liaison
• Assistance with proposals, planning,
reporting
• Collaboration within science domains
• Performance engineering
• Application tuning
• Data analytics
• Data management services
• Workshops & seminars
• Customized training programs
• On-line content & user guides
• Educational and industry outreach
programs
ALCF Service Offerings
6. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Programming Models and Development Environment
Languages:
– Full language support with IBM XL and GNU compilers
– Languages: Fortran, C, C++, Python
MPI:
– Based on MPICH2 1.0.x base code:
• MPI-IO supported
• One-sided communication supported
• No process management (MPI_Spawn(), MPI_Connect(), etc)
– Utilizes the 3 different BG/P networks for different MPI functions
Threads:
– OpenMP 2.5
– NPTL Pthreads
Linux development environment:
– Compute Node Kernel provides look and feel of a Linux environment
• POSIX routines (with some restrictions: no fork() or system())
• BG/P adds pthread support, additional socket support
– Supports statically and dynamically linked libraries (static is default)
– Cross compile since login nodes and compute nodes have different processor & OS
7. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Supported Libraries and Programs
Program Location Description
TotalView /soft/apps/totalview-8.5.0-0 Multithreaded, multiprocess source code debugger for high
performance computing.
Coreprocessor /soft/apps/coreprocessor.pl A tool to debug and provide postmortem analysis of dead applications.
TAU-2.17 /soft/apps/tau A portable profiling and tracing toolkit for performance analysis of
parallel programs written in Fortran, C++, and C
HPCT /soft/apps/hpct_bgp MPI profiling and tracing library, which collects profiling and tracing
data for MPI programs.
7
Program Location Description
armci /bgsys/drivers/ppcfloor/comm The Aggregate Remote Memory Copy (ARMCI) library
HDF5 /soft/apps/hdf5-1.6.6 The Hierarchical Data Format (HDF) is a model for managing and
storing data.
NetCDF /soft/apps/netcdf-3.6.2 A set of software libraries and machine-independent data formats
that supports the creation, access, and sharing of array-oriented
scientific data.
Parallel NetCDF /soft/apps/parallel-netcdf-
1.0.2
A library providing high-performance I/O while still maintaining file-
format compatibility with Unidata's NetCDF.
mercurial-0.9.5 /soft/apps/mercurial-0.9.5 A distributed version-control system
Scons /soft/apps/scons-0.97 A cross-platform substitute for the classic Make utility
tcl-8.4.14 /soft/apps/tcl-8.4.14 A dynamic programming language,
8. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010 8
DOE INCITE Program
Innovative and Novel Computational Impact on Theory and Experiment
Solicits large computationally intensive research projects
– To enable high-impact scientific advances
– Call for proposal opened once per year (call closed 6/30/2010)
– INCITE Program web site: www.er.doe.gov/ascr/incite
Open to all scientific researchers and organizations
– Scientific Discipline Peer Review
– Computational Readiness Review
Provides large computer time & data storage allocations
– To a small number of projects for 1-3 years
– Academic, Federal Lab and Industry, with DOE or other support
Primary vehicle for selecting principal science projects for the
Leadership Computing Facilities (60% of time at Leadership Facilities)
In 2010, 35 INCITE projects allocated more than 600M CPU hours at the
ALCF
9. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
DOE ALCC Program
ASCR Leadership Computing Challenge
Allocations for projects of special interest to DOE with an emphasis
on high risk, high payoff simulations in areas of interest to the
department’s energy mission (30% of the core hours at Leadership
Facilities)
Awards granted in June, 2010
– http://www.er.doe.gov/ascr/facilities/alcc.htm
10 awards at ALCF in 2010 for 300+ million core hours
9
10. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Discretionary Allocations
Time is available for projects without INCITE or ALCC allocations!
ALCF Discretionary allocations provide time for:
– Porting, scaling, and tuning applications
– Benchmarking codes and preparing INCITE proposals
– Preliminary science runs prior to an INCITE award
– Early Science Program
To apply go to the ALCF allocations page
– www.alcf.anl.gov/support/gettingstarted
10
11. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010 11
2009 INCITE Allocations at ALCF
35projects
600+ M CPU Hours
12. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
ALCF Projects Span Many Domains
12
Life Sciences
U CA-San Diego
Applied Math
Argonne Nat’l Lab
Physical Chemistry
U CA-Davis
Nanoscience
Northwestern U
Engineering Physics
Pratt & Whitney
Biology
U Washington
13. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Science Projects Using ALCF
Resources
Heat/fluid dynamics for advanced burner reactor design
Reactive MD Simulation of Nickel Fracture
Simulation of Two Phase Flow Combustion in Gas Turbines
Next-generation Community Climate System Model
Computational Protein Structure Prediction and Design
Gating Mechanism of Membrane Proteins
Validation of Type Ia supernova models
Probing the Non-Scalable Nano Regime in Catalytic Nanoparticles
Lattice Quantum Chromodynamics
13
14. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Computational Nuclear Engineering
Code: Nek5000
Improve the Safety, Efficiency and
Cost of Liquid-Metal-Cooled Fast
Reactors through computation
High resolution 3D simulation of fluid
flow and heat transfer in full reactor
core sub-assembly
Simulation requires full pin
assembly, cannot use symmetries
– 217 Pin configuration
– 2.95 million spectral elements
– 1 billion grid points
14
Paul Fischer, Argonne National Lab
15. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Sulfur Segregation-Induced Embrittlement of Nickel
Important for next-generation nuclear reactors
• Experiments at Argonne discovered a relation between S segregation-
induced embrittlement & amorphization of Ni
• 48 million-atom reactive molecular-dynamic simulation (the largest
ever) on 65,536 IBM BlueGene/P processors (50 million cpu-hours)
at Argonne reveals an atomistic mechanism that links amorphization
& embrittlement
USC-Purdue-CSUN-Harvard-LANL-LLNL OASCR-BES-
NNSA SciDAC on Stress Corrosion Cracking
16. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Sulfur Segregation-Induced Embrittlement of Nickel
Brittle cleavage with
S segregation
Ductile tearing in
nanocrystalline Ni
17. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Large Eddy Simulation of Two Phase Flow
Combustion in Gas Turbines
Code: AVBP
Improve airplane and helicopter
engine designs – reduce design cost,
improve efficiency
Study the two phase flows and
combustion in gas turbine engines
Results:
– First and only simulations of a full
annular chamber
– Identified instability mechanisms that
reduce efficiency in gas turbines
17
Thierry Poinsot, CERFACS
18. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Climate Simulations
18
Warren Washington, NCAR
Code: CCSM (climate simulation code used by the DOE and NSF
climate change experiments)
Ultra-high resolution atmosphere simulations:
– CAM-HOMME atmospheric component
– 1/8th degree (12.km avg grid) coupled with land model at 1/4th degree
and ocean/ice
– Testing up to 56K cores, 0.5 simulated years per day with full I/O
– ½ degree finite-volume CAM with tropospheric chemistry and 399
tracer
19. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Computational Protein Structure
Prediction and Protein Design
Code: Rosetta
Rapidly determine an accurate,
high-resolution structure of any
protein sequence up to 150-
200 residues
Incorporating sparse
experimental NMR data into
Rosetta to allow larger proteins
Comparison of computational and experimentally
derived protein structure
19
David Baker, University of Washington
20. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Gating Mechanism of Membrane Proteins
Code: NAMD with periodicity and
particle-mesh Ewald method
Understand how proteins work so
we can alter them to change their
function
Validated the atomic models of
Kv1.2 and first to calculate the
gating charge in the two functional
states
NAMD scaled out to 4K nodes, twice
as far as before
20
Benoit Roux, ANL, University of Chicago
21. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
FLASH
Code: FLASH
Investigating Type 1a supernovae
which are used as a “standard
candle” in astronomy
Answered critical question on
critical process in Type Ia
supernovae
– First simulated buoyancy-driven
turbulent nuclear combustion in the
fully-developed turbulent regime while
also simultaneously resolving the
Gibson scale
– Reveals complex structure
– Requires higher resolution studies
22
Don Lamb, University of Chicago
22. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
Probing the Non-Scalable Nano Regime in
Catalytic Nanoparticles
Code: GPAW
Gold has exciting catalytic capabilities that are
still poorly understood
– e.g. CO into CO2
First principle calculations of
the actual nano-particles are
needed
23
Jeff Greeley, Argonne National Laboratory
23. Argonne National
Laboratory Recent Advances at ALCF – Blue Gene Consortium Meeting -- 16 Nov 2010
The Next Generation ALCF System: BG/Q
DOE has approved our acquisition in 2012 of “Mira” a 10 Petaflops
Blue Gene/Q system. An evolution of the Blue Gene architecture
with:
– Over 750k cores
– 16 cores/node
– 1 GB of memory per core, nearly a TB of memory in aggregate
– 70 PB of disk with 470 GB/s of I/O bandwidth
– Power efficient, water cooled
Argonne and Livermore worked closely with IBM over the last few
years to help develop the specifications for this next generation Blue
Gene system
16 Projects Accepted into the Early Science Program
Applications running on the BG/P should run immediately on the
BG/Q, but may see better performance by exposing greater levels of
parallelism at the node level
24
Editor's Notes
Table: Check-marks means those simulations are done. 217 pin case : Equivalent tpLES of channel flow in achannel of length Lz ~ 90000h, where h is the channel half-height, save that this geometry is complexTwo INCITE projectsReactor Core HydrodynamicsPaul FischerPredictions of Thermal Striping in Sodium Cooled ReactorsAndrew Siegel
All-atom MDThe first to calculate the gating charge that is transferred across the membrane upon transition of the protein between the the two states: open and closedALCF work extended the scalability of NAMD out to 4k nodes, twice as far as beforecell membrane represents the physical and functional boundary between living organisms and their environment
Residual dipolar coupling data from NMR Residual dipolar coupling (RDC) data provide long-range orientational restraints between parts, particularly secondary structure elements, of a protein structure. While RDC alone is insufficient for experimentally solving a structure, incorporating them in Rosetta narrows down search space by eliminating all protein conformations incompatible with the RDC data. NMR NOESY experiments give short-range interatomic distances. However, to extract the interatomic distances, all the peaks in the NOESY spectrum require to be assigned. Assigning the peaks is a very time consuming process and can take several weeks or months for larger proteins. In collaboration with the Northeast Structural Genomics Center, we have developed methods that will allow rapid screening of Rosetta models against unassigned NOESY data.Figure 3. Schematic of the hot-spot grafting methodology. The Fc domain is colored green in all frames. A. Interactions of the Fc domain of human IgG1 with the Fc-binding domain of protein A (1L6X). Magenta colored residues represent key interactions within this interface. Dotted lines represent hydrogen bonds B. Computed hot-spot set of phenylalanine (Phe) that has been diversified by small docking perturbations. Original Phe from Protein A is colored in magenta C. Rotamers of glutamine (Gln) that still allow beneficial interactions with the Fc fragment. D. de novo designed scaffold with unrelated backbone structure that has the key residues Phe and Gln incorporated. The in silico engineered scaffold is originally from a DNA-binding protein (1NH9).Technique will be used to target: In particular, we will target the L1R protein from poxviruses (including smallpox and monkey-pox) and the hemagglutinin protein from H1N1 influenza viruses, common to the Spanish and swine flu.
All-atom MDThe first to calculate the gating charge that is transferred across the membrane upon transition of the protein between the the two states: open and closedALCF work extended the scalability of NAMD out to 4k nodes, twice as far as beforecell membrane represents the physical and functional boundary between living organisms and their environment