Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SUSE - Keeping Pace with Growing High Performance Computing Needs


Published on

SUSE offers a highly scalable, high performance open source computing system that enables users to harness the power of the supercomputer for day-to-day business. SUSE accelerates innovation with new solutions developed and delivered with a strong ecosystem of partners in the OpenHPC community. We will discuss the latest exciting initiatives within this ecosystem which improve scalability and performance, realize faster time to value and answer the needs of future HPC system.

Published in: Software
  • Be the first to comment

SUSE - Keeping Pace with Growing High Performance Computing Needs

  1. 1. Keeping Pace with Growing High Performance Computing Needs SUSE Linux Enterprise Server for High Performance Computing Jeff Reser
  2. 2. Simplifying lives with a highly scalable, high performance open source operating system that harnesses the power of the supercomputer for your day-to-day business Solving computational, data-intensive, or numerically-intensive tasks Reducing the time and effort required to set up and maintain HPC clusters Ensuring that all components of the HPC stack work together © Copyright 2016 SUSE LLC. All rights reserved. 2 SUSE Linux Enterprise Server for High Performance Computing 12 SP2
  3. 3. What’s New in SLES for HPC 3 • Realize faster time to value with HPC Module packaging that includes leading-edge HPC components and enables easier selling, simplified maintenance and more focus – with affordable pricing • Accelerate innovation with added support for the Ceph File System support, enabling you to transform your enterprise storage infrastructure to reduce costs while providing unlimited scalability • Now commercially available as part of Intel HPC Orchestrator – Intel used SLES plus a number of OpenHPC community components • Microsoft Azure Test Drive program includes SLES for HPC for a free trial • ARM64 partnering with OpenHPC, Cavium, ARM, Applied Micro • Increase efficiency with Intel Omni-Path Architecture (OPA) support for performance and throughput
  4. 4. 4 Challenges
  5. 5. Top HPC Challenges and Needs Control Infrastructure Optimize Operations Increase Performance System performance needs to be increased amid IT budgets and staff reductions HPC is business-critical, but managed by a separate department with a scientific flavor Need support and interoperability across the entire HPC stack, without dependencies on a single hardware vendor 5 © Copyright 2016 SUSE LLC. All rights reserved.
  6. 6. Top HPC Market Challenges HPC market still developing Stack components provided by various vendors Applications need to be updated to leverage HPC Mix of small and big vendors Segmented into commercial and scientific 6 © Copyright 2016 SUSE LLC. All rights reserved.
  7. 7. 7 Market Evolution
  8. 8. Linux on 99.4% of the Top 500 Supercomputers Linux Unix Mixed Windows BSD Mac 70% 60% 50% 40% 30% 20% 10% 0% 80% 90% 100% 1994 TODAY 8 Linux in HPC Half of the world’s 20 largest supercomputers run on SUSE 50% © Copyright 2016 SUSE LLC. All rights reserved. 13.4% SUSE OS Share, including Cray Linux Environment
  9. 9. Growing HPC Adoption 9 Universities Government Academia & Research Institutions Financial Services: Risk Analysis Oil & Gas Exploration, Production Semiconductor: Electronics Design Automation Life Sciences: Pharmaceutical & Biomedical Manufacturing: Aerospace & Automotive Departmental and workgroup compute clusters “High Productivity Computing” YESTERDAY TODAY TOMORROW © Copyright 2016 SUSE LLC. All rights reserved.
  10. 10. Worldwide HPC Server Market (according to IDC) 10 CAGR 2015-2019: • 10.1% Supercomputer (>$500K) • 0.7% Divisional ($250K-$500K) • 9.5% Departmental ($100K-$250K) • 8.7% Workgroup (<$100K) • HPC still expected to be a strong growth market • Growing recognition of HPC’s strategic value is helping to drive high-end sales • Low-end buyers are back into a growth mode • Recognition of HPC’s strategic and economic value will drive the exascale race • The formative HPDA (High Performance Data Analysis) market will expand opportunities for vendors (plus machine learning, cognitive and IoT)
  11. 11. 11 Community
  12. 12. OpenHPC Open Source Community 12 • Linux Foundation project – SUSE is a founding member (now 30+ full members) • Provides common platform – standard HPC stack – for collaboration and innovation • Provides full HPC stack (~300 packages) on top of SLES • Build with SUSE build service: • Simplifies installation, configuration, and maintenance of a custom software stack © Copyright 2016 SUSE LLC. All rights reserved.
  13. 13. OpenHPC Ecosystem OEMs Users ISV-OSV 13 © Copyright 2016 SUSE LLC. All rights reserved.
  14. 14. Partnering with Intel 14 Intel distributes SUSE Linux Enterprise Server for High Performance Computing as part of the Intel HPC Orchestrator SUSE is the commercial Linux supported by the Intel HPC stack • Delivers a fully supported stack for HPC workloads • Accelerates research/innovation and time to insight • Aligns new optimized components driven by the broader • Simplifies ongoing maintenance and support 14 © Copyright 2016 SUSE LLC. All rights reserved.
  15. 15. SUSE Joins Microsoft Enterprise Cloud Alliance and Test Drive Program 15 SUSE is the only supported enterprise Linux HPC workload on Microsoft Azure public cloud • Cloud-based SLES, including automated updates, to enterprise customers using the Microsoft Azure cloud platform • SLES for HPC is part of the Microsoft Azure Test Drive program, freely available via Azure Marketplace 15 © Copyright 2016 SUSE LLC. All rights reserved.
  16. 16. 16 HPC in Practice
  17. 17. 17 MULTI-CORE PROCESSOR SUPPORT TECHNOLOGY HIGH PRODUCTIVITY COMPUTING ACADEMIC AND RESEARCH COOPERATION SUSE Highlights of High Performance Computing 17 © Copyright 2016 SUSE LLC. All rights reserved. Intel, AMD, POWER, ARM Kernel, Lustre, Ceph OpenHPC, Intel, IBM, SGI, HPE, Dell, Bull, NEC, Cray, Cisco, Fujitsu Total Oil & Gas LRZ/SuperMUC, BSC/MareNostrum, Tokyo Institute of Technology/TSUBAME, Beijing Computing Center, NASA/Pleiades
  18. 18. HPC Impacts Our Lives Through Space Research © Copyright 2016 SUSE LLC. All rights reserved. Understanding how solar activity affects us on Earth • Pleiades supercomputer simulations help explain NASA’s IRIS (Interface Region Imaging Spectrograph) solar observatory findings • Combined with high-performance computer modeling, simulation, and analysis back on Earth, scientists hope to increase their understanding of solar dynamics • Solar simulations through numerical modeling on Pleiades, located at the NASA Advanced Supercomputing (NAS) facility at Ames Research Center (runs SUSE Linux Enterprise) 18
  19. 19. HPC Impacts Our Lives With Better Preparation for Seismic Events © Copyright 2016 SUSE LLC. All rights reserved. Understanding the effects of seismic activity • Geophysicists use earthquake simulation software to investigate seismic waves beneath the Earth’s surface • Goal is to simulate earthquakes as accurately as possible to be better prepared for future events • Calculations involved in this kind of simulation are so complex that they push even supercomputers to their limits • Using SuperMUC, they simulated the vibrations inside the geometrically complex Merapi volcano on the island of Java 19
  20. 20. HPC Impacts Our Lives Through Weather & Climate Forecasting © Copyright 2016 SUSE LLC. All rights reserved. From saving lives to just planning the weekend • National Center for Atmospheric Research (NCAR) has a new system – Cheyenne, running SUSE Linux – for advancing atmospheric and Earth science • Helps scientists improve predictions of a range of phenomena, from hour-by-hour risks associated with thunderstorm outbreaks to the timing of the 11-year solar cycle and its potential impacts • Cheyenne will be a critical tool for researchers studying climate change, severe weather, geomagnetic storms, seismic activity, air quality, wildfires, and other important geoscience topics 20
  21. 21. HPC Impacts Our Lives Through Oil and Gas Exploration © Copyright 2016 SUSE LLC. All rights reserved. From getting around with our cars to just staying at home in front of a gas fireplace • HPC accelerates oil and gas exploration with processing of seismic data and simulation of deposit fluids • As new oil and gas reserves become more elusive, companies like Total rely increasingly on HPC to find opportunities in an ocean of seismic data. • With SUSE Linux Enterprise Server, Total enjoys the optimal combination of performance, price and efficiency. 21
  22. 22. HPC Impacts Our Lives Through Media and Entertainment 22 From action-packed cinematic special effects to games • SUSE Linux Enterprise runs on supercomputers in creating special effects in movies and video games • All with demanding requirements for ultra- high computing for an amazing user and viewer experience • Linux is the OS of choice for Hollywood animation and special effects departments, used in production of more than 30 blockbuster films © Copyright 2016 SUSE LLC. All rights reserved.
  23. 23. HPC Impacts Our Lives Through Social Phenomena Analysis 23 Used for numerous and varied purposes, from medicine to earthquake warnings, to social phenomenon analysis • TSUBAME touted as the “supercomputer for everyone” • Even though its main work is research in academia, the door is also wide open for use by industry • Predicting traffic congestion or share prices, simulating human organs, or forecasting the weather “The excellent management that SUSE Linux Enterprise Server provides is one of the key factors behind Tsubame 2.5’s success.” – Professor Satoshi Matsuoka, Global Scientific Information and Computing Center, Tokyo Institute of Technology © Copyright 2016 SUSE LLC. All rights reserved.
  24. 24. HPC Impacts Our Lives Through Medical Research 24 PSC’s game-changing supercomputer is being used to advance medical research • Visualize data in a way that transforms understanding of public health issues • Calculate the electronic structure of a large organic semiconductor molecule with applications in solar power cells • Model the possible benefits of flu vaccine choice • Genetic sequencing of various species • Assemble 378 billion base pairs of bacterial DNA from the intestines of healthy patients and those with diabetes “SUSE Linux Enterprise Server is the only distribution that supports the full capabilities of the SGI machine. It was a no-brainer for this application. We use it. We recommend it. SUSE has a newer kernel than other options, making it the best choice.” – Jim Kasdorf, Director of Special Projects at PSC © Copyright 2016 SUSE LLC. All rights reserved.
  25. 25. Banking & Financial Services Rely on High Performance Computing 25 The ways in which research applications and financial services applications rely on Linux and HPC are in many cases quite similar • Risk modeling in determining aggregate risk in financial portfolios • Fraud detection in real time as millions of transactions are processed between disparate systems o Find suspicious patterns in related data sets o Create and deploy new fraud models into event flows quickly and with minimal effort • High-frequency trading applications execute trades milliseconds before competitors – which can mean millions in additional returns • Pricing and regulatory compliance applications • Big data applications like customer profiling and predictive analytics © Copyright 2016 SUSE LLC. All rights reserved. Beijing Rural Commercial Bank Banking & Financial Services firms running SUSE Linux Enterprise Server
  26. 26. Technical Update 26
  27. 27. The SUSE HPC Stack 27 Hardware SUSE Linux Enterprise Server © Copyright 2016 SUSE LLC. All rights reserved. Storage & File Systems Networking Message Passing Interface (MPI) EXT3 BTRFS XFS OCFS2 pNFS NFS Ceph FS pNFS IBRIX Lustre GPFS 10G OFED TCP Offload ParTec HPE SGI Intel MPICH openMPI SUSE supported SUSE partner 40G Queuing / Management Tools Applications PBS Pro IBM Spectrum LSF Moab Bright CM Grid Engine
  28. 28. SUSE HPC Features, 1 of 2 • Asynchronous I/O (AIO) Form of I/O processing that permits other processing to continue before the transmission has finished • Modular I/O Scheduler Algorithm most suitable for workload can be chosen dynamically • Multi-core/hyper-threading processor support Execute threads in parallel within each individual processor • Intel I/O Acceleration Offloads the CPU towards the network card, thus allowing the system to continue processing data while I/O is taking place Advanced I/O Processing • CFQ Complete Fair Queuing, suitable for a wide variety of applications, default I/O scheduler • Deadline I/O scheduler A valuable choice on a server with intelligent hardware, or if latency is the primary goal • Anticipatory I/O scheduler Suitable for file servers but does not work as well for database workloads • No-op Better suited for systems with storage solutions that have their own cache and their own virtualization Modular I/O Schedule Flexible/Pluggable I/O Scheduler 28 © Copyright 2016 SUSE LLC. All rights reserved.
  29. 29. SUSE HPC Features, 2 of 2 • Lustre-enabled kernel Parallel distributed file system for large-scale cluster computing & optimal performance • CPU and system activity management CPUset System, CPUset command line tool, Sysstat package, IRQbalance • Diagnose complex applications SystemTap for diagnosing apps, LTTng 2.0, an open source tracing framework to help gain insights into system behavior Performance and Management • OpenFabrics Enterprise Distribution (OFED) Remote Direct Memory Access (RDMA) switched fabric technologies – high-speed data transport technologies for server and storage connectivity High-speed Data Transport 29 • Computer memory design used in multiprocessors, where the memory access time depends on the memory location relative to a processor • NUMA provides excellent scalability and performance for Intel Itanium and AMD Opteron architectures • Auto-tuning available when apps aren’t NUMA-aware Non-Uniform Memory Architecture (NUMA) © Copyright 2016 SUSE LLC. All rights reserved.
  30. 30. SUSE Linux Enterprise Server for High Performance Computing Harness the power of supercomputing and solve your most demanding computational and data-intensive problems. • Accelerate innovation with a strong ecosystem of partners. • Optimize scalability and performance with Linux clustering. • Realize faster time to value through simplified configuration of HPC workloads. 50% Reduction in administration 20% Boost in productivity 99.999% Stable and reliable © Copyright 2016 SUSE LLC. All rights reserved. 30
  31. 31. 31 Outlook
  32. 32. Outlook • OpenHPC community contributions will continue to grow • Foster growing partner ecosystem • Wider HPC adoption across many industries • Increase in hybrid cloud deployments mixed with on-premise • More definition around HPC Module packaging © Copyright 2016 SUSE LLC. All rights reserved. 32
  33. 33. High Performance Computing Ecosystem Partnering OpenHPC community ARM Microsoft Intel’s enterprise HPC based on OpenHPC community & SLES Modularize HPC to focus on specific needs HPC partners leverage SLES SLES is the foundation for HPC © Copyright 2016 SUSE LLC. All rights reserved. 33
  34. 34. 34 Learn more at 34 Questions? © Copyright 2016 SUSE LLC. All rights reserved.
  35. 35. Resources SUSE Linux Enterprise Server for High Performance Computing Products and Services for HPC • • ormance_Computing Campaigns and Programs for HPC • computing HPC Success Stories • 35 SUSE Webinar • 223349 HPC Partners • • hpc-stack/ • us/marketplace/partners/suse/suselinuxent erpriseserver12optimizedforhighperforman cecompute/
  36. 36. 37 Unpublished Work of SUSE LLC. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE LLC. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third- party trademarks are the property of their respective owners.