Successfully reported this slideshow.
Your SlideShare is downloading. ×

HPC_Week_01.pdf

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
00 Motivational Lecture.pptx
00 Motivational Lecture.pptx
Loading in …3
×

Check these out next

1 of 28 Ad
Advertisement

More Related Content

Recently uploaded (20)

Advertisement

HPC_Week_01.pdf

  1. 1. HIGH PERFORMANCE COMPUTING 18 BS(CS) DEPARTMENT OF COMPUTER SCIENCE Engr. MIR MUHAMMAD mir.juno@quest.edu.pk
  2. 2. Course Objective  The goal of this course is to provide a survey of high performance computing.  The course will discuss topics related to accessing high performance computing resources, developing applications for those resources, and executing developed applications. Wednesday, September 15, 2021 2
  3. 3. Learning Aspects..  The students will be able to design, analysis, and implementation, of high performance computational science and engineering applications.  The course focuses on advanced computer architectures, parallel algorithms, parallel languages, and performance- oriented computing Wednesday, September 15, 2021 3
  4. 4. Books  TEXT BOOKS:  ”Introduction to Parallel Processing”, Latest Edition, By: M Sasikumar, Dinesh Shikhare, P.Ravi Prakash  “Parallel Programming in C with MPI and OpenMP” Latest Edition, By: Michael J. Quinn  “Introduction to Parallel Computing”, Latest Edition, By: Annanth Grama, Anshul Gupta, George karypis, Vipin Kumar Wednesday, September 15, 2021 4
  5. 5. Books  REFERENCE BOOKS:  “Grid Computing: A practical Guide to Technology and Applications”, Latest Edition, By: Ahmer Abbas Wednesday, September 15, 2021 5
  6. 6. Course Contents  INTRODUCTION TO HPC:  Importance of HPC in Science and Engineering, Introduction to Parallel Computers and Computational Overheads  ARCHITECTURES AND BUILDING BLOCKS:  Supercomputer Architecture, Vector Machines, Parallel Processors, Data Parallel Processors, Single-Instruction-Multiple- Data, Multiple-Instruction-Multiple-Data, Pipelining, Vectorization, Parallelization, Comparison of Serial, Parallel and Vector Architectures Wednesday, September 15, 2021 6
  7. 7. Course Contents  PERFORMANCE MEASURES AND MODELS:  Speed-up limitations, Theoretical Timings, Efficiencies, Amdahl's Law and Extensions, Scaled Speed-up, Pipeline Speed-up, Data Dependency Reduction. Data flow, Loop reordering, Loop optimizations, Implementation Principal of Locality, Caches and Buffers.  PARALLEL ALGORITHMS:  Purely Parallel Algorithms, Parallelization of Algorithms, Parallel linear algebra routines, Cyclic Reduction; Iterative Algorithms (Jacobi, Gauss-Seidel and Red-Black Orderings); Divide-and-Conquer Algorithms, Adaptive Quadrature, Correct Termination, Block Decomposition Methods, Massively Data Parallel Algorithms, Array notation. Wednesday, September 15, 2021 7
  8. 8. Course Contents  PARALLEL PROGRAMMING PACKAGES:  Parallel Languages and Compilers, Fortran90 and HPC Fortran, Parallel and Vector C Code, Massage passing interface (MPI), Parallel Virtual Machine (PVM), Layout, Align, Replicate, Masking, Shifting, Spreading, Broadcasting, Forall Loops, Levels of Abstraction, Models of Computation and Parallel Overheads, Levels of Abstraction, Multiple Program Counters in Hardware; Multi- Thread Models, with Primary Sources of Overhead;; Task-Parallel versus Data-Parallel Programming Models; Further Sources of Overhead; Loop Transformations; Data Transformations; Dependence Analysis; Compiler Strategies. Wednesday, September 15, 2021 8
  9. 9. Course Contents  RESULTS AND BENCHMARKING:  Memory Architecture and Memory Access Times and Associated Sources of Overhead; Multi-Process Execution Model; Performance Tuning via Overhead Reduction; Task Scheduling; Data Partitioning and its Effect on Performance.  Restructuring for Parallel Performance Wednesday, September 15, 2021 9
  10. 10. Let us begin … Wednesday, September 15, 2021 10
  11. 11. Introduction to High-Performance Computing  What is High Performance Computing?  High‐speed computing. Originally implemented only in supercomputers for scientific research  Used for scientific research and computational science  Main area of discipline is developing parallel processing algorithms and software so that programs can be divided into small independent parts and can be executed simultaneously by separate processors  HPC systems have shifted from supercomputer to computing clusters Wednesday, September 15, 2021 11
  12. 12. Introduction to High-Performance Computing Wednesday, September 15, 2021 12
  13. 13. What is Cluster Cluster is a group of machines interconnected in a way that they work together as a single system  Terminology  Node – individual machine in a cluster  Head/Master node – connected to both the private network of the cluster and a public network and are used to access a given cluster. Responsible for providing user an environment to work and distributing task among other nodes  Compute nodes – connected to only the private network of the cluster and are generally used for running jobs assigned to them by the head node(s) Wednesday, September 15, 2021 13
  14. 14. Types of Cluster  Storage  Storage clusters provide a consistent file system image  Allowing simultaneous read and write to a single shared file system  High‐availability (HA)  ƒ Provide continuous availability of services by eliminating single points of failure  Load‐balancing  ƒ Sends network service requests to multiple cluster nodes to balance the requested load among the cluster nodes  High‐performance  ƒ Use cluster nodes to perform concurrent calculations  Allows applications to work in parallel to enhance the performance of the applications  Also referred to as computational clusters or grid computing Wednesday, September 15, 2021 14
  15. 15. Benefits of Cluster  Reduced Cost  The price of off‐the‐shelf consumer desktops has fallen in recent years, and this drop in price has corresponded with a vast increase in their processing power and performance. The average desktop PC today is many times more powerful than the first mainframe computers.  Processing Power  The parallel processing power of a high‐performance cluster can, in many cases, prove more cost effective than a mainframe with similar power. This reduced price‐per‐unit of power enables enterprises to get a greater ROI(Return On Investment).  Scalability  Perhaps the greatest advantage of computer clusters is the scalability they offer. While mainframe computers have a fixed processing capacity, computer clusters can be easily expanded as requirements change by adding additional nodes to the network Wednesday, September 15, 2021 15
  16. 16. Benefits of Cluster  Improved Network Technology  In clusters, computers are typically connected via a single virtual local area network (VLAN), and the network treats each computer as a separate node. Information can be passed throughout these networks with very little lag, ensuring that data doesn’t bottleneck between nodes.  Availability  When a mainframe computer fails, the entire system fails. However, if a node in a computer cluster fails, its operations can be simply transferred to another node within the cluster, ensuring that there is no interruption in service Wednesday, September 15, 2021 16
  17. 17. When We Need High Performance Computing?  Case -1: Complete a time-consuming operation in less time  I am an automotive engineer  I need to design a new car that consumes less gasoline  I’d rather have the design completed in 6 months than in 2 years  I want to test my design using computer simulations rather than building very expensive prototypes and crashing them  Case 2: Complete an operation under a tight deadline  I work for a weather prediction agency  I am getting input from weather stations/sensors  I’d like to predict tomorrow’s forecast today Wednesday, September 15, 2021 17
  18. 18. When We Need High Performance Computing?  Case 3: Perform a high number of operations per seconds  I am an engineer at Amazon.com  My Web server gets 1,000 hits per seconds  I’d like my web server and databases to handle 1,000 transactions per seconds so that customers do not experience bad delays Wednesday, September 15, 2021 18
  19. 19. Where are we using HPC?  Used to solve complex modeling problems in a spectrum of disciplines  Topics include: o Artificial intelligence o Physical oceanography o Climate modeling o Plasma physics o Automotive engineering o Quantum physics o Cryptographic analysis o Quantum chemistry o Geophysics o Nuclear physics o Molecular biology o Solid state physics o Molecular dynamics o Structural dynamics  HPC is currently applied to business uses as well  data warehouses  transaction processing Wednesday, September 15, 2021 19
  20. 20.  From concept to engineering, from design to test and manufacturing, from weather prediction to medical discoveries, our day to day life depends more and more on HPC simulations  Safer products, accurate predictions, research, etc.  High-performance compute clusters provide the most efficient, flexible, cost effective HPC environment for any HPC simulation Wednesday, September 15, 2021 20
  21. 21. Changing times  From 1986 – 2002, microprocessors were speeding like a rocket, increasing in performance an average of 50% per year  Since then, it’s dropped to about 20% increase per year.. Wednesday, September 15, 2021 21
  22. 22. The Problem  Up to now, performance increases have been attributed to increasing density of transistors  But there are inherent problems  A little Physics lesson:  Smaller transistors = faster processors  Faster processors = increased power consumption  Increased power consumption = increased heat  Increased heat = unreliable processors Wednesday, September 15, 2021 22
  23. 23. An intelligent solution  Move away from single‐core systems to multicore processors  “core” = processing unit  Introduction of parallelism..!!  But …  Adding more processors doesn’t help much if programmers aren’t aware of them..!!  … or don’t know how to use them  Serial programs don’t benefit from this approach (in most cases) Wednesday, September 15, 2021 23
  24. 24. Parallel Computing  Form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently i.e. "in parallel“  So, we need to rewrite serial programs so that they’re parallel.  Write translation programs that automatically convert serial progra ms into parallel programs.  This is very difficult to do.  Success has been limited Wednesday, September 15, 2021 24
  25. 25. Parallel Computing  Example  Compute n values and add them together.  Serial solution: Wednesday, September 15, 2021 25
  26. 26. Parallel Computing  Example  We have p cores, p much smaller than n.  Each core performs a partial sum of approximately n/p values. Wednesday, September 15, 2021 26
  27. 27. Parallel Computing  Example  After each core completes execution of the code, is a private variable my_sum contains the sum of the values computed by its calls to Compute_next_value.  Ex., n = 200, then • Serial – will take 200 addition • Parallel (for 8 cores) – Each core will perform n/p = 25 addition – And master will perform 8 more addition + 8 receive operation – Total 41 operation. Wednesday, September 15, 2021 27
  28. 28. Parallel Computing  Some coding constructs can be recognized by an automatic program generator, and converted to a parallel construct.  However, it’s likely that the result will be a very inefficient program.  Sometimes the best parallel solution is to step back and devise an en tirely new algorithm.  Parallel computer programs are more difficult to write than sequential programs  Potential problems  Race condition (output depending on sequence or timing of other events)  Communication and synchronization between the different subtasks Wednesday, September 15, 2021 28

×