B9 cmis


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

B9 cmis

  2. 2. HPCNOTJUSTFOR ROCKET SCIENTIST• Earlier the word supercomputer to someone was monstrouslycomplicated machines solving problems no one really understands.• supercomputing was for the serious scientists and engineers whoneeded to crunch numbers as fast as possible.Today we have a different world. The custom supercomputer ofyesteryear has given way to commodity-based supercomputing , orwhat is now called High Performance Computing (HPC).28/11/2012HPC2
  3. 3. Featuresof HPC• HPC is now considered as an essential part of business success.• Can use the same hardware found in Web servers and even desktopworkstations.• Cost effective• Provide a great competitive advantage by designing new products,optimizing manufacturing and delivery processes, solving productionproblems, mining data , and simulating everything from businessprocess to shipping crates all in an effort to become morecompetitive, profitable , and “green.”28/11/2012HPC3
  4. 4. TheFormsof HPC✓ The commodity HPC cluster✓ Dedicated supercomputer✓ HPC cloud computing28/11/2012 HPCHPC4
  5. 5. WhoUsesHPC Today?• The worldwide HPC market is growing rapidly. According to IDC, the totalHPC market was $10 billion in 2007 and is expected to hit $11.7 billionaround 2012.The following is a list of major areas where HPC has a significantpresence: Bio-sciences and the human genome: Drug discovery, diseasedetection/prevention.Computer aided engineering (CAE): Automotive design and testing,transportation, structural, mechanical design Chemical engineering: Process and molecular design Digital content creation (DCC) and distribution: Computer aidedgraphics in film and media28/11/2012HPC5
  6. 6.  Economics/financial: Wall Street risk analysis, portfolio management,automated trading Electronic design and automation (EDA): Electronic component designand verification Geosciences and geo-engineering: Oil and gas exploration andreservoir modeling Mechanical design and drafting: 2D and 3D design and verification,mechanical modeling Defense and energy: Nuclear stewardship, basic and applied research Government labs: Basic and applied research University/academic: Basic and applied research Weather forecasting: Near term and climate/earth modeling28/11/2012 HPC6
  7. 7. WhoShouldBe UsingHPC?• High Performance Computing represents a tremendous competitiveedge in the marketplace because it can give users the ability to quicklymodel and then manipulate a product or process to see the impact ofvarious decisions before they are made.• Existing products and processes can be optimized to reduce cost, andnew avenues of development can be explored at lower cost and withfaster times to market.• So anyone interested in faster times to solution, better science, informeddecisions, more competitive products can use HPC.28/11/2012 HPC7
  8. 8. Enter theCommodityClusterDrawbacks and problems in HPC : First the price was steep- in seven figures the entire system wasintegrated from top to bottom by a single company leading to highcost.As the performance was increasing the cost of designing andfabricating custom processor and other components was alsoincreasing. considered office processors, no one really considered x86s to begood at floating point calculations. In addition to getting faster, supercomputer-specific processors alsogot more expensive to manufacture. Indeed, the expense became sohigh that it was no longer feasible to design and fabricatesupercomputer-specific processors.28/11/2012 HPC8
  9. 9. Hardwareand softwaremovement.1. As commodity processors from companies like AMD improved,collections of servers began to outpace even the fastestsupercomputers. The move to commodity hardware had begun.2. Many of the software tools needed to support a UNIX-likeenvironment were freely available under the GNU software license.3. Another enabler of the commodity cluster was the Internet . TheInternet helped in two ways:• it ignited the demand for Web servers.• it opened up an avenue for international collaboration, helping solvemany of the early HPC issues.28/11/2012 HPC9
  10. 10. HOWDoesa Cluster LookLike?• Head or master node. This node is normally a gateway to a sharednetwork where users “log in” to the cluster.• The head node has one or more networks with which itcommunicates to the worker (or compute) nodes.• In terms of networks, a cluster may have as little as one privatenetwork, which is usually Gigabit Ethernet (GigE), although InfiniBand is becoming a popular option.28/11/2012 HPC10
  11. 11. “Typical”clusterconfigurationModelTypical Cluster Configuration Contents• There’s a head node that maycontain a large amount ofstorage that is shared via thenetwork by all the workernodes.• A varying number of workernodes communicate with eachother and the head node overthe available privatenetworks.28/11/2012 HPC11
  12. 12. REALLIVECLUSTER!!• A real live cluster often lookslike that shown in figure. Thenodes are placed in standardequipment racks and usuallyrequire a data centerenvironment (raised floor,chilled air, and so on).28/11/2012 HPC12
  13. 13. TypicalIU ServerV/S Bladed SystemTypical “1U server” Bladed system28/11/2012 HPC13
  14. 14. Categoriesof cluster• Clusters are often broken into two categories:1. Capability2. Capacity• A capability cluster is designed to handle (or be capable of handling)large compute jobs that may employ every node in the cluster.• Capacity clusters, on the other hand, are those that are used todeliver a certain amount of computing capacity to the end users.28/11/2012 HPC14
  15. 15. IfYou NeedSpeed!!• High performance clusters are used where time to solution isimportant.• To increase computing throughput, HPC clusters are used in a varietyof ways.1 compute farm2 parallel computing28/11/2012HPC15
  16. 16. Computefarm• The first and easiest way is to allow the cluster to act as a computefarm.• Depending on the resources, all the jobs may run at the same timeor some may wait in the queue while other jobs finish.• This type of computing is local to a cluster node, which means thenode doesn’t communicate with other nodes, but may need highspeed file system access28/11/2012 HPC16
  17. 17. ComputeFarmProcess28/11/2012 HPC17
  18. 18. Parallelprogramrunning• Another way to use clusters is to break large HPC jobs up into smallersub-jobs and run each sub-job on different nodes.• This process is done at the software level• The most popular method for creating cluster programs is to use theMPI (Message Passing Interface) library and insert communicationcalls in the program..• The program employs all the processors and must communicatewith the other nodes.• The parallel program will run much faster than using a single node28/11/2012HPC18
  19. 19. ParallelProgram Process28/11/2012 HPC19
  20. 20. Of Cores, Threads,and Nodes• Cores are processing units within a modern day processor. A typicalmotherboard may hold 1, 2, or 4 processors and is contained in anode.• A typical node may have 2 processors, each with 6 cores for a total of12 cores per node.• programs on a multi-core node are run in a Symmetric Multi-Processing (SMP) mode. This designation means that multipleprograms can be running at the same time on various cores.• Threaded programming is parallel programming designed for asingle SMP node.28/11/2012 HPC20
  21. 21. AbouttheData!!• In addition to the parallel computation there may also be a need forparallel data access.• If all the nodes were required to read and write data from the samefile at the same time. A bottleneck would develop and limit the speedof your program.• For this reason, parallel file systems have been developed. Thesesolutions often involve additional hardware and software.28/11/2012 HPC21
  22. 22. ProcessorsAnd Nodes1)Processor is a workhorse of the cluster.2)Nodes are each server in a cluster.3)Cluster nodes may have 8 to 16 cores per nodes.4)The choice of processor is very important because clusterinstallations are often rely on scalable processor performance.28/11/2012 HPC22
  23. 23. ProcessorsAnd Nodes(contd..)5) Depending upon the design of the cluster , nodes are:(i) fat nodes (lots of cores, disk and memory)(ii) thin nodes ( small cores, disk and memory )28/11/2012 HPC23
  24. 24. IPMI• The Intelligent Platform Management Interface specification definesa set of common interfaces to a computer system. Systemadministrators can use IPMI to monitor system health and managethe system.28/11/2012 HPC24
  25. 25. Co-processors• They are called array processors or GP-GPU ( General PurposeGraphical Processing Units ) or video cards . These devices canaccelerate certain types of mathematical operations .28/11/2012 HPC25
  26. 26. Interconnect• It is a port that attaches one devices to another. Their performanceis rated by :-a) Latency [ fastest time in which a single byte can be sent ]b) Bandwidth [max. data rate ]c) N/2 Packet Size [throughput curve rise rate]28/11/2012 HPC26
  27. 27. Interconnect(contd..)d) Messaging [ tells how much messages per second an interconnectcan send ]e) Bi-section width [ this rates how well the switch supports manysimultaneous conversations ]28/11/2012 HPC27
  28. 28. Deploymentof HPC1) Infini Band (IB)2) 10Gigabit Ethernet28/11/2012 HPC28
  29. 29. RememberingTheStorage• Almost all clusters require HPC storage. The simplest method is touse the head node as an NFS server and RAID sub-systems.• HPC applications create large amt of data and it is to be ensured thatan archiving system is available to many data centre.• Flash modules can be a integrated directly onto motherboards.28/11/2012 HPC29
  30. 30. RackingAnd Stacking• Computer nodes can take the form of IU servers or blade systems.Blade systems often have a slightly higher costs, but offers moremanageability, density and some power and cooling redundancy. It isto be ensured that all cluster equipment is ‘rack mountable’ and canbe placed in a standard 19 inch equipment racks.28/11/2012 HPC30
  31. 31. FindingCluster Software• Linux is by far the most common OS that HPC users choose and otheroptions include Solaris software .• The GNU/Linux core is a open software and can be freely copied . Ithas allowed HPC users to create apps, build drivers and lots more .28/11/2012 HPC31
  32. 32. HPC Glue: Cluster Software• For a successful software running several types of software tasksinclude administration, programming, debugging, job schedulingand provision of nodes.• Programming is perhaps the most important aspect of cluster and itstools include MPI (Message Passing Interface ). MPI allows programsto talk to one another over cluster networks.• In addition to MPI , programmer requires compilers, debuggers andpropellers .28/11/2012 HPC32
  33. 33. FileSystemsNow a days parallel file systems developed for HPC systems. Theyallow for multi node input and output and are designed to match aparticular cluster. Parallel file system e.g. lustier from SunMicrosystems.28/11/2012 HPC33
  34. 34. SharingIs Caring: HPC ResourceSchedules• The resource scheduler is a critical part of the cluster because it wouldbe impossible to share resources without same form of load balancingtools . There are several tools freely available resource schedulers e.g.Sun Grid Engine from Sun Microsystems , Torque, Lava etc.28/11/2012 HPC34
  35. 35. Provisioning: CreatingTheClusterTurning raw hardware into a functioning cluster is calledprovisioning and it involves installing and configuring the headnode and worker nodes of the cluster . There are 2 main methodsthat are used :I. LocalII. Remote28/11/2012 HPC35
  36. 36. ClusterToolKits• The free available cluster distributions that provide turn keyprovisioning ready to run software are :-A. Sun HPC Software , Linux EditionB. Rocks ClustersC. Oscar28/11/2012 HPC36
  37. 37. YOUTHANK28/11/2012 HPC37