This document provides information on building a high performance computing cluster, including definitions of supercomputers, why they are needed, types of supercomputers, and steps for building a cluster. It outlines identifying the application, selecting hardware and software components, installation, configuration, testing, and maintenance. Homemade and commercial clusters are compared, and opportunities for generating revenue from clusters are discussed. Additional online resources for learning more are provided at the end.

1. This Presentation is protected by
Creative Common License v2.0

2. Agenda Domain : High Performance Computing
Presentation Level : Beginner
Prerequisite : Familiarity with Linux
-What is a Supercomputer?
-Why do we need it?
-Types of Supercomputers
-The Recipe of building a cluster
-Basic Concepts
-Identifying the Application
-Selection of Raw Materials
-Preparation
-Configuration
-Deployment
-Testing
-Maintenance
-Home made vs. commercial clusters
-Making money from clusters
-Other resources and links

3. What is a Supercomputer?
“ An extremely fast computer that can perform hundreds of millions of instructions
per second.”
-A powerful system built from collection of special purpose hardware
-It is designed for a specific application
-Processing power range is very high
-There is no standard for supercomputer specification
-Works on parallel processing schema

4. Best Supercomputers of the world

5. Indian Supercomputers
PARAM Padma is C-DAC's next
generation high performance scalable
computing cluster, currently with a peak
computing power of One Teraflop.
KABRU is a 144 node (Xeon DP) Linux cluster.
Though it is a very fast supercomputer, it is not
the fastest in the world. With a sustained
performance of 1002.3 GFlops(reached on
October 13th 2004) of double precision
arithmetic it is the second fastest supercomputer
in India and the fastest supercomputer in India
belonging to an academic institution.
--IMSc

6. Why do we need it?
“Obviously, we need it for more processing power!!!”
- We use it where computation can be parallel.
- We use it where “divide and conquer” is prominent in algorithms
- We use it for High Performance/Availability computing
- We use it for distributed computing
In India, CDAC uses supercomputers for research in:
Bioinfomatics, Computational Structural Mechanics , Computational
Atmospheric Science, Evolutionary Computing , Computational Chemistry etc.

7. Types of Supercomputers
“Two broad categories: Tightly coupled parallel systems & Loosely Coupled
Clusters”
Modern supercomputing clusters:
- High performance (HP) clusters
- Load-leveling clusters
- Web-service clusters
- Storage clusters
- Database clusters
A special type is Single System Image (SSI) Clusters

8. The Recipe for building a cluster
“…before you make soup, you need hunger to enjoy it…”
Let’s brush up the basic concepts:
- Linux Installation Basics
- DHCP
- Network Boot (via PXE Boot or Etherboot)
- Interconnect

9. The Recipe for building a cluster
[Identifying the Application]
“Why would you need a supercomputer? Hey , I need it just for fun!”
Building a cluster for:
- High Performance (HP) need
- High Availability (HA) need
or
-“just need it for fun”
“There is really lot of fun when you write & test your algorithms on a cluster…”

10. The Recipe for building a cluster
[Selection of raw material]
“Innovators build great things from non-great elements !”
Selection of Hardware:
- Few number of old mother boards
- Enough number of processors to sit on the boards
- Minimum of 32Mb RAM per board
- Network support via On-Board or External NIC cards
- At least one Hard disk & CD-ROM Drive
- Either BIOS support for Network Boot or A floppy drive for each board
“mean to say, just get few boxes from your friends if you feel lazy to build your own
hardware…”

11. The Recipe for building a cluster
[Selection of raw material]
“Innovators build great things from non-great elements !”
Selection of software:
- The OpenSSI (http://www.openssi.org)
- OSCAR (http://oscar.sourceforge.net/)
- TFTP
- Etherboot (http://rom-o-matic.net/5.2.4/ )
“mean to say, just get few boxes from your friends if you feel lazy to build your own
hardware…”

12. The OpenSSI
“the most fantastic product I have ever seen !”
It has internode communication, clusterwide process management, clusterwide devices, a cluster
filesystem, clusterwide IPC (pipes, fifos, msgqueues, semaphores, etc.) and clusterwide tcp/ip networking.

13. The Recipe for building a cluster
[Preparation]
“Question : Dedicated Cluster or Temporary Cluster ?”
- A clean install of base OS (Fedora Core 3)
on PCs that has bootable device
- A clean Network Configuration

14. The Recipe for building a cluster
[Configuration]
- Download and unpack OpenSSI (http://www.openssi.org)
- Go through the DOCS
- ./install does everything for year
Enter a clustername.
Enter a node number between 1 and 125.
Select a Network Interface Card (``NIC'') for the cluster interconnect.
Select (P)XE or (E)therboot as the network boot protocol for this node.
Select whether you want to enable root filesystem failover.
- For Adding nodes, run openssi-config-node. Select ``Add a new node''.
“ Remember that node 1 is called the init node…”

15. The Recipe for building a cluster
[Configuration]
- Essentials
# cluster -v
(To check the membership of nodes in cluster with their status)
# bash-ll
(The shell that performs load leveling.
/etc/sysconfig/loadlevellist can also be added to specify special processes to be loadlevelled and run
service loadlevel restart )
# ssi-ksync
(To rebuild the ramdisk to include the driver and update the network boot images)
# onnode <node_number> <command>
(To run a specific command on a specific node)

16. The Recipe for building a cluster
[Testing]
while(no_of_processes < 1000)
{
if (fork()==0)
{
run_relevent_algorithm();
exit(1);
}
no_of_processes++;
}
“…This piece of code can do wonders…Its fun to keep adding zeros in the loop...”

17. The Recipe for building a cluster
[Maintenance]
- Make sure there is no IP conflict if network is shared
- Perform regular efficiency audit on network
- Put proper firewall for security
“ you actually don’t need to bother much for maintenance…”

18. Home made vs. commercial clusters
“The obvious difference is in the looks…”

19. Making money from clusters
“Are’nt you interested in this…”
-Host web servers , file servers etc.
-Create supercomputer for fun, give access to yours friends
-Provide low-cost high performance computing facility to research institutes
-Convert offices and academic institutions into night-time research facility

20. Other resources and links
“go ahead find out more…”
Download this presentation and various other interesting things at:
http://www.parolkar.com/download.aspx
Other links:
http://www.openssi.org
http://www.beowulf.org
http://sourceforge.net/projects/ci-linux
http://linux-ha.org
http://www.openmosix.org

21. Questions
?

22. Thank You!!!
I am just a click away
abhishek@parolkar.com