Cluster management (supercomputer)

Cluster Management (Supercomputer) in Native GNU/ Linux
4 April 2015, Jakarta, Indonesia
1 http://tifosilinux.wordpress.com
Hary Cahyono
Published April 2015
Abstract
Originally, “clusters” and “high performance computing” were synonymous. Today, the meaning of the
word “cluster” has expanded beyond high-performance to include high-availability (HA) and load-
balancing (LB) clusters, then you have to ensure that you have skill to manage your entire cluster. Skill of
programming software like MPI and another like that is a must. You’ll need to keep your cluster running.
Cluster management includes both routine system administration tasks and monitoring the health of your
cluster. Fortunately, there are several method came from packages that can be used to simplify these
tasks.
Keywords: Supercomputer, Native Linux
Preface
There were various tools in order to support
activity to manage our cluster. OpenMosix,
OSCAR, ROCKS, Beowulf, whatever method or
cluster kits that you have been used [1]. We’ll
need third party software in order to make it
easier.
By choosing the two alternative from all
prospect, C3, a set of commands for
administering multiple systems and PVFS, one
of the parallel filesystems that make clustering
I/O easier.
Theory
Both C3 and PVFS can be used with
federated clusters as well as simple clusters.
Cluster Command and Control (C3) is a set of
about a dozen command-line utilities used to
execute common management tasks. These
commands were designed to provide a look and
feel similar to that of issuing commands on a
single machine. The commands are both secure
and scale reliably. Each command is actually a
Python script. The c3-4.0.1.tar.gz is the file that I
have been used. Once you have met the
prerequisites, you can download, unpack, and
install C3. They are come with powerful
commands like cexec, cget, ckill, cpush, crm,
cshutdown, clist, cname, and cnum [2].
Increasingly, tasks that are computationally
expensive also involve a large amount of I/O,
frequently accessing either large data sets or
large databases. Selecting a filesystem for a
cluster is a balancing act. There are number
characteristics that can be used to compare
filesystems, including robustness, failure
recovery, journaling, enhanced security, and
reduced latency. Unfortunately, NFS is not very
efficient. In particular, it has not been optimized

for the types of I/O often needed with many
high-performance cluster applications. Figure 1.
That is, you must decide which machines will be
I/O servers (these directories are where the
actual pieces of a data file will be stored at
mount point), metadata server (these will
containing all of informations like permission,
path, fs type, etc will be stored) and which
machines will be clients.
Result and Discussion
The default configuration of C3 recorded all
of hostname nodes is /etc/c3.conf.
Figure 2. cpush and cexec <mkdir> command is
used to move a file to each node on the cluster
and created directory on each machine.
Since the old version of PVFS would not
compile correctly under my Red Hat 9.0. You
are also get the suitable patch and PVFS kernel
as well as the packages. In the PVFS
installation, as noted, this version the kernel
module sources need to be patched.
Figure 3. Patch the appropriate kernel and
PVFS.
Next, configuring the metadata server. Create
the metadata configuration files and place them
in the directory, creates the two configuration
files .iodtab and .pvfsdir contains permission
information for the metadata directory.
Fortunately, PVFS provides a script to simplify
the process. Figure 4, 5, and 6.

I/O server setup. To set up the I/O servers,
you need to create a data directory on the
appropriate machines, create a configuration
file, and then push the configuration file, along
with the other I/O server software, to the
appropriate machines. In my case, all the nodes
in the cluster including the head node are I/O
servers.
Client setup is little more involved. For each
client, you’ll need to create a PVFS device file,
copy over the kernel module, create a mount
point and a PVFS table, and copy over the
appropriate executable along with any other
utilities you might need on the client machine. In
my case, all nodes including the head ar
configured as clients. Figure 7, 8, 9, and 10.
Using an extra mknod command.
Running PVFS. Finally, now that you have
everything installed, you can start PVFS. You
need to start the appropriate daemons on the
appropriate machines and load the kernel
module. To load the kernel module, use the
insmod command.
Figure 11. PVFS should be up and running.
The mounted PVFS will be included in the
listing given with the mount command after
cexec /sbin/mount.pvfs hary:/pvfs-meta
/mnt/pvfs from head node. Figure 12.
This should work on each node.

Conclusion
Although PVFS (supported with C3’s
command) eliminates the bottleneck inherent in
a single file server approach such as NFS. This
is only partial listing of what is available. Since
PVFS does not provide redundancy, does not
support symbolic or hard links, and it does not
provide fsck-like utility. But while they may be
ideal for some uses, they may be problematic for
others .
Acknowledgments
First and foremost, credit goes to Allah SWT
and this proceeding would not exist if not for the
GNU/ Linux community.
References
[1] Cahyono, Hary. (2015). “(UPDATE) RAIH
DUNIA DENGAN SUPERKOMPUTER DI
LINUX NATIVE”. 2015: 1-59. Retrieved 4
April 2015.
[2] Sloan, Joseph D. (2004). High Performance
– LINUX CLUSTER with OSCAR, Rocks,
openMosix & MPI. United States of America:
O’Reilly Media.
Hary Cahyono
Email : h4ry.oop@gmail.com
Phone: 085695042489
Skype: hary_122
BBM: 7943F602

Cluster management (supercomputer)

More Related Content

What's hot

Similar to Cluster management (supercomputer)

More from Hary HarysMatta

Recently uploaded

Cluster management (supercomputer)