SlideShare a Scribd company logo
1 of 4
Download to read offline
Cluster Management (Supercomputer) in Native GNU/ Linux
4 April 2015, Jakarta, Indonesia
1 http://tifosilinux.wordpress.com
Cluster Management (Supercomputer) in Native GNU/ Linux
Hary Cahyono
Published April 2015
Abstract
Originally, “clusters” and “high performance computing” were synonymous. Today, the meaning of the
word “cluster” has expanded beyond high-performance to include high-availability (HA) and load-
balancing (LB) clusters, then you have to ensure that you have skill to manage your entire cluster. Skill of
programming software like MPI and another like that is a must. You’ll need to keep your cluster running.
Cluster management includes both routine system administration tasks and monitoring the health of your
cluster. Fortunately, there are several method came from packages that can be used to simplify these
tasks.
Keywords: Supercomputer, Native Linux
Preface
There were various tools in order to support
activity to manage our cluster. OpenMosix,
OSCAR, ROCKS, Beowulf, whatever method or
cluster kits that you have been used [1]. We’ll
need third party software in order to make it
easier.
By choosing the two alternative from all
prospect, C3, a set of commands for
administering multiple systems and PVFS, one
of the parallel filesystems that make clustering
I/O easier.
Theory
Both C3 and PVFS can be used with
federated clusters as well as simple clusters.
Cluster Command and Control (C3) is a set of
about a dozen command-line utilities used to
execute common management tasks. These
commands were designed to provide a look and
feel similar to that of issuing commands on a
single machine. The commands are both secure
and scale reliably. Each command is actually a
Python script. The c3-4.0.1.tar.gz is the file that I
have been used. Once you have met the
prerequisites, you can download, unpack, and
install C3. They are come with powerful
commands like cexec, cget, ckill, cpush, crm,
cshutdown, clist, cname, and cnum [2].
Increasingly, tasks that are computationally
expensive also involve a large amount of I/O,
frequently accessing either large data sets or
large databases. Selecting a filesystem for a
cluster is a balancing act. There are number
characteristics that can be used to compare
filesystems, including robustness, failure
recovery, journaling, enhanced security, and
reduced latency. Unfortunately, NFS is not very
efficient. In particular, it has not been optimized
Cluster Management (Supercomputer) in Native GNU/ Linux
4 April 2015, Jakarta, Indonesia
2 http://tifosilinux.wordpress.com
for the types of I/O often needed with many
high-performance cluster applications. Figure 1.
That is, you must decide which machines will be
I/O servers (these directories are where the
actual pieces of a data file will be stored at
mount point), metadata server (these will
containing all of informations like permission,
path, fs type, etc will be stored) and which
machines will be clients.
Result and Discussion
The default configuration of C3 recorded all
of hostname nodes is /etc/c3.conf.
Figure 2. cpush and cexec <mkdir> command is
used to move a file to each node on the cluster
and created directory on each machine.
Since the old version of PVFS would not
compile correctly under my Red Hat 9.0. You
are also get the suitable patch and PVFS kernel
as well as the packages. In the PVFS
installation, as noted, this version the kernel
module sources need to be patched.
Figure 3. Patch the appropriate kernel and
PVFS.
Next, configuring the metadata server. Create
the metadata configuration files and place them
in the directory, creates the two configuration
files .iodtab and .pvfsdir contains permission
information for the metadata directory.
Fortunately, PVFS provides a script to simplify
the process. Figure 4, 5, and 6.
Cluster Management (Supercomputer) in Native GNU/ Linux
4 April 2015, Jakarta, Indonesia
3 http://tifosilinux.wordpress.com
I/O server setup. To set up the I/O servers,
you need to create a data directory on the
appropriate machines, create a configuration
file, and then push the configuration file, along
with the other I/O server software, to the
appropriate machines. In my case, all the nodes
in the cluster including the head node are I/O
servers.
Client setup is little more involved. For each
client, you’ll need to create a PVFS device file,
copy over the kernel module, create a mount
point and a PVFS table, and copy over the
appropriate executable along with any other
utilities you might need on the client machine. In
my case, all nodes including the head ar
configured as clients. Figure 7, 8, 9, and 10.
Using an extra mknod command.
Running PVFS. Finally, now that you have
everything installed, you can start PVFS. You
need to start the appropriate daemons on the
appropriate machines and load the kernel
module. To load the kernel module, use the
insmod command.
Figure 11. PVFS should be up and running.
The mounted PVFS will be included in the
listing given with the mount command after
cexec /sbin/mount.pvfs hary:/pvfs-meta
/mnt/pvfs from head node. Figure 12.
This should work on each node.
Cluster Management (Supercomputer) in Native GNU/ Linux
4 April 2015, Jakarta, Indonesia
4 http://tifosilinux.wordpress.com
Conclusion
Although PVFS (supported with C3’s
command) eliminates the bottleneck inherent in
a single file server approach such as NFS. This
is only partial listing of what is available. Since
PVFS does not provide redundancy, does not
support symbolic or hard links, and it does not
provide fsck-like utility. But while they may be
ideal for some uses, they may be problematic for
others .
Acknowledgments
First and foremost, credit goes to Allah SWT
and this proceeding would not exist if not for the
GNU/ Linux community.
References
[1] Cahyono, Hary. (2015). “(UPDATE) RAIH
DUNIA DENGAN SUPERKOMPUTER DI
LINUX NATIVE”. 2015: 1-59. Retrieved 4
April 2015.
[2] Sloan, Joseph D. (2004). High Performance
– LINUX CLUSTER with OSCAR, Rocks,
openMosix & MPI. United States of America:
O’Reilly Media.
Hary Cahyono
Email : h4ry.oop@gmail.com
Phone: 085695042489
Skype: hary_122
BBM: 7943F602

More Related Content

What's hot

Slide sistem operasi
Slide sistem operasiSlide sistem operasi
Slide sistem operasitheogrouptech
 
Union FileSystem - A Building Blocks Of a Container
Union FileSystem - A Building Blocks Of a ContainerUnion FileSystem - A Building Blocks Of a Container
Union FileSystem - A Building Blocks Of a ContainerKnoldus Inc.
 
Namespaces and cgroups - the basis of Linux containers
Namespaces and cgroups - the basis of Linux containersNamespaces and cgroups - the basis of Linux containers
Namespaces and cgroups - the basis of Linux containersKernel TLV
 
Proxmox Talk - Linux Fest Northwest 2018
Proxmox Talk - Linux Fest Northwest 2018Proxmox Talk - Linux Fest Northwest 2018
Proxmox Talk - Linux Fest Northwest 2018Richard Clark
 
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxCon
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConAnatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxCon
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
 
Linux container, namespaces & CGroup.
Linux container, namespaces & CGroup. Linux container, namespaces & CGroup.
Linux container, namespaces & CGroup. Neeraj Shrimali
 
Manage Virtual Machines with WebVirtMgr on openSUSE
Manage Virtual Machines with WebVirtMgr on openSUSEManage Virtual Machines with WebVirtMgr on openSUSE
Manage Virtual Machines with WebVirtMgr on openSUSEDendy P. Delly
 
Oracle VM3: Virtuelle Maschinen per Script erstellen
Oracle VM3: Virtuelle Maschinen per Script erstellenOracle VM3: Virtuelle Maschinen per Script erstellen
Oracle VM3: Virtuelle Maschinen per Script erstellenTrivadis
 
Linux con europe_2014_f
Linux con europe_2014_fLinux con europe_2014_f
Linux con europe_2014_fsprdd
 
Docker navjot kaur
Docker navjot kaurDocker navjot kaur
Docker navjot kaurNavjot Kaur
 
OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...
OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...
OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...OpenNebula Project
 
Academy PRO: Docker. Lecture 3
Academy PRO: Docker. Lecture 3Academy PRO: Docker. Lecture 3
Academy PRO: Docker. Lecture 3Binary Studio
 
Freenas Tutorial EuroBSDCon 2012
Freenas Tutorial EuroBSDCon 2012Freenas Tutorial EuroBSDCon 2012
Freenas Tutorial EuroBSDCon 2012Dru Lavigne
 
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...OpenNebula Project
 
Lavigne aug11 bsdmag
Lavigne aug11 bsdmagLavigne aug11 bsdmag
Lavigne aug11 bsdmagDru Lavigne
 
Unix Automation using centralized configuration management tool
Unix Automation using centralized configuration management toolUnix Automation using centralized configuration management tool
Unix Automation using centralized configuration management toolTorrid Networks Private Limited
 
Practical Introduction To Linux
Practical Introduction To LinuxPractical Introduction To Linux
Practical Introduction To LinuxZeeshan Rizvi
 

What's hot (20)

Slide sistem operasi
Slide sistem operasiSlide sistem operasi
Slide sistem operasi
 
Union FileSystem - A Building Blocks Of a Container
Union FileSystem - A Building Blocks Of a ContainerUnion FileSystem - A Building Blocks Of a Container
Union FileSystem - A Building Blocks Of a Container
 
Namespaces and cgroups - the basis of Linux containers
Namespaces and cgroups - the basis of Linux containersNamespaces and cgroups - the basis of Linux containers
Namespaces and cgroups - the basis of Linux containers
 
Proxmox Talk - Linux Fest Northwest 2018
Proxmox Talk - Linux Fest Northwest 2018Proxmox Talk - Linux Fest Northwest 2018
Proxmox Talk - Linux Fest Northwest 2018
 
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxCon
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConAnatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxCon
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxCon
 
Linux container, namespaces & CGroup.
Linux container, namespaces & CGroup. Linux container, namespaces & CGroup.
Linux container, namespaces & CGroup.
 
Manage Virtual Machines with WebVirtMgr on openSUSE
Manage Virtual Machines with WebVirtMgr on openSUSEManage Virtual Machines with WebVirtMgr on openSUSE
Manage Virtual Machines with WebVirtMgr on openSUSE
 
Sistem operasi
Sistem operasiSistem operasi
Sistem operasi
 
Oracle VM3: Virtuelle Maschinen per Script erstellen
Oracle VM3: Virtuelle Maschinen per Script erstellenOracle VM3: Virtuelle Maschinen per Script erstellen
Oracle VM3: Virtuelle Maschinen per Script erstellen
 
Linux con europe_2014_f
Linux con europe_2014_fLinux con europe_2014_f
Linux con europe_2014_f
 
Docker navjot kaur
Docker navjot kaurDocker navjot kaur
Docker navjot kaur
 
OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...
OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...
OpenNebulaConf 2016 - ONEDock: Docker as a hypervisor in ONE by Carlos de Alf...
 
Academy PRO: Docker. Lecture 3
Academy PRO: Docker. Lecture 3Academy PRO: Docker. Lecture 3
Academy PRO: Docker. Lecture 3
 
Freenas Tutorial EuroBSDCon 2012
Freenas Tutorial EuroBSDCon 2012Freenas Tutorial EuroBSDCon 2012
Freenas Tutorial EuroBSDCon 2012
 
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
OpenNebulaConf 2016 - Building a GNU/Linux Distribution by Daniel Dehennin, M...
 
Ilf2013
Ilf2013Ilf2013
Ilf2013
 
Lavigne aug11 bsdmag
Lavigne aug11 bsdmagLavigne aug11 bsdmag
Lavigne aug11 bsdmag
 
Ubuntu server guide
Ubuntu server guideUbuntu server guide
Ubuntu server guide
 
Unix Automation using centralized configuration management tool
Unix Automation using centralized configuration management toolUnix Automation using centralized configuration management tool
Unix Automation using centralized configuration management tool
 
Practical Introduction To Linux
Practical Introduction To LinuxPractical Introduction To Linux
Practical Introduction To Linux
 

Similar to Cluster management (supercomputer)

Open stack implementation
Open stack implementation Open stack implementation
Open stack implementation Soumyajit Basu
 
Performance improvement by
Performance improvement byPerformance improvement by
Performance improvement byIJCNCJournal
 
Dru lavigne servers-tutorial
Dru lavigne servers-tutorialDru lavigne servers-tutorial
Dru lavigne servers-tutorialDru Lavigne
 
final proposal-Xen based Hypervisor in a Box
final proposal-Xen based Hypervisor in a Boxfinal proposal-Xen based Hypervisor in a Box
final proposal-Xen based Hypervisor in a BoxParamkusham Shruthi
 
How to install_and_configure_r_on_a_linux_server
How to install_and_configure_r_on_a_linux_serverHow to install_and_configure_r_on_a_linux_server
How to install_and_configure_r_on_a_linux_serversushantbit04
 
Unix Automation using centralized configuration management tool
Unix Automation using centralized configuration management toolUnix Automation using centralized configuration management tool
Unix Automation using centralized configuration management toolTorrid Networks Private Limited
 
Net app ecmlp2495163
Net app ecmlp2495163Net app ecmlp2495163
Net app ecmlp2495163forum4user
 
Piattaforma Web Linux completa dai sorgenti
Piattaforma Web Linux completa dai sorgentiPiattaforma Web Linux completa dai sorgenti
Piattaforma Web Linux completa dai sorgentiGiulio Destri
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...Codemotion
 
ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES
ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES
ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES csandit
 
4 implementation
4 implementation4 implementation
4 implementationhanmya
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014Carlo Bonamico
 
Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5Carlos Eduardo
 
Using openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual MachinesUsing openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual MachinesKris Buytaert
 
Building Toward an Open and Extensible Autonomous Computing Platform Utilizi...
Building Toward an Open and Extensible  Autonomous Computing Platform Utilizi...Building Toward an Open and Extensible  Autonomous Computing Platform Utilizi...
Building Toward an Open and Extensible Autonomous Computing Platform Utilizi...Phil Cryer
 
Lavigne bsdmag-jan2012
Lavigne bsdmag-jan2012Lavigne bsdmag-jan2012
Lavigne bsdmag-jan2012Dru Lavigne
 
maXbox Starter87
maXbox Starter87maXbox Starter87
maXbox Starter87Max Kleiner
 
Presentation linux on power
Presentation   linux on powerPresentation   linux on power
Presentation linux on powersolarisyougood
 

Similar to Cluster management (supercomputer) (20)

Open stack implementation
Open stack implementation Open stack implementation
Open stack implementation
 
Performance improvement by
Performance improvement byPerformance improvement by
Performance improvement by
 
Dru lavigne servers-tutorial
Dru lavigne servers-tutorialDru lavigne servers-tutorial
Dru lavigne servers-tutorial
 
final proposal-Xen based Hypervisor in a Box
final proposal-Xen based Hypervisor in a Boxfinal proposal-Xen based Hypervisor in a Box
final proposal-Xen based Hypervisor in a Box
 
How to install_and_configure_r_on_a_linux_server
How to install_and_configure_r_on_a_linux_serverHow to install_and_configure_r_on_a_linux_server
How to install_and_configure_r_on_a_linux_server
 
Unix Automation using centralized configuration management tool
Unix Automation using centralized configuration management toolUnix Automation using centralized configuration management tool
Unix Automation using centralized configuration management tool
 
First steps on CentOs7
First steps on CentOs7First steps on CentOs7
First steps on CentOs7
 
Net app ecmlp2495163
Net app ecmlp2495163Net app ecmlp2495163
Net app ecmlp2495163
 
Piattaforma Web Linux completa dai sorgenti
Piattaforma Web Linux completa dai sorgentiPiattaforma Web Linux completa dai sorgenti
Piattaforma Web Linux completa dai sorgenti
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
 
ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES
ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES
ENHANCING PERFORMANCE OF AN HPC CLUSTER BY ADOPTING NONDEDICATED NODES
 
4 implementation
4 implementation4 implementation
4 implementation
 
OpenQrm
OpenQrmOpenQrm
OpenQrm
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014
 
Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5Instalando Cacti no CentOS 5
Instalando Cacti no CentOS 5
 
Using openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual MachinesUsing openQRM to Manage Virtual Machines
Using openQRM to Manage Virtual Machines
 
Building Toward an Open and Extensible Autonomous Computing Platform Utilizi...
Building Toward an Open and Extensible  Autonomous Computing Platform Utilizi...Building Toward an Open and Extensible  Autonomous Computing Platform Utilizi...
Building Toward an Open and Extensible Autonomous Computing Platform Utilizi...
 
Lavigne bsdmag-jan2012
Lavigne bsdmag-jan2012Lavigne bsdmag-jan2012
Lavigne bsdmag-jan2012
 
maXbox Starter87
maXbox Starter87maXbox Starter87
maXbox Starter87
 
Presentation linux on power
Presentation   linux on powerPresentation   linux on power
Presentation linux on power
 

More from Hary HarysMatta

Strongswan ipsec vpn_linux_strongswan_ip
Strongswan ipsec vpn_linux_strongswan_ipStrongswan ipsec vpn_linux_strongswan_ip
Strongswan ipsec vpn_linux_strongswan_ipHary HarysMatta
 
Letsencrypt with pfsense
Letsencrypt with pfsenseLetsencrypt with pfsense
Letsencrypt with pfsenseHary HarysMatta
 
Membangun Aplikasi Web Data Analysis dengan Framework Django
Membangun Aplikasi Web Data Analysis dengan Framework DjangoMembangun Aplikasi Web Data Analysis dengan Framework Django
Membangun Aplikasi Web Data Analysis dengan Framework DjangoHary HarysMatta
 
GA (Google Analytics) Masterclass
GA (Google Analytics) MasterclassGA (Google Analytics) Masterclass
GA (Google Analytics) MasterclassHary HarysMatta
 
Pengantar jaringan komputer
Pengantar jaringan komputerPengantar jaringan komputer
Pengantar jaringan komputerHary HarysMatta
 
Ayat jurnal penyesuaian dan ayat jurnal balik
Ayat jurnal penyesuaian dan ayat jurnal balikAyat jurnal penyesuaian dan ayat jurnal balik
Ayat jurnal penyesuaian dan ayat jurnal balikHary HarysMatta
 
Ikhtisar pengantar akuntansi dan manajemen keuangan2
Ikhtisar pengantar akuntansi dan manajemen keuangan2Ikhtisar pengantar akuntansi dan manajemen keuangan2
Ikhtisar pengantar akuntansi dan manajemen keuangan2Hary HarysMatta
 
Update - Superkomputer dengan Native GNU/ Linux - Final
Update - Superkomputer dengan Native GNU/ Linux - FinalUpdate - Superkomputer dengan Native GNU/ Linux - Final
Update - Superkomputer dengan Native GNU/ Linux - FinalHary HarysMatta
 
Superkomputer dengan Native Linux(encrypted)
Superkomputer dengan Native Linux(encrypted)Superkomputer dengan Native Linux(encrypted)
Superkomputer dengan Native Linux(encrypted)Hary HarysMatta
 
Esensi Sholat Berjama'ah
Esensi Sholat Berjama'ahEsensi Sholat Berjama'ah
Esensi Sholat Berjama'ahHary HarysMatta
 
Cloud Computing Dengan VMWARE
Cloud Computing Dengan VMWARECloud Computing Dengan VMWARE
Cloud Computing Dengan VMWAREHary HarysMatta
 
Panduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat Kita
Panduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat KitaPanduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat Kita
Panduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat KitaHary HarysMatta
 
Panduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi Thread
Panduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi ThreadPanduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi Thread
Panduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi ThreadHary HarysMatta
 
Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2
Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2
Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2Hary HarysMatta
 
Metode Praktis Penetapan Nisab Zakat
Metode Praktis Penetapan Nisab ZakatMetode Praktis Penetapan Nisab Zakat
Metode Praktis Penetapan Nisab ZakatHary HarysMatta
 

More from Hary HarysMatta (16)

Strongswan ipsec vpn_linux_strongswan_ip
Strongswan ipsec vpn_linux_strongswan_ipStrongswan ipsec vpn_linux_strongswan_ip
Strongswan ipsec vpn_linux_strongswan_ip
 
Letsencrypt with pfsense
Letsencrypt with pfsenseLetsencrypt with pfsense
Letsencrypt with pfsense
 
Membangun Aplikasi Web Data Analysis dengan Framework Django
Membangun Aplikasi Web Data Analysis dengan Framework DjangoMembangun Aplikasi Web Data Analysis dengan Framework Django
Membangun Aplikasi Web Data Analysis dengan Framework Django
 
GA (Google Analytics) Masterclass
GA (Google Analytics) MasterclassGA (Google Analytics) Masterclass
GA (Google Analytics) Masterclass
 
Pengantar jaringan komputer
Pengantar jaringan komputerPengantar jaringan komputer
Pengantar jaringan komputer
 
Ayat jurnal penyesuaian dan ayat jurnal balik
Ayat jurnal penyesuaian dan ayat jurnal balikAyat jurnal penyesuaian dan ayat jurnal balik
Ayat jurnal penyesuaian dan ayat jurnal balik
 
Ikhtisar pengantar akuntansi dan manajemen keuangan2
Ikhtisar pengantar akuntansi dan manajemen keuangan2Ikhtisar pengantar akuntansi dan manajemen keuangan2
Ikhtisar pengantar akuntansi dan manajemen keuangan2
 
Update - Superkomputer dengan Native GNU/ Linux - Final
Update - Superkomputer dengan Native GNU/ Linux - FinalUpdate - Superkomputer dengan Native GNU/ Linux - Final
Update - Superkomputer dengan Native GNU/ Linux - Final
 
Superkomputer dengan Native Linux(encrypted)
Superkomputer dengan Native Linux(encrypted)Superkomputer dengan Native Linux(encrypted)
Superkomputer dengan Native Linux(encrypted)
 
Esensi Sholat Berjama'ah
Esensi Sholat Berjama'ahEsensi Sholat Berjama'ah
Esensi Sholat Berjama'ah
 
Cloud Computing Dengan VMWARE
Cloud Computing Dengan VMWARECloud Computing Dengan VMWARE
Cloud Computing Dengan VMWARE
 
Panduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat Kita
Panduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat KitaPanduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat Kita
Panduan Evaluasi Sholat dan Hadits Shahih - Mengevaluasi Cara Sholat Kita
 
Panduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi Thread
Panduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi ThreadPanduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi Thread
Panduan Java sebagai Pemrograman Berorientasi Objek dan Implementasi Thread
 
Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2
Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2
Implementasi Kannel dengan modem Wavecom di Linux OpenSuSE 11.2
 
Mengenal ORACLE PL/ SQL
Mengenal ORACLE PL/ SQLMengenal ORACLE PL/ SQL
Mengenal ORACLE PL/ SQL
 
Metode Praktis Penetapan Nisab Zakat
Metode Praktis Penetapan Nisab ZakatMetode Praktis Penetapan Nisab Zakat
Metode Praktis Penetapan Nisab Zakat
 

Recently uploaded

Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 

Recently uploaded (20)

Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 

Cluster management (supercomputer)

  • 1. Cluster Management (Supercomputer) in Native GNU/ Linux 4 April 2015, Jakarta, Indonesia 1 http://tifosilinux.wordpress.com Cluster Management (Supercomputer) in Native GNU/ Linux Hary Cahyono Published April 2015 Abstract Originally, “clusters” and “high performance computing” were synonymous. Today, the meaning of the word “cluster” has expanded beyond high-performance to include high-availability (HA) and load- balancing (LB) clusters, then you have to ensure that you have skill to manage your entire cluster. Skill of programming software like MPI and another like that is a must. You’ll need to keep your cluster running. Cluster management includes both routine system administration tasks and monitoring the health of your cluster. Fortunately, there are several method came from packages that can be used to simplify these tasks. Keywords: Supercomputer, Native Linux Preface There were various tools in order to support activity to manage our cluster. OpenMosix, OSCAR, ROCKS, Beowulf, whatever method or cluster kits that you have been used [1]. We’ll need third party software in order to make it easier. By choosing the two alternative from all prospect, C3, a set of commands for administering multiple systems and PVFS, one of the parallel filesystems that make clustering I/O easier. Theory Both C3 and PVFS can be used with federated clusters as well as simple clusters. Cluster Command and Control (C3) is a set of about a dozen command-line utilities used to execute common management tasks. These commands were designed to provide a look and feel similar to that of issuing commands on a single machine. The commands are both secure and scale reliably. Each command is actually a Python script. The c3-4.0.1.tar.gz is the file that I have been used. Once you have met the prerequisites, you can download, unpack, and install C3. They are come with powerful commands like cexec, cget, ckill, cpush, crm, cshutdown, clist, cname, and cnum [2]. Increasingly, tasks that are computationally expensive also involve a large amount of I/O, frequently accessing either large data sets or large databases. Selecting a filesystem for a cluster is a balancing act. There are number characteristics that can be used to compare filesystems, including robustness, failure recovery, journaling, enhanced security, and reduced latency. Unfortunately, NFS is not very efficient. In particular, it has not been optimized
  • 2. Cluster Management (Supercomputer) in Native GNU/ Linux 4 April 2015, Jakarta, Indonesia 2 http://tifosilinux.wordpress.com for the types of I/O often needed with many high-performance cluster applications. Figure 1. That is, you must decide which machines will be I/O servers (these directories are where the actual pieces of a data file will be stored at mount point), metadata server (these will containing all of informations like permission, path, fs type, etc will be stored) and which machines will be clients. Result and Discussion The default configuration of C3 recorded all of hostname nodes is /etc/c3.conf. Figure 2. cpush and cexec <mkdir> command is used to move a file to each node on the cluster and created directory on each machine. Since the old version of PVFS would not compile correctly under my Red Hat 9.0. You are also get the suitable patch and PVFS kernel as well as the packages. In the PVFS installation, as noted, this version the kernel module sources need to be patched. Figure 3. Patch the appropriate kernel and PVFS. Next, configuring the metadata server. Create the metadata configuration files and place them in the directory, creates the two configuration files .iodtab and .pvfsdir contains permission information for the metadata directory. Fortunately, PVFS provides a script to simplify the process. Figure 4, 5, and 6.
  • 3. Cluster Management (Supercomputer) in Native GNU/ Linux 4 April 2015, Jakarta, Indonesia 3 http://tifosilinux.wordpress.com I/O server setup. To set up the I/O servers, you need to create a data directory on the appropriate machines, create a configuration file, and then push the configuration file, along with the other I/O server software, to the appropriate machines. In my case, all the nodes in the cluster including the head node are I/O servers. Client setup is little more involved. For each client, you’ll need to create a PVFS device file, copy over the kernel module, create a mount point and a PVFS table, and copy over the appropriate executable along with any other utilities you might need on the client machine. In my case, all nodes including the head ar configured as clients. Figure 7, 8, 9, and 10. Using an extra mknod command. Running PVFS. Finally, now that you have everything installed, you can start PVFS. You need to start the appropriate daemons on the appropriate machines and load the kernel module. To load the kernel module, use the insmod command. Figure 11. PVFS should be up and running. The mounted PVFS will be included in the listing given with the mount command after cexec /sbin/mount.pvfs hary:/pvfs-meta /mnt/pvfs from head node. Figure 12. This should work on each node.
  • 4. Cluster Management (Supercomputer) in Native GNU/ Linux 4 April 2015, Jakarta, Indonesia 4 http://tifosilinux.wordpress.com Conclusion Although PVFS (supported with C3’s command) eliminates the bottleneck inherent in a single file server approach such as NFS. This is only partial listing of what is available. Since PVFS does not provide redundancy, does not support symbolic or hard links, and it does not provide fsck-like utility. But while they may be ideal for some uses, they may be problematic for others . Acknowledgments First and foremost, credit goes to Allah SWT and this proceeding would not exist if not for the GNU/ Linux community. References [1] Cahyono, Hary. (2015). “(UPDATE) RAIH DUNIA DENGAN SUPERKOMPUTER DI LINUX NATIVE”. 2015: 1-59. Retrieved 4 April 2015. [2] Sloan, Joseph D. (2004). High Performance – LINUX CLUSTER with OSCAR, Rocks, openMosix & MPI. United States of America: O’Reilly Media. Hary Cahyono Email : h4ry.oop@gmail.com Phone: 085695042489 Skype: hary_122 BBM: 7943F602