Ibm spectrum scale fundamentals workshop for americas part 1 components architectures-n-install

Spectrum Scale 4.1 System Administration
Spectrum Scale Components,
Architectures
(with basic Install and configuration)
© Copyright IBM Corporation 2015

Unit objectives
After completing this unit, you should be able to:
• Describe the Components of a Spectrum Scale Cluster
• Describe common Spectrum Scale Architectures
• Describe the Value of Spectrum Scale solutions
• Describe the Installation checklist
• Explain System requirements for installation
• Explain IBM and Client Responsibilities
• List the Basic Software packages required
– Creating a cluster
– Creating Network Shared Disks (NSD)
– Creating a file system
– Mounting a file system

Describe the Components of a Spectrum Scale Cluster
1- A Server Architecture to install Spectrum Scale on
2- A Storage architecture to create a file system on
3- A network architecture for Cluster Inter-communication
4- A network architecture to support client access to File System data
5- A set of protocols to manage client data access
6- An Authentication mechanism to manage access and authorization to data
7- An Operating System to run Spectrum Scale Software on
8- Spectrum Scale Software
9- Spectrum Scale Server Licenses
10- Spectrum Scale Cluster Quorum Devices
11- Spectrum Scale Client Licenses (Optional)

Spectrum Scale License Types
• Server
– Shares Data
– Cluster Administrative Functions
– Serving Data: Spectrum Scale and NFS
• Client
– Consumes data.
• FPO
– Perform NSD server functions for sharing data with other nodes that
have Spectrum Scale server or FPO license.
• In Spectrum Scale 4.1 licensing is extended to Spectrum Scale
Standard, Express, and Advanced Editions

What is a Spectrum Scale cluster?
• A set of nodes all running Spectrum Scale software configured
into a single grouping.
• All nodes have common view of the data.
• Nodes are tightly coupled.
SAN

Software Defined Storage means
• Spectrum Scale (GPFS 4.1) is
a Software product
• It provides flexibility to choose
the right server & storage
technology
• It allows you to mix and match
storage to support a wide
variety of application
workloads.
• It supports virtually any
Network Architecture allowing
Use of Fibre Channel SAN,
SAS, TCP/IP and InfiniBand
RDMA to transfer data.
IBM and Business Partner Use Only
Storage
Storage Storage
TCP/IP or Infinband RDMA Network
Storage Network
TCP/IP or Infiniband Network
TCP/IP or Infinband Network
NSD
Servers
Application
Nodes
Application
Nodes
Describe Common Spectrum Scale Architectures

Elastic Storage Server Architectures
• These will be covered in detail on Day 2

Describe the Value of Spectrum Scale solutions
For enterprises that are swamped by unstructured data IBM Spectrum Scale software lets you share
the storage infrastructure while it automatically moves file and object data to the optimal storage
tier as quickly as possible, i.e. move data automatically between flash, disk and tape.
Spectrum Scale enjoys highly differentiated value:
 Virtually limitless scaling to 9 quintillion files and yottabytes of data.
 Provides high performance - over 400 GB/s - and simultaneous access to a common set of shared data.
 Potential for Over 400 GB/s performance.
 Software Defined Storage lets you build your infrastructure your way.
 Easy to scale with relatively inexpensive commodity hardware while maintaining world Class storage
management capabilities.
 Use any combination of flash, spinning disk and tape.
 Use a variety of cluster models that include Storage Area Networks (SAN), Network
 Shared Disk and Shared Nothing clusters.
 Add more storage capacity without affecting the application to greatly simplify administration.
 Integrated Life Cycle Management (ILM) Tools automatically move data based on policies.
 This can dramatically reduce operational costs as fewer administers can manage larger storage
infrastructures.
 Provides global data access across geographic distances and unreliable WAN connections.
 Proven reliability with use in the most demanding commercial applications.
 Protects data from most security breaches, unauthorized access, or being lost, stolen, or improperly
discarded with native file encryption for data at rest and secure erase.

Explain System requirements for Spectrum Scale 4.1
installation
Licensing
Spectrum Scale has two types of licenses, a Server license and a Client license (licenses are priced per
processor core). For each node in a Spectrum Scale cluster, the customer determines the appropriate
number of Spectrum Scale Server licenses or Spectrum Scale Client licenses that correspond to the
way Spectrum Scale is used on that node (a node is defined as one operating system instance on a
single computer or running in a virtual partition). For further information, see the related Spectrum Scale
FAQs:
Operating systems support
Spectrum Scale is supported on AIX, Linux, and Windows. Consult the Spectrum Scale FAQ
information center
for the latest list of AIX environments, Linux distributions, Linux kernel versions, OpenSSL
levels, and Windows environments. OS should be installed and properly configured & networked.
Hardware support
Spectrum Scale for POWER® is supported on both AIX and Linux. The Spectrum Scale for Linux
Multiplatform and x86 Architecture products run Linux clusters that are based on selected servers.
Spectrum Scale for Windows Multiplatform is supported on Windows Server 2008 on 64-bit
architectures (AMD x64 / EM64T).
Hardware Readiness
All Hardware must be racked, running a supported operating system, and fully networked. All
Storage should be racked, and connected or network connected to the servers for use by
Spectrum Scale.
Software Packages Readiness
Spectrum Scale Software packages to support the above mentioned Server and OS
architectures should be downloaded and available for install.

Describe the Installation checklist
1. Server installations are complete
2. Network installations are complete
3. Routing, DNS, NTP, Authentication are complete
4. Storage hardware is installed and accessible by the Servers
5. OS is registered and configured
6. Spectrum Scale Licenses are registered and acquired
7. Spectrum Scale software is available
8. Installation Guide is available for reference

Pre-install steps: Operating system configuration
• Use the FAQ section of the Spectrum Scale documentation to
verify
– OS being installed to ensure that it is supported
– Prerequisite software is installed.
• Get nodes properly on the network(s)
– Ensure that the hostnames and IP addresses are correct and correctly
recorded in DNS entries and/or in hosts entries.
• Verify node-to-node rsh/ssh communications
– rsh/ssh without password challenge over the node’s hostname as
well as fully qualified domain name, including rsh/ssh to self
– Disable login banners (touch .hushlogin)
– #1 cause of installation issues.
• Synchronize all node clocks [ not required, but recommended]
– Use ntp (ntpdate <time_server>)

Pre-install steps: Storage configuration
• Attach disks to nodes
– SAN model all nodes are attached to disk.
– NSD model only NSD servers have disk access.
– Mixed Model: Some NSD and some direct attached application nodes.
• Verify drivers, firmware, and other software levels as
recommended by the storage manufacturer are used.
– Array configuration
• Plan RAID stripe size versus file system blocksize.
– All these steps are independent of Spectrum Scale.

Explain IBM and Client Responsibilities
IBM and Business Partner Use Only
IBM Provides the solution to integrate with your Network Requirements
IBM Lab Services provides the install and base file system creation
IBM Lab Services provides Initial Knowledge transfer & documentation
IBM Pre Sales provides guidance on new workload integration sizing and solution consult
IBM Provides Support and maintenance
The Client Responsibilities:
- Red Hat Updates and peripheral management and updates
- 3rd Party Software Integration, Management and Maintenance
- 3rd Party Hardware Integration, Management and Maintenance
- Spectrum Scale Client License management and ongoing Spectrum Scale Cluster Management
Common Maintenance Practices:
- Red Hat Updates
- File System or File Set creation
- Export Management
- File System Replication
- Solution feature management
- Capacity Monitoring
- Support engagement
- Cluster Upgrade Scheduling
- Authentication Service Management
IBM will remain accessible as your trusted solution advisor

List the Basic Software packages required
This depends on whether you are installing Spectrum Scale or Elastic
Storage Server As Elastic Storage Server requires additional packages for
the UI and GNR software.
Bellow we will focus on the packages required for installing a Basic Spectrum
Scale Cluster on a VMware based lab (as we have in our demos today).

Installation: Steps to installing a Spectrum Scale
system
Installation checklist
1. Get nodes properly installed
2. Install Spectrum Scale code
3. Create the Spectrum Scale cluster
4. Start Spectrum Scale and verify the status of all nodes
5. Create NSDs
6. Create file system

Installation: Steps to installing a Spectrum Scale
system
• Basic installation checklist
1. Get nodes properly installed
2. Install Spectrum Scale code
3. Create the Spectrum Scale cluster
4. Start Spectrum Scale and verify the status of all nodes
5. Create NSDs
6. Create file system

Installation: Install the Spectrum Scale code on AIX
• Install Spectrum Scale packages to all nodes in cluster
– Use installp to install the following packages
• gpfs.base
• gpfs.docs.data
• gpfs.msg.en_US
• [[[Add 4.1 Packages ]]]
– Apply Latest Patch Level (PTF)
• Patches are available at “IBM Fix Central”

Installation: Install the Spectrum Scale code on
Linux
• Two steps to the Linux install
– Use RPM to install the packages
• gpfs.base-3.5.0-0.x86_64.rpm
• gpfs.docs-3.5.0-0.noarch.rpm
• gpfs.gpl-3.5.0-0.noarch.rpm
• gpfs.libsrc-3.5.0-0.noarch.rpm
• gpfs.msg.en_US-3.5.0-0.noarch.rpm
• [[[Add 4.1 Packages ]]]
– Build the Spectrum Scale portability layer
• The portability layer allows you to run Spectrum Scale with multiple Linux
Kernel levels.
• Linux environment only.

Spectrum Scale portability layer (GPL)
• This portion of Spectrum Scale code is open source.
• Allows Spectrum Scale to run on many different kernels
– Unique for each kernel and platform.
• Package gpfs.gpl-3.5.0-0.noarch.rpm
• Only for Linux nodes
– mmfslinux portability module.
• Enables communication between Spectrum Scale kernel
modules and the Linux kernel.

Installation: Installing Linux GPL
• Install Spectrum Scale RPM’s
• Building Spectrum Scale portability layer
cd /usr/lpp/mmfs/src/
make Autoconfig
make World
make InstallImages
– Optional method to distribute module to other nodes of same Linux
version: make rpm
– Use the rpm to install on other nodes with the same machine type and
kernel. Output file is created in:
/root/rpmbuild/RPMS/x86_64/gpfs.gplbin-`uname -r`-
<GPFS Version>.<CPU Arch>.rpm

Managing Licenses
• Configure using mmchlicense command
mmchlicense {client| fpo | server} [--accept] -N
{Node[,Node...] | NodeFile | NodeClass}
• View License status
mmlslicense [-L]
© Copyright IBM Corporation 2015© Copyright IBM Corporation 2008
Summary information
---------------------
Number of nodes defined in the cluster:
2
Number of nodes with server license designation:
2
Number of nodes with client license designation:
0
Number of nodes still requiring server license
designation: 0
Number of nodes still requiring client license
designation: 0

Create cluster: Create a Spectrum Scale cluster
• To create a Spectrum Scale cluster:
– Install Spectrum Scale on one or more nodes
– Run the mmcrcluster command
• View the status of a cluster using the mmlscluster
command.
• Two nodes in cluster have an additional responsibility the are
the primary and secondary cluster configuration data servers.
– Note: These nodes are required for configuration changes only.

Create cluster: mmcrcluster
• Usage:
mmcrcluster -N {NodeDesc[,NodeDesc...] | NodeFile}
-p PrimaryServer
[-s SecondaryServer]
[-r RemoteShellCommand]
[-R RemoteFileCopyCommand]
[-C ClusterName] [-U DomainName]
[-A] [-c ConfigFile]
• Nodefile - an input file with a list of node names and
designations.
• Spectrum Scale cluster configuration server node(s)
– Primary and secondary.
• Cluster configuration data is stored in the text file:
/var/mmfs/gen/mmsdrfs
NOTE: Do NOT Edit /var/mmfs/gen/mmsdrfs

Create cluster mmcrcluster: Node descriptor
• Node designation format:
NodeName:NodeDesignations:AdminNodeName
• Can be inserted on the command line or placed in a file.
• NodeName is the hostname or IP address to be used for node-to-
node communication.
• NodeDesignation is an optional separated “-” list of node roles
– Quorum or nonquorum
– Client or manager
– Example: node1:manager-quorum
• Must have at least one quorum and one manager node in the cluster.
• The AdminNodeName is optional
– A DNS name to be used by Spectrum Scale Admin commands
– The default value of AdminNodeName is NodeName.

Create cluster mmcrcluster: config file
• The config file:
– Contains subset of parms in sample file
/usr/lpp/mmfs/samples/mmfs.cfg.sample
– For example, pagepool, maxblocksize, maxFilesToCache
and maxStatCache.
• Used for:
– Creating similar clusters
– Test clusters
– Disaster recovery.

Spectrum Scale systems administration: Adding
node(s)
• Command:
mmaddnode -N {NodeDesc[,NodeDesc...] | NodeFile}
– Must have root authority
– May be run from any node in the Spectrum Scale cluster
– Ensure proper authentication (.rhosts or ssh key exchanges)
– Install Spectrum Scale onto new node
– Decide designation(s) for new node, for example, Manager and Quorum.

Spectrum Scale systems administration: Install
license
• Configure using mmchlicense command
mmchlicense {client| fpo | server} [--accept] -N
{Node[,Node...] | NodeFile | NodeClass}
• View License status
mmlslicense [-L]
© Copyright IBM Corporation 2015© Copyright IBM Corporation 2008
Summary information
---------------------
Number of nodes defined in the cluster: 2
Number of nodes with server license designation: 2
Number of nodes with client license designation: 0
Number of nodes still requiring server license designation: 0
Number of nodes still requiring client license designation: 0

Create cluster system directories used by
Spectrum Scale
•/usr/lpp/mmfs - Spectrum Scale installation directory
– /usr/lpp/mmfs/bin – Location for Spectrum Scale executables (binaries and
scripts)
– /usr/lpp/mmfs/src – Location of the Spectrum Scale portability layer source
(Linux only)
•/var/mmfs - Spectrum Scale config data directory
– /var/mmfs/gen – Location for critical Spectrum Scale config data
– /var/mmfs/etc – Location for Spectrum Scale specific user scripts and cluster
specific custom config files (mmfs.cfg and cluster.preferences)
[mmchconfig]
•/var/adm/ras – System logs
• Windows root directory
– C:windowsSUA (pre-GPFS 4.1)
– SUA is no longer supported for Windows nodes in Spectrum Scale 4.1
– SUA in Spectrum Scale 4.1 is replaced by Cygwin and is required prior to install
on Windows nodes

Create cluster mmlscluster
# mmlscluster
GPFS cluster information
========================
GPFS cluster name: rr_beta.ppd.pok.ibm.com
GPFS cluster id: 13882466803121521035
GPFS UID domain: rich_domain
Remote shell command: /usr/bin/ssh
Remote file copy command: /usr/bin/scp
GPFS cluster configuration servers:
-----------------------------------
Primary server: c35f1n01.ppd.pok.ibm.com
Secondary server: c35f1n03.ppd.pok.ibm.com
Node Daemon node name IP address Admin node name Designation
-----------------------------------------------------------------------------------------------
1 c35f1n01.ppd.pok.ibm.com 192.168.110.1 c35f1n01.ppd.pok.ibm.com
4 c13sun16g.ppd.pok.ibm.com 192.168.120.16 c13sun16g.ppd.pok.ibm.com quorum-manager

Create cluster mmlsconfig
# mmlsconfig
Configuration data for cluster rr_beta.ppd.pok.ibm.com:
-------------------------------------------------------
clusterName rr_beta.ppd.pok.ibm.com
clusterId 13882390374179224464
clusterType lc
autoload no
useDiskLease yes
maxFeatureLevelAllowed 906
pagepool 256M <<< changed from default
[node1] <<< Node specific
maxMBpS 5000
File systems in cluster rr_beta.ppd.pok.ibm.com:
------------------------------------------------

Create cluster: Change Spectrum Scale cluster
options
• Use the mmchcluster command to change cluster wide
settings
– Examples:
• Cluster name
• Change RemoteShellCommand and/or RemoteFileCopyCommand
settings.
• Use the mmchconfig to set or change other Spectrum Scale
daemon options.

Installation: Start Spectrum Scale cluster:
mmstartup
• Start Spectrum Scale and verify all nodes join
Usage: mmstartup [-a | -N {Node[,Node...] | NodeFile |
NodeClass}] [-E EnvVar=value ...]
– Three scopes for command:
• Individual node mmstartup -N node1
• Some nodes mmstartup -N {Node[,Node..] | NodeFile | NodeClass}
• Entire cluster mmstartup -a
• Verify cluster state using the mmgetstate command:
# mmgetstate -a
• Node states are:
– Active: Up and ready for business
– Arbitrating: A node is trying to form quorum with the other available nodes.
– Down: Spectrum Scale daemon is not running on the node.
– Unknown: Unknown value. Node cannot be reached or some other error
occurred.

mmgetstate
• Usage: mmgetstate [-L] [-s] [-v] [-a | -N
{Node[,Node...] | NodeFile | NodeClass}]
# mmgetstate –a -L -s
Node number Node name Quorum Nodes up Total nodes GPFS state Remarks
------------------------------------------------------------------------------------
1 c5plnx101e 3 3 7 active quorum node
3 c5plnx103e 0 0 7 down
4 c5plnx104e 3 3 7 active
7 c5plnx107e 0 0 7 down
Summary information
---------------------
Number of nodes defined in the cluster: 6
Number of local nodes active in the cluster: 4
Number of remote nodes joined in this cluster: 0
Number of quorum nodes defined in the cluster: 3
Number of quorum nodes active in the cluster: 3
Quorum = 3, Quorum achieved

Create cluster view active configuration: mmfsadm
• Get Spectrum Scale version
– # mmdiag --version
=== mmdiag: version ===
Current GPFS build: “4.1.0.2 ".
Built on Sep 9 2014 at 15:34:22
Running 46 minutes 22 secs
• To view the values of all the active configuration parameters use the mmfsadm
command
– # mmdiag --config
=== mmdiag: config ===
allowDeleteAclOnChmod 1
assertOnStructureError 0
cipherList EMPTY
! clusterId 661835417728118983
! clusterName perf7-c3-int64.fvtdomain.net
consoleLogEvents 0
<…snip…>

Spectrum Scale logging
• Logs all Spectrum Scale startup, shutdown, mount,
configuration changes and errors
• Location: /var/adm/ras
• Cleared every 10 days or 10 files
•TimeStamped according to Spectrum Scale startup
• Two linked files:
– /var/adm/ras/mmfs.log.latest  latest Spectrum Scale
startup
– /var/adm/ras/mmfs.log.previous  previous Spectrum Scale
startup
• With Spectrum Scale 4.1, Spectrum Scale can now log specific
messages to the syslog on Linux
– The mmchconfig command controls what is sent to the syslog© Copyright IBM Corporation 2015

Spectrum Scale log Example
# cat /var/adm/ras/mmfs.log.latest
Fri Jan 13 18:04:24 EST 2014 runmmfs starting
Removing old /var/adm/ras/mmfs.log.* files:
./mmfs.log.2014.11.22.13.47.39.c5an02
./mmfs.log.2014.11.28.14.54.39.c5an02
./mmfs.log.2014.12.02.14.28.30.c5an02
./mmfs.log.2014.12.01.16.36.06.c5an02
Unloading modules from /usr/lpp/mmfs/bin
Loading modules from /usr/lpp/mmfs/bin
Module Size Used by
mmfslinux 170404 1 mmfs
tracedev 15796 2 mmfs,mmfslinux
Removing old /var/mmfs/tmp files:
./complete.map.2014.12.08.10.59.45.c5an02
Fri Jan 13 18:04:27 2014: mmfsd initializing. {Version: 4.1.0.1 Built: Aug 17 2014
16:06:05} ...
Fri Jan 13 18:04:28 2014: Connecting to 192.168.11.3 c5an03g
Fri Jan 13 18:04:28 2014: Connected to 192.168.11.3 c5an03g
Fri Jan 13 18:04:28 2014: Connecting to 192.168.11.7 c5an07g
Fri Jan 13 18:04:28 2014: Connected to 192.168.11.7 c5an07g
Fri Jan 13 18:04:28 EST 2014 /var/mmfs/etc/gpfsready invoked
Fri Jan 13 18:04:28 2014: mmfsd ready

Installation: Create Spectrum Scale disks
• NSD stands for Network Shared Disk
• The term NSD is used two places in Spectrum Scale
– NSD for defining a disk
– NSD protocol for network access to lun (More later).
• Steps to define an NSD
– Choose a node that has direct access to the storage.
– Identify the disks to be used by Spectrum Scale
• On AIX you can use lscfg and look for hdisks
• On Linux you can run cat/proc/partitions
• This may vary based on storage type.
– Choose a storage type for each disk:
• dataOnly, metadataOnly, dataAndMetadata, descOnly or
localCache.

Installation: Create Spectrum Scale disks
• Use the mmcrnsd command to create the NSD’s
– Create a NSD StanzaFile (latest format)
• All of the disk create commands use StanzaFile as an input file. This file
describes disk/node/usage relationships.
– Create backup copy of the StanzaFile file
– Execute the mmcrnsd command
Usage: mmcrnsd -F StanzaFile [-v {yes | no}]
– Verify the NSD’s were created using mmlsnsd command

mmcrnsd: Input StanzaFile file
• Upon successful completion of the mmcrnsd command, the
StanzaFile file is rewritten to reflect changes made by the
command, as follows:
– If an NSD stanza is found to be in error, the stanza is commented out.
– If an nsd=NsdName clause is not specified, and an NSD name is
generated by Spectrum Scale, an nsd= clause is inserted into the
corresponding stanza.
• You must have write access to the directory where the
StanzaFile file is located in order to rewrite the created NSD
information.

mmcrnsd: Example input StanzaFile file
#Format
# %nsd: device=DiskName
# nsd=NsdName
# servers=ServerList
# usage={dataOnly | metadataOnly |
dataAndMetadata | descOnly}
# failureGroup=FailureGroup
# pool=StoragePool
%nsd: device=/dev/vpatha
nsd=mynsd_1
servers=node1,node2
usage=dataAndMetadata
%nsd: device=/dev/vpathb
nsd=mynsd_2
servers=node2,node1
usage=dataAndMetadata

View NSD information: mmlsnsd
•mmlsnsd command:
Usage:
mmlsnsd [-a | -F | -f Device | -d
"DiskName[;DiskName...]"]
[-L | -m | -M | -X] [-v]
• Some option descriptions
-m: Map the NSD name to its disk device name in /dev on the local
node and, if applicable, on the primary and backup NSD server nodes.
-M: Map the NSD names to its disk device name in /dev on all nodes.
This is a slow operation and its usage is suggested for problem
determination only.
-X: Map the NSD name to its disk device name in /dev on the local
node and, if applicable, on the NSD server nodes, showing the
information with extended information in the NSD volume id and the
remarks fields. This is a slow operation and is suggested only for
problem determination.

mmlsnsd: List NSDs (after filesystem creation)
# mmlsnsd -f gpFSreef -m
Disk name NSD volume ID Device Node name Remarks
----------------------------------------------------------------------------------
mynsd_1 0972858243664131 /dev/vpatha node1.ppd.pok.ibm.com server node
mynsd_1 0972858243664131 /dev/vpathf node2.ppd.pok.ibm.com server node
mynsd_2 0972858243664134 /dev/vpathb node1.ppd.pok.ibm.com server node
mynsd_2 0972858243664134 /dev/vpathg node2.ppd.pok.ibm.com server node
mynsd_3 0972858243664137 /dev/vpathf node1.ppd.pok.ibm.com server node
mynsd_3 0972858243664137 /dev/vpathk node2.ppd.pok.ibm.com server node
mynsd_4 0972858743664143 /dev/vpathd node1.ppd.pok.ibm.com server node
mynsd_4 0972858743664143 /dev/vpathh node2.ppd.pok.ibm.com server node

Installation: Creating a file system
• Before creating a file system you need to determine:
1. Which NSD’s are to be used for the file system
2. Storage pool configuration
3. File system block size
4. Replication plan and failure group configuration

Installation: Create a file system using mmcrfs
• Use the StanzaFile (after mmcrnsd) as input to the mmcrfs command.
• The mmcrfs command
– Usage:
mmcrfs Device {"DiskDesc[;DiskDesc...]" | -F StanzaFile}[-A {yes | no | automount}]
[-B BlockSize] [-D {posix | nfs4}][-E {yes | no}] [-i InodeSize]
[-j {cluster | scatter}][-k {posix | nfs4 | all}]
[-K {no | whenpossible | always}][-L LogFileSize]
[-m DefaultMetadataReplicas][-M MaxMetadataReplicas] [-n NumNodes]
[-Q {yes | no}][-r DefaultDataReplicas] [-R MaxDataReplicas]
[-S {yes | no | relatime}] [-T Mountpoint] [-t DriveLetter][-v {yes | no}]
[-z {yes | no}] [--filesetdf][--inode-limit
MaxNumInodes[:NumInodesToPreallocate]]
[--metadata-block-size MetadataBlockSize][--mount-priority Priority]
[--perfileset-quota][--version VersionString]
[--version VersionString]
• Decide on mount point and file system block size.
• Parameters that cannot be changed once the file system is created:
– File System Blocksize (-B)
– Maximum data and metadata replication settings (-R, –M)
– Number of nodes cannot be changed for a storage pool (-n).

Installation: Create Spectrum Scale file system
• Issue the mmcrfs command combining options
– Example:
mmcrfs fs1 -F StanzaFile –B 1M –T /fs1
– This command will format the disks for use by Spectrum Scale and
write a disk descriptor file to each disk in the file system.
• To see file system characteristics:
– Mount on all nodes
mmmount fs1 –a
– View the parameters for a file system
mmlsfs fs1
– View the disks in a file system
mmlsdisk fs1

mmlsfs: List file system parameters
# mmlsfs gpFSreef
flag value description
---- -------------- -----------------------------------------------------
-s roundRobin Stripe method
-f 32768 Minimum fragment size in bytes
-i 512 Inode size in bytes
-I 32768 Indirect block size in bytes
-m 2 Default number of metadata replicas
-M 2 Maximum number of metadata replicas
-r 1 Default number of data replicas
-R 2 Maximum number of data replicas
-j cluster Block allocation type
-D posix File locking semantics in effect
-k posix ACL semantics in effect
-a 1048576 Estimated average file size
-n 32 Estimated number of nodes that will mount file system
-B 1048576 Block size
-Q none Quotas enforced
none Default quotas enabled
-F 786438 Maximum number of inodes
-V 8.01 File system version. Highest supported version: 8.02
-u yes Support for large LUNs?
-z no Is DMAPI enabled?
-E yes Exact mtime default mount option
-S no Suppress atime default mount option
-d c5an02_r1_1;c5an02_r1_2;c5an02_r1_3;c5an02_r1_4;c5an07_r1_1;c5an07_r1_2;c5an07_r1_3;c5an07_r1_4
Disks in file system
-A no Automatic mount option
-o none Additional mount options

Mount and unmount a file system
• Commands work for local or remote clusters
• mmmount
Usage:
mmmount {Device | DefaultMountPoint | DefaultDriveLetter | all |
all_local | all_remote}
[-o MountOptions] [-a | -N {Node[,Node...] | NodeFile |
NodeClass}]
or
mmmount Device {MountPoint | DriveLetter} [-o MountOptions]
[-a | -N {Node[,Node...] | NodeFile | NodeClass}]
• mmumount/mmunmount
Usage:
mmunmount {Device | MountPoint | DriveLetter | all | all_local |
all_remote} [-f]
[-a | -N {Node[,Node...] | NodeFile | NodeClass}]
or
mmunmount Device -f -C {all_remote | ClusterName} [-N Node[,Node...]]

Controlling file system mounts
• Do not mount any file systems on startup
touch /var/mmfs/etc/ignoreStartupMount
• To prevent all startup mounts or just particular file systems
touch /var/mmfs/etc/ignoreStartupMount.<devicename>
• Example if you have file systems fs1 and fs1 and you do not
want to mount fs2 on this node:
touch /var/mmfs/etc/ignoreStartupMount.fs2

Controlling file system mounts: Mount options
• If you want to set special mount options on a node
• Create the definition file
/var/mmfs/etc/localMountOptions.fs1
• File Contains options, to mount read only the file would contain
one line
ro

Installation: Stopping Spectrum Scale
• Usage
mmshutdown [-t UnmountTimeout] [-a | -N {Node[,Node...] | NodeFile | NodeClass}]
• Shutdown Spectrum Scale use the mmshutdown command
– Three scopes for command:
• Individual node mmshutdown –N node1
• Some nodes mmshutdown –N {Node[,Node...] | NodeFile | NodeClass}
• Entire cluster mmshutdown -a
– Results in all Spectrum Scale file systems being unmounted and
Spectrum Scale daemons stopped.

mmshutdown
# mmshutdown
Sun Jan 15 01:57:05 EST 2014: mmshutdown: Starting force unmount of
GPFS file systems
cat /var/adm/Sun Jan 15 01:57:10 EST 2014: mmshutdown: Shutting down
GPFS daemons
Shutting down!
'shutdown' command about to kill process 30142
ras/mmfs.log.l Unloading modules from /usr/lpp/mmfs/bin
Unloading module mmfs
Unloading module mmfslinux
Unloading module tracedev
Sun Jan 15 01:57:18 EST 2014: mmshutdown: Finished
From GPFS Log (/var/adm/ras/mmfs.log.latest) :
Sun Jan 15 01:57:10 2014: mmfsd is shutting down.
Sun Jan 15 01:57:10 2014: Reason for shutdown: Normal shutdown
/var/mmfs/etc/mmfsdown.scr: Subsystem: mmfs Status: inoperative
/var/mmfs/etc/mmfsdown.scr: /bin/umount -a -f -t gpfs

Review
• We have learned:
– How to verify the environment is ready for Spectrum Scale
– What software is required to install Spectrum Scale
– How to create a Spectrum Scale cluster using mmcrcluster
– How to create a Network Shared Disk (NSD)
– How to create a Spectrum Scale file system

Exercise 1
Spectrum Scale Installation and
Setup
Exercise

Unit summary
Having completed this unit, you should be able to:
• Installation checklist
• System requirements for installation
• Software packages required
• Creating a cluster
• Creating Network Shared Disks (NSD)
• Creating a file system
• Mounting a file system

Ibm spectrum scale fundamentals workshop for americas part 1 components architectures-n-install

In this document

More Related Content

What's hot

Similar to Ibm spectrum scale fundamentals workshop for americas part 1 components architectures-n-install

More from xKinAnx

Recently uploaded

Ibm spectrum scale fundamentals workshop for americas part 1 components architectures-n-install