Introduction to IBM Tivoli Storage Manager

1. INTO “ TSM ”
About this …….
This Document is like an entry to the world of Backup storage Management of Tivoli it is not an
IBM document it is collective document from internet articles and presentations of basics and
concepts of the product of TSM, so this is like basics only but for administration you must use
the IBM administration documents which are downloadable from IBM website.
History
We must mention and give credit to the main company which made this storage product it was
not IBM it was in the name “ADSM” Adstar Distributed Storage Manager till IBM took it over
and enhance it, so you can notice that in the main core of the configuration and commands you
will find the name DSM.
Introduction
IBM TSM is a client /server program that provides centralized automated data protection and
storage management solutions in a multivendor computer environment
IT has various versions but the real transition occurred between 5.x to 6.x since the metadata
limit rose from 530 GB to unlimited and our most concern is with version 6.4 as our company
A.T.Lease uses now this version as server and Node clients. Before getting the real structure
and management we need first to know some important terminologies and definitions which
are used in policies and backup strategy.
1 | P a g e

2. Basic concepts
Backup – Restore: “This is the Type where ATLEASE rely on”
The backup process copies data from client node to TSM server storage. The server retains
versions of a file according to policy, and replaces older versions of the file with newer versions.
The first backup of any client is considered a "full" backup as TSM is considered a forever
incremental backup solution. During the first backup the entire OS is scanned and sent to the
TSM server storage. During the next backup TSM again scans the entire OS for changes and this
time only backs up the files that have changed from the previous backup [incrementally on a
day‐to‐day basis].
Archive – Retrieve: “Not used in ATLEASE “
The archive process creates a copy of files and stores them for a specific time. Typically used for
maintaining copies of vital records for legal or historical purposes. In the past backup and
archive were interchangeable however within TSM they have become separate functions. Each
object archived regardless of versioning will have its own entry in the TSM database causing
bloat of the database and effecting performance. Typical running a backup with a longer
retention period [via a management class] will achieve long term storage without bloating the
TSM server database.
Versioning:
Versioning are multiple copies of same file in chronology, so as to explain in it easy way follow
the below scenario:
During backup
• If no corresponding object on the server, new object becomes the active version
• If corresponding object already exists on the server, new object becomes active version
and the existing active version is deactivated
• Extraneous versions are marked for expiration (base date set to 0)
• The number of allowed versions of a backup object is determined by the object's
management class and copy group
2 | P a g e

3. Basicconcepts…..
e a
p
m
m
e
o
s
o
e u
Retention policy:
IT is simply how many days you want to keep copies OR how many copies you want to retain
The final retention depends upon whichever condition of the above is satisfied
The default is 30 days for backup, 1 year for archive and default no. of versions is 4
Expiration:
Versions keep changing states: active Inactive  obsolete
You need to start a process to remove pointers of obsolete files in Meta data DB whereas this
process is called Expiration
Reclamation: “like defrag entation in windows”
Once you have expired files fro your sequential devic s the space has invalid data ,so to se it
you hav to move d ta from on / or more equential devices to other devices
The below figure ex lains the n rmal cycle f backup in TSM and it will demonstrate the last
definitions from backup, expiration and reclamation.
3 | P a g e

4. Basicconcepts…..
Collocation:
In order to reduce no. of sequential devices you need to collect data, it is also needed to avoid
reclamation situations
Collocation types (by node, by group, by application)
Data Migration Hierarchy:
1‐ RANDOM STORAGE DEVICES.
2‐ RANDOM STORAGE DEVICES / SAN / NAS.
3‐ SEQUENTIAL STORAGE DEVICES.
Data Migration:
So data can be migrated from random to random or random or sequential but cannot vice versa
(sequential cannot migrate to random).
Data migration can be triggered and stopped by these two values:
‐ High threshold value (default is 90% )
‐ Low Threshold value (default is 70% )
And here comes the storage Pool Role and why it is the main core in TSM
Storage hierarchy exploits attributes of different device types and increase:
‐ Performance, Concurrent access and Cost
‐ As well as the ability to remove media for remote storage
Organization of storage into storage pools supports automatic, policy based
Management of stored objects
4 | P a g e

5. Let’s Start….
After the last introduction of the basic concepts and definitions of the TSM environment, now it
is time to wrap up our sleeves and start to the real work and we will start with big stone TSM
server from architecture, components and files as well as the ATLEASE structure and how to
access the server then we will take about the storage solution “Tape library” after that we will
cover the TSM client and policies then at the end with Troubleshooting.
The Main Points in the next section:
I‐ TSM Server
II‐ Storage pool
III‐ TSM Client
IV‐ policy
V‐ Troubleshooting
But before we start lets mention the main advantages and why using this complicated solution before
getting into details
TSM Pros:
• Automated, policy‐driven storage management
• Distributed, heterogeneousclients
• Centralizedstorage‐managementservers
• Support for numerous storage devices
• Comprehensivestorage‐managementfunction
‐ Backup/recovery ‐Archive/retrieve
‐ Space management (HSM)
‐ Data protection for specific applications
‐ Application Programming Interface (API)
‐ Contentmanagement
5 | P a g e

6. I‐TSM SERVER
TSMComponents:
• Storage pools
• Meta data DB
It contains the following (Object ID , index, version, status of backup, time stamp , users &
groups ,file space name , etc..)
• Configured Settings … (Policy, schedule, etc..)
• Recovery logs
TSM server Important Files:
• Dsmserv.exe
• Dsmserv.dsk (FQN,logv,dbvinfo)
• Init serv.log(server installation logs)
• Devcnfg.out (device class info , info of DB bkups, export info)
• Vollist.out(volume library info , info of DB bkups export info)
• Dsmserv.out( parameters ,settings for instance / server)
6 | P a g e

7. I‐TSMSERVER…….
Methods of accessing TSM server:
1‐ TSM GUI.
a. Web console.
b. ISC Admin. (Integrated Solution console)
Logon to the TSM server “192.168.254.7” open any browser preferred Mozilla
https://localhost:16311/ibm/console/logon.jsp
Username: tipadmin
Password: password
2‐ Administrative command line.
To open the Administrative command line go to All Programs‐>Tivoli storage manager ‐>
Administrative command line.
Username: admin
Password: admin
7 | P a g e

8. II‐Storage Pool
TSM Storage solution: “Storage Pools”
In ATLEASE we use Tape Library where the TSM integrate with it and assign volumes to storage
pools which we create in the server as example for that we need to make daily backup for
certain client servers “ nodes “ (file server , exchange , SQL , etc..)
So first of all we make a daily policy then make a storage pool for it as example daily storage
pool and TSM interact with library and put volumes scratch tapes and so on
8 | P a g e

9. II‐StoragePool…..
How to access storage pool
Go to the web console  storage devices view storage devices
We have in ATLEASE 10 storage pools in which in our environment we make a storage pool for
each policy, and we don’t control the amount of tapes or volumes we leave it to the TSM to
integrate automatically with Tap Library
9 | P a g e

10. III‐TSM Client
Before we start with TSM Client we need to know these definitions and terms to give us better estimate
Node:
A node is a container within the TSM Backup system that holds all of the data you back up. This
Includes all file spaces, which in turn, includes active & inactive backed‐up files, and any
archives you’ve created. Example (File server daily Node Incremental Bkup)
Scheduler:
A component of the TSM Client that is installed on your system that transfers your files to the
TSM backup system at the scheduled time.
Options file:
The component of the TSM Client that defines the scope & behavior of your backups & other
Settings (dsm.opt).as example for that dsm.opt of file server
Active File:
A file that still exists on your system that is within the scope of your scheduled backups. The
Most recent version of active files is always available for restore, regardless of the retention
Period
Inactive File:
A formerly active file that is either deleted from your system, or is now outside of the scope of
Your scheduled backups (due to changes to your INCLUDE / EXCLUDE statements in your
options file). An active file cannot become inactive until an incremental backup occurs that
notices the change.
File Space:
A container within the TSM Backup system that holds all active & inactive files, and any
archives you've created, for a particular drive or partition on your system. File spaces are
unique per machine name and require manual deletion if you ever want to delete them.
10 | P a g e

11. III‐TSM Client……
BA Client Important files:
• dsm.opt(node name, server address server TCP port and other configurable parameters)
• dsm.exe
• dsmc.exe
• dsmadmc.exe (administrative utility)
ATLEASE TSM Backup System Overview
Each of these servers are client agents for the TSM server but in fact it act as more than 1 agent
As example if we take the File server it Backup archive client but not one node it has 4 nodes
a‐ Daily node
b‐ Weekly node
 Incremental node attached with the daily policy to take same
Retention and it is the daily‐Tape‐pool.
 Full backup node attached with weekly policy
c‐ Monthly node Full backup node attached with monthly policy
d‐ Yearly node  Full backup node attached with yearly policy
So each node has separate option file, separate schedule to run in, separate retention policy
And stored in different storage pool according to the policy which lay in.
11 | P a g e

12. III‐TSM Client……
From the TSM perspective there are different types of clients BA client as example the File
server and TDP client as oracle server. And each has its usage and way of backup so below will
show the difference between BA client and TDP.
1‐ backup/archive client (BA Client)
The Tivoli Storage Manager backup/archive client is designed to back up and restore, archive, and
retrieve client file system data. Therefore, the client can back up any non‐database and database
files. Tivoli Storage Manager Clients use standard operating system functions to access files
within file systems, but they are not required to understand any logical structure that might exist
Within a file. This is not a limitation specific to Tivoli Storage Manager; all other client backup
products that work at the file system level operate similarly.
The way TSM sees Oracle and other database systems affects how they are backed up. Each
database appears as an individual file on the server or client file systems. A Tivoli Storage
Manager backup/archive client running on an.
Oracle server or client can back up and restore, archive, and retrieve entire Oracle databases. It
cannot back up smaller increments such as tables. Other than the issues of size and replication,
using a Tivoli Storage Manager backup/archive client to back up Oracle databases is
straightforward. Each database is a self‐contained data file that is backed up and restored.
Tivoli Storage Manager restores a database in its entirety because it is just a file for TSM.
If a database is deleted or corrupted, it is a simple task for Tivoli Storage Manager to restore the
most recent or any previous backup version of this database from the Tivoli Storage Manager
server to the Oracle server or client.
The TSM backup/archive client, however, does not meet all requirements for an ideal storage
management solution in Database environment.
Following are some of the drawbacks you might experience when using the TSM backup/archive
client for Oracle as example:
• Consider a 5 GB database those changes every day. The TSM backup‐archive client will back up
the full 5 GB even if only a 2 MB document has changed. You can waste a lot of time and storage
Space using this strategy.
• Many databases need to operate twenty four hours a day, seven days a week, so they are in use
all the time and a consistent backup cannot be taken. The alternative is to quiescent the Oracle
database and take backups, but this would result in server unavailability, which is not good for
Business.
12 | P a g e

13. III‐TSM Client……
2‐ Tivoli Data Protection: (TDP)
The TDP for Oracle application client provides an integrated solution for performing full backup
and restore operations on Oracle databases. It is a client application that provides full backup of
online databases and restore of full databases to the original or different location. It is
supported on these platforms:
• Microsoft Windows NT and Windows 2000 (on Intel)
• IBM AIX
• HP‐UX
• Sun Solaris
TDP for Oracle is not intended as a substitute for the standard TSM backup/archive client. TDP
for Oracle cannot be used to back up or restore any non‐database data, such as history files or
any other system configuration files. Those files need to be backed up by the Tivoli Storage
Manager backup/archive client. Therefore, the two client types work together to provide full
data protection for your Oracle environment. The TDP for Oracle application client and the
Tivoli Storage Manager backup/archive client can run simultaneously on the same Oracle
server, however, they are totally separate clients as far as the Tivoli Storage Manager server is
concerned.
TDP for Oracle and RMAN
The Oracle Recovery Manager (RMAN) provides consistent and secure backup, restore, and
recovery performance for Oracle databases. While the Oracle RMAN initiates a backup or
restore, TDP for Oracle acts as the interface to the TSM server (Version 3 or above). The TSM
server then applies administrator‐defined storage management policies to the data.
• TDP for Oracle provides the following actions and operations:
• Full back up function for the following while online or offline:
‐ Databases
‐ Table spaces
‐ Data files
‐ Archive log files
‐ Control files
• Full database restore while offline
• Table space and data file restore while online or offline
• Change of TDP client password
13 | P a g e

14. o
m
y
u
m
o
u
a
o
p
n
m
e
IV‐POLICY
PolicyComponents:
• Versioningdetails
• Retention details
• Schedule details
• Storage pools
• Backup‐archivedetails
Policy domains and policy sets
‐ A policy domain is a group of clients with similar requirements for backing up and
archiving data.
‐ Policy domains contain one or more policy sets. An administrator uses policy
domains to manage a group of client nodes in a logical way.
For example, a policy domain ight include:
• A department, physical location, etc..
• In our case in ATLEASE we make the design according to the retention and repetition
time example (Daily – Weekly – Quarterly)
Tivoli St rage Mana er includes a default p licy domain named Standard. At first, your client
node might be associated with the default policy domain. However, your admi istrator can
define additional policy domains if there are groups of sers with unique backup and archive
require ents.
A policy set is a group of one or more management cl sses. Each olicy domain can hold
many policy sets. The administrator uses a policy set to implement different anagement
classes based on b siness and user needs. Only one of these policy sets can b active at a
time. This is called the active policy set. Each policy set contains a default management class
and an number of additional management classes.
Management Class:
A container within the TSM Backup system that defines data retention & number of versions.
There can be, and usually is, different behavior for active & inactive files. Backed‐up files &
archives are "bound" to a management class that defines this behavior.
14 | P a g e

15. IV‐POLICY……
The Normal Work flow of the backup process including server/client and apply policy
The steps are as follows:
1‐ A client initiates a backup, archive, or migration operation. The file involved in the
operation is bound to a management class. The management class is either the default
or one specified for the file in client options (the client's include‐exclude list).
2‐ If the file is a candidate for backup, archive, or migration based on information in the
management class, the client sends the file and file information to the server.
3‐ The server checks the management class that is bound to the file to determine the
destination, the name of the TSM storage pool where the server initially stores the file.
For backed‐up and archived files, destinations are assigned in the backup and archive
copy groups, which are within management classes. For space‐managed files,
destinations are assigned in the management class itself. The storage pool can be a
group of disk volumes, tape volumes, or optical volumes.
4‐ The server stores the file in the storage pool that is identified as the storage destination.
5‐ The Tivoli Storage Manager server saves information in its database about each file that
it backs up, archives, or migrates.
15 | P a g e

16. IV‐POLICY……
Now we need to map this to ATLEASE environment so let’s follow the below scenario where we
take one of the policies check concerning nodes
So in the daily policy we put the nodes to apply on them same policies and rules ( retention
versions, expiration) as example the Daily policy it contains (exchange daily , exchange Saturday
full, SQL,MBX,WEB)
And this policy is linked with one storage pool Daily tape Pool in the tape library
In ATLESAE we work with default management class in all our domain policies the Standard
And as the picture below you can manage the backup settings for the policy
16 | P a g e

17. V‐Troubleshooting
Before getting in to troubleshooting and check the return codes there is a standard way of
problem solving in IBM or sending service request (SR)
IBM Problem Solving (SR)
Determining the business impact
When you report a problem to IBM, you are asked to supply a severity level.
Therefore, you must understand and assess the business impact of the problem
You are reporting.
Describing the problem and gathering background information
When explaining a problem to IBM, it is helpful to be as specific as possible. Include all relevant
background information so that IBM Software Support specialists can help you solve the
problem efficiently.
To save time, know the answers to these questions:
1‐What software versions were you running when the problem occurred?
2‐Do you have logs, traces, and messages that are related to the problem Symptoms?
3‐Can the problem be re‐created? If so, what steps led to the failure?
4‐ Have any changes been made to the system? For example, hardware, operating System
Networking software, and so on.
5‐Are you using a workaround for this problem? If so, be prepared to explain it
17 | P a g e

18. V‐Troubleshooting……..
The return codes:
The backup‐archive command‐line interface and the scheduler exit with return codes that
accurately reflect the success or failure of the client operation.
Scripts, batch files, and other automation facilities can use the return code from the
command‐line interface. For operations that use the Tivoli Storage Manager scheduler, the
return codes are shown in the output of the QUERY EVENT administrative command.
In general, the return code is related to the highest severity message during the client
operation.
• If the highest severity message is informational (ANSnnnnI), then return code is 0.
• If the highest severity message is a warning (ANSnnnnW), then the return code is 8.
• If the highest severity message is an error (ANSnnnnE or ANSnnnnS), then the return
code is 12.
• The exception to the above rules is warning or error messages that individual files
could not be processed. For such a skipped file, the return code is 4. For cases where
the return code is not 0, you can examine the dsmerror.log file (and, for scheduled
events, the dsmsched.log file).
18 | P a g e

19. V‐Troubleshooting……..
For a description of the return codes and their meanings, see the following table.
Code Explanation
0 All operations completed successfully.
4 The operation completed successfully, but some files were not processed.
There were no other errors or warnings. This return code is very
common.
Files are not processed for various reasons. The most common reasons
are:
‐ The file satisfies an entry in an exclude list.
‐ The file was in use by another application and could not be
accessed by the client.
‐ The file changed during the operation to an extent prohibited by
the copy
8 The operation completed with at least one warning message. For
scheduled events, the status is Completed. Review dsmerror.log (and
dsmsched.log for scheduled events) to determine what warning messages
were issued and to assess their impact on the operation.
12 The operation completed with at least one error message (except for error
messages for skipped files). For scheduled events, the status is Failed.
Review the dsmerror.log file (and dsmsched.log file for scheduled events)
to determine what error messages were issued and to assess their impact
on the operation. As a general rule, this return code means that the error
was severe enough to prevent the successful completion of the operation.
For example, an error that prevents an entire drive from being processed
yields return code 12.
other For scheduled operations where the scheduled action is COMMAND, the
return code is the return code from the command that was executed. If
the return code is 0, the status of the scheduled operation is Completed. If
the return code is nonzero, then the status is Failed. Some commands
might issue a nonzero return code to indicate success. For these
commands, you can avoid a Failed status by wrapping the command in a
script that invokes the command, interprets the results, and exits with
return code 0 if the command was successful (the script should exit with a
nonzero return code if the command failed). Then ask your Tivoli Storage
manager server
administrator to modify the schedule definition to invoke your script
instead of the command.
19 | P a g e

20. V‐Troubleshooting……..
The most important tool in trouble shooting is the query command and here is some important
commands which were given by Mr. Hazem Mohamed ATLEASE IT Administrator
TSM Commands:
audit library LIB1 checkl=b ‐> ( for audit library, Note : please make sure not processes running on
Library )
q pro ‐‐> for query process
q ses ‐‐> for query session
q libvol f=d ‐‐> for query Library status
q vol f=d ‐‐> for query all taps status
q node ‐‐> for query all node and policy
q actlog begindate=‐1 > C:Q_ACTLOG1.txt ‐‐‐‐> ( Export actlog to txt file form TSM command line )
reclaim stgpool storage pool name ‐‐> ( for manual reclamation)
reclaim stgpool monthly‐tape‐pool duration=2
dsmj ‐servername=oracle‐ba‐monthly ‐‐> for take BA backup from oracle application. ( AIX Command )
# ps ‐aef | grep dsmc ‐‐‐> ( check schdual services running or Not "AIX Command ).
# nohup dsmc sched ‐servername=oracle‐ba 2> /dev/null & ‐‐> ( Start up Daily TDP backup inc.
scheduled service ) AIX Command
# nohup dsmc sched ‐servername=oracle‐ba‐monthly 2> /dev/null & ‐‐> ( Start up Expire Monthly
backup scheduled service ) AIX Command
nohup dsmc sched ‐servername=oracle‐tdp‐monthly 2> /dev/null & ‐‐> ( Start up Monthly TDP
backup Full scheduled service ) AIX Command
20 | P a g e

21. V‐Troubleshooting……..
Volume status result :
STatus
Specifies that output is restricted by volume status. This parameter is optional. You can specify multiple
status values by separating values with commas and no intervening spaces. If you do not specify a value
for this parameter, output is not restricted by volume status. Possible values are:
ONline
Display random access volumes that are available to the server.
OFfline
Display random access volumes that are not available to the server.
EMPty
Display sequential access volumes that have no data.
PENding
Display volumes with a status of PENDING. These could be sequential‐access volumes from which all
files have been deleted, but for which the time specified by the REUSEDELAY parameter on the DEFINE
STGPOOL command has not elapsed. These volumes could also be random‐access disk volumes that
were deleted, but that still contain discarded data that is waiting to be shredded. After the data is
shredded, the volume will be physically deleted.
FILling
Display sequential access volumes that the server has written to but has not yet filled to capacity.
FULl
Display sequential access volumes that the server has filled.
21 | P a g e