The document provides an overview of initiatives and findings related to improving backup and restore processes at Century Link. Key accomplishments include developing health check scripts for TSM servers to reduce downtime and improve backup success rates. New findings identified opportunities to streamline TSM tape storage utilization by removing expired nodes and eliminating duplicate data to save on tapes. Issues were also found with some TSM server storage pools not having offsite copies. Suggested changes to address issues included evaluating backup tools, server configurations, and separating test/dev environments. Help was requested to approve removing expired and duplicate data.

1. August 4th

2. Objective : This project is a TECHNICAL INSIGHT
of the Improvement Opportunities in the
BackUp and Restore Area for Century Link .
 Accomplishments.
 New Findings –
 Help Needed.

3. Initiatives
1) Developed Health Check Script & implemented health
checks of TSM SERVERS as a process every six hours.
2) TSM STORAGEPOOL Script implemented
3) Implemented Journal Based Backup to resolve the
issue on server "QTCMHPNTIAP01".
Benefits
1) Pro-active monitoring of TSM Servers Every Six Hours
– Has Reduced the Downtime Risk of TSM Servers and
improved the Back up Success Rate from
91% to 97% .
2) This script has helped in Evaluating the storage Pool
space to accommodate fresh back ups from oracle with
out having them to Fail – This has resulted in Improving
the Oracle Backup Success Rate.
3) This technic can be implemented on servers which
carry more numbers files.
This helps in reducing the Backup Window from
For Eg 2 Million Files which take 5 working days can be
backed up in approximately 2 hours.

4. New Findings - Streamline of TSM TAPE StorageNew Findings - Streamline of TSM TAPE Storage
UtilizationUtilization
◦ .
NODENAME
110330ELXDNP49EL
110330ELXDNP50EL
110330ELXDNT47EL
110330ELXDNT48EL
110413ELXDENVMTC057L
110413EQTDENVMDT143L
110417EQTDENVD188N
110417EQTDENVD189N
110509ERASCALSS
 We have around 573 Expired Nodes to be
removed from TSM SERVERS
 Removing expired nodes data from TSM
SERVERS will release 1000 Tapes
approximately .
 Can benefit the account from XX $$ Value.
Tape Data associated to
Decommissioned servers is not
managed (Deleted)– Resulting into
More Tapes Consumption.
Note : Not aware of any Hidden Associated Risk to delete
the nodes data

5. New – Findings Scope (Intel) : Configuration Issues resulting in
to Data Duplication
NTOMT78CS NTOMT78A NCLUNTOMT78AN NCLUNTOMT78SQL1N No'S'Drive AIOMB52J
QTOMASQLDB4CS QTOMASQLDB4A BCLUQTOMASQLDB4AN BCLUQTOMASQLDB4SQL1N No'S'Drive AIOMB53J
QTOMASQLDB4CS QTOMASQLDB4B BCLUQTOMASQLDB4BN No'R'Drive BCLUQTOMASQLDB4SQL2N AIOMB53J
qtomasql16acs QTOMASQL16A DCLUQTOMASQL16AN DCLUQTOMASQL16SQL1N DCLUQTOMASQL16SQL2N AIOMB54J
QTOMASQL14CS QTOMASQL14A MCLUQTOMASQL14AN MCLUQTOMASQL14SQL1N No'S'Drive AIOMB55J
QTOMASQL11CS QTOMASQL11A DCLUQTOMASQL11AN DCLUQTOMASQL11SQL1N No'S'Drive AIOMB54J
QTDENSQL1CS QTDENSQL1B HQTDENSQL1BN No'R'Drive HCLUQTDENSQL1SQL2N AIDNB866
QTDENSQL2CS QTDENSQL2A HQTDENSQL2AN HCLUQTDENSQL2SQL1N No'S'Drive AIDNB866
QTDENSQL2CS QTDENSQL2B HQTDENSQL2BN No'R'Drive HCLUQTDENSQL2SQL2N AIDNB866
QTDENSQL3CS QTDENSQL3A ACLUQTDENSQL3AN ACLUQTDENSQL3SQL1N No'S'Drive AIDNB864
Identified few Cluster
nodes with TB’s of
duplicated Data :Getting
Rid of this data will save
around 500 Tapes
Approximately.

6. SCHEDULE_NAME COMMAND ACTIVE IssuesinthisTSMSERVER
BACKUP_DEVCONF badevconf YES
BACKUP_STG_ARCHIVE_DISKbastgarchive_diskoffsite_tapeYES
BACKUP_STG_ARCHIVE_TAPEbastgarchive_tapeoffsite_tapeYES
BACKUP_STG_DBA_DISK bastgdba_diskdba_offsiteYES
BACKUP_STG_DBA_FILE bastgdba_filedba_offsiteYES
BACKUP_STG_DBA_TAPE bastgdba_tape_t10kdba_offsiteYES
BACKUP_STG_GOLD_DISK bastggold_diskgold_offsiteYES
BACKUP_STG_GOLD_FILE bastggold_filegold_offsiteYES
BACKUP_STG_GOLD_TAPE bastggold_tape_t10kgold_offsiteYES
AIDNB866-2
Utilization 0%
Utilization 0%
NoBackupStgforDBA_TAPE
NoBackupStgforDBA_VTL
NoBackupStgfor
DBA_XIV_VTL
New – Findings : Configuration Issues resulting in to TSM SERVERTSM SERVER
Storage pools having no offsite copy.Storage pools having no offsite copy.
Approximately 40% of the
Data doesn’t have an
offsite Copy.

7.  What all new tools/technologies can be implemented to resolve few issues?
 Tivoli Monitoring to be in place.
 Use Journal Based Backup wherever required.
 Implementing LANFREE for the servers which has 500GB and above data.
 What all changes needed in DESIGN to have better environment.
 TSM SERVER Configuration needs to be thoroughly checked.
 TEST/DEV to be separated with different Storagepools.
 Password Expiration.
 Help Needed
 Approval for removing the expired & Duplicate data .
 Once Approved, the plan of action will be submitted after working with the US Counter part.

8.  S3
– Save Storage Space [YES CUBE]
 Team Responsibilities
 Weekend Tasks
 Technical KT

9.  Finding the nodes which has more backup data. [List Collected]
 Through check on that host’s each object and checking with customer or AIP whether
these files/objects really need for backup.
 Ex:

10.  AIOMB08R
◦ NSUOMT102S 326.3 GB
◦ NSUOMT800S 2.8 TB
◦ NECOMT106L 379.6 GB
 AIOMB09R
◦ BHPOMP267H 372.5 GB
 AIOMB54J
◦ DSUOMP34QS 187.5 GB
◦ DSUOMP34QS 206.2 GB

11.  Taking individual responsibility on the remedy tickets and resolve them with in the SLA.
Ex:

 More attention on monitoring the TSM Storage Pool migrations to reduce the production
database backup failures count.

12. ◦ Indentified File systems which has no backup from long time and working with Server
Owner’s/AIP’s to get confirmation to remove the data from TSM Storage pool which has
occupied TB’S of space.[Report will be generated by Friday and Weekend Shift Team is
working on this ]
◦ Irrespective of daily missed/failed backups; we have initiated digging in to servers which has
no backup in last 1-5 Days. [This is to ensure that we should not have any of missed/failed
backups repeated on Monday.]

13.  We have started Knowledge Share within the team members on the topics
raised by team
◦ Duration : 1 Hour
◦ Frequency : 2/Week
◦ Topics : Everything related to BACKUP/SAN/AIX
 Topics Covered
◦ LANFREE BACKUPS
◦ NAS BACKUPS